feat(tts): share one TLS context across sessions and keep TTS HTTP connections for 120 s

CLAUDE.md

@@ -52,7 +52,7 @@ aai.settings.api_key = "your-key"
 - `aai.AsyncDictationTranscriber` — Asyncio counterpart of `DictationTranscriber`; `open_live()` returns an `AsyncDictationLiveSession`. Owns an HTTP pool: use `async with` or `await aclose()`
 - `aai.DictationConfig` — Dictation options: `sample_rate`, `channels`, `language_codes`, `keyterms_prompt`, `llm_instruction`, `rewrite_mode` (`aai.DictationRewriteMode.fast` default | `smart`). Rejects unknown fields
 - `aai.DictationResponse` — Dictation result: `.text`, `.words` (`DictationWord` with `text` + `confidence`), `.confidence`, `.llm_response`, `.llm_error`, `.audio_duration_ms`, `.session_id`, `.request_time_ms`, `.sync_time_ms`, and `.final_text` (the LLM rewrite, falling back to `.text`)
-- `aai.TextToSpeech` — Text-to-Speech HTTP API: one text per request, audio out. Methods: `synthesize()` (complete audio as a `TTSResponse`), `stream()` (a `TTSStream` context manager yielding bytes as the API generates them), `close()` / `with` for a client built from `api_key=`. Host from `aai.settings.tts_base_url` (`https://tts.us.assemblyai.com`; set `https://tts.eu.assemblyai.com` for the EU zone)
+- `aai.TextToSpeech` — Text-to-Speech HTTP API: one text per request, audio out. Methods: `synthesize()` (complete audio as a `TTSResponse`), `stream()` (a `TTSStream` context manager yielding bytes as the API generates them), `close()` / `with` for a client built from `api_key=`. Host from `aai.settings.tts_base_url` (`https://tts.us.assemblyai.com`; set `https://tts.eu.assemblyai.com` for the EU zone). Requests use the client's own TTS pool (`client.tts_http_client`, built when the `TextToSpeech` is), which keeps idle connections 120 s by default (`aai.settings.tts_keepalive_expiry`; unset, a shorter `keepalive_expiry` applies), so consecutive sentences reuse one connection. Every TTS pool shares one process-wide TLS context. A passed `client=` stays the caller's: `client.close()` closes both pools
 - `aai.AsyncTextToSpeech` — Asyncio counterpart of `TextToSpeech`; `synthesize()` is a coroutine and `stream()` returns an `AsyncTTSStream`. Owns an HTTP pool: use `async with` or `await aclose()`
 - `aai.TTSSession` — Text-to-Speech WebSocket session: one connection, text in as it is written, audio out. `with` connects and waits for `Begin` (`session.begin`); `speak(text)` yields one request's audio; `generate(text)` / `flush()` -> flush_id / `audio(flush_id)` for streamed-in text and pipelining (`audio()` with no id follows the request being written, from another thread); `cancel()` for barge-in (ends the iteration at once); `keep_alive()`; `on(TTSEvents.X, handler(event))` for `Begin`, `Audio` (switches audio from pull to push), `FlushDone`, `WordBoundaries`, `Cancelled`, `Termination`, `Error`. Exit sends `Terminate` and sets `session.termination` (`None` after a failure). Refusals and failures raise `TTSError` (`status_code` None, `request_id` = session id). Raw PCM/G.711 only; 2,000 chars per generate unless `Begin` reports another `limits.max_generate_text_length`; ~100 chars/s sustained. `token=` for temporary tokens
 - `aai.AsyncTTSSession` — Asyncio counterpart: `async with`, coroutine sends, `async for chunk in session.speak(text)`
@@ -325,7 +325,9 @@ it ends," raise `keepalive_expiry` (e.g. to 120, the sync audio cap) so one `war
 covers the whole clip; otherwise call `warm()` shortly before `transcribe()` (it is
 idempotent, so re-warming to refresh is fine). It only helps the *first* request: sync
 calls share the client's connection pool, so later `transcribe()` calls already reuse the
-open connection. Note `keepalive_expiry` applies to the whole client (sync and async).
+open connection. Note `keepalive_expiry` applies to the whole client (sync and async) except
+Text-to-Speech requests, whose own pool keeps idle connections 120s, or `keepalive_expiry` when
+that is shorter (`tts_keepalive_expiry` overrides both).
 
 ## Streaming (real-time)
 

README.md

@@ -894,7 +894,7 @@ except aai.DictationError as error:
 
 `aai.TextToSpeech` speaks one text per HTTP request on the Text-to-Speech API. `synthesize()` returns the complete audio; `stream()` returns it as the API generates it, so playback can start long before the sentence ends. A text is 1 to 5,000 characters. Options go in a `TTSConfig`: the `voice_id` (a voice of the AssemblyAI voice library by its slug: `aai-jane` by default, `aai-alba`, `aai-anna`, `aai-charles`, `aai-eve`, `aai-george`, `aai-jean`, `aai-mary`, `aai-michael`, `aai-paul`, `aai-vera` and hundreds more; or a voice cloned by your account, by its UUID or slug), the `model_id` (`assemblyai-tts-1`), the `language` (`english`, `spanish`, `german`, `french`, `italian`, `portuguese`), the `sample_rate` (8000, 16000, 22050, 24000 by default, 44100, 48000), the `encoding` (`pcm_s16le` by default, `pcm_mulaw`/`pcm_alaw` for telephony, `mp3`) and the `container` (`raw` or `wav`). A per-call `TTSConfig` replaces the client's default config whole. Voices are created and managed in the AssemblyAI dashboard.
 
-Requests go to `aai.settings.tts_base_url`, `https://tts.us.assemblyai.com` by default; set it to `https://tts.eu.assemblyai.com` to keep them in the EU data zone.
+Requests go to `aai.settings.tts_base_url`, `https://tts.us.assemblyai.com` by default; set it to `https://tts.eu.assemblyai.com` to keep them in the EU data zone. They share a connection pool of their own that keeps an idle connection for 120 seconds by default, so a request that follows the previous one within that time, such as a voice agent's next sentence, reuses its connection instead of opening a new one. If you set `aai.settings.keepalive_expiry` shorter than that, this pool uses it too; `aai.settings.tts_keepalive_expiry` sets this pool's idle time on its own. A client you pass as `client=` stays yours to close: `client.close()` closes both of its pools.
 
 <details>
   <summary>Synthesize a WAV file</summary>
@@ -2042,6 +2042,7 @@ aai.settings.dictation_http_timeout = 300.0
 aai.settings.tts_base_url = "https://tts.eu.assemblyai.com"
 aai.settings.tts_http_timeout = 30.0  # stream(): each socket operation
 aai.settings.tts_synthesize_http_timeout = 610.0  # synthesize(): the wait for the whole clip
+aai.settings.tts_keepalive_expiry = 120.0  # seconds an idle connection stays in the TTS pool (unset: 120, or keepalive_expiry when shorter)
 ```
 
 Set these before creating a client: `aai.TextToSpeech()` (like every client) copies the settings when it is built, so a later change applies to the next instance, not to an existing one. Every setting can also come from the environment with the `ASSEMBLYAI_` prefix, such as `ASSEMBLYAI_TTS_BASE_URL`.

assemblyai/__version__.py

@@ -1 +1 @@
-__version__ = "1.8.2"
+__version__ = "1.9.0"

assemblyai/async_client.py

@@ -5,7 +5,15 @@
 from typing_extensions import Self
 
 from . import types
-from .client import _MISSING_API_KEY_ERROR, _build_headers, _build_limits
+from .client import (
+    _CLOSED_CLIENT_ERROR,
+    _MISSING_API_KEY_ERROR,
+    _TTS_POOL_HOLDERS,
+    _build_headers,
+    _build_limits,
+    _build_tts_limits,
+    _shared_tts_ssl_context,
+)
 
 
 class AsyncClient:
@@ -62,14 +70,20 @@ def __init__(
         async def _store_response(response: httpx.Response) -> None:
             self._last_response = response
 
+        self._event_hooks = {"response": [_store_response]}
         self._http_client = httpx.AsyncClient(
             base_url=self._settings.base_url,
             headers=_build_headers(self._settings),
             timeout=self._settings.http_timeout,
             limits=_build_limits(self._settings),
-            event_hooks={"response": [_store_response]},
+            event_hooks=self._event_hooks,
         )
 
+        # Built when an `AsyncTextToSpeech` takes this client, or on first
+        # use, so a client that never speaks pays nothing for it.
+        self._tts_http_client: Optional[httpx.AsyncClient] = None
+        self._closed = False
+
     @property
     def last_response(self) -> Optional[httpx.Response]:
         """
@@ -102,10 +116,64 @@ def http_client(self) -> httpx.AsyncClient:
 
         return self._http_client
 
-    async def aclose(self) -> None:
-        """Closes the underlying HTTP connection pool."""
+    @property
+    def tts_http_client(self) -> httpx.AsyncClient:
+        """
+        The HTTP client of the Text-to-Speech API, built when an
+        `AsyncTextToSpeech` takes this client or on first use.
+
+        A connection pool of its own, whose idle connections stay 120
+        seconds by default (see `settings.tts_keepalive_expiry`); see
+        `Client.tts_http_client`. `aclose()` closes it with `http_client`.
+
+        Raises:
+            RuntimeError: if the client was closed before the pool was ever
+                built.
+        """
+
+        client = self._ensure_tts_http_client()
+        if client is None:
+            raise RuntimeError(_CLOSED_CLIENT_ERROR)
+
+        return client
 
-        await self._http_client.aclose()
+    def _ensure_tts_http_client(self) -> Optional[httpx.AsyncClient]:
+        """
+        Returns the Text-to-Speech pool, building it if it is not built yet,
+        or `None` if the client was closed before the pool was ever built.
+
+        `AsyncTextToSpeech` calls it when it is constructed, so its first
+        request does not wait for the build.
+        """
+
+        client = self._tts_http_client
+        if client is None and not self._closed:
+            client = self._tts_http_client = httpx.AsyncClient(
+                base_url=self._settings.tts_base_url,
+                headers=_build_headers(self._settings),
+                timeout=self._settings.tts_http_timeout,
+                limits=_build_tts_limits(self._settings),
+                event_hooks=self._event_hooks,
+                verify=_shared_tts_ssl_context(),
+            )
+            _TTS_POOL_HOLDERS.add(self)
+
+        return client
+
+    def _forget_tts_http_client(self) -> None:
+        """Drops the Text-to-Speech pool unclosed; see `client._forget_tts_pools_in_child`."""
+
+        self._tts_http_client = None
+
+    async def aclose(self) -> None:
+        """Closes the underlying HTTP connection pools."""
+
+        self._closed = True
+        try:
+            await self._http_client.aclose()
+        finally:
+            if self._tts_http_client is not None:
+                await self._tts_http_client.aclose()
 
     async def __aenter__(self) -> Self:
         return self

assemblyai/client.py

@@ -1,6 +1,9 @@
+import os
+import ssl
 import sys
 import threading
-from typing import ClassVar, Dict, Optional
+import weakref
+from typing import Any, ClassVar, Dict, Optional
 
 import httpx
 
@@ -36,6 +39,111 @@ def _build_limits(settings: types.Settings) -> httpx.Limits:
     )
 
 
+# Seconds the Text-to-Speech pool keeps an idle connection when neither
+# `tts_keepalive_expiry` nor `keepalive_expiry` is set.
+_DEFAULT_TTS_KEEPALIVE_EXPIRY = 120.0
+
+
+def _tts_keepalive_expiry(settings: types.Settings) -> float:
+    """
+    Returns the seconds the Text-to-Speech pool keeps an idle connection:
+    `settings.tts_keepalive_expiry` when set, else 120, or
+    `settings.keepalive_expiry` when that is set lower (someone who lowered
+    it because their network drops idle connections gets the same here).
+    """
+
+    if settings.tts_keepalive_expiry is not None:
+        return settings.tts_keepalive_expiry
+    if settings.keepalive_expiry is not None:
+        return min(settings.keepalive_expiry, _DEFAULT_TTS_KEEPALIVE_EXPIRY)
+    return _DEFAULT_TTS_KEEPALIVE_EXPIRY
+
+
+def _build_tts_limits(settings: types.Settings) -> httpx.Limits:
+    """
+    Builds the limits of the Text-to-Speech pool: no cap on connections, as
+    from `_build_limits`, and idle connections kept
+    `_tts_keepalive_expiry(settings)` seconds. Shared with `AsyncClient`.
+    """
+
+    return httpx.Limits(keepalive_expiry=_tts_keepalive_expiry(settings))
+
+
+# What httpx raises from a request on a closed client, raised as well when a
+# client closed before it built its Text-to-Speech pool is asked for that pool.
+_CLOSED_CLIENT_ERROR = "Cannot send a request, as the client has been closed."
+
+
+# The context `_shared_tts_ssl_context()` returns once it has been built.
+_tts_ssl_context: Optional[ssl.SSLContext] = None
+_TTS_SSL_CONTEXT_LOCK = threading.Lock()
+
+
+def _shared_tts_ssl_context() -> ssl.SSLContext:
+    """
+    Returns the TLS context of every Text-to-Speech pool in the process,
+    built on the first call. Shared with `AsyncClient`.
+
+    It is the context httpx would build for the pool by default (certifi's CA
+    bundle, or `SSL_CERT_FILE` / `SSL_CERT_DIR` when set), built once instead
+    of once per pool. A build loads and parses the CA bundle, about 15 ms on an
+    idle machine with OpenSSL 3.0 and several times that on a busy one, and
+    each `AsyncTextToSpeech` built without `client=`, like each `TextToSpeech`
+    built from `api_key=`, has a pool of its own. One context serves any
+    number of pools, threads and event loops; each connection keeps its own
+    TLS state.
+
+    Only HTTP/1.1 pools may use it: httpcore sets the context's ALPN protocols
+    at every connect, to `http/1.1` for an HTTP/1.1 pool.
+
+    The CA certificates and environment it reflects are those in place at the
+    first call; a later change takes effect in a new process.
+    """
+
+    global _tts_ssl_context
+    context = _tts_ssl_context
+    if context is None:
+        with _TTS_SSL_CONTEXT_LOCK:
+            context = _tts_ssl_context
+            if context is None:
+                context = _tts_ssl_context = httpx.create_ssl_context()
+    return context
+
+
+# Every `Client` and `AsyncClient` that has built a Text-to-Speech pool, for
+# `_forget_tts_pools_in_child()`.
+_TTS_POOL_HOLDERS: "weakref.WeakSet[Any]" = weakref.WeakSet()
+
+
+def _forget_tts_pools_in_child() -> None:
+    """
+    Runs in a child process right after `fork()`, and drops every
+    Text-to-Speech pool the child inherited, so the child's next request
+    builds a pool of its own.
+
+    An inherited pool holds the parent's idle connections, kept up to 120
+    seconds by default: a request made before the fork, such as a warm-up in
+    a `gunicorn --preload` app, leaves one there. Processes sending on one
+    connection at once garble each other's responses. The pools are dropped
+    rather than closed: another thread of the parent may have held a pool's
+    lock at the moment of the fork, and an async pool needs its event loop.
+    Each process closes its own socket descriptors, so the parent's
+    connections stay intact.
+
+    Dropping relies on httpx not closing a client when it is garbage
+    collected, which holds from httpx 0.22, the floor in `setup.py`. httpx
+    0.19 to 0.21 close it in `Client.__del__`, which takes the pool's lock.
+    """
+
+    for holder in list(_TTS_POOL_HOLDERS):
+        holder._forget_tts_http_client()
+    _TTS_POOL_HOLDERS.clear()
+
+
+if hasattr(os, "register_at_fork"):  # POSIX only
+    os.register_at_fork(after_in_child=_forget_tts_pools_in_child)
+
+
 _MISSING_API_KEY_ERROR = (
     "Please provide an API key: set the ASSEMBLYAI_API_KEY environment variable, "
     "set aai.settings.api_key, or pass api_key= to the transcriber or client."
@@ -79,14 +187,21 @@ def __init__(
         def _store_response(response):
             self._last_response = response
 
+        self._event_hooks = {"response": [_store_response]}
         self._http_client = httpx.Client(
             base_url=self.settings.base_url,
             headers=_build_headers(self._settings),
             timeout=self.settings.http_timeout,
             limits=_build_limits(self._settings),
-            event_hooks={"response": [_store_response]},
+            event_hooks=self._event_hooks,
         )
 
+        # Built when a `TextToSpeech` takes this client, or on first use, so
+        # a client that never speaks pays nothing for it.
+        self._tts_http_client: Optional[httpx.Client] = None
+        self._tts_http_client_closed = False
+        self._tts_http_client_lock = threading.Lock()
+
     @property
     def last_response(self) -> Optional[httpx.Response]:
         """
@@ -119,6 +234,85 @@ def http_client(self) -> httpx.Client:
 
         return self._http_client
 
+    @property
+    def tts_http_client(self) -> httpx.Client:
+        """
+        The HTTP client of the Text-to-Speech API, built when a
+        `TextToSpeech` takes this client or on first use.
+
+        A connection pool of its own, whose idle connections stay 120
+        seconds by default (see `settings.tts_keepalive_expiry`): a voice
+        agent's next sentence reuses the connection of the previous one
+        instead of opening a new one. `http_client` is unaffected.
+        Responses update `last_response` as those of `http_client` do.
+        `close()` closes it with `http_client`.
+
+        Raises:
+            RuntimeError: if the pool was closed before it was ever built.
+        """
+
+        client = self._ensure_tts_http_client()
+        if client is None:
+            raise RuntimeError(_CLOSED_CLIENT_ERROR)
+
+        return client
+
+    def _ensure_tts_http_client(self) -> Optional[httpx.Client]:
+        """
+        Returns the Text-to-Speech pool, building it if it is not built yet,
+        or `None` if the pool was closed before it was ever built.
+
+        `TextToSpeech` calls it when it is constructed, so its first request
+        does not wait for the build.
+        """
+
+        client = self._tts_http_client
+        if client is None:
+            with self._tts_http_client_lock:
+                client = self._tts_http_client
+                if client is None and not self._tts_http_client_closed:
+                    client = self._tts_http_client = httpx.Client(
+                        base_url=self._settings.tts_base_url,
+                        headers=_build_headers(self._settings),
+                        timeout=self._settings.tts_http_timeout,
+                        limits=_build_tts_limits(self._settings),
+                        event_hooks=self._event_hooks,
+                        verify=_shared_tts_ssl_context(),
+                    )
+                    _TTS_POOL_HOLDERS.add(self)
+
+        return client
+
+    def _forget_tts_http_client(self) -> None:
+        """Drops the Text-to-Speech pool unclosed; see `_forget_tts_pools_in_child`."""
+
+        self._tts_http_client = None
+        # Another thread of the parent may have held the lock at the fork.
+        self._tts_http_client_lock = threading.Lock()
+
+    def _close_tts_http_client(self) -> None:
+        """Closes the Text-to-Speech pool, and keeps it from being built later."""
+
+        with self._tts_http_client_lock:
+            self._tts_http_client_closed = True
+            client = self._tts_http_client
+        if client is not None:
+            client.close()
+
+    def close(self) -> None:
+        """
+        Closes both HTTP connection pools, `http_client` and `tts_http_client`.
+
+        A request after it raises httpx's closed-client `RuntimeError`. Leave
+        the default client (`Client.get_default()`) open: every instance built
+        without `client=` or `api_key=` shares it.
+        """
+
+        try:
+            self._http_client.close()
+        finally:
+            self._close_tts_http_client()
+
     @classmethod
     def get_default(cls, api_key_required: bool = True):
         """