[aleks-mitov]

What this speeds up

Two connection costs that an app built on the SDK pays before its first audio. The API's own latency metrics do not include either one.

1. Session connects build a TLS context every time. Each TTSSession and AsyncTTSSession connect built a new TLS context, which loads and parses the trusted CA certificates before the handshake starts. With a CA file (the system Pythons of Debian and Ubuntu, among others) that takes about 13 to 17 ms on a quiet machine and several times longer on a busy one. Sessions now share one process-wide context: the streaming client's write-first context, built by the first connect (_shared_tls_context() in streaming/v3/_tls.py). The streaming STT client is unchanged.
2. HTTP requests reconnect after 5 s idle. TextToSpeech and AsyncTextToSpeech used the SDK's shared pool, which keeps httpx's 5 s idle expiry, so a voice agent's next sentence more than 5 s after the last one paid for a new TCP and TLS connection. Text-to-Speech requests now use a pool of their own (Client.tts_http_client / AsyncClient.tts_http_client) that keeps idle connections for 120 seconds by default (settings.tts_keepalive_expiry). A TextToSpeech or AsyncTextToSpeech builds that pool when it is constructed, so its first request does not wait for it, and every TTS pool in the process connects with one TLS context (httpx's own default context, built once by _shared_tts_ssl_context()), so a new pool loads no certificates. It still speaks HTTP/1.1: HTTP/2 measured no faster.

Measured

Interleaved A/B of master against this branch: the two SDKs ran in alternating processes, with staggered arrivals and one request in flight, against a staging deployment of the Text-to-Speech API in the client's AWS region (us-west-2). Client: system Python 3.12.3 with OpenSSL 3.0.13 and a 120-certificate CA file, on a shared 8-vCPU machine. Each process's first connect is excluded because both arms build a context there. Deltas are branch minus master, with bootstrap 95% intervals.

What (n per arm)	master p50 / p95	this branch p50 / p95	delta p50 [95% CI]	delta p95 [95% CI]
TTSSession connect to Begin (40)	45.8 / 122.9 ms	23.8 / 37.2 ms	-22.1 [-50.7, -19.3]	-85.7 [-163.3, -37.8]
AsyncTTSSession connect to Begin (40)	47.4 / 103.9 ms	23.3 / 31.2 ms	-24.0 [-39.7, -20.0]	-72.7 [-157.7, -55.2]
TextToSpeech.stream() first audio, requests 6 s apart (40)	54.2 / 89.3 ms	42.8 / 72.8 ms	-11.4 [-25.2, -6.6]	-16.5 [-31.6, +1.9]

• Sessions: 10 alternating rounds of 5 connects per arm, 0.7 s apart. Time from text to first audio did not change (29.1 vs 28.6 ms p50 sync, 29.2 vs 27.1 async), so the whole difference is in the connect. Load average 23.9 p50 (9.5 to 52.0).
• HTTP: the two arms ran in parallel processes, offset by 3 s. Every master request opened a new TLS connection (1.0 handshakes per request). Every request on this branch after the first reused its connection (0.0). Load average 16.5 p50 (9.3 to 19.9). The p95 interval includes zero. This row times later requests on one instance: the bench builds the pool before its timed loop and drops each process's first request, so it shows connection reuse, not the first request. The next table covers the first request.

First request of a new instance

The first push of this PR built the TTS pool on first use, which moved httpx's TLS context build (about 15 ms) into the first request of every instance. This table times that first request with every pool build inside the timed region. Each sample builds a new TextToSpeech(api_key=...) or AsyncTextToSpeech(api_key=...) (a client and pool per instance, the README's per-use pattern), starts the timer just before construction and stops it at the first audio chunk of its first stream(). Nothing touches the pool before the timer starts. Three arms ran in rotating order: master, the first push (7a5ba6a, pool built on first use) and this branch now. 12 rounds of 5 instances per process per arm, 0.7 s apart, one request in flight, same staging deployment and client. Load average 10.7 p50 (3.8 to 17.2). The rows are the instances after the first in each process (n=48 per arm). Deltas are this branch minus master, with bootstrap 95% intervals.

What (n=48 per arm)	master p50 / p95	first push p50 / p95	now p50 / p95	now minus master p50 [95% CI]
TextToSpeech: first request to first audio	50.8 / 79.6 ms	70.1 / 130.0 ms	46.6 / 78.6 ms	-4.2 [-6.5, -0.1]
TextToSpeech: construction to first audio	70.1 / 145.4 ms	91.8 / 214.7 ms	69.0 / 122.9 ms	-1.1 [-18.0, +5.9]
AsyncTextToSpeech: first request to first audio	55.8 / 93.0 ms	67.0 / 140.2 ms	51.0 / 88.6 ms	-4.8 [-11.5, +3.3]
AsyncTextToSpeech: construction to first audio	83.0 / 181.1 ms	86.5 / 259.1 ms	69.4 / 141.3 ms	-13.6 [-52.1, +4.7]

• The first push was slower than master on the first request: +19.3 ms p50 [14.2, 36.7] sync and +11.1 ms [3.9, 27.6] async. This branch now is no slower than master. The small p50 gains are at or near noise, so read them as parity, not as a speedup.
• The first instance of each process (n=12 per arm) builds the shared context once, at construction: construction takes 46.8 vs 30.2 ms p50 sync and 49.0 vs 31.3 ms async, and its first request to first audio matches master (49.7 vs 51.9 ms sync, 61.7 vs 67.1 ms async).
• Construction alone, in process (n=30 per run, three runs, load average 6.8 to 6.9): a later AsyncTextToSpeech() or TextToSpeech(api_key=...) takes 12.7 to 13.4 ms p50 against 12.7 ms on master, and reading tts_http_client after it costs nothing. On the first push that read cost 22 ms p50 (n=30, load average 18.6).

Confidence and caveats:

• The host was heavily loaded (load average 4 to 52 on 8 vCPUs across the runs), which makes building a context slower than it is on a quiet machine. On the same Python, a context build takes 12.8 to 15.4 ms p50 (n=30 each, run three times). That build time is roughly what a quiet client saves per session connect. The loaded numbers above show what a busy client saves.
• A Python whose OpenSSL has no CA file configured, such as the python-build-standalone builds that uv installs, builds a session context in about 0.1 ms. An earlier run on that Python showed no change, as expected.
• The HTTP gain comes from skipping the handshakes, so it grows with distance. In-region it is a few milliseconds. From another region each reconnect costs at least two network round trips (TCP, then TLS 1.3). From round-trip times that lower bound is about 120 ms from us-west-2 to us-east-1 and about 300 ms to eu-north-1. Full TCP plus TLS reconnects measured from the same us-west-2 host to AWS STS endpoints took 182 ms p50 to us-east-1 and 466 ms p50 to eu-north-1 (n=20 each), so the real saving can be larger.
• Production numbers will not move until apps upgrade the SDK.

Compatibility and rollout

• Client side only. Nothing changes on the wire, in the API, or for the streaming STT and transcription clients.
• Idle time of the TTS pool: 120 s by default, or settings.keepalive_expiry when that is set shorter, so someone who lowered it because their network drops idle connections gets the same on TTS. A keepalive_expiry raised past 120 s leaves the TTS pool at 120 s. settings.tts_keepalive_expiry, when set, wins either way. The README and the settings' docstrings say so.
• Dependency: httpx 0.22 or later, up from 0.19. The fork handler (below) drops inherited pools unclosed, and httpx 0.19 to 0.21 close a client in Client.__del__, which takes the pool's lock: a child could deadlock when the dropped pool was collected. httpx 0.22, already the lowest version the test matrix runs, has no __del__. Keeping the dropped pools referenced instead only moves the deadlock to the child's interpreter exit (reproduced on 0.19, 0.20 and 0.21.3) and keeps the parent's sockets open in every child.
• New public surface: settings.tts_keepalive_expiry (env ASSEMBLYAI_TTS_KEEPALIVE_EXPIRY, None by default, >= 0 when set), Client.tts_http_client, AsyncClient.tts_http_client and Client.close(). Version 1.9.0.
• When pools are built: a TextToSpeech or AsyncTextToSpeech builds its client's TTS pool at construction. A Client or AsyncClient that never speaks builds none. Every TTS pool in the process uses one TLS context, httpx's default (certifi's CA bundle, or SSL_CERT_FILE / SSL_CERT_DIR when set), read at the first build; a later change takes effect in a new process. The other APIs' pool keeps building its own context per client, as before.
• Closing: TextToSpeech(api_key=...).close() and AsyncClient.aclose() close both pools. On a sync Client you passed as client=, the new client.close() closes both; the README and docstrings say so. A request after closing raises httpx's closed-client RuntimeError, including when the pool was never built. Constructing an instance on a closed client does not raise; its first request does.
• Idle connections: no request is retried, so a POST is never sent twice. The client never reuses a connection idle longer than the pool's idle time, well under the server's own idle timeout. httpx has no idle reaper, so an idle connection stays open until the next request or close() discards it, or the server closes it. Before reusing an idle connection, httpcore checks it and drops one the server has closed, so a server-side close never fails a request (tested, sync and async). 120 s is also under common NAT idle timeouts (4 minutes on Azure, 350 s on an AWS NAT gateway). A middlebox that silently drops idle connections sooner, with no FIN or RST, would make a reused connection wait for the read timeout. Lower tts_keepalive_expiry in that environment.
• Fork: a forked child drops the TTS pools it inherited (os.register_at_fork), so a request made before a fork, such as a gunicorn --preload warm-up, never leaves parent and child sending on one connection. The child builds its own pool on its next request. Dropping never closes the pool, so it takes none of its locks (needs httpx 0.22 or later, see above). POSIX only, since Windows has no fork.
• The shared session TLS context reflects the CA certificates and SSL_CERT_FILE / SSL_CERT_DIR in place at the first connect. A later change takes effect in a new process.
• Left out on purpose: a helper that keeps a session open ahead of a conversation's turns, and automatic KeepAlive frames. Both hold API capacity while idle.

Tests

• test_streaming_tls.py: the shared session context is built once, also when 8 threads ask for it at the same time, and keeps verification and the write-first socket class. Real TLS 1.3 handshakes through the sync and asyncio session connects against a local websockets server use the shared context on every connect.
• test_tts_session.py / test_tts_session_async.py: two sessions over wss:// pass the same context to websockets, under the installed release's parameter name. A ws:// host passes none.
• test_tts.py / test_tts_async.py: the TTS pool is separate from the other APIs' pool, keeps idle connections 120 s by default, uses HTTP/1.1, is built when a TextToSpeech / AsyncTextToSpeech is constructed (also on the default client) and not by a Client alone, is built only once under concurrent first use, carries last_response, and closes with an owned client or Client.close(); a passed client stays open. Every TTS pool shares one verifying TLS context, built once under 8 concurrent first calls, while the other APIs' pool keeps its own. A closed client constructs without raising and its first request raises.
• Against a local HTTP/1.1 server (tests/unit/tts_http_server.py) that counts connections: consecutive requests reuse one connection at the default 120 s; a request after a 0.2 s expiry opens a new one; with a server that closes each connection 0.2 s after its response and a 120 s expiry, a request 0.5 s later succeeds on a new connection, sync and async. Those last two fail with httpx.ReadError when httpcore's check for idle connections the server closed is disabled, so they cover that case. A forked child uses a pool and connection of its own, and the parent keeps reusing its connection afterwards. No test depends on a timing margin.
• Fork handler without forking: a dropped sync pool that has sent a request is collected without close() being called, and a dropped open async pool is collected without the unclosed-client warning. Both fail on httpx 0.19, 0.20 and 0.21.3 and pass from 0.22. A third test keeps setup.py's httpx floor at 0.22 or later; it fails on the old 0.19 floor.
• Idle time: a keepalive_expiry of 2 s gives the TTS pool 2 s (sync and async), one of 300 s leaves it at 120 s, and a set tts_keepalive_expiry wins over both.
• CI: the test matrix (Python 3.9 to 3.12, plus the websockets 11, httpx 0.22 and 0.24, and pydantic 1.10 floors on 3.11), ruff and the version check ran on the first push. The mypy job fails before it runs: actions/setup-python@v4 finds no Python 3.9 for the ubuntu-latest image (Ubuntu 26.04), a runner change outside this diff. ci(lint): run the mypy job on ubuntu-24.04, which has Python 3.9 #285 pins that job to ubuntu-24.04. Run locally as CI runs it (mypy 1.5.1, --python-version 3.9, no deps installed, --follow-imports=silent --ignore-missing-imports) on the 15 changed files, it is now clean. An earlier version of this description said that run was clean. It was not: it reported 8 errors, one added by this PR (a str | bytes host in a test URL) and 7 already on master in files this PR touches (test_tts.py error attributes, test_streaming_tls.py Optional[str]). All 8 are fixed, so the mypy job will pass on these files once the runner is fixed.
• Full suite at 0803e4a, through the repo's tox envs as CI runs them: 958 passed in each of py39-latest, py311-latest, py311-httpx0.22, py311-httpx0.24, py311-pydantic1.10, py311-websockets11.0 and py312-latest. mypy 1.5.1 as CI runs it (Python 3.9, the 9 changed package files) is clean, and also on Python 3.12. ruff format --check and ruff check (0.3.5) are clean. The connection tests passed 20 of 20 repeats at load average ~9 on an earlier push.

1.9.0 release notes

To paste into the GitHub release when 1.9.0 is cut:

• Text-to-Speech HTTP requests (TextToSpeech, AsyncTextToSpeech) use a connection pool of their own, client.tts_http_client, which keeps idle connections 120 seconds by default, so a voice agent's next sentence reuses the connection. If you set keepalive_expiry shorter, this pool uses it too. tts_keepalive_expiry sets it on its own.
• Headers, event hooks and mocks you attach to client.http_client no longer apply to Text-to-Speech requests. Attach them to client.tts_http_client.
• Client.close() closes both pools.
• TTSSession and AsyncTTSSession connects share one TLS context.
• Requires httpx 0.22 or later (was 0.19).

🤖 Generated with Claude Code

https://claude.ai/code/session_01KcyY6MAEUVjqBt5pMpetvD

[aurpsis-aai]

Holding approval for one fix. TLS is not weakened: both shared contexts keep cert and hostname verification and the stdlib minimum TLS version. The STT client is unchanged, close/aclose cover both pools, and the mypy failure is the runner (setup-python can't find 3.9 on Ubuntu 26.04) before mypy runs; mypy 1.5.1 is clean on the changed files locally.

Blocking (public SDK):

• client.py:98-120, the fork handler: setup.py allows httpx>=0.19. On httpx 0.19 and 0.20, Client.__del__ calls close(), so dropping the pool in the forked child closes it and takes the pool's _thread_lock, which is the deadlock the docstring says it avoids. It also warns "Unclosed AsyncClient" in every child. CI only tests 0.22 and 0.24, so it can't catch this. Fix: keep the dropped clients referenced in the child, or raise the floor to httpx>=0.21.

Should fix (not blocking):

• types.py:174: tts_keepalive_expiry ignores a keepalive_expiry the user set explicitly. Someone who lowered it because a NAT drops idle connections silently now gets 120 s on TTS, and a dropped connection hangs until the 30 s read timeout, with no retry. Consider Optional, defaulting to min(keepalive_expiry, 120) when the user set it.
• types.py:175: the public docstring names the API load balancer's one-hour idle timeout. That's backend detail that will go stale; just say the server keeps connections longer than 120 s.
• MIGRATION.md:225: the note sits under "Migrating to 1.0", so 1.8 to 1.9 users won't see it. Put it in the 1.9.0 release notes, along with: headers, hooks and mocks on client.http_client no longer apply to TTS requests.
• The mypy job: fix it in its own lint.yml PR (pin ubuntu-24.04, or drop 3.9) rather than merging past a red required check.

-- Jeeves

[aleks-mitov]

Thanks. Each point, now at 0803e4a:

• Fork handler: fixed in 431b741, which raises the floor to httpx>=0.22. 0.21 is not enough: httpx 0.21.3 still closes a client in Client.__del__, and keeping the dropped clients referenced only moved the hang to the child's interpreter exit on 0.19, 0.20 and 0.21.3 (reproduced with the pool lock held at fork). The new tests fail on those three versions and pass from 0.22.
• tts_keepalive_expiry: fixed in 0803e4a, so it is None by default and the TTS pool then keeps 120 s, or keepalive_expiry when that is shorter.
• Docstring: fixed in 8d071ca, which drops the load balancer and tells the user to lower the setting if a proxy or NAT closes idle connections sooner.
• MIGRATION.md: fixed in 6643453, which takes the note out of the 1.0 guide; the 1.9.0 release notes, with the client.http_client headers, hooks and mocks item and the httpx floor, are drafted at the end of the PR description.
• mypy runner: ci(lint): run the mypy job on ubuntu-24.04, which has Python 3.9 #285 pins that job to ubuntu-24.04.

[aurpsis-aai]

Approved at 0803e4a. All four points are fixed. The httpx floor is now 0.22, so dropping the TTS pools in a forked child no longer closes them (0.19-0.21 close in del), and tox tracks the new floor. With tts_keepalive_expiry unset, TTS keeps 120 s, or keepalive_expiry when that is shorter. The docstring no longer names server infrastructure, and the misplaced MIGRATION note is gone. Merge after #285 so mypy runs green here. Put two lines in the 1.9.0 release notes: the dependency floor is now httpx>=0.22, and headers, hooks or mocks set on client.http_client no longer apply to TTS requests.

-- Jeeves