From d606059eb46a6f55fc020e87a437884f1e0733ab Mon Sep 17 00:00:00 2001 From: Aleksandar Mitov <140423361+aleks-mitov@users.noreply.github.com> Date: Wed, 21 Oct 2026 21:49:54 +0000 Subject: [PATCH 1/3] feat(tts): name every voice by voice_id and follow the API's latest contract The Text-to-Speech API retires the `voice` parameter: every voice is named by `voice_id` (a library slug such as `aai-jane`, a voice UUID, or one of the account's voice slugs), and a request that sends `voice` is refused 422 invalid_parameter. `X-TTS-Voice` gives way to `X-TTS-Voice-Id`, which carries the voice used (`aai-jane` when the request named none). - TTSConfig has no `voice` field. Passing it is refused like any unknown field, with a message that points to `voice_id`. - TTSResponse and TTSStream.info expose `voice_id` from `X-TTS-Voice-Id` (None from a server that does not send it yet) and `region` from `X-TTS-Region`; `X-TTS-Voice` is no longer read. - The biometric terms check is documented where errors are: 403 biometric_terms_not_accepted and 503 auth_unavailable, in the TTSError and extend_biometric_consent docstrings, the README and CLAUDE.md. - A streamed body cut after the first audio raises a retryable TTSError (status_code 200, the request's request_id, the httpx error as its cause) instead of a bare httpx error. - synthesize() reads for up to 610 s, over the API's 600 s request limit, so the API's own 504 session_expired arrives as a TTSError; connecting and sending use tts_http_timeout. - The pydantic version is probed with `model_validator` (v2 only), as the dictation and streaming models do: the `types.py` flag is False on pydantic 2 without pydantic-settings, where the v1 branch defined a class-based Config on a v2 model. `protected_namespaces=()` stops the `model_id` warning on pydantic 2.0 to 2.9. - Docstrings: voice_id forms and zones, the language rule, the streamed values of voice_expires_at and billed_characters, audio_duration_ms on responses without audio, empty audio. `container` takes a string, as `encoding` does; `audio` is left out of the repr. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01KcyY6MAEUVjqBt5pMpetvD --- CLAUDE.md | 18 +- README.md | 22 ++- assemblyai/tts/v1/_base.py | 19 +- assemblyai/tts/v1/api.py | 60 ++++-- assemblyai/tts/v1/async_api.py | 30 ++- assemblyai/tts/v1/async_client.py | 9 +- assemblyai/tts/v1/client.py | 22 ++- assemblyai/tts/v1/models.py | 153 ++++++++++----- assemblyai/types.py | 6 +- tests/unit/test_tts.py | 300 ++++++++++++++++++++++++++++-- tests/unit/test_tts_async.py | 153 ++++++++++++++- 11 files changed, 675 insertions(+), 117 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index bd92aee..76c63d8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -54,9 +54,9 @@ aai.settings.api_key = "your-key" - `aai.DictationResponse` — Dictation result: `.text`, `.words` (`DictationWord` with `text` + `confidence`), `.confidence`, `.llm_response`, `.llm_error`, `.audio_duration_ms`, `.session_id`, `.request_time_ms`, `.sync_time_ms`, and `.final_text` (the LLM rewrite, falling back to `.text`) - `aai.TextToSpeech` — Text-to-Speech HTTP API: one text per request, audio out. Methods: `synthesize()` (complete audio as a `TTSResponse`), `stream()` (a `TTSStream` context manager yielding bytes as the API generates them). Host from `aai.settings.tts_base_url` (`https://tts.us.assemblyai.com`; set `https://tts.eu.assemblyai.com` for the EU zone) - `aai.AsyncTextToSpeech` — Asyncio counterpart of `TextToSpeech`; `synthesize()` is a coroutine and `stream()` returns an `AsyncTTSStream`. Owns an HTTP pool: use `async with` or `await aclose()` -- `aai.TTSConfig` — Text-to-Speech options: `voice` (preset name) or `voice_id` (a cloned or library voice), `model_id`, `language`, `sample_rate`, `encoding` (`aai.TTSEncoding`: `pcm_s16le` default | `pcm_mulaw` | `pcm_alaw` | `mp3`), `container` (`aai.TTSContainer`: `raw` default | `wav`), `bit_rate`. Rejects unknown fields -- `aai.TTSResponse` — Text-to-Speech result: `.audio` (bytes) plus the response headers: `.request_id`, `.model`, `.voice`, `.language`, `.encoding`, `.sample_rate`, `.container`, `.bit_rate`, `.content_type`, `.voice_expires_at`, `.billed_characters`, `.audio_duration_ms`. `TTSStream.info` carries the same fields without `audio`; `audio_duration_ms` is `None` on a stream -- `aai.TTSError` — Raised on any Text-to-Speech refusal: `.status_code`, `.error_code` (stable API code), `.param`, `.request_id`, `.retryable`, `.retry_after`, `.errors` +- `aai.TTSConfig`: Text-to-Speech options: `voice_id` (every voice: a library slug such as `aai-jane`, the default, a voice UUID, or one of the account's voice slugs; there is no `voice` field), `model_id` (`assemblyai-tts-1`), `language`, `sample_rate`, `encoding` (`aai.TTSEncoding`: `pcm_s16le` default | `pcm_mulaw` | `pcm_alaw` | `mp3`), `container` (`aai.TTSContainer`: `raw` default | `wav`), `bit_rate`, `extend_biometric_consent`. Rejects unknown fields +- `aai.TTSResponse`: Text-to-Speech result: `.audio` (bytes) plus the response headers: `.request_id`, `.region`, `.model`, `.voice_id` (from `X-TTS-Voice-Id`), `.language`, `.encoding`, `.sample_rate`, `.container`, `.bit_rate`, `.content_type`, `.voice_expires_at`, `.billed_characters`, `.audio_duration_ms`. `TTSStream.info` carries the same fields without `audio`; `audio_duration_ms` is `None` on a stream +- `aai.TTSError`: Raised on any Text-to-Speech refusal, and on a stream cut after its first audio (`status_code` 200, retryable): `.status_code`, `.error_code` (stable API code, such as `voice_not_found`, `biometric_terms_not_accepted` (403), `auth_unavailable` (503)), `.param`, `.request_id`, `.retryable`, `.retry_after`, `.errors` - `assemblyai.streaming.v3.RealTimeTranscriber` — Real-time streaming with event-based API (threaded) - `assemblyai.streaming.v3.AsyncRealTimeTranscriber` — Asyncio-native counterpart; same options/events - `aai.LLMGateway` — LLM Gateway client. Resources: `models.list()`, `chat.completions.create()` (incl. `stream=True`), `understanding.create()`/`.validate()` @@ -459,25 +459,25 @@ async with AsyncRealTimeTranscriber(RealTimeTranscriberOptions(token=token_from_ ## Text-to-Speech -One text per HTTP request. `synthesize()` returns the whole audio; `stream()` yields it as it is generated. An API refusal is always `aai.TTSError`; network failures are httpx exceptions, and `httpx.RemoteProtocolError` from a stream loop means the body was cut after the first audio, so retry the request. Settings are copied when a client is built: set `tts_base_url` before creating `TextToSpeech`. `synthesize()` waits for the whole clip under `settings.tts_synthesize_http_timeout` (120 s); `stream()` uses `settings.tts_http_timeout` (30 s) per socket operation. +One text per HTTP request. `synthesize()` returns the whole audio; `stream()` yields it as it is generated. Every voice is named by `voice_id` (`aai-jane` by default; `TTSConfig(voice=...)` is refused). An API failure is always `aai.TTSError`, including a stream cut after the first audio (`status_code` 200, `retryable`; retry the whole request); network failures and client timeouts are httpx exceptions. `403 biometric_terms_not_accepted` refuses the account's own cloned voices until it accepts the biometric data terms in the dashboard. Settings are copied when a client is built: set `tts_base_url` before creating `TextToSpeech`. `synthesize()` waits for the whole clip under `settings.tts_synthesize_http_timeout` (610 s read, over the API's 600 s limit); `stream()` uses `settings.tts_http_timeout` (30 s) per socket operation. ```python -tts = aai.TextToSpeech(config=aai.TTSConfig(voice="jane")) +tts = aai.TextToSpeech(config=aai.TTSConfig(voice_id="aai-jane")) -# A per-call config replaces the default whole: repeat the voice in it -wav = tts.synthesize("Hello.", aai.TTSConfig(voice="jane", container=aai.TTSContainer.wav)).audio +# A per-call config replaces the default whole: repeat the voice_id in it +wav = tts.synthesize("Hello.", aai.TTSConfig(voice_id="aai-jane", container=aai.TTSContainer.wav)).audio with tts.stream("Hello.") as speech: for chunk in speech: # raw 24 kHz s16le mono by default play(chunk) # Telephony: 8 kHz mu-law (chunk sizes vary; re-frame to 160 bytes for 20 ms frames) -telephony = aai.TTSConfig(voice="jane", encoding=aai.TTSEncoding.pcm_mulaw, sample_rate=8000) +telephony = aai.TTSConfig(voice_id="aai-jane", encoding=aai.TTSEncoding.pcm_mulaw, sample_rate=8000) with tts.stream("Hello.", telephony) as speech: for chunk in speech: send_to_call(chunk) -# A cloned or library voice by id +# Another library voice, or one of the account's cloned voices by its UUID or slug tts.synthesize("Hello.", aai.TTSConfig(voice_id="aai-alba")) ``` diff --git a/README.md b/README.md index e2e41b9..2ea345a 100644 --- a/README.md +++ b/README.md @@ -892,7 +892,9 @@ except aai.DictationError as error: ### **Text-to-Speech Examples** -`aai.TextToSpeech` speaks one text per HTTP request on the Text-to-Speech API. `synthesize()` returns the complete audio; `stream()` returns it as the API generates it, so playback can start long before the sentence ends. A text is 1 to 5,000 characters. Options go in a `TTSConfig`: a preset `voice` (`jane` by default, `alba`, `anna`, `charles`, `eve`, `george`, `jean`, `mary`, `michael`, `paul`, `vera`) or a `voice_id` (a voice cloned by your account, or a voice of the AssemblyAI voice library by its slug such as `aai-jane`), the `language` (`english`, `spanish`, `german`, `french`, `italian`, `portuguese`), the `sample_rate` (8000, 16000, 22050, 24000 by default, 44100, 48000), the `encoding` (`pcm_s16le` by default, `pcm_mulaw`/`pcm_alaw` for telephony, `mp3`) and the `container` (`raw` or `wav`). A per-call `TTSConfig` replaces the client's default config whole. Voices are created and managed in the AssemblyAI dashboard. +`aai.TextToSpeech` speaks one text per HTTP request on the Text-to-Speech API. `synthesize()` returns the complete audio; `stream()` returns it as the API generates it, so playback can start long before the sentence ends. A text is 1 to 5,000 characters. Options go in a `TTSConfig`: the `voice_id`, the `model_id` (`assemblyai-tts-1`, or the language's default model when left out), the `language` (`english`, `spanish`, `german`, `french`, `italian`, `portuguese`), the `sample_rate` (8000, 16000, 22050, 24000 by default, 44100, 48000), the `encoding` (`pcm_s16le` by default, `pcm_mulaw`/`pcm_alaw` for telephony, `mp3`), the `container` (`raw` or `wav`), the MP3 `bit_rate` and `extend_biometric_consent`. A per-call `TTSConfig` replaces the client's default config whole. + +Every voice is named by `voice_id`: a voice of the AssemblyAI voice library by its slug (`aai-jane` by default, `aai-alba`, `aai-anna`, `aai-charles`, `aai-eve`, `aai-george`, `aai-jean`, `aai-mary`, `aai-michael`, `aai-paul`, `aai-vera`, and the rest of the library), a voice's ID (a UUID), or the slug of one of your own voices. `TTSConfig` has no `voice` field. Without `language` a request speaks the voice's own language; the eleven built-in voices listed above speak all six. Voices are created and managed in the AssemblyAI dashboard, which shows the value that names each one. A cloned voice of your account needs the account to have accepted the biometric data terms (the data controls page of the dashboard), and `extend_biometric_consent=True` renews its consent for another 60 days. Requests go to `aai.settings.tts_base_url`, `https://tts.us.assemblyai.com` by default; set it to `https://tts.eu.assemblyai.com` to keep them in the EU data zone. @@ -908,17 +910,17 @@ tts = aai.TextToSpeech() response = tts.synthesize( "Hello from AssemblyAI.", - aai.TTSConfig(voice="jane", container=aai.TTSContainer.wav), + aai.TTSConfig(voice_id="aai-jane", container=aai.TTSContainer.wav), ) # A config given per call replaces the client's default config whole. with open("hello.wav", "wb") as out: out.write(response.audio) -print(response.model, response.voice, response.audio_duration_ms, "ms") +print(response.model, response.voice_id, response.audio_duration_ms, "ms") ``` -`synthesize()` waits for the whole audio (`aai.settings.tts_synthesize_http_timeout`, 120 s by default), so an API refusal is always a `TTSError`, never a cut response, and the WAV header carries real sizes. Network failures raise httpx exceptions, as everywhere in the SDK. Python's `wave` module reads PCM WAV only; a mu-law or A-law WAV needs another reader, or use `container="raw"` for telephony codecs. The response headers are on the result: `request_id`, `model`, `voice`, `language`, `encoding`, `sample_rate`, `container`, `bit_rate`, `billed_characters` and `audio_duration_ms`. +`synthesize()` waits for the whole audio (`aai.settings.tts_synthesize_http_timeout`, 610 s by default, a little over the API's own 600-second limit on a request), so an API failure is always a `TTSError`, never a cut response, and the WAV header carries real sizes. Network failures and client timeouts raise httpx exceptions, as everywhere in the SDK. Python's `wave` module reads PCM WAV only; a mu-law or A-law WAV needs another reader, or use `container="raw"` for telephony codecs. The response headers are on the result: `request_id`, `region`, `model`, `voice_id` (the voice used, `aai-jane` when the request named none), `language`, `encoding`, `sample_rate`, `container`, `bit_rate`, `voice_expires_at`, `billed_characters` and `audio_duration_ms`. A text with nothing to speak (only emoji, for example) returns empty audio and bills nothing. @@ -930,7 +932,7 @@ import assemblyai as aai aai.settings.api_key = "" -tts = aai.TextToSpeech(config=aai.TTSConfig(voice="alba")) +tts = aai.TextToSpeech(config=aai.TTSConfig(voice_id="aai-alba")) with tts.stream("Streaming straight to the speakers.") as speech: print("speaking with", speech.info.model, "at", speech.info.sample_rate, "Hz") @@ -938,7 +940,7 @@ with tts.stream("Streaming straight to the speakers.") as speech: play(chunk) # raw 24 kHz 16-bit mono PCM ``` -`stream()` returns once the API has sent the first audio, usually well under a second. Chunk sizes vary and are not aligned to samples or frames: for 16-bit PCM carry an odd trailing byte over to the next chunk, and re-frame into fixed-size frames (20 ms of 8 kHz mu-law is 160 bytes) if your sink needs them. Breaking out of the loop early (a barge-in) drops that connection, so the next request pays a new TLS handshake, about 100 ms more to its first audio. The API commits to `200` with the first audio, so a failure after that cuts the body instead of changing the status: `httpx.RemoteProtocolError` is raised from the loop, and the whole request should be retried. +`stream()` returns once the API has sent the first audio, usually well under a second. Chunk sizes vary and are not aligned to samples or frames: for 16-bit PCM carry an odd trailing byte over to the next chunk, and re-frame into fixed-size frames (20 ms of 8 kHz mu-law is 160 bytes) if your sink needs them. Breaking out of the loop early (a barge-in) drops that connection, so the next request pays a new TLS handshake, about 100 ms more to its first audio. The API commits to `200` with the first audio, so a failure after that cuts the body instead of changing the status: the loop raises a `TTSError` with `status_code` 200, `retryable` `True` and the request's `request_id` (the httpx error is its `__cause__`), and the whole request should be retried. On a stream, `info.billed_characters` holds once the audio is read to its end (a cut on the API's side bills nothing), and `info.voice_expires_at` is the expiry as the request started. @@ -961,6 +963,8 @@ with aai.TextToSpeech(config=telephony).stream("Your appointment is confirmed.") send_to_call(chunk) ``` +A cloned voice works on the host of the data zone it was created in, and only once the account has accepted the biometric data terms on the data controls page of the dashboard: until then requests with it are refused `403 biometric_terms_not_accepted`. +
@@ -982,7 +986,7 @@ except aai.TTSError as e: time.sleep(e.retry_after or 1) ``` -`error_code` is the API's stable code (`invalid_parameter`, `invalid_request` for a voice the model does not speak, `voice_not_found`, `rate_limited`, `at_capacity`, ...); branch on it rather than on the message. `retryable` and `retry_after` (seconds) say whether and when to send the same request again. +`error_code` is the API's stable code (`invalid_parameter`, `invalid_request` for a voice the model does not speak, `voice_not_found`, `biometric_terms_not_accepted`, `rate_limited`, `at_capacity`, `auth_unavailable`, ...); branch on it rather than on the message. `retryable` and `retry_after` (seconds) say whether and when to send the same request again. `biometric_terms_not_accepted` (403) refuses one of your own cloned voices, or `extend_biometric_consent`, until the account accepts the biometric data terms on the data controls page of the dashboard; `auth_unavailable` (503) means the credential or those terms could not be checked just now, and is retryable. A `TTSError` with `status_code` 200 is a streamed response cut after its first audio.
@@ -1963,8 +1967,8 @@ aai.settings.dictation_http_timeout = 300.0 # Text-to-Speech: the host (US by default, EU below) and the timeouts aai.settings.tts_base_url = "https://tts.eu.assemblyai.com" -aai.settings.tts_http_timeout = 30.0 # stream(): each socket operation -aai.settings.tts_synthesize_http_timeout = 120.0 # synthesize(): the wait for the whole clip +aai.settings.tts_http_timeout = 30.0 # stream(): each socket operation; synthesize(): connect and send +aai.settings.tts_synthesize_http_timeout = 610.0 # synthesize(): the wait for the whole clip ``` Set these before creating a client: `aai.TextToSpeech()` (like every client) copies the settings when it is built, so a later change applies to the next instance, not to an existing one. Every setting can also come from the environment with the `ASSEMBLYAI_` prefix, such as `ASSEMBLYAI_TTS_BASE_URL`. diff --git a/assemblyai/tts/v1/_base.py b/assemblyai/tts/v1/_base.py index 54328fa..1eb9eff 100644 --- a/assemblyai/tts/v1/_base.py +++ b/assemblyai/tts/v1/_base.py @@ -2,10 +2,13 @@ from typing import Any, Dict, Optional +import httpx + from ... import _audio +from ...types import Settings from .models import TTSConfig -__all__ = ["check_config", "check_text", "request_body"] +__all__ = ["check_config", "check_text", "request_body", "synthesize_timeout"] def check_config(owner: str, config: Optional[TTSConfig]) -> None: @@ -31,6 +34,20 @@ def check_text(text: object) -> None: raise TypeError(f"text must be a str, not {type(text).__name__}") +def synthesize_timeout(settings: Settings) -> httpx.Timeout: + """ + The httpx timeout of a buffered request (`synthesize()`). + + Its one read waits for the whole audio, so the read timeout is + `tts_synthesize_http_timeout`; connecting, sending the body and waiting + for a pooled connection use `tts_http_timeout`, as a streamed request does. + """ + + return httpx.Timeout( + settings.tts_http_timeout, read=settings.tts_synthesize_http_timeout + ) + + def request_body(text: str, config: TTSConfig) -> Dict[str, Any]: """The JSON body of a speech request: `text` plus every option that is set.""" diff --git a/assemblyai/tts/v1/api.py b/assemblyai/tts/v1/api.py index 0cccc3d..31dd8a6 100644 --- a/assemblyai/tts/v1/api.py +++ b/assemblyai/tts/v1/api.py @@ -1,4 +1,4 @@ -from typing import Any, Dict, Iterator, Optional +from typing import Any, Dict, Iterator, Optional, Union import httpx @@ -12,6 +12,12 @@ # answers have no JSON body, so `retryable` is decided from the status alone. _RETRYABLE_STATUSES = frozenset({408, 429, 500, 502, 503, 504}) +# What httpx raises from a streamed body that ends before the audio is +# complete: the API closing the connection (HTTP/1.1) or resetting the stream +# (HTTP/2) after a failure, a dropped connection, or no audio for a whole read +# timeout. +_CUT_ERRORS = (httpx.RemoteProtocolError, httpx.ReadError, httpx.ReadTimeout) + def _int_header(headers: httpx.Headers, name: str) -> Optional[int]: value = headers.get(name) @@ -25,8 +31,9 @@ def _header_fields(headers: httpx.Headers) -> Dict[str, Any]: return { "request_id": headers.get("x-request-id"), + "region": headers.get("x-tts-region"), "model": headers.get("x-tts-model"), - "voice": headers.get("x-tts-voice"), + "voice_id": headers.get("x-tts-voice-id"), "language": headers.get("x-tts-language"), "encoding": headers.get("x-tts-encoding"), "sample_rate": _int_header(headers, "x-tts-sample-rate"), @@ -43,6 +50,24 @@ def _info_from_headers(headers: httpx.Headers) -> TTSResponseInfo: return TTSResponseInfo(**_header_fields(headers)) +def _cut_error(info: TTSResponseInfo, error: Exception) -> TTSError: + """ + The `TTSError` for a streamed body that ended before the audio was complete. + + The API commits to `200` with the first audio, so a failure after it cuts + the body: the request is retryable from the start, and a cut caused on the + API's side is not billed. Raised `from` the httpx error. + """ + + return TTSError( + "the audio stream was cut after the first audio " + f"({type(error).__name__}: {error}); retry the whole request", + status_code=200, + request_id=info.request_id, + retryable=True, + ) + + def _error_from_response(response: httpx.Response) -> TTSError: """ Builds a `TTSError` from a non-200 response. @@ -105,13 +130,14 @@ class TTSStream: Returned by `TextToSpeech.stream()`. Use it as a context manager, or call `close()`, so the connection is released: a fully read stream's connection returns to the pool, a partially read one is dropped. `info` holds what - the response headers say (model, voice, encoding, sample rate, request + the response headers say (model, voice id, encoding, sample rate, request id); `iter_bytes()` yields the audio as it arrives and `read()` collects all of it. The API commits to `200` with the first audio, so a failure after that - cuts the body instead of changing the status: httpx raises - `RemoteProtocolError` from the iteration. Retry the whole request. + cuts the body instead of changing the status: the iteration raises a + `TTSError` with `status_code` `200`, `retryable` `True` and + `info.request_id`, from the httpx error. Retry the whole request. Chunks are not aligned to samples: for 16-bit PCM, carry an odd trailing byte over to the next chunk. """ @@ -130,11 +156,19 @@ def __iter__(self) -> Iterator[bytes]: return self.iter_bytes() def iter_bytes(self) -> Iterator[bytes]: - """Yields the audio bytes as they arrive.""" + """ + Yields the audio bytes as they arrive. - for chunk in self._response.iter_bytes(): - if chunk: - yield chunk + Raises: + TTSError: if the body is cut before the audio is complete. + """ + + try: + for chunk in self._response.iter_bytes(): + if chunk: + yield chunk + except _CUT_ERRORS as error: + raise _cut_error(self.info, error) from error def read(self) -> bytes: """Reads the rest of the audio and returns it.""" @@ -152,7 +186,7 @@ def synthesize( *, base_url: str, body: Dict[str, Any], - timeout: float, + timeout: Union[float, httpx.Timeout], ) -> TTSResponse: """ Posts a buffered speech request and returns the complete audio. @@ -164,7 +198,8 @@ def synthesize( client: the HTTP client (carries the `Authorization` header). base_url: the Text-to-Speech API base URL, e.g. `https://tts.us.assemblyai.com`. body: the JSON body: `text` and the options of a `TTSConfig`. - timeout: the httpx timeout of each socket operation. + timeout: the httpx timeout; its read timeout is the wait for the + whole audio. Raises: `TTSError` on any non-200 response. """ @@ -195,7 +230,8 @@ def stream_speech( the API has started speaking. The caller reads the audio from the returned `TTSStream` and closes it. - Raises: `TTSError` if the request fails before any audio is sent. + Raises: `TTSError` if the request fails before any audio is sent; the + returned stream raises it if the body is cut after the first audio. """ request = client.build_request( diff --git a/assemblyai/tts/v1/async_api.py b/assemblyai/tts/v1/async_api.py index 1c1c073..07269cc 100644 --- a/assemblyai/tts/v1/async_api.py +++ b/assemblyai/tts/v1/async_api.py @@ -1,9 +1,11 @@ -from typing import Any, AsyncIterator, Dict +from typing import Any, AsyncIterator, Dict, Union import httpx from .api import ( + _CUT_ERRORS, ENDPOINT_SPEECH, + _cut_error, _error_from_response, _header_fields, _info_from_headers, @@ -20,8 +22,9 @@ class AsyncTTSStream: the connection is released (a fully read stream's connection returns to the pool, a partially read one is dropped). `info` holds what the response headers say; `aiter_bytes()` yields the audio as it arrives and `aread()` - collects all of it. A failure after the first audio cuts the body: - httpx raises `RemoteProtocolError` from the iteration. + collects all of it. A failure after the first audio cuts the body: the + iteration raises a `TTSError` with `status_code` `200`, `retryable` + `True` and `info.request_id`, from the httpx error. """ def __init__(self, response: httpx.Response, info: TTSResponseInfo) -> None: @@ -38,11 +41,19 @@ def __aiter__(self) -> AsyncIterator[bytes]: return self.aiter_bytes() async def aiter_bytes(self) -> AsyncIterator[bytes]: - """Yields the audio bytes as they arrive.""" + """ + Yields the audio bytes as they arrive. - async for chunk in self._response.aiter_bytes(): - if chunk: - yield chunk + Raises: + TTSError: if the body is cut before the audio is complete. + """ + + try: + async for chunk in self._response.aiter_bytes(): + if chunk: + yield chunk + except _CUT_ERRORS as error: + raise _cut_error(self.info, error) from error async def aread(self) -> bytes: """Reads the rest of the audio and returns it.""" @@ -63,7 +74,7 @@ async def synthesize( *, base_url: str, body: Dict[str, Any], - timeout: float, + timeout: Union[float, httpx.Timeout], ) -> TTSResponse: """ Posts a buffered speech request and returns the complete audio. @@ -97,7 +108,8 @@ async def stream_speech( The asyncio counterpart of `api.stream_speech`. - Raises: `TTSError` if the request fails before any audio is sent. + Raises: `TTSError` if the request fails before any audio is sent; the + returned stream raises it if the body is cut after the first audio. """ request = client.build_request( diff --git a/assemblyai/tts/v1/async_client.py b/assemblyai/tts/v1/async_client.py index 70a1ce9..95e9ee7 100644 --- a/assemblyai/tts/v1/async_client.py +++ b/assemblyai/tts/v1/async_client.py @@ -2,7 +2,7 @@ from ... import async_client as _async_client from . import async_api -from ._base import check_config, check_text, request_body +from ._base import check_config, check_text, request_body, synthesize_timeout from .async_api import AsyncTTSStream from .models import TTSConfig, TTSResponse @@ -50,7 +50,7 @@ def __init__( the global `settings` and closed on `aclose()`. Pass a client to share one pool between instances. config: Default options. A per-call `config` replaces it whole, - so repeat the voice in a per-call config. + so repeat the `voice_id` in a per-call config. api_key: The API key to authenticate with. Builds and owns an `AsyncClient` from it. Given alongside `client`, it takes precedence: a client is built from a copy of that client's @@ -104,7 +104,7 @@ async def synthesize( self._client.http_client, base_url=self._client.settings.tts_base_url, body=body, - timeout=self._client.settings.tts_synthesize_http_timeout, + timeout=synthesize_timeout(self._client.settings), ) async def stream( @@ -120,7 +120,8 @@ async def stream( `async with`, or `await aclose()` it. Raises: - TTSError: if the API refuses the request before any audio. + TTSError: if the API refuses the request before any audio, or + (from the iteration) cuts the body after it. """ check_text(text) check_config(type(self).__name__, config) diff --git a/assemblyai/tts/v1/client.py b/assemblyai/tts/v1/client.py index 93a4c8f..6004219 100644 --- a/assemblyai/tts/v1/client.py +++ b/assemblyai/tts/v1/client.py @@ -2,7 +2,7 @@ from ... import client as _client from . import api -from ._base import check_config, check_text, request_body +from ._base import check_config, check_text, request_body, synthesize_timeout from .api import TTSStream from .models import TTSConfig, TTSResponse @@ -28,10 +28,10 @@ class TextToSpeech: aai.settings.api_key = "your-key" - tts = aai.TextToSpeech(config=aai.TTSConfig(voice="alba")) + tts = aai.TextToSpeech(config=aai.TTSConfig(voice_id="aai-alba")) with open("hello.wav", "wb") as out: - wav = aai.TTSConfig(voice="alba", container=aai.TTSContainer.wav) + wav = aai.TTSConfig(voice_id="aai-alba", container=aai.TTSContainer.wav) out.write(tts.synthesize("Hello from AssemblyAI.", wav).audio) with tts.stream("Streaming straight to the speakers.") as speech: @@ -53,7 +53,7 @@ def __init__( Args: client: The HTTP client to use. Defaults to the shared default client. config: Default options. A per-call `config` replaces it whole, - so repeat the voice in a per-call config. + so repeat the `voice_id` in a per-call config. api_key: The API key to authenticate with. Builds a `Client` for this instance. Given alongside `client`, it takes precedence: a client is built from a copy of that client's settings with @@ -94,7 +94,10 @@ def synthesize( Nothing is sent back until the audio is complete, so a long text takes several seconds and an API failure is always a `TTSError`, - never a cut response. Use `stream()` when the first audio matters. + never a cut response. The read timeout, + `settings.tts_synthesize_http_timeout`, outlasts the API's own + 600-second limit on a request. Network failures and client timeouts + raise httpx exceptions. Use `stream()` when the first audio matters. Args: text: 1 to 5,000 characters. @@ -114,7 +117,7 @@ def synthesize( self._client.http_client, base_url=self._client.settings.tts_base_url, body=body, - timeout=self._client.settings.tts_synthesize_http_timeout, + timeout=synthesize_timeout(self._client.settings), ) def stream( @@ -127,7 +130,9 @@ def stream( Returns once the API has sent the first audio. Iterate the returned `TTSStream` for the bytes, and use it as a context manager (or call - `close()`) so the connection is released. + `close()`) so the connection is released. A body cut after the first + audio raises `TTSError` (`status_code` `200`, `retryable`) from the + iteration. Args: text: 1 to 5,000 characters. @@ -135,7 +140,8 @@ def stream( the instance's default `config`. Raises: - TTSError: if the API refuses the request before any audio. + TTSError: if the API refuses the request before any audio, or + (from the iteration) cuts the body after it. """ check_text(text) check_config(type(self).__name__, config) diff --git a/assemblyai/tts/v1/models.py b/assemblyai/tts/v1/models.py index 9724cc2..b85c6c3 100644 --- a/assemblyai/tts/v1/models.py +++ b/assemblyai/tts/v1/models.py @@ -6,16 +6,20 @@ """ from enum import Enum -from typing import List, Optional, Union +from typing import Any, List, Optional, Union -from ...types import AssemblyAIError, pydantic_v2 +try: + # pydantic v2 import (`model_validator` exists only in v2) + from pydantic import BaseModel, ConfigDict, Field, model_validator -# pydantic 1.10 also exports `ConfigDict`, so the version is taken from -# `types.py`, which probes a v2-only name, instead of an import attempt here. -if pydantic_v2: - from pydantic import BaseModel, ConfigDict -else: - from pydantic import BaseModel + pydantic_v2 = True +except ImportError: + # pydantic v1 import (fallback for Python < 3.14) + from pydantic import BaseModel, Field, root_validator + + pydantic_v2 = False + +from ...types import AssemblyAIError __all__ = [ "TTSConfig", @@ -32,12 +36,24 @@ class TTSError(AssemblyAIError): Error raised when a Text-to-Speech request fails. `error_code` is the API's stable `code` (`invalid_parameter`, - `voice_not_found`, `rate_limited`, `at_capacity`, ...): branch on it, not - on the message, which may be reworded. `retryable` says whether the same - request can succeed later and `retry_after` (seconds) when to send it; - `param` names the field at fault and `errors` lists every failing field + `voice_not_found`, `biometric_terms_not_accepted`, `rate_limited`, + `at_capacity`, `auth_unavailable`, ...): branch on it, not on the message, + which may be reworded. `retryable` says whether the same request can + succeed later and `retry_after` (seconds) when to send it; `param` names + the field at fault and `errors` lists every failing field (`{"message", "param"}`) of an `invalid_parameter` refusal. `request_id` is the value to quote to support. + + `biometric_terms_not_accepted` (`403`, not retryable) refuses one of the + account's own cloned voices, or `extend_biometric_consent`, until the + account accepts the biometric data terms on the data controls page of the + AssemblyAI dashboard. `auth_unavailable` (`503`, retryable) means the + credential, or those terms, could not be checked just now. + + A streamed response cut after its first audio raises a `TTSError` with + `status_code` `200`, no `error_code`, `retryable` `True` and the + request's `request_id`; the httpx error is its `__cause__`. Retry the + whole request. """ def __init__( @@ -92,33 +108,55 @@ def __str__(self): return self.value +def _refuse_voice_field(data: Any) -> Any: + """ + Refuses a `voice` key with a message that points to `voice_id`, the field + that names every voice. Other unknown fields are refused by + `extra="forbid"`. + """ + + if isinstance(data, dict) and "voice" in data: + raise ValueError( + "TTSConfig has no field `voice`: name the voice with `voice_id`, " + 'such as voice_id="aai-jane"' + ) + return data + + class TTSConfig(BaseModel): """ Options for a Text-to-Speech request. - Every field is optional: the API speaks English with its default voice at - 24 kHz as raw 16-bit PCM when nothing is set. Unknown fields are rejected - rather than silently dropped, so a typo surfaces as a validation error. + Every field is optional: the API speaks English with its default voice + (`aai-jane`) at 24 kHz as raw 16-bit PCM when nothing is set. Unknown + fields are rejected rather than silently dropped, so a typo surfaces as a + validation error. Every voice is named by `voice_id`: there is no `voice` + field. The API's `speed` (only 1) and `word_boundaries` (served by + `/v1/speech/events`, which this client does not call) are left out. """ - voice: Optional[str] = None - """The name of an AssemblyAI voice: `jane` (the default), `alba`, `anna`, - `charles`, `eve`, `george`, `jean`, `mary`, `michael`, `paul` or `vera`, - each in every language. Case-insensitive. Not together with `voice_id`.""" - voice_id: Optional[str] = None - """A voice by its id: a voice cloned by the account (its UUID or slug) or a - voice of the AssemblyAI voice library by its global slug, such as - `aai-jane`. Sets the request's language to the voice's own unless - `language` is given. Not together with `voice`.""" + """The voice: a voice of the AssemblyAI voice library by its slug, such as + `aai-jane` (the default), `aai-alba` or `aai-michael`; a voice's ID (a + UUID, in any letter case); or the slug of one of the account's own voices, + in lowercase. The AssemblyAI dashboard shows the value that names each + voice. A voice belongs to the data zone it was created in: on the other + zone's host it is `voice_not_found`.""" model_id: Optional[str] = None """The model to run the request on, such as `assemblyai-tts-1`. Defaults to - the region's default model of the request's language.""" + the region's default model of the request's language. A cloned voice must + be paired with it.""" language: Optional[str] = None - """The language, by its English name: `english` (the default), `spanish`, - `german`, `french`, `italian` or `portuguese`. Case-insensitive.""" + """The language, by its English name: `english`, `spanish`, `german`, + `french`, `italian` or `portuguese`. Case-insensitive. Defaults to the + voice's own language (English for the default `aai-jane`). The eleven + built-in voices (`aai-jane`, `aai-alba`, `aai-anna`, `aai-charles`, + `aai-eve`, `aai-george`, `aai-jean`, `aai-mary`, `aai-michael`, + `aai-paul`, `aai-vera`) speak all six; another voice speaks a language + other than its own only where its model can, and is refused + `invalid_parameter` on `language` otherwise.""" sample_rate: Optional[int] = None "Output sample rate in Hz: 8000, 16000, 22050, 24000 (the default), 44100 or 48000." @@ -127,26 +165,43 @@ class TTSConfig(BaseModel): """Output codec: a `TTSEncoding`, or its name as a string. The API also takes the aliases `mulaw` and `alaw`. Defaults to `pcm_s16le`.""" - container: Optional[TTSContainer] = None - "`raw` (the default) or `wav`, with a PCM or G.711 encoding only." + container: Optional[Union[TTSContainer, str]] = None + """`raw` (the default) or `wav`, with a PCM or G.711 encoding only: a + `TTSContainer`, or its name as a string (case-insensitive).""" bit_rate: Optional[int] = None """MP3 bit rate in bits per second: 32000, 48000, 64000 (the default), 96000 or 128000; at most 64000 at 8 kHz. With `encoding=mp3` only.""" extend_biometric_consent: Optional[bool] = None - """`True` renews the biometric consent of the cloned voice in `voice_id`, - which only the voice's owner may do. A cloned voice expires 60 days after - its consent was last given.""" + """`True` attests renewed biometric consent for the cloned voice in + `voice_id` on behalf of the account that owns it, and the API then moves + the voice's expiry to 60 days after the request started (best effort; + `voice_expires_at` shows the result). A cloned voice expires 60 days after + its consent was last given. Refused `invalid_parameter` without + `voice_id`, `not_owner` on a voice the account does not own (a library + voice included), and `403 biometric_terms_not_accepted` until the account + accepts the biometric data terms in the dashboard.""" if pydantic_v2: - model_config = ConfigDict(extra="forbid") + # `model_id` is the API's field name: `protected_namespaces=()` keeps + # pydantic 2.0 to 2.9 from warning about its `model_` prefix on import. + model_config = ConfigDict(extra="forbid", protected_namespaces=()) + + @model_validator(mode="before") + @classmethod + def _refuse_voice(cls, data: Any) -> Any: + return _refuse_voice_field(data) else: class Config: extra = "forbid" + @root_validator(pre=True) + def _refuse_voice(cls, values: Any) -> Any: + return _refuse_voice_field(values) + class TTSResponseInfo(BaseModel): """ @@ -158,11 +213,16 @@ class TTSResponseInfo(BaseModel): request_id: Optional[str] = None "The request's id, from `X-Request-Id`. Quote it in support requests." + region: Optional[str] = None + "The region that served the request, such as `us-east-1`, from `X-TTS-Region`." + model: Optional[str] = None "The model that spoke, such as `assemblyai-tts-1`." - voice: Optional[str] = None - "The voice that spoke, such as `jane`; `custom` for a voice named by `voice_id`." + voice_id: Optional[str] = None + """The voice that spoke, from `X-TTS-Voice-Id`: the request's `voice_id`, + or the default voice's slug (such as `aai-jane`) when the request named + none. `None` from a server that does not send the header yet.""" language: Optional[str] = None "The language used, lowercase." @@ -184,20 +244,29 @@ class TTSResponseInfo(BaseModel): voice_expires_at: Optional[int] = None """For a cloned voice of the account that expires: its expiry as a Unix - time in seconds, after the request's consent renewal if it asked for one. - `None` for a preset or library voice.""" + time in seconds. On `synthesize()` it is the expiry after the request's + consent renewal, if it asked for one; on `stream()` it is the expiry as + the request started, before any renewal (a later request shows the + renewed one). `None` for a library voice, another account's voice and a + voice that does not expire.""" billed_characters: Optional[int] = None - "The characters the request bills." + """The characters the request bills. On `stream()` the header leaves with + the first audio, before the outcome is known: it holds once the audio is + read to its end, and a response cut by a failure on the API's side bills + `0`. Reconcile billing on `request_id`.""" audio_duration_ms: Optional[int] = None - """The duration of the audio in milliseconds. Sent on a buffered response - (`synthesize()`) only: a streamed response does not know it up front.""" + """The duration of the audio in milliseconds. Sent on a complete response + only: every `synthesize()`, and a response without audio. A streamed + response with audio does not know it up front.""" class TTSResponse(TTSResponseInfo): """The result of `TextToSpeech.synthesize()`: the whole audio, with its headers.""" - audio: bytes + audio: bytes = Field(..., repr=False) """The audio in the requested encoding and container, complete. Its - duration is `audio_duration_ms`.""" + duration is `audio_duration_ms`. A text with nothing to speak (only emoji, + for example) gives no audio: empty bytes, or a WAV header without samples, + with `billed_characters` `0`. Left out of the repr.""" diff --git a/assemblyai/types.py b/assemblyai/types.py index 940c8f1..f722658 100644 --- a/assemblyai/types.py +++ b/assemblyai/types.py @@ -166,10 +166,10 @@ class Settings(BaseSettings): "The base URL for the Text-to-Speech API (used by `TextToSpeech`/`AsyncTextToSpeech`). Set `https://tts.eu.assemblyai.com` to keep requests in the EU data zone." tts_http_timeout: float = 30.0 - "The HTTP timeout for streamed Text-to-Speech requests (`stream()`). Like every httpx timeout it bounds each socket operation (connecting, the wait for the first audio, each read), not the response as a whole." + "The HTTP timeout for streamed Text-to-Speech requests (`stream()`). Like every httpx timeout it bounds each socket operation (connecting, the wait for the first audio, each read), not the response as a whole. Buffered requests (`synthesize()`) use it to connect and send." - tts_synthesize_http_timeout: float = 120.0 - "The HTTP timeout for buffered Text-to-Speech requests (`synthesize()`), whose one read waits for the complete audio. Sized for a text at the 5,000-character limit; the server's own cap is 600 seconds." + tts_synthesize_http_timeout: float = 610.0 + "The read timeout of buffered Text-to-Speech requests (`synthesize()`), whose one read waits for the complete audio. A little over the API's own 600-second limit on a request, so a request that reaches it gets the API's `504 session_expired` as a `TTSError`, not a client timeout." polling_interval: float = Field(default=3.0, gt=0.0) "The default polling interval for long-running requests (e.g. polling the `Transcript`'s status)" diff --git a/tests/unit/test_tts.py b/tests/unit/test_tts.py index 7940998..a459940 100644 --- a/tests/unit/test_tts.py +++ b/tests/unit/test_tts.py @@ -5,12 +5,14 @@ """ import json +from typing import Any, Iterator, Optional import httpx import pytest from pytest_httpx import HTTPXMock import assemblyai as aai +from assemblyai.tts.v1 import models aai.settings.api_key = "test" @@ -23,7 +25,7 @@ "x-request-id": "0199c6f1-2d0b-7c3e-9a41-5b6f0e1d2c3a", "x-tts-region": "us-east-1", "x-tts-model": "assemblyai-tts-1", - "x-tts-voice": "jane", + "x-tts-voice-id": "aai-jane", "x-tts-language": "english", "x-tts-encoding": "pcm_s16le", "x-tts-sample-rate": "24000", @@ -59,6 +61,34 @@ def _sent_body(httpx_mock: HTTPXMock) -> dict: return json.loads(requests[0].content) +class _RecordingStream(httpx.SyncByteStream): + """A response body that yields `chunks`, then raises `error` if one is + given, and records whether it was closed.""" + + def __init__(self, *chunks: bytes, error: Optional[Exception] = None) -> None: + self._chunks = chunks + self._error = error + self.closed = False + + def __iter__(self) -> Iterator[bytes]: + yield from self._chunks + if self._error is not None: + raise self._error + + def close(self) -> None: + self.closed = True + + +def _field_names(model: Any) -> set: + return set(model.model_fields if models.pydantic_v2 else model.__fields__) + + +def _peer_closed() -> httpx.RemoteProtocolError: + return httpx.RemoteProtocolError( + "peer closed connection without sending complete message body" + ) + + def test_synthesize_posts_a_buffered_request_to_the_tts_host(httpx_mock: HTTPXMock): # GIVEN a TTS host that answers audio _mock_ok(httpx_mock, **{"x-tts-audio-duration-ms": "20"}) @@ -75,8 +105,9 @@ def test_synthesize_posts_a_buffered_request_to_the_tts_host(httpx_mock: HTTPXMo # AND the audio and the headers are on the response assert response.audio == _AUDIO assert response.request_id == "0199c6f1-2d0b-7c3e-9a41-5b6f0e1d2c3a" + assert response.region == "us-east-1" assert response.model == "assemblyai-tts-1" - assert response.voice == "jane" + assert response.voice_id == "aai-jane" assert response.language == "english" assert response.encoding == "pcm_s16le" assert response.sample_rate == 24000 @@ -90,13 +121,15 @@ def test_synthesize_posts_a_buffered_request_to_the_tts_host(httpx_mock: HTTPXMo def test_config_fields_go_in_the_body_and_unset_ones_stay_out(httpx_mock: HTTPXMock): # GIVEN a default config on the client and a per-call config _mock_ok(httpx_mock) - tts = aai.TextToSpeech(config=aai.TTSConfig(voice="alba", language="english")) + tts = aai.TextToSpeech( + config=aai.TTSConfig(voice_id="aai-alba", language="english") + ) # WHEN a per-call config is given tts.synthesize( "Hola.", aai.TTSConfig( - voice_id="aai-juan", + voice_id="aai-adela", model_id="assemblyai-tts-1", sample_rate=8000, encoding=aai.TTSEncoding.pcm_mulaw, @@ -107,7 +140,7 @@ def test_config_fields_go_in_the_body_and_unset_ones_stay_out(httpx_mock: HTTPXM # THEN it replaces the default, enums serialize to their values assert _sent_body(httpx_mock) == { "text": "Hola.", - "voice_id": "aai-juan", + "voice_id": "aai-adela", "model_id": "assemblyai-tts-1", "sample_rate": 8000, "encoding": "pcm_mulaw", @@ -118,9 +151,13 @@ def test_config_fields_go_in_the_body_and_unset_ones_stay_out(httpx_mock: HTTPXM def test_default_config_is_used_when_no_per_call_config(httpx_mock: HTTPXMock): _mock_ok(httpx_mock) - aai.TextToSpeech(config=aai.TTSConfig(voice="alba")).synthesize("Hi.") + aai.TextToSpeech(config=aai.TTSConfig(voice_id="aai-alba")).synthesize("Hi.") - assert _sent_body(httpx_mock) == {"text": "Hi.", "voice": "alba", "stream": False} + assert _sent_body(httpx_mock) == { + "text": "Hi.", + "voice_id": "aai-alba", + "stream": False, + } def test_stream_yields_audio_as_it_arrives_and_reports_the_headers( @@ -137,6 +174,7 @@ def test_stream_yields_audio_as_it_arrives_and_reports_the_headers( assert _sent_body(httpx_mock) == {"text": "Hello there.", "stream": True} assert b"".join(chunks) == _AUDIO assert speech.info.model == "assemblyai-tts-1" + assert speech.info.voice_id == "aai-jane" assert speech.info.sample_rate == 24000 assert speech.info.audio_duration_ms is None @@ -259,7 +297,7 @@ def test_tts_base_url_setting_picks_the_host(httpx_mock: HTTPXMock): def test_setting_config_to_none_restores_the_defaults(httpx_mock: HTTPXMock): _mock_ok(httpx_mock) - tts = aai.TextToSpeech(config=aai.TTSConfig(voice="alba")) + tts = aai.TextToSpeech(config=aai.TTSConfig(voice_id="aai-alba")) tts.config = None # type: ignore[assignment] tts.synthesize("Hi.") @@ -283,16 +321,234 @@ def test_json_error_without_retryable_falls_back_to_the_status(httpx_mock: HTTPX def test_voice_expiry_header_is_reported(httpx_mock: HTTPXMock): + # GIVEN one of the account's cloned voices, named by its UUID + voice_id = "01a0da61-138c-77c7-ab98-421431ed74e8" _mock_ok( - httpx_mock, **{"x-tts-voice": "custom", "x-tts-voice-expires-at": "1791480600"} + httpx_mock, + **{"x-tts-voice-id": voice_id, "x-tts-voice-expires-at": "1791480600"}, ) - response = aai.TextToSpeech().synthesize("Hi.", aai.TTSConfig(voice_id="aai-x")) + response = aai.TextToSpeech().synthesize("Hi.", aai.TTSConfig(voice_id=voice_id)) - assert response.voice == "custom" + # THEN the voice and its expiry come from the headers + assert response.voice_id == voice_id assert response.voice_expires_at == 1791480600 +def test_voice_id_is_read_from_x_tts_voice_id_only(httpx_mock: HTTPXMock): + # GIVEN a server that sends the old X-TTS-Voice header and no X-TTS-Voice-Id + headers = {k: v for k, v in _OK_HEADERS.items() if k != "x-tts-voice-id"} + httpx_mock.add_response( + url=SPEECH_URL, + method="POST", + status_code=httpx.codes.OK, + content=_AUDIO, + headers={**headers, "x-tts-voice": "jane"}, + ) + + response = aai.TextToSpeech().synthesize("Hi.") + + # THEN voice_id is unknown, and the retired header is not exposed + assert response.voice_id is None + assert not hasattr(response, "voice") + assert "voice" not in _field_names(aai.TTSResponse) + + +def test_voice_is_not_a_config_field_and_the_error_names_voice_id(): + # GIVEN the retired `voice` field, with or without voice_id + for kwargs in ({"voice": "jane"}, {"voice": "jane", "voice_id": "aai-jane"}): + # THEN the config refuses it and points to voice_id + with pytest.raises(ValueError, match="voice_id"): + aai.TTSConfig(**kwargs) # type: ignore[arg-type] + + assert "voice" not in _field_names(aai.TTSConfig) + + +def test_the_apis_refusal_of_voice_is_a_tts_error(httpx_mock: HTTPXMock): + # GIVEN the API's refusal of a body that sends `voice` + message = "voice: voice is not accepted; name the voice with voice_id=aai-jane" + httpx_mock.add_response( + url=SPEECH_URL, + method="POST", + status_code=httpx.codes.UNPROCESSABLE_ENTITY, + json={ + **_ERROR_BODY, + "message": message, + "error": message, + "param": "voice", + "errors": [{"message": message, "param": "voice"}], + }, + ) + + with pytest.raises(aai.TTSError) as exc_info: + aai.TextToSpeech().synthesize("Hi.") + + assert exc_info.value.error_code == "invalid_parameter" + assert exc_info.value.param == "voice" + assert exc_info.value.retryable is False + + +def test_biometric_terms_refusal_is_a_tts_error(httpx_mock: HTTPXMock): + # GIVEN an account that has not accepted the biometric data terms + message = ( + "This account has not accepted the biometric data terms. Please visit " + "https://www.assemblyai.com/dashboard/settings/data-controls to review " + "and accept them." + ) + httpx_mock.add_response( + url=SPEECH_URL, + method="POST", + status_code=httpx.codes.FORBIDDEN, + json={ + "code": "biometric_terms_not_accepted", + "message": message, + "param": "voice_id", + "request_id": "req-403", + "errors": None, + "retryable": False, + "retry_after_seconds": None, + "error": message, + }, + ) + + # WHEN one of its own cloned voices is used + with pytest.raises(aai.TTSError) as exc_info: + aai.TextToSpeech().synthesize( + "Hi.", aai.TTSConfig(voice_id="01a0da61-138c-77c7-ab98-421431ed74e8") + ) + + # THEN the refusal is final until the terms are accepted + err = exc_info.value + assert err.status_code == 403 + assert err.error_code == "biometric_terms_not_accepted" + assert err.param == "voice_id" + assert err.retryable is False + assert "data-controls" in str(err) + + +def test_auth_unavailable_is_retryable(httpx_mock: HTTPXMock): + httpx_mock.add_response( + url=SPEECH_URL, + method="POST", + status_code=httpx.codes.SERVICE_UNAVAILABLE, + json={ + "code": "auth_unavailable", + "message": "Could not confirm the account's biometric data terms", + "retryable": True, + "retry_after_seconds": 1, + }, + headers={"retry-after": "1"}, + ) + + with pytest.raises(aai.TTSError) as exc_info: + aai.TextToSpeech().stream("Hi.", aai.TTSConfig(voice_id="my-voice")) + + assert exc_info.value.error_code == "auth_unavailable" + assert exc_info.value.retryable is True + assert exc_info.value.retry_after == 1 + + +def test_a_stream_cut_after_the_first_audio_is_a_retryable_tts_error( + httpx_mock: HTTPXMock, +): + # GIVEN a streamed answer that the API cuts after its first audio + body = _RecordingStream(_AUDIO, error=_peer_closed()) + httpx_mock.add_callback( + lambda request: httpx.Response(200, headers=_OK_HEADERS, stream=body), + url=SPEECH_URL, + method="POST", + ) + + # WHEN the audio is read + received = [] + with pytest.raises(aai.TTSError) as exc_info: + with aai.TextToSpeech().stream("Hello there.") as speech: + for chunk in speech: + received.append(chunk) + + # THEN the first audio arrived, then a retryable error with the request id + assert b"".join(received) == _AUDIO + err = exc_info.value + assert err.status_code == 200 + assert err.error_code is None + assert err.retryable is True + assert err.request_id == _OK_HEADERS["x-request-id"] + assert isinstance(err.__cause__, httpx.RemoteProtocolError) + # AND the connection was released + assert body.closed + + +def test_stream_read_raises_the_same_error_on_a_cut(httpx_mock: HTTPXMock): + httpx_mock.add_callback( + lambda request: httpx.Response( + 200, + headers=_OK_HEADERS, + stream=_RecordingStream(_AUDIO, error=httpx.ReadError("reset")), + ), + url=SPEECH_URL, + method="POST", + ) + + with aai.TextToSpeech().stream("Hello there.") as speech: + with pytest.raises(aai.TTSError) as exc_info: + speech.read() + + assert exc_info.value.retryable is True + assert isinstance(exc_info.value.__cause__, httpx.ReadError) + + +def test_a_refused_stream_releases_its_connection_if_its_body_fails( + httpx_mock: HTTPXMock, +): + # GIVEN a refusal whose error body is cut while it is read + body = _RecordingStream(b'{"code": ', error=httpx.ReadError("reset")) + httpx_mock.add_callback( + lambda request: httpx.Response( + 503, headers={"content-type": "application/json"}, stream=body + ), + url=SPEECH_URL, + method="POST", + ) + + # THEN the network error surfaces and the connection is released + with pytest.raises(httpx.ReadError): + aai.TextToSpeech().stream("Hi.") + + assert body.closed + + +def test_region_is_reported_and_the_audio_stays_out_of_the_repr( + httpx_mock: HTTPXMock, +): + _mock_ok(httpx_mock) + + response = aai.TextToSpeech().synthesize("Hi.") + + assert response.region == "us-east-1" + assert "audio=" not in repr(response) + assert repr(_AUDIO) not in repr(response) + + +def test_container_takes_the_enum_or_a_string(httpx_mock: HTTPXMock): + _mock_ok(httpx_mock) + _mock_ok(httpx_mock) + tts = aai.TextToSpeech() + + tts.synthesize("Hi.", aai.TTSConfig(container="WAV")) + tts.synthesize("Hi.", aai.TTSConfig(container=aai.TTSContainer.raw)) + + bodies = [json.loads(r.content) for r in httpx_mock.get_requests()] + assert bodies[0]["container"] == "WAV" + assert bodies[1]["container"] == "raw" + + +def test_model_id_needs_no_protected_namespace_warning(): + if models.pydantic_v2: + assert aai.TTSConfig.model_config["protected_namespaces"] == () + + assert aai.TTSConfig(model_id="assemblyai-tts-1").model_id == "assemblyai-tts-1" + + def test_encoding_takes_the_enum_or_a_string_alias(httpx_mock: HTTPXMock): _mock_ok(httpx_mock) _mock_ok(httpx_mock) @@ -318,13 +574,25 @@ def test_synthesize_and_stream_use_their_own_timeouts(httpx_mock: HTTPXMock): with tts.stream("Hi.") as speech: speech.read() - timeouts = [r.extensions["timeout"]["read"] for r in httpx_mock.get_requests()] - assert timeouts == [99.0, 7.0] + # The buffered read waits for the whole audio; everything else, and the + # streamed request, use the per-operation timeout + synthesize, stream = [r.extensions["timeout"] for r in httpx_mock.get_requests()] + assert synthesize == {"connect": 7.0, "read": 99.0, "write": 7.0, "pool": 7.0} + assert stream == {"connect": 7.0, "read": 7.0, "write": 7.0, "pool": 7.0} + + +def test_the_synthesize_read_timeout_outlasts_the_apis_request_limit(): + # The API answers 504 session_expired after 600 seconds + assert aai.settings.tts_synthesize_http_timeout > 600 def test_config_rejects_unknown_fields(): with pytest.raises(ValueError): - aai.TTSConfig(voice="jane", speed=2) # type: ignore[call-arg] + aai.TTSConfig(sample_rte=8000) # type: ignore[call-arg] + + # `speed` exists on the API, but only as 1: the config leaves it out + with pytest.raises(ValueError): + aai.TTSConfig(speed=1) # type: ignore[call-arg] def test_wrong_config_type_is_a_type_error(): @@ -342,9 +610,9 @@ def test_text_must_be_a_str(): def test_swapped_text_and_config_is_named(): with pytest.raises(TypeError, match="text comes first"): - aai.TextToSpeech().stream(aai.TTSConfig(voice="jane"), "Hi.") # type: ignore[arg-type] + aai.TextToSpeech().stream(aai.TTSConfig(voice_id="aai-jane"), "Hi.") # type: ignore[arg-type] def test_a_dict_config_is_a_type_error_not_an_attribute_error(): with pytest.raises(TypeError, match="TTSConfig"): - aai.TextToSpeech().synthesize("Hi.", {"voice": "jane"}) # type: ignore[arg-type] + aai.TextToSpeech().synthesize("Hi.", {"voice_id": "aai-jane"}) # type: ignore[arg-type] diff --git a/tests/unit/test_tts_async.py b/tests/unit/test_tts_async.py index a775886..589df0b 100644 --- a/tests/unit/test_tts_async.py +++ b/tests/unit/test_tts_async.py @@ -6,6 +6,7 @@ """ import json +from typing import AsyncIterator, Optional import httpx import pytest @@ -25,7 +26,7 @@ "content-type": "audio/pcm;rate=24000;channels=1", "x-request-id": "req-async", "x-tts-model": "assemblyai-tts-1", - "x-tts-voice": "jane", + "x-tts-voice-id": "aai-alba", "x-tts-language": "english", "x-tts-encoding": "pcm_s16le", "x-tts-sample-rate": "24000", @@ -50,10 +51,29 @@ def _sent_body(httpx_mock: HTTPXMock) -> dict: return json.loads(requests[0].content) +class _RecordingStream(httpx.AsyncByteStream): + """A response body that yields `chunks`, then raises `error` if one is + given, and records whether it was closed.""" + + def __init__(self, *chunks: bytes, error: Optional[Exception] = None) -> None: + self._chunks = chunks + self._error = error + self.closed = False + + async def __aiter__(self) -> AsyncIterator[bytes]: + for chunk in self._chunks: + yield chunk + if self._error is not None: + raise self._error + + async def aclose(self) -> None: + self.closed = True + + async def test_synthesize_matches_the_threaded_twin(httpx_mock: HTTPXMock): _mock_ok(httpx_mock, **{"x-tts-audio-duration-ms": "20"}) - async with aai.AsyncTextToSpeech(config=aai.TTSConfig(voice="alba")) as tts: + async with aai.AsyncTextToSpeech(config=aai.TTSConfig(voice_id="aai-alba")) as tts: response = await tts.synthesize("Hello there.") request = httpx_mock.get_requests()[0] @@ -61,11 +81,12 @@ async def test_synthesize_matches_the_threaded_twin(httpx_mock: HTTPXMock): assert request.headers["authorization"] == "test" assert _sent_body(httpx_mock) == { "text": "Hello there.", - "voice": "alba", + "voice_id": "aai-alba", "stream": False, } assert response.audio == _AUDIO assert response.request_id == "req-async" + assert response.voice_id == "aai-alba" assert response.sample_rate == 24000 assert response.audio_duration_ms == 20 @@ -80,6 +101,7 @@ async def test_stream_yields_audio_and_reports_the_headers(httpx_mock: HTTPXMock assert _sent_body(httpx_mock) == {"text": "Hello there.", "stream": True} assert b"".join(chunks) == _AUDIO assert speech.info.model == "assemblyai-tts-1" + assert speech.info.voice_id == "aai-alba" async def test_stream_aread_collects_the_whole_audio(httpx_mock: HTTPXMock): @@ -118,7 +140,7 @@ async def test_errors_are_tts_errors(httpx_mock: HTTPXMock): async with aai.AsyncTextToSpeech() as tts: with pytest.raises(aai.TTSError) as exc_info: - await tts.synthesize("Hi.", aai.TTSConfig(voice_id="nope")) + await tts.synthesize("Hi.", aai.TTSConfig(voice_id="aai-nobody")) assert exc_info.value.error_code == "voice_not_found" assert exc_info.value.param == "voice_id" assert exc_info.value.request_id == "req-2" @@ -147,3 +169,126 @@ async def test_a_shared_client_is_left_open_on_aclose(httpx_mock: HTTPXMock): async def test_wrong_config_type_is_a_type_error(): with pytest.raises(TypeError): aai.AsyncTextToSpeech(config=aai.DictationConfig()) # type: ignore[arg-type] + + +async def test_biometric_terms_refusal_and_auth_unavailable(httpx_mock: HTTPXMock): + httpx_mock.add_response( + url=SPEECH_URL, + method="POST", + status_code=httpx.codes.FORBIDDEN, + json={ + "code": "biometric_terms_not_accepted", + "message": "This account has not accepted the biometric data terms.", + "param": "extend_biometric_consent", + "request_id": "req-403", + "errors": None, + "retryable": False, + "retry_after_seconds": None, + }, + ) + httpx_mock.add_response( + url=SPEECH_URL, + method="POST", + status_code=httpx.codes.SERVICE_UNAVAILABLE, + json={"code": "auth_unavailable", "message": "Retry", "retryable": True}, + headers={"retry-after": "1"}, + ) + renew = aai.TTSConfig(voice_id="my-voice", extend_biometric_consent=True) + + async with aai.AsyncTextToSpeech() as tts: + with pytest.raises(aai.TTSError) as exc_info: + await tts.synthesize("Hi.", renew) + assert exc_info.value.status_code == 403 + assert exc_info.value.error_code == "biometric_terms_not_accepted" + assert exc_info.value.retryable is False + + with pytest.raises(aai.TTSError) as exc_info: + await tts.stream("Hi.", renew) + assert exc_info.value.error_code == "auth_unavailable" + assert exc_info.value.retryable is True + assert exc_info.value.retry_after == 1 + + +async def test_a_stream_cut_after_the_first_audio_is_a_retryable_tts_error( + httpx_mock: HTTPXMock, +): + # GIVEN a streamed answer that the API cuts after its first audio + body = _RecordingStream( + _AUDIO, + error=httpx.RemoteProtocolError( + "peer closed connection without sending complete message body" + ), + ) + httpx_mock.add_callback( + lambda request: httpx.Response(200, headers=_OK_HEADERS, stream=body), + url=SPEECH_URL, + method="POST", + ) + + received = [] + async with aai.AsyncTextToSpeech() as tts: + with pytest.raises(aai.TTSError) as exc_info: + async with await tts.stream("Hello there.") as speech: + async for chunk in speech: + received.append(chunk) + + # THEN the first audio arrived, then a retryable error with the request id + assert b"".join(received) == _AUDIO + err = exc_info.value + assert err.status_code == 200 + assert err.retryable is True + assert err.request_id == "req-async" + assert isinstance(err.__cause__, httpx.RemoteProtocolError) + # AND the connection was released + assert body.closed + + +async def test_stream_aread_raises_the_same_error_on_a_cut(httpx_mock: HTTPXMock): + httpx_mock.add_callback( + lambda request: httpx.Response( + 200, + headers=_OK_HEADERS, + stream=_RecordingStream(_AUDIO, error=httpx.ReadTimeout("no audio")), + ), + url=SPEECH_URL, + method="POST", + ) + + async with aai.AsyncTextToSpeech() as tts: + async with await tts.stream("Hello there.") as speech: + with pytest.raises(aai.TTSError) as exc_info: + await speech.aread() + + assert exc_info.value.retryable is True + assert isinstance(exc_info.value.__cause__, httpx.ReadTimeout) + + +async def test_synthesize_and_stream_use_their_own_timeouts(httpx_mock: HTTPXMock): + _mock_ok(httpx_mock) + _mock_ok(httpx_mock) + client = aai.AsyncClient(api_key="test") + client.settings.tts_http_timeout = 7.0 + client.settings.tts_synthesize_http_timeout = 99.0 + + async with client: + tts = aai.AsyncTextToSpeech(client=client) + await tts.synthesize("Hi.") + async with await tts.stream("Hi.") as speech: + await speech.aread() + + synthesize, stream = [r.extensions["timeout"] for r in httpx_mock.get_requests()] + assert synthesize == {"connect": 7.0, "read": 99.0, "write": 7.0, "pool": 7.0} + assert stream == {"connect": 7.0, "read": 7.0, "write": 7.0, "pool": 7.0} + + +async def test_an_owned_pool_is_closed_on_aclose(): + tts = aai.AsyncTextToSpeech(api_key="test") + + await tts.aclose() + + assert tts.client.http_client.is_closed + + +async def test_voice_is_refused_by_the_config(): + with pytest.raises(ValueError, match="voice_id"): + aai.AsyncTextToSpeech(config=aai.TTSConfig(voice="alba")) # type: ignore[call-arg] From 9ea91486475681b03fa60a3d857dbbba1baa47a4 Mon Sep 17 00:00:00 2001 From: Aleksandar Mitov <140423361+aleks-mitov@users.noreply.github.com> Date: Wed, 21 Oct 2026 22:19:51 +0000 Subject: [PATCH 2/3] fix(tts): cover async refused-stream release and pydantic detection; correct cut billing and error docs - test_tts_async: a refused stream whose error body fails releases its connection (mirrors the sync test; fails without the aclose in async_api.stream_speech). - test_tts: TTSConfig is a native model of the installed pydantic (read from pydantic.VERSION), so reverting models.py to the types.pydantic_v2 flag fails on pydantic 2 without pydantic-settings. - CLAUDE.md: audio_duration_ms on a stream is None only with audio; a streamed request with nothing to speak gets it (0). - CLAUDE.md: httpx exceptions come from synthesize() and from stream() until it returns; ReadError, ReadTimeout and RemoteProtocolError while a stream is iterated become TTSError (status_code 200). - README, _cut_error and billed_characters docstrings: a cut bills 0 on a server failure, the text before the current sentence on session_expired, and every character on audio_backlog, per the API reference. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01KcyY6MAEUVjqBt5pMpetvD --- CLAUDE.md | 4 ++-- README.md | 2 +- assemblyai/tts/v1/api.py | 7 +++++-- assemblyai/tts/v1/models.py | 5 +++-- tests/unit/test_tts.py | 17 +++++++++++++++++ tests/unit/test_tts_async.py | 21 +++++++++++++++++++++ 6 files changed, 49 insertions(+), 7 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 76c63d8..d93cdf9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -55,7 +55,7 @@ aai.settings.api_key = "your-key" - `aai.TextToSpeech` — Text-to-Speech HTTP API: one text per request, audio out. Methods: `synthesize()` (complete audio as a `TTSResponse`), `stream()` (a `TTSStream` context manager yielding bytes as the API generates them). Host from `aai.settings.tts_base_url` (`https://tts.us.assemblyai.com`; set `https://tts.eu.assemblyai.com` for the EU zone) - `aai.AsyncTextToSpeech` — Asyncio counterpart of `TextToSpeech`; `synthesize()` is a coroutine and `stream()` returns an `AsyncTTSStream`. Owns an HTTP pool: use `async with` or `await aclose()` - `aai.TTSConfig`: Text-to-Speech options: `voice_id` (every voice: a library slug such as `aai-jane`, the default, a voice UUID, or one of the account's voice slugs; there is no `voice` field), `model_id` (`assemblyai-tts-1`), `language`, `sample_rate`, `encoding` (`aai.TTSEncoding`: `pcm_s16le` default | `pcm_mulaw` | `pcm_alaw` | `mp3`), `container` (`aai.TTSContainer`: `raw` default | `wav`), `bit_rate`, `extend_biometric_consent`. Rejects unknown fields -- `aai.TTSResponse`: Text-to-Speech result: `.audio` (bytes) plus the response headers: `.request_id`, `.region`, `.model`, `.voice_id` (from `X-TTS-Voice-Id`), `.language`, `.encoding`, `.sample_rate`, `.container`, `.bit_rate`, `.content_type`, `.voice_expires_at`, `.billed_characters`, `.audio_duration_ms`. `TTSStream.info` carries the same fields without `audio`; `audio_duration_ms` is `None` on a stream +- `aai.TTSResponse`: Text-to-Speech result: `.audio` (bytes) plus the response headers: `.request_id`, `.region`, `.model`, `.voice_id` (from `X-TTS-Voice-Id`), `.language`, `.encoding`, `.sample_rate`, `.container`, `.bit_rate`, `.content_type`, `.voice_expires_at`, `.billed_characters`, `.audio_duration_ms`. `TTSStream.info` carries the same fields without `audio`; its `audio_duration_ms` is `None` on a stream with audio, and `0` on a streamed request with nothing to speak (a complete response, which carries `X-TTS-Audio-Duration-Ms`) - `aai.TTSError`: Raised on any Text-to-Speech refusal, and on a stream cut after its first audio (`status_code` 200, retryable): `.status_code`, `.error_code` (stable API code, such as `voice_not_found`, `biometric_terms_not_accepted` (403), `auth_unavailable` (503)), `.param`, `.request_id`, `.retryable`, `.retry_after`, `.errors` - `assemblyai.streaming.v3.RealTimeTranscriber` — Real-time streaming with event-based API (threaded) - `assemblyai.streaming.v3.AsyncRealTimeTranscriber` — Asyncio-native counterpart; same options/events @@ -459,7 +459,7 @@ async with AsyncRealTimeTranscriber(RealTimeTranscriberOptions(token=token_from_ ## Text-to-Speech -One text per HTTP request. `synthesize()` returns the whole audio; `stream()` yields it as it is generated. Every voice is named by `voice_id` (`aai-jane` by default; `TTSConfig(voice=...)` is refused). An API failure is always `aai.TTSError`, including a stream cut after the first audio (`status_code` 200, `retryable`; retry the whole request); network failures and client timeouts are httpx exceptions. `403 biometric_terms_not_accepted` refuses the account's own cloned voices until it accepts the biometric data terms in the dashboard. Settings are copied when a client is built: set `tts_base_url` before creating `TextToSpeech`. `synthesize()` waits for the whole clip under `settings.tts_synthesize_http_timeout` (610 s read, over the API's 600 s limit); `stream()` uses `settings.tts_http_timeout` (30 s) per socket operation. +One text per HTTP request. `synthesize()` returns the whole audio; `stream()` yields it as it is generated. Every voice is named by `voice_id` (`aai-jane` by default; `TTSConfig(voice=...)` is refused). An API failure is always `aai.TTSError`, including a stream cut after the first audio (`status_code` 200, `retryable`; retry the whole request). A network failure or client timeout is an httpx exception from `synthesize()`, and from `stream()` until it returns; while a stream is iterated, `httpx.ReadError`, `httpx.ReadTimeout` and `httpx.RemoteProtocolError` become that `TTSError` (`status_code` 200, the httpx error as `__cause__`). `403 biometric_terms_not_accepted` refuses the account's own cloned voices until it accepts the biometric data terms in the dashboard. Settings are copied when a client is built: set `tts_base_url` before creating `TextToSpeech`. `synthesize()` waits for the whole clip under `settings.tts_synthesize_http_timeout` (610 s read, over the API's 600 s limit); `stream()` uses `settings.tts_http_timeout` (30 s) per socket operation. ```python tts = aai.TextToSpeech(config=aai.TTSConfig(voice_id="aai-jane")) diff --git a/README.md b/README.md index 2ea345a..8e33d7f 100644 --- a/README.md +++ b/README.md @@ -940,7 +940,7 @@ with tts.stream("Streaming straight to the speakers.") as speech: play(chunk) # raw 24 kHz 16-bit mono PCM ``` -`stream()` returns once the API has sent the first audio, usually well under a second. Chunk sizes vary and are not aligned to samples or frames: for 16-bit PCM carry an odd trailing byte over to the next chunk, and re-frame into fixed-size frames (20 ms of 8 kHz mu-law is 160 bytes) if your sink needs them. Breaking out of the loop early (a barge-in) drops that connection, so the next request pays a new TLS handshake, about 100 ms more to its first audio. The API commits to `200` with the first audio, so a failure after that cuts the body instead of changing the status: the loop raises a `TTSError` with `status_code` 200, `retryable` `True` and the request's `request_id` (the httpx error is its `__cause__`), and the whole request should be retried. On a stream, `info.billed_characters` holds once the audio is read to its end (a cut on the API's side bills nothing), and `info.voice_expires_at` is the expiry as the request started. +`stream()` returns once the API has sent the first audio, usually well under a second. Chunk sizes vary and are not aligned to samples or frames: for 16-bit PCM carry an odd trailing byte over to the next chunk, and re-frame into fixed-size frames (20 ms of 8 kHz mu-law is 160 bytes) if your sink needs them. Breaking out of the loop early (a barge-in) drops that connection, so the next request pays a new TLS handshake, about 100 ms more to its first audio. The API commits to `200` with the first audio, so a failure after that cuts the body instead of changing the status: the loop raises a `TTSError` with `status_code` 200, `retryable` `True` and the request's `request_id` (the httpx error is its `__cause__`), and the whole request should be retried. On a stream, `info.billed_characters` holds once the audio is read to its end. A cut bills nothing when a server failure caused it, the text before the sentence being spoken when the request reached the API's 600-second limit (`session_expired`), and every character when the client read too slowly (`audio_backlog`). `info.voice_expires_at` is the expiry as the request started. diff --git a/assemblyai/tts/v1/api.py b/assemblyai/tts/v1/api.py index 31dd8a6..cba83d3 100644 --- a/assemblyai/tts/v1/api.py +++ b/assemblyai/tts/v1/api.py @@ -55,8 +55,11 @@ def _cut_error(info: TTSResponseInfo, error: Exception) -> TTSError: The `TTSError` for a streamed body that ended before the audio was complete. The API commits to `200` with the first audio, so a failure after it cuts - the body: the request is retryable from the start, and a cut caused on the - API's side is not billed. Raised `from` the httpx error. + the body: the request is retryable from the start. A cut caused by a + server failure is not billed; a `session_expired` cut (the API's + 600-second limit) bills the text before the sentence being spoken when it + expired, and an `audio_backlog` cut (a client reading too slowly) bills + every character. Raised `from` the httpx error. """ return TTSError( diff --git a/assemblyai/tts/v1/models.py b/assemblyai/tts/v1/models.py index b85c6c3..ac5ac5e 100644 --- a/assemblyai/tts/v1/models.py +++ b/assemblyai/tts/v1/models.py @@ -253,8 +253,9 @@ class TTSResponseInfo(BaseModel): billed_characters: Optional[int] = None """The characters the request bills. On `stream()` the header leaves with the first audio, before the outcome is known: it holds once the audio is - read to its end, and a response cut by a failure on the API's side bills - `0`. Reconcile billing on `request_id`.""" + read to its end. A response cut by a server failure bills `0`, and one + cut by `session_expired` (the API's 600-second limit) bills less. + Reconcile billing on `request_id`.""" audio_duration_ms: Optional[int] = None """The duration of the audio in milliseconds. Sent on a complete response diff --git a/tests/unit/test_tts.py b/tests/unit/test_tts.py index a459940..4cc955a 100644 --- a/tests/unit/test_tts.py +++ b/tests/unit/test_tts.py @@ -8,6 +8,7 @@ from typing import Any, Iterator, Optional import httpx +import pydantic import pytest from pytest_httpx import HTTPXMock @@ -595,6 +596,22 @@ def test_config_rejects_unknown_fields(): aai.TTSConfig(speed=1) # type: ignore[call-arg] +def test_config_is_a_native_model_of_the_installed_pydantic(): + # The default install is pydantic 2 without pydantic-settings, so the + # major version is read from pydantic itself + if int(pydantic.VERSION.split(".")[0]) >= 2: + assert models.pydantic_v2 is True + assert "Config" not in vars(aai.TTSConfig) + assert aai.TTSConfig.model_config["extra"] == "forbid" + assert aai.TTSConfig.model_config["protected_namespaces"] == () + else: + assert models.pydantic_v2 is False + assert aai.TTSConfig.__config__.extra == "forbid" + + with pytest.raises(ValueError): + aai.TTSConfig(sample_rte=8000) # type: ignore[call-arg] + + def test_wrong_config_type_is_a_type_error(): with pytest.raises(TypeError): aai.TextToSpeech(config=aai.DictationConfig()) # type: ignore[arg-type] diff --git a/tests/unit/test_tts_async.py b/tests/unit/test_tts_async.py index 589df0b..14f7d0d 100644 --- a/tests/unit/test_tts_async.py +++ b/tests/unit/test_tts_async.py @@ -263,6 +263,27 @@ async def test_stream_aread_raises_the_same_error_on_a_cut(httpx_mock: HTTPXMock assert isinstance(exc_info.value.__cause__, httpx.ReadTimeout) +async def test_a_refused_stream_releases_its_connection_if_its_body_fails( + httpx_mock: HTTPXMock, +): + # GIVEN a refusal whose error body is cut while it is read + body = _RecordingStream(b'{"code": ', error=httpx.ReadError("reset")) + httpx_mock.add_callback( + lambda request: httpx.Response( + 503, headers={"content-type": "application/json"}, stream=body + ), + url=SPEECH_URL, + method="POST", + ) + + # THEN the network error surfaces and the connection is released + async with aai.AsyncTextToSpeech() as tts: + with pytest.raises(httpx.ReadError): + await tts.stream("Hi.") + + assert body.closed + + async def test_synthesize_and_stream_use_their_own_timeouts(httpx_mock: HTTPXMock): _mock_ok(httpx_mock) _mock_ok(httpx_mock) From cdbf8cbc67272f03d1e3f343970cf2c8cc08fa67 Mon Sep 17 00:00:00 2001 From: Aleksandar Mitov <140423361+aleks-mitov@users.noreply.github.com> Date: Wed, 21 Oct 2026 22:39:58 +0000 Subject: [PATCH 3/3] docs(tts): the synthesize timeout in CLAUDE.md is 610 s Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01KcyY6MAEUVjqBt5pMpetvD --- CLAUDE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CLAUDE.md b/CLAUDE.md index 76cf840..4b33e03 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -459,7 +459,7 @@ async with AsyncRealTimeTranscriber(RealTimeTranscriberOptions(token=token_from_ ## Text-to-Speech -One text per HTTP request. `synthesize()` returns the whole audio; `stream()` yields it as it is generated. An API refusal is always `aai.TTSError`, raised by `synthesize()` or by entering or first reading a stream (`stream()` itself sends nothing). A network failure or client timeout is an httpx exception from `synthesize()`, and from a stream until its first audio arrives (connecting, sending, the wait for the first audio, reading a refusal's body). Once the audio flows, `httpx.RemoteProtocolError`, `httpx.ReadError` and `httpx.ReadTimeout` from the stream loop become `TTSError` (`status_code` 200, `retryable=True`, the httpx error as `__cause__`), so retry the request. Settings are copied when a client is built: set `tts_base_url` before creating `TextToSpeech`. `synthesize()` waits for the whole clip under `settings.tts_synthesize_http_timeout` (120 s); `stream()` uses `settings.tts_http_timeout` (30 s) per socket operation. +One text per HTTP request. `synthesize()` returns the whole audio; `stream()` yields it as it is generated. An API refusal is always `aai.TTSError`, raised by `synthesize()` or by entering or first reading a stream (`stream()` itself sends nothing). A network failure or client timeout is an httpx exception from `synthesize()`, and from a stream until its first audio arrives (connecting, sending, the wait for the first audio, reading a refusal's body). Once the audio flows, `httpx.RemoteProtocolError`, `httpx.ReadError` and `httpx.ReadTimeout` from the stream loop become `TTSError` (`status_code` 200, `retryable=True`, the httpx error as `__cause__`), so retry the request. Settings are copied when a client is built: set `tts_base_url` before creating `TextToSpeech`. `synthesize()` waits for the whole clip under `settings.tts_synthesize_http_timeout` (610 s read, the API's 600 s cap plus a margin); `stream()` uses `settings.tts_http_timeout` (30 s) per socket operation. ```python tts = aai.TextToSpeech(config=aai.TTSConfig(voice_id="aai-jane"))