Skip to content

fix: size async uploads from logical stream positions - #276

Open
Apeksha9389 wants to merge 1 commit into
AssemblyAI:masterfrom
Apeksha9389:fix/logical-upload-length
Open

Apeksha9389 wants to merge 1 commit into
AssemblyAI:masterfrom
Apeksha9389:fix/logical-upload-length

Conversation

@Apeksha9389

Copy link
Copy Markdown

AsyncTranscriber.upload_file() calculates a file object's Content-Length from os.fstat(stream.fileno()). For standard-library gzip, bz2 and lzma readers, that descriptor describes the compressed file, while read() yields decoded bytes. The header therefore disagrees with the uploaded body. After a logical read offset, it can even become zero while audio remains.

Measure the stream's logical end and restore its position instead of using the backing descriptor. Run upload-body preparation on a worker thread because seeking a wrapped stream can perform decoding or disk I/O. Streams whose size cannot be obtained still use chunked transfer encoding. Caller-owned streams remain open.

Validation on Python 3.12: 748 unit tests pass, and Ruff check/format pass on the changed files. Six regressions using real gzip/bz2/lzma files, with and without an initial read, fail on unchanged master because the header differs from the body length. A further test verifies that a paused size probe leaves the event loop responsive. No provider requests were made.

Prepared with AI assistance; reproduced and tested locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant