Repository navigation
feat(rs-client): Add automatic resumable uploads - #700
Merged
Merged
Conversation
Resume eligible buffer and file uploads from authoritative server progress, with configurable thresholds and temporary-file compression. Keep automatic uploads available without the manual protocol feature.
This comment has been minimized.
This comment has been minimized.
Separate payload preparation and reader lifetime from protocol recovery. Stop and join each request reader before returning its result, report local IO errors directly, and keep progress probes in a dedicated retry loop.
Unify direct fallbacks, make sizing optional, and limit resumables to standalone uploads.
# Conflicts: # Cargo.lock # clients/rust/src/error.rs Co-Authored-By: Lorenzo Cian <17258265+lcian@users.noreply.github.com>
Semver Impact of This PRThis PR lands in these workspaces:
📋 Changelog PreviewThis is how your changes will appear in the changelog. New Features
🤖 This preview updates automatically when you update the PR. |
…ated docs Co-Authored-By: Lorenzo Cian <17258265+lcian@users.noreply.github.com>
Keep both test dependencies and support gzip when spooling resumable payloads.
Limit automatic resumables to buffers and stable paths. Reopen source paths and named spool files per attempt to remove reader coordination.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
lcian
marked this pull request as ready for review
October 28, 2026 16:03
This comment has been minimized.
This comment has been minimized.
jan-auer
reviewed
Oct 29, 2026
lcian
marked this pull request as draft
October 29, 2026 12:25
lcian
marked this pull request as ready for review
October 29, 2026 12:26
lcian
added a commit
that referenced
this pull request
Oct 29, 2026
Fixes the changelog preview added in #737, which fails at startup on every PR ([run on #700](https://github.com/getsentry/objectstore/actions/runs/40873702019): "likely failed because of a workflow file issue"). Craft's reusable preview job requests `contents: read`, `pull-requests: write` and `statuses: write`. A called workflow can't get more permissions than its caller, and our caller only granted the first two. This adds `statuses: write`, matching the old per-client preview workflows. The preview runs from the base branch (`pull_request_target`), so it starts working on other PRs once this merges. #skip-changelog --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Lorenzo Cian <17258265+lcian@users.noreply.github.com>
jan-auer
approved these changes
Oct 30, 2026
jan-auer
left a comment
Member
There was a problem hiding this comment.
Approved provided the below comments are addressed.
Keep optional resumable exports together and move the dead-code allowance into the module.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit c6b8bef. Configure here.
…746) Compressed resumable uploads now spool to anonymous temporary files and read each retry at explicit offsets. This avoids leaving named spool files behind after a process crash and lets an earlier HTTP request finish reading without affecting a retry's position. Stacked on #700. Reads run on Tokio's blocking pool in 4 KiB chunks, using Unix positional reads or Windows `seek_read`. Each chunk is zero-initialized and converted to `Bytes` without copying its contents. Source-path uploads continue opening independent readers. Regression coverage checks overlapping readers, cancellation with a pending read, and premature EOF. Client tests pass with all features and with default features disabled on macOS; the Windows implementation has not been exercised on Windows.
lcian
commented
Nov 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Changelog Entry
Session::putnow uses resumable uploads automatically for buffers and paths to regular files of 256 MiB or more, and resends only what the server hasn't stored after a failed writeUsecase::with_resumable_threshold_bytesor per request with.resumable_threshold_bytes(...);Nonedisables automatic resumable uploads.ClientBuilder::spool_dirto choose where payloads compressed on the fly are spooled (defaults to the system temp dir).resumable-upload-apifeature only gates the manual resumable upload API. This new behavior is enabled regardless of that feature flag.Details
Session::putnow uses resumable uploads automatically for eligible uploads, matching the Python client (#667). An upload is eligible if:put,put_path)many()keep their existing batch or direct routing and ignore the threshold.Open file handles (
put_file), streams, generic readers, non-regular paths and inputs of unknown size use a direct upload.The API and behavior mirrors the Python client:
Path and reader lifetime. Each attempt opens an independent reader. Source paths must remain readable, unchanged, and must not be renamed, removed, or replaced until the upload completes. Compressed payloads spool to named temporary files and retries reopen them. The HTTP layer may retain old readers after the upload returns or is dropped, delaying handle closure and temporary-storage reclamation.
For reviewers
ErrorvariantsResumableUploadUnavailable,UploadOffsetMismatchand the chunk-granularity variant are no longer feature-gated. SinceErroris#[non_exhaustive](feat(rs-client)!: Mark Error as non-exhaustive #706), this is additive.put.rs(routing),put/payload.rs(preparation and readers) andput/automatic.rs(recovery loop).Closes FS-561