Repository navigation
perf(racer): remove small-object read bottlenecks in the sidecar path - #958
Merged
Merged
Conversation
…or sleeps A scope cancelled during a service turn was only noticed by the scan at the end of the next poll, so the reactor slept a full IDLE_WAIT with the cancel unsubmitted. Every client request cancels its disconnect watch before responding, which added about 1ms per request at low load. Reactor::wait now queues pending cancellations before it blocks.
Every worker polls its ingress inbox on each turn. With one mutex over all targets, 20 workers and the handoff acceptor serialized on it, and the acceptor blocked on the futex about once per accepted connection, capping the node's client connection rate. Give each target its own lock and keep an already registered waker instead of cloning it on every poll.
The Stat lane had four slots and a queue of 16, so a caller that Stats before every Get (such as the racer-object sidecar) failed with ErrUnavailable once about 20 requests were in flight, well below MaxConnections. The Stat lane and idle connection pool now follow MaxConnections, and the queue matches the bulk lane.
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The pre-wait cancellation scan can evaluate deadlines against the wrong clock environment.
1 open issue
What changed in this PR
Improves cached small-object reads through the Racer sidecar by reducing scheduling delays and contention.
Changes:
- Submits cancellations before sleeping and drains connection backlogs within budget.
- Uses independent handoff target locks and avoids redundant waker clones.
- Scales SDK Stat concurrency and idle connections with
MaxConnections.
| File | Description |
|---|---|
pkg/racersdk/client.go |
Expands Stat capacity and connection pooling. |
pkg/racersdk/client_stat_test.go |
Tests lane sizing and idle limits. |
cmd/racer-dataplane/src/client/tests.rs |
Tests backlog draining and wake behavior. |
cmd/racer-dataplane/src/client/listener.rs |
Accepts multiple connections within budget. |
cmd/racer-dataplane/runtime/src/reactor/tests.rs |
Tests cancellation submission before waiting. |
cmd/racer-dataplane/runtime/src/reactor.rs |
Adds pre-wait cancellation scanning and readiness probing. |
cmd/racer-dataplane/flow/src/admission.rs |
Separates target locks and optimizes waker reuse. |
Review effort: Balanced
A nonzero wait queues ASYNC_CANCEL for scopes whose check fails, and scope checks read the ambient clock. Enter the reactor's captured environment for that scan, as poll_budgeted already does, so a caller in another environment cannot cancel live I/O or miss an expired deadline.
…e is pending A pending local wake makes Reactor::wait return without sleeping, so scanning every in-flight scope first only adds full-table work to budgeted busy turns. Peek the local wake flag without consuming it and leave the scan to the next poll_budgeted turn; submission and the swap-based early return are unchanged.
Racer holds an ingress connection slot for every open client connection, including idle keep-alive ones, until its 30 second request timeout. Since the Stat lane and the idle pool were both sized from MaxConnections, a burst of concurrent Stats could leave every ingress slot held by pooled sockets, and Get, which always dials a fresh connection, would stall until Racer reclaimed them. Keep the Stat lane at MaxConnections, but cap the idle pool at four connections, or MaxConnections if smaller. Add a regression test where the server admits as many connections as the client budget: a Stat burst followed by Get now succeeds instead of timing out.
Jordan Olshevski (jveski)
marked this pull request as ready for review
October 21, 2026 16:37
Jordan Olshevski (jveski)
enabled auto-merge
October 21, 2026 16:43
Cameron Childress (cchildress)
approved these changes
Oct 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This fixes several bottlenecks that capped small-object reads through the racer-object sidecar when data is already cached in node memory. The dataplane reactor now submits cancellations from scopes cancelled during a service turn before it sleeps, which removes a 1ms stall on most requests at low load. The handoff acceptor now drains the client accept backlog with its remaining budget, and the ingress handoff locks each worker target independently so accepting connections no longer contends with every worker's poll. The SDK's Stat lane is now sized from MaxConnections instead of a fixed 4 slots and 16 queued requests, which caused 503s once more than about 20 GETs were in flight.