Add cross-scheme slow-start warm-up ramp for newly added servers by rajvarun77 · Pull Request #3526 · apache/brpc

rajvarun77 · 2026-09-05T19:31:53Z

What & Why

A server that just joined the cluster or restarted is "cold" (empty caches, unwarmed JIT, unestablished connection pools). Every load balancer hands it a full traffic share immediately, causing cold-start tail-latency spikes — and latency-feedback schemes (la, p2c_ewma from #3367) can even oscillate: the cold server scores badly, gets starved, its stats decay, it gets slammed again. This PR adds an opt-in cross-scheme slow-start ramp, the same remedy Envoy ships as slow_start.

Usage

-lb_warmup_ms=30000          # ramp window; 0 (default) disables warm-up entirely
-lb_warmup_curve=1.0         # shape: share = max(min_weight, progress^curve)
-lb_warmup_min_weight=0.1    # initial share, validated (0, 1]

Design

Join timestamp is stamped per server at AddServer (deliberately not on Socket — sockets are shared across channels, so a socket-level stamp would leak one channel's membership change into another's ramp).
Ramp math centralized in load_balancer.{h,cpp} (WarmupMultiplier / WarmupAccept), one implementation for all policies.
la and p2c_ewma multiply the ramp into their weight so it composes with latency scoring instead of fighting it; rr, wrr, random, and consistent hashing divert probabilistically (chash moves to the next ring node, temporarily diverting part of the hash affinity).
The ramp is never applied on the last-chance path: a warming server that is the only choice still gets picked, so warm-up cannot manufacture EHOSTDOWN.
Re-adding a server after membership removal restarts the ramp; transient disconnects do not; all servers ramping together at channel init is a no-op (shares stay equal).

Tests & Docs

8 cases in test/brpc_lb_warmup_unittest.cpp (disabled-by-default, ramp math incl. curve shaping and configurable floor, per-policy share convergence for rr/wrr/random/la/p2c/chash, last-chance exemption, re-add restart); full brpc_load_balancer_unittest (17/17) passing. Documented in docs/cn/client.md and docs/en/client.md.

cc @chenBright @wwbmmm

When a server joins a LoadBalancer (scale-up, restart, redeploy), every policy immediately sends it a full traffic share while its caches, JIT and connection pools are still cold, spiking tail latency; latency-feedback policies (la/p2c) then punish the cold server and oscillate between starving and slamming it. -lb_warmup_ms (default 0, disabled) ramps a newly added server from about 10% of its normal share to 100% over the window and -lb_warmup_curve (default 1, linear) shapes the ramp, similar to Envoy slow_start's aggression parameter. The ramp math lives once in load_balancer.{h,cpp} and works off a per-server join timestamp recorded when the server is added: la and p2c multiply the ramp into their weights so it composes with latency scoring, while rr/wrr/random/consistent-hashing divert selections probabilistically to the next candidate. Re-adding a removed server restarts the ramp; transient disconnections do not change LB membership and keep it; servers added together at channel init ramp together with unchanged relative shares. When disabled the only per-selection cost is one gflag branch per candidate. Includes unit tests (ramp math, disabled-by-default, reduced-share integration for rr/wrr/chash/la/p2c, re-join restart) and docs in cn/en client.md.

Promote the hardcoded 0.1 warm-up floor to a validated gflag in (0, 1], mirroring Envoy slow_start's min_weight_percent, and document it in docs/{cn,en}/client.md.

Copilot

🟡 Changes recommended

Flag validation and test flag-isolation should be tightened to match documented behavior and established repository test practices.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds an opt-in slow-start (“warm-up”) ramp that applies consistently across multiple load balancer policies, reducing cold-start tail latency and oscillations when servers newly join or rejoin membership. The ramp logic is centralized in load_balancer.{h,cpp} and then composed into selection/weighting in each policy; docs and a dedicated unit test suite are included.

Changes:

Introduce -lb_warmup_ms, -lb_warmup_curve, -lb_warmup_min_weight plus centralized WarmupMultiplier/WarmupAccept helpers in src/brpc/load_balancer.{h,cpp}.
Apply warm-up behavior across rr/wrr/random/consistent-hash (probabilistic diversion) and la/p2c (weight-based composition).
Add test/brpc_lb_warmup_unittest.cpp and document the feature in docs/{en,cn}/client.md.

File summaries

File	Description
test/brpc_lb_warmup_unittest.cpp	Adds unit coverage for ramp math and per-policy behavior (rr/wrr/random/la/p2c/chash).
src/brpc/load_balancer.h	Declares warm-up gflag + exposes warm-up helper APIs for policies.
src/brpc/load_balancer.cpp	Implements warm-up math/acceptance and defines new gflags.
src/brpc/policy/round_robin_load_balancer.h	Tracks per-server join timestamps for warm-up.
src/brpc/policy/round_robin_load_balancer.cpp	Applies warm-up acceptance during rr selection.
src/brpc/policy/randomized_load_balancer.h	Tracks per-server join timestamps for warm-up.
src/brpc/policy/randomized_load_balancer.cpp	Applies warm-up acceptance during random selection.
src/brpc/policy/weighted_round_robin_load_balancer.h	Adds join timestamp to server entries for warm-up.
src/brpc/policy/weighted_round_robin_load_balancer.cpp	Applies warm-up acceptance in wrr selection path.
src/brpc/policy/consistent_hashing_load_balancer.h	Adds per-node join timestamp carried on the ring.
src/brpc/policy/consistent_hashing_load_balancer.cpp	Assigns join timestamps to replicas and diverts along ring probabilistically during warm-up.
src/brpc/policy/locality_aware_load_balancer.h	Adds join timestamp into weight calculation state.
src/brpc/policy/locality_aware_load_balancer.cpp	Stamps join time and discounts weight while warming.
src/brpc/policy/p2c_ewma_load_balancer.h	Adds join timestamp to per-node stats.
src/brpc/policy/p2c_ewma_load_balancer.cpp	Composes warm-up multiplier into p2c score/weighting.
docs/en/client.md	Documents slow-start flags, behavior, and policy interactions.
docs/cn/client.md	Adds Chinese documentation for slow-start flags and semantics.

Review details

Files reviewed: 17/17 changed files
Comments generated: 2
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

rajvarun77 · 2026-09-06T10:20:05Z

+static bool ValidateWarmupMinWeight(const char*, double v) {
+    return v > 0.0 && v <= 1.0;
+}
+BRPC_VALIDATE_GFLAG(lb_warmup_curve, PassValidate);
+BRPC_VALIDATE_GFLAG(lb_warmup_min_weight, ValidateWarmupMinWeight);


Done in 251b7a9: ValidateWarmupCurve rejects non-positive values; help text updated; covered by the new flag_validation test.

rajvarun77 · 2026-09-06T10:20:07Z

+class LbWarmupTest : public ::testing::Test {
+protected:
+    void SetUp() override {
+        _saved_warmup_ms = brpc::FLAGS_lb_warmup_ms;
+        _saved_curve = brpc::FLAGS_lb_warmup_curve;
+        _saved_min_weight = brpc::FLAGS_lb_warmup_min_weight;
+    }
+    void TearDown() override {
+        brpc::FLAGS_lb_warmup_ms = _saved_warmup_ms;
+        brpc::FLAGS_lb_warmup_curve = _saved_curve;
+        brpc::FLAGS_lb_warmup_min_weight = _saved_min_weight;
+    }
+
+    int64_t _saved_warmup_ms;
+    double _saved_curve;
+    double _saved_min_weight;
+};


Done in 251b7a9: fixture now holds a GFLAGS_NAMESPACE::FlagSaver member instead of the manual save/restore.

The ramp formula progress^lb_warmup_curve is only meaningful for a positive exponent; reject other values at flag-parse time instead of silently falling back to a linear ramp. The test fixture now relies on GFLAGS_NAMESPACE::FlagSaver to restore flags, as other tests in the repo do, and a new test covers the validators of both flags.

Copilot

🔵 Needs a closer look

The warm-up duration conversion in WarmupMultiplierImpl can overflow int64_t, which should be clamped to avoid incorrect behavior for large lb_warmup_ms values.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

src/brpc/load_balancer.cpp:66

WarmupMultiplierImpl computes warmup_us via FLAGS_lb_warmup_ms * 1000L, which can overflow int64_t for very large flag values and silently disable warm-up (or produce incorrect progress). Consider clamping the multiplication to avoid overflow so the flag behaves monotonically across its full range.

Files reviewed: 17/17 changed files
Comments generated: 0 new
Review effort level: Lite

Copilot

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

Suppressed comments (1)

src/brpc/policy/locality_aware_load_balancer.cpp:1

PR description says the join timestamp is stamped at AddServer to avoid restarting warm-up on non-membership events. Stamping _join_time_us in the Weight constructor risks restarting the warm-up whenever Weight instances are recreated/reinitialized (e.g., internal rebuilds) rather than only when membership changes. A more robust approach is to stamp once at the LB membership add path and pass/retain that timestamp in Weight so it only resets on remove+add.

// Licensed to the Apache Software Foundation (ASF) under one

rajvarun77 · 2026-09-07T22:29:53Z

+double WarmupMultiplierImpl(int64_t join_time_us, int64_t now_us) {
+    const int64_t warmup_us = FLAGS_lb_warmup_ms * 1000L;
+    if (warmup_us <= 0 || join_time_us <= 0) {
+        return 1.0;
+    }


Done in ee137b2: ValidateWarmupMs rejects values outside [0, INT64_MAX/1000]; covered in flag_validation.

rajvarun77 · 2026-09-07T22:29:55Z

        SocketId server_id = GetServerInNextStride(s->server_list, filter, tls_temp);
+        bool warmup_pass = true;
+        if (remain_servers > 1 && FLAGS_lb_warmup_ms > 0) {
+            warmup_pass = WarmupAccept(
+                s->server_list[s->server_map.at(server_id)].join_time_us,
+                in.begin_time_us);
+        }


Done in ee137b2: GetServerInNextStride now returns the selected index via an out-param; both server_map.at() lookups in SelectServer are gone.

rajvarun77 · 2026-09-07T22:29:56Z

+        // Time when the server was added, for the warm-up ramp. Not part
+        // of ordering/equality so that re-adding an existing server keeps
+        // its original stamp.


Done in ee137b2: comment now states that AddBatch merges with std::set_union, so a duplicate AddServer keeps the existing node and its stamp, while remove + add rebuilds the nodes with a fresh stamp.

Copilot

🔵 Needs a closer look

It changes selection behavior across multiple core load balancer policies and needs final human validation of routing semantics and rollout risk.

Review details

Files reviewed: 17/17 changed files
Comments generated: 2
Review effort level: Lite

rajvarun77 · 2026-09-07T22:29:57Z


+### 慢启动（预热）
+
+新加入集群或刚重启的server往往是“冷”的（缓存未命中、JIT未编译、连接池未建立），立即承担全量流量会推高其延时甚至过载。设置-lb_warmup_ms大于0（默认为0，即关闭）后，新加入负载均衡器的server先获得一小部分正常流量份额（-lb_warmup_min_weight，默认0.1），并在该时间窗口内线性爬升到100%。该机制对rr、wrr、random、la、p2c和一致性哈希均生效：la和p2c把爬升系数乘入权重，与延时评分自然叠加而不会互相干扰；其余算法按该系数概率性地把请求转给其他server（一致性哈希转给环上的下一个节点，预热期间会有部分请求偏离原有的哈希亲和性）。


Done in ee137b2: 改为“逐步爬升”。

rajvarun77 · 2026-09-07T22:29:59Z

+// Probabilistic form of WarmupMultiplier for policies without changable
+// weights: returns true with probability WarmupMultiplier(...).


Done in ee137b2.

- Validate lb_warmup_ms in [0, INT64_MAX/1000] so the conversion to microseconds cannot overflow. - Return the selected index from WeightedRoundRobinLoadBalancer:: GetServerInNextStride and use it in SelectServer instead of two server_map.at() lookups on the selection path. - Clarify when a consistent-hashing node keeps or resets its join stamp. - Doc wording (the ramp is only linear for curve == 1) and a typo.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

rajvarun77 added 2 commits September 5, 2026 15:28

Make warm-up floor configurable via -lb_warmup_min_weight

f66b190

Promote the hardcoded 0.1 warm-up floor to a validated gflag in (0, 1], mirroring Envoy slow_start's min_weight_percent, and document it in docs/{cn,en}/client.md.

wwbmmm requested a lite review from Copilot September 6, 2026 09:42

Copilot started reviewing on behalf of wwbmmm September 6, 2026 09:43 View session

Copilot AI reviewed Sep 6, 2026

View reviewed changes

wwbmmm requested a lite review from Copilot September 7, 2026 13:29

Copilot started reviewing on behalf of wwbmmm September 7, 2026 13:36 View session

Copilot AI reviewed Sep 7, 2026

View reviewed changes

wwbmmm requested a lite review from Copilot September 7, 2026 14:33

Copilot AI reviewed Sep 7, 2026

View reviewed changes

Copilot started reviewing on behalf of wwbmmm September 7, 2026 15:02 View session

wwbmmm requested a lite review from Copilot September 7, 2026 19:43

Copilot started reviewing on behalf of wwbmmm September 7, 2026 19:44 View session

Copilot AI reviewed Sep 7, 2026

View reviewed changes

wwbmmm requested a lite review from Copilot September 8, 2026 03:02

Copilot AI reviewed Sep 8, 2026

wwbmmm requested a lite review from Copilot September 8, 2026 04:11

Copilot AI reviewed Sep 8, 2026

Navigation Menu

Uh oh!

Add cross-scheme slow-start warm-up ramp for newly added servers - #3526

Add cross-scheme slow-start warm-up ramp for newly added servers#3526
rajvarun77 wants to merge 4 commits into
apache:masterfrom
rajvarun77:lb-warmup

rajvarun77 commented Sep 5, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

rajvarun77 Sep 6, 2026

Uh oh!

rajvarun77 Sep 6, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

rajvarun77 Sep 7, 2026

Uh oh!

rajvarun77 Sep 7, 2026

Uh oh!

rajvarun77 Sep 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

rajvarun77 Sep 7, 2026

Uh oh!

rajvarun77 Sep 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		### 慢启动（预热）

		新加入集群或刚重启的server往往是“冷”的（缓存未命中、JIT未编译、连接池未建立），立即承担全量流量会推高其延时甚至过载。设置-lb_warmup_ms大于0（默认为0，即关闭）后，新加入负载均衡器的server先获得一小部分正常流量份额（-lb_warmup_min_weight，默认0.1），并在该时间窗口内线性爬升到100%。该机制对rr、wrr、random、la、p2c和一致性哈希均生效：la和p2c把爬升系数乘入权重，与延时评分自然叠加而不会互相干扰；其余算法按该系数概率性地把请求转给其他server（一致性哈希转给环上的下一个节点，预热期间会有部分请求偏离原有的哈希亲和性）。

		// Probabilistic form of WarmupMultiplier for policies without changable
		// weights: returns true with probability WarmupMultiplier(...).

Uh oh!

Conversation

rajvarun77 commented Sep 5, 2026

What & Why

Usage

Design

Tests & Docs

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

rajvarun77 Sep 6, 2026

Choose a reason for hiding this comment

Uh oh!

rajvarun77 Sep 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Suppressed comments (1)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

rajvarun77 Sep 7, 2026

Choose a reason for hiding this comment

Uh oh!

rajvarun77 Sep 7, 2026

Choose a reason for hiding this comment

Uh oh!

rajvarun77 Sep 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

rajvarun77 Sep 7, 2026

Choose a reason for hiding this comment

Uh oh!

rajvarun77 Sep 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants