feat(pbs): dial builders outside the config for ePBS requests by JasonVranek · Pull Request #506 · Commit-Boost/commit-boost-client

JasonVranek · 2026-10-07T04:05:50Z

Builds on #505. There, a bid or preferences request whose auth data matches no relay entry gets 400, so a validator key whose builder config names a builder the operator has not added to Commit-Boost gets no bids from it. This PR has Commit-Boost dial the builder the auth data names, behind an SSRF guard, and routes auth data written as a builder URL. It also answers the signed block with 202 once it has been forwarded, whatever the builders answer, which addresses a review comment on #505.

What it does

URL-form auth data. Besides the hostname (the builder-specs default), auth data that is an http or https URL matches the first relay entry with the same scheme, host and port. The path and the pubkey in the relay URL do not count.
Builder parameters. Either form may end in ? and form-encoded parameters for the builder, such as builder-a.example.com?filter=1, a way for a proposer to state a preference like transaction filtering. Commit-Boost routes on the part before ? and forwards the signed auth, parameters included, unchanged, so the parameters arrive signed by the proposer. A builder that does not accept them answers 400, so a proposer sets them only for builders that accept them. Without a ?, routing is unchanged.
Dialing builders outside the config. A bid or preferences request whose auth data matches no relay entry goes to the builder it names: https://<hostname> on the default port, or the URL's scheme, host and port. There is no setting to turn it off. Dialed requests count under relay_id="dial", and each dial logs the builder's origin and the addresses it connects to. A key without builder config sends Commit-Boost's own hostname, so Commit-Boost dials https://<its hostname> on port 443: 400 when that hostname resolves to a loopback or private address, as it usually does, and otherwise one request to whatever serves HTTPS on that host, which is Commit-Boost only behind a TLS proxy, where the loop guard below stops it.
The signed block gets 202 once it has been forwarded, whatever the builders answer. This answers a review comment on epbs builder api endpoints #505: relays may do nothing with these requests, so an apparent failure should not be reported as one, and the beacon node gossips the block anyway. The block still goes only to configured relays, since Commit-Boost keeps no auction state; a dialed builder gets it over gossip.
A test for epbs builder api endpoints #505's JSON passthrough. epbs builder api endpoints #505 returns the relay's bid body unchanged when the beacon node asks for the encoding the relay answered in. A new mock relay option serves its JSON bid pretty-printed, and a test checks that the beacon node gets exactly those bytes.

The ePBS docs page gets a section on builders outside the config, which also says what a key without builder config gets, the URL form and ? parameters in the routing reference, 500 in the metrics table for preferences only, and troubleshooting rows for both 400s and for that warning.

Scope

Commit-Boost now makes outbound requests to hosts named in request data. The proposer signs that data, but Commit-Boost does not verify the signature, so the target is treated as untrusted, and anyone who can reach Commit-Boost's PBS port can choose it:

The host is resolved once, after the timing headers and the deadline are checked; an IP address needs no lookup. The lookup and the dial share the bid's remaining time (timeout_register_validator_ms for preferences). If the name does not resolve, the answer is 400 and nothing is dialed. A lookup or dial setup that runs out of time counts as a builder that did not answer: no bid (204), or 500 for preferences.
At most 32 dial lookups run at once, and further requests get the same "did not answer" result. A lookup that times out keeps its blocking thread until the resolver returns, and configured relays' own lookups share that pool, so without the cap, requests naming slow-resolving hosts could stall the relays' DNS.
The dial connects only to the checked addresses (reqwest resolve_to_addrs), ignores HTTP(S)_PROXY and ALL_PROXY (a proxy would resolve the host again), follows no redirects, and sends none of the operator's relay headers.
A request carrying X-CommitBoost-Version, which every Commit-Boost dial sends, never leads to a dial, so a dial that reaches a Commit-Boost, this one included, goes no further. Configured relays still serve such a request, so chained Commit-Boosts keep working.
Auth data comes from the request, and a dialed builder's response comes from a host the request chose, so Commit-Boost logs both escaped, keeps a logged error body to 1 KiB, and logs a dial target as its origin only, without any user:password@.
A public address of the host Commit-Boost runs on is not recognized as its own. A dial to it reaches services listening on all interfaces, as a POST of an SSZ body to a fixed builder-API path.
Each dial builds its own HTTP client, because the address pin is per client, so it pays a client build and a fresh connection. Configured relays are unaffected.

Testing

cargo test --all-features: 402 passed (392 on part 1). Unit: URL-form auth data decoding (only http and https; opaque data such as builder-a:prod and host:port parse as URLs and are refused), the dial target for each auth data form (URL, hostname, IPv6 hostname, a hostname with ? parameters, an internal address, a non-hostname, a non-HTTP scheme), every resolved address checked, the lookup cap refusing a hostname but not an IP address, the dial pinned to the given address with redirects refused, the blocked-address table, and URL-form routing to relay entries and routing on the part before ?. End to end: a bid dialed with its auth, ? parameters included, unchanged, a bid naming Commit-Boost's own URL dialed once and answered with the second hop's 400, a request from a Commit-Boost not dialed but still served by a configured relay, preferences dialed, an unmatched localhost refused as loopback, and no lookup once the deadline has passed. A builder's 400 and 401 reach the beacon node even with an error body over the 1 KiB read cap. The signed block gets 202 when every builder rejects it, and a pretty-printed JSON bid reaches the beacon node byte for byte.
Reverting each change fails a test: dropping the Commit-Boost version header from dialed requests fails the dial-to-self test, which then loops until the deadline and answers 204, putting the 500 for a block no builder accepted back fails the all-reject test, and removing epbs builder api endpoints #505's passthrough arm fails only the passthrough test.
Not covered by a test: the proxy bypass (the test would set HTTP_PROXY in a process that runs other tests in parallel), the escaping in logs and the 1 KiB cap on a logged error body, the refusal when the dial setup leaves no time, and the lookup's timing: its timeout and the time it leaves the dial (a local lookup finishes inside tokio's 1 ms timer tick, so a name slow enough to matter needs the network).
Kurtosis, on the full stack at part 4's tip, which carries this PR's code: the cb-testing modes pass, and so does the live bid-stream gate against Helix. In the block-submission mode, the signed block was forwarded 17 times and buildoor accepted all 17. Dialing ran on a devnet too, with a test-only build that turns off the private-address check (cb-testing --assert dial; Docker addresses are all private). Each key's auth data was http://buildoor:8080, which no relay entry matches, so Commit-Boost dialed buildoor 95 times, for bid and preferences requests. All 17 observed slots were built from those bids, no dial was refused, and none looped.

Pin the lighthouse crates to sigp/lighthouse unstable at 31d8cfd, whose gloas containers hash as EIP-7495 progressive containers, matching the consensus specs. Mirror lighthouse's own [patch.crates-io] entries for the progressive ssz stack (a consumer does not inherit a dependency's patches) and pin the blstrs_plus patch to an explicit rev. lighthouse dropped `TestRandom` for an `arbitrary` generator, so `TestRandomSeed` now draws from `arbitrary::Arbitrary`. Validated BLS points do not randomize under `arbitrary`, so tests that need a real key use `BlsSecretKey::random()`. Also adapt to `ForkName::Heze` and to `ExecutionRequests` no longer implementing `ssz::Decode`.

Tests discovered a free port, dropped the listener and let the server rebind it, leaving a window where another process could take the port. PbsService and SigningService gain `run_with_listener`, the mock SSV servers take a bound listener, and the legacy suites hand their listeners straight to the server. Also add `wait_for_ready`, which polls /status instead of sleeping a fixed 100 ms.

From the Gloas fork the beacon node calls three builder-API endpoints on Commit-Boost instead of get_header and get_payload: - POST /eth/v1/builder/execution_payload_bid/{slot}/{parent_hash}/{parent_root}/{proposer_pubkey} sends the request to the one relay its auth data names and returns that relay's bid (200), its 400 or 401, or 204 when it has no bid or fails. Commit-Boost does not validate or rank the bid: the beacon node checks it, and each request names exactly one builder. - POST /eth/v1/builder/builder_preferences/{proposer_pubkey} forwards to the relay the auth data names and returns that builder's answer. - POST /eth/v1/builder/beacon_blocks forwards the SSZ block bytes unparsed to every configured relay and answers 202 when one accepts. Requests must carry Eth-Consensus-Version: gloas. A body without Content-Type is JSON and a request without Accept gets JSON, as builder-specs requires. Bid requests need Date-Milliseconds and X-Timeout-Ms; Commit-Boost clamps the deadline to one slot, keeps proposer_deadline_buffer_ms back for the return trip and sends the rest to the builder as its own X-Timeout-Ms, or returns 204 when nothing is left. Routing follows builder-specs #168: auth data matches a relay when it equals the lowercase hostname of the relay URL. The first match in config order wins and unmatched auth data is a 400. Commit-Boost never verifies auth data; the builder does. Relay-supplied amounts are saturated. The new operator page (docs/get_started/epbs.md) covers writing each validator key's builder config through the keymanager API, with a copy-paste call, plus routing, timing, metrics and troubleshooting. Unreleased, shipping from v0.12.0-rc1.

…links to

condition and lint test

Auth data that is an http(s) URL now also routes to the relay entry with the same scheme, host and port. Either form may end in `?` and parameters for the builder: Commit-Boost routes on the part before `?` and forwards the signed auth, parameters included, unchanged. When no relay entry matches, Commit-Boost dials the builder the auth data names for a bid or preferences request: `https://<hostname>`, or the URL. The target is resolved once, and the lookup and the dial share the request's budget. A name that does not resolve, or any address that is not public unicast, gets 400; a lookup or dial setup that runs out of time counts as a builder that did not answer. At most 32 lookups run at once, since a timed-out lookup keeps its blocking thread. The dial goes only to the checked addresses, with no proxy and no redirects. A request from a Commit-Boost never leads to a dial, so a dial that reaches a Commit-Boost, this one included, goes no further. Auth data, a dialed builder's error body and a dial target come from the request, so they are logged escaped, the body capped at 1 KiB and the target as its origin. The signed block gets 202 once it has been forwarded, whatever the builders answer, since the beacon node gossips it anyway. A block whose bid came from a dialed builder, which is not a relay entry, would otherwise always get 500. When no builder accepts the block, Commit-Boost logs a warning.

proposer_deadline_buffer_ms must now be greater than 0 as well as less than one slot. With 0 the builder gets the whole deadline, so a bid it sends at the deadline reaches the beacon node late. The range is one check in PbsConfig::validate, so the service, a hot reload and a custom PBS module's loader all refuse it. The test fixtures use the default, 50. The metrics catalog documents the ePBS endpoint values, the codes the ePBS endpoints return to the beacon node, the relay_id="dial" label, and that ePBS bids set the relay gauges. The ePBS page corrects two builder config bullets: a top-level min_bid does not floor an entry that sets its own, and the per-client max_execution_payment defaults are dropped. A builder Commit-Boost refuses to dial is not counted under relay_id="dial".

ninaiiad · 2026-10-08T14:18:43Z

+    let mut client =
+        reqwest::Client::builder().no_proxy().redirect(reqwest::redirect::Policy::none());


we could reuse a client for all these dials that can't be routed to existing preconfigured relay clients
this optimises repeat dials to same addresses outside of the config + no new client created

added to 7524262

A client per dial existed only because resolve_to_addrs pins addresses per client. The shared client's resolver now checks every address it resolves, so a dial still connects only to checked addresses, and a second dial to a builder reuses the first one's connection instead of a new TCP and TLS handshake. Dial hosts are chosen by the request, so the client keeps at most one idle connection per host, for one slot. The check before the dial is unchanged, so a blocked target still gets 400 before anything is sent.

vladimir-ea · 2026-10-10T18:05:14Z

+pub fn decode_auth_data_url(data: &[u8]) -> Option<Url> {
+    let url = Url::parse(std::str::from_utf8(data).ok()?).ok()?;
+    matches!(url.scheme(), "http" | "https").then_some(url)
+}


I guess this means that the auth data must be the builder URL only? I thought I saw somewhere suggested that the auth data would be the builder URL as a prefix followed by arbitrary other data?

test_get_execution_payload_bid_dial() covers this case

vladimir-ea · 2026-10-10T18:15:21Z

+        .or_else(|| {
+            let url =
+                Url::parse(&format!("https://{}/", std::str::from_utf8(address).ok()?)).ok()?;
+            (url.host_str()?.as_bytes() == address).then_some(url)


is this round trip check necessary? from_utf8() is loss-less

added a comment saying so in 241b59a

vladimir-ea · 2026-10-10T18:19:44Z

+
+/// Run for each new connection to a hostname (an IP address skips it): refuses
+/// it while every lookup slot is taken or if any address is disallowed
+struct DialResolver;


is it worth caching resolved addresses here for some interval? also, given that this is all asynchronous a background task could be spawned to recheck already seen addresses - would save the dns resolution on the request path.

added in 60717c0

decode_auth_data_url only ever gets the auth data address, the part before any `?`, so its parameter and doc now say so. dial_target's comparison of the parsed host with the address passes only a hostname already in the builder-specs default form, since parsing also accepts a port, userinfo, a path, upper case and numeric IPv4.

A dial to a hostname looked the host up twice in a row on the request path: once in dial_relay's check, and again in DialResolver when the connection opened. The resolver now takes the check's addresses while they are under 2 s old, checking them again, and looks the host up itself otherwise, so a later connection still sees a DNS change. The addresses are kept with port 0, as the resolver's own lookup returns them, so a dial to the same host on another port connects to its own.

JasonVranek added 7 commits October 5, 2026 21:51

docs(epbs): keep the builder config anchor the troubleshooting table …

2a792e6

…links to

address review comments

e6af0fd

test: wait for PBS's startup status check in test_mux to fix race

bc3c590

condition and lint test

JasonVranek added this pull request to stack #507 October 7, 2026 04:11

JasonVranek mentioned this pull request Oct 7, 2026

epbs builder api endpoints #505

Merged

JasonVranek added 2 commits October 7, 2026 11:30

normalize gloas version header

867fd19

Merge branch 'epbs-pr1' into epbs-pr2

2b22fd2

JasonVranek requested a review from a team October 7, 2026 22:30

ninaiiad reviewed Oct 8, 2026

View reviewed changes

Base automatically changed from epbs-pr1 to main October 8, 2026 15:02

JasonVranek added 2 commits October 8, 2026 08:18

Merge main after #505

540295e

This was referenced Oct 9, 2026

ePBS (Gloas) support, part 3: validators' builder config from the Commit-Boost config #508

Closed

ePBS (Gloas) support, part 3: bid streaming #509

Open

vladimir-ea reviewed Oct 10, 2026

View reviewed changes

JasonVranek added 2 commits October 10, 2026 13:28

Navigation Menu

Repository navigation

feat(pbs): dial builders outside the config for ePBS requests - #506

feat(pbs): dial builders outside the config for ePBS requests#506
JasonVranek wants to merge 14 commits into
mainfrom
epbs-pr2

JasonVranek commented Oct 7, 2026 •

edited

Loading

Uh oh!

ninaiiad Oct 8, 2026

Uh oh!

JasonVranek Oct 8, 2026

Uh oh!

vladimir-ea Oct 10, 2026

Uh oh!

JasonVranek Oct 10, 2026

Uh oh!

vladimir-ea Oct 10, 2026

Uh oh!

JasonVranek Oct 10, 2026

Uh oh!

vladimir-ea Oct 10, 2026

Uh oh!

JasonVranek Oct 10, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		let mut client =
		reqwest::Client::builder().no_proxy().redirect(reqwest::redirect::Policy::none());

Conversation

JasonVranek commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What it does

Scope

Testing

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

JasonVranek commented Oct 7, 2026 •

edited

Loading