Skip to content

Latest commit

 

History

History
304 lines (224 loc) · 12.7 KB

wire-protocol.md

File metadata and controls

304 lines (224 loc) · 12.7 KB

RSK Wire Protocol & Sync Framing

The rskj-specific RLPx framing, message encoding, and sync protocol a client must speak to talk to RSK peers, plus discovery liveness. Interop-load-bearing but (except where noted) not consensus-critical — getting these wrong stalls sync rather than forking state.

Part of the rskj compatibility catalogue; section numbers (§N) are historical and shared across the whole catalogue — the index in java-compatibility.md maps every § to its file.

1. Multi-Frame (Chunked) RLPx Messages

Source: rskj/.../rlpx/FrameCodec.java, rskj/.../rlpx/MessageCodec.java

Standard Ethereum clients (geth, reth) send each P2P message as a single RLPx frame regardless of size. The Java RSK node splits messages larger than ~32 KB into multiple frames, each with its own header and MAC.

How chunking works

MessageCodec.splitMessageToFrames slices msg.getEncoded() into chunks of maxFramePayloadSize (32,768 bytes). Each chunk becomes a Frame object that shares the same contextId and message type (code).

The first frame carries totalFrameSize = msg.getEncoded().length in its header-data. Continuation frames carry only the contextId.

Header-data encoding

Bytes 3-15 of the decrypted 16-byte frame header contain RLP-encoded header-data:

Frame kind header-data RLP element count
Normal rlp([0]) 1
Chunked-first rlp([0, contextId, totalFrameSize]) 3
Continuation rlp([0, contextId]) 2

ptype prefix on every frame body

FrameCodec.writeFrame prepends the RLP-encoded message type (ptype) to the encrypted body of every frame, not just the first:

byte[] ptype = RLP.encodeInt((int) frame.type);
int totalSize = frame.size + ptype.length;   // frame-size in header
// ...
enc.processBytes(ptype, ...);                // written first
// then the payload chunk follows

For message code 24 (0x18), ptype is a single byte 0x18. The receiver must strip this prefix from every frame body before concatenating the payload chunks.

Key consequence: totalFrameSize from the header-data equals msg.getEncoded().length and does not include the per-frame ptype bytes. After stripping ptype from each frame, the concatenated payload must equal exactly totalFrameSize bytes.

Rust implementation

See crates/networking/src/rlpx/frame.rs:

  • FrameType enum identifies frame kinds via parse_header_data.
  • ChunkedFirst extracts the protocol_id (ptype) and begins assembly.
  • ChunkedContinuation strips the ptype and appends only the data portion.
  • Assembly completes when assembly_buf.len() >= totalFrameSize.

3. Compressed Block Headers (RSKIP-351)

Source: rskj/.../core/BlockHeader.java, RSKIP-351

Recent RSK blocks replace the 256-byte logsBloom field (RLP field index 6) with a shorter extensionData blob containing rlp([version, hash]). This significantly reduces header size.

The Rust Header::decode in crates/core/src/types/header.rs peeks at the RLP header of field 6:

  • If it is a non-list of exactly 256 bytes -> standard logsBloom.
  • Otherwise -> compressed extensionData; store the raw bytes and set logsBloom to default.

4. RSK Sub-Protocol Message Wrapping

Source: rskj/.../net/messages/Message.java

RSK messages use a double-wrapped RLP structure that differs from standard Ethereum eth sub-protocol messages:

RLP([                          // outer list  (RskMessage.encode)
  RLP([                        // inner list  (Message.getEncoded)
    type_byte,                 // e.g. 10 for BlockHeadersResponse
    RLP_String(body_params)    // body as an RLP string (not list)
  ])
])

The body is encoded as an RLP string (blob), not a list. Inside that string, the actual parameters are a nested RLP list. RskMessage::decode in crates/networking/src/protocol/rsk.rs handles this unwrapping.

Unknown message types

The Java node sends many message types beyond what a light client needs (Transactions, NewBlockHashes, GetBlockHeaders from peers, etc.). These are handled gracefully via RskSubMessage::Unknown(u8) to avoid crashing sessions.


4a. Snapshot sync messages, and extension by trailing elements

Source: rskj/.../net/messages/Snap*Message.java, crates/networking/src/protocol/snap.rs

Snapshot sync adds six messages, ids 20-25, all framed as RLP([id, params]) where params is an RLP string holding an encoded list -- the same string-not-list shape as section 4, and the thing rustock got wrong until #239.

Message Type ID Params
SnapStateChunkRequest 20 blockNumber, from, chunkSize
SnapStateChunkResponse 21 chunk, blockNumber, from, to, complete
SnapStatusRequest 22 (empty list)
SnapStatusResponse 23 blocks[], difficulties[], trieSize
SnapBlocksRequest 24 blockNumber
SnapBlocksResponse 25 blocks[], difficulties[]

Trailing elements are ignored, which is what makes these extensible

rskj's decoders read a fixed number of elements from each message and never consult the list length again, so an element appended after the last one they read is accepted and ignored. That is what lets these messages grow without a flag day, and it is already relied on twice: the served-range extension appends a fifth element to the status message, and the refusal code below appends a sixth to message 21.

The refusal code (SnapStateChunkResponse, a sixth element)

params = RLP([ chunk, blockNumber, from, to, complete, refusal ])
                                                       ^^^^^^^

Refusal in crates/networking/src/protocol/snap.rs:

value variant meaning
0 None not a refusal -- the chunk is in the payload
1 UnknownBlock no block at that number on this chain
2 StateRootMismatch this chain has a different state root at that height
3 StateNotStored the state was known but is no longer stored
4 PastTheEnd the offset is past the end of the trie
5 OffsetNotOnGrid the offset is not a multiple of the server's chunk granularity

Without it an empty payload is ambiguous: "nothing at that offset, ask elsewhere", "nothing for you at all", and "your request was malformed" look identical, and a client can only tell them apart by retrying. None is misbehaviour, so none is charged against the peer -- Refusal::worth_asking_again decides only whether another round trip to the same peer is useful.

rskj never reads it. A client must treat its absence as "no reason given", and a server that does not implement it sends rskj's five elements.

Specified upstream in RSKIP-696.


5. Descending Header Delivery

Source: rskj/.../net/NodeBlockProcessor.java (processBlockHeadersRequest)

RSK peers return BlockHeadersResponse in descending block number order (highest first). The server starts at the requested hash and walks getParentHash() backward for count blocks.

SyncManager::handle_headers_response in crates/sync/src/lib.rs reverses the list before processing so headers are stored in ascending order.


6. Skeleton-Based Forward Sync Protocol

Source: rskj/.../net/sync/ (multiple state classes)

The Java rskj node uses a multi-phase skeleton-based sync, not a simple backward download. The Rust light client mirrors this approach.

Sync phases

Phase Java class Rust equivalent
1. Find connection point FindingConnectionPointSyncState SyncState::FindingConnectionPoint
2. Download skeleton DownloadingSkeletonSyncState SyncState::DownloadingSkeleton
3. Download headers DownloadingHeadersSyncState SyncState::DownloadingHeaders
4. Download bodies DownloadingBodiesSyncState (skipped — light client)

Phase 1: Connection point (binary search)

Uses BlockHashRequest (type 8) / BlockHashResponse (type 18) to binary-search for the highest block that both the local store and the peer share. Search range: [0, peerBestBlockNumber]. Converges in O(log N) steps.

The Java ConnectionPointFinder uses the algorithm:

  • mid = start + (end - start) / 2
  • Peer responds with the hash at mid
  • If we have it locally: start = mid (search higher)
  • If we don't: end = mid (search lower)
  • When end - start <= 1: connection point = start

Phase 2: Skeleton download

Sends SkeletonRequest (type 16) with the connection point height. The peer responds with SkeletonResponse (type 13) containing a list of BlockIdentifier(hash, number) at intervals of CHUNK_SIZE = 192 blocks.

Server-side skeleton construction (NodeBlockProcessor.processSkeletonRequest):

  • Rounds start down to nearest multiple of 192
  • Adds entries every 192 blocks up to maxSkeletonChunks * 192 (default: 3840)
  • Appends the best block as the final entry
  • Result: at most 21 entries covering up to 3840 blocks per skeleton round

Phase 3: Chunk-by-chunk header download

For each pair of adjacent skeleton points [S_{i-1}, S_i]:

  • Request count = S_i.number - max(S_{i-1}.number, connectionPoint) headers starting from S_i.hash
  • Peer responds with headers in descending order
  • Client reverses to ascending, validates each header against its parent
  • Since chunks are processed in order from the connection point forward, every header has a known parent and can be fully verified

When all chunks in a skeleton are processed and the node is still behind the peer, a new skeleton is requested from the current head.

Key constants (from RskSystemProperties.java)

Constant Value Meaning
CHUNK_SIZE 192 Headers per chunk / skeleton interval
maxSkeletonChunks 20 Max chunks per skeleton round
timeoutWaitingRequest 30 s Per-request timeout

Message types used

Message Type ID Fields
BlockHashRequest 8 id, height
BlockHashResponse 18 id, hash
SkeletonRequest 16 id, startNumber
SkeletonResponse 13 id, blockIdentifiers[]
BlockHeadersRequest 9 id, hash, count
BlockHeadersResponse 10 id, headers[]

All extend MessageWithId and use the RSK double-wrapped RLP encoding described in section 4.

Rust implementation

See crates/sync/src/lib.rs:

  • SyncState enum drives the state machine
  • SyncHandler forwards inbound messages to SyncService via an mpsc channel
  • SyncService::start() runs the event loop (timer ticks + channel events)
  • SyncManager::handle_headers_response() validates and stores each chunk

Discovery NodeTable eviction (liveness, non-consensus)

rskj's Kademlia bucket is full at KademliaOptions.BUCKET_SIZE = 16. When a new node would overflow a bucket, Bucket.addNode returns the least-recently-seen entry (getOldestEntry, ordered by BucketEntry.lastSeenTime) as an eviction candidate rather than dropping it; PeerExplorer.addConnection then opens a NodeChallenge that pings the candidate. If the candidate pongs, the challenge is cancelled and the newcomer is discarded (live peers are kept); if it stays silent until the ping request expires, removeConnection evicts it (PeerExplorer.java lines ~553–562, removeConnections/removeConnection).

rustock does not replicate the asynchronous challenge machinery. Instead NodeTable (crates/networking/src/discovery/table.rs) stores a last_seen Instant per entry (mirroring BucketEntry.lastSeenTime) and, on a full-bucket insert, evicts the least-recently-seen entry only if it has gone stale (EVICTION_STALENESS = 90s); otherwise the newcomer is rejected. This is behavior-equivalent because rustock's discovery loop already pings every table node every 15s, so an entry whose last_seen is older than several ping rounds is precisely a node that failed its "challenge". A bucket of still-responsive nodes is left intact, matching rskj's bias toward known-good peers. last_seen is liveness-only and is never serialized — the wire/disk DiscoveryNode is unchanged. This subsystem is not consensus-critical (it only affects which peers we talk to, never state), so exact replication is unnecessary.

References