Skip to content

Latest commit

 

History

History
175 lines (131 loc) · 8.44 KB

rskj-import.md

File metadata and controls

175 lines (131 loc) · 8.44 KB

Importing an rskj database

Rustock can import a synced rskj database instead of syncing from peers. On RSK mainnet at block 9,230,008 this takes about an hour, against days of syncing.

Why it is a conversion, not a copy

The two nodes store the same chain in structurally different ways, so the databases cannot be swapped:

rskj rustock
container one RocksDB per datasource directory (blocks/, unitrie/, receipts/) one RocksDB, 7 column families
blocks hash -> RLP(header, txs, uncles) in one value split across headers and block_bodies
index MapDB file of Java-serialised BlockInfo block_numbers + total_difficulty
state unitrie/: hash -> node message trie_nodes, byte-identical values

RocksDB cannot adopt another database's directory as a column family. And the canonical-chain flag and cumulative difficulty are not in a key-value store at all — they live in a MapDB file encoded with Java object serialisation.

The state trie copies verbatim: a trie node's serialised form is identical in both implementations, which is why their state roots agree. This is not an assumption — importing over a partially synced node found 20% of nodes already present, and trie keys are content-addressed, so a key only collides when the value is byte-identical.

Running an import

Extract the rskj database (only two datasources are needed; receipts are regenerated by execution and blooms are derived):

gzip -dc snapshot.tgz | tar xf - \
    --wildcards 'database/mainnet/blocks/*' 'database/mainnet/unitrie/*'

Stop the node, then:

rustock --import-rskj /path/to/database/mainnet \
        --data-dir /var/lib/rustock \
        --import-metrics import.csv

Then rebuild chain metadata with the fast path:

rustock --metadata-in-memory --data-dir /var/lib/rustock

Phases run in this order, each resumable because each flushes before reporting itself complete:

  1. unitrie -> trie_nodes (verbatim copy)
  2. blocks -> headers + block_bodies (RLP split)
  3. compaction (before the read-heavy phase, not after)
  4. metadata -> block_numbers + total_difficulty (derived)

Options

Source and output

Flag Default Meaning
--import-rskj <DIR> — Directory holding blocks/ and unitrie/; runs the import and exits
--import-metrics <CSV> — Append per-phase phase,scanned,written,skipped,bytes,elapsed_secs
--import-metadata-only off Skip the trie and block phases; rebuild metadata only

Write tuning

The defaults are the fastest safe combination; these flags exist to turn optimisations off. They apply only through BlockStore::open_for_import, so the node's own behaviour is unchanged — these trades are safe only because a failed import is re-run rather than repaired.

Flag Default Why that default
--import-skip-existing off Keys are content-addressed, so rewriting an existing entry writes identical bytes. The lookup is a performance trade, not correctness. At 20% overlap it cost ~85% of throughput. Turn on only when topping up a nearly-complete datadir.
--import-enable-wal off The WAL recovers unflushed writes after a crash; a failed import is re-run from the source instead. Enabling it writes every entry twice.
--import-write-buffer-mb <N> 256 Larger memtables mean fewer, bigger flushes and less write amplification, at the cost of dirty data in RAM.
--import-auto-compaction off Compacting mid-load repeatedly rewrites data that is still changing. One pass runs before the metadata phase instead.
--import-parallelism <N> 0 = CPU count Background flush/compaction threads. The scan itself is single-threaded.

Diagnostics

Flag Meaning
--probe-unitrie <HASHES> with --probe-source <DIR> Look up node hashes in an rskj unitrie. Answers whether a snapshot is archival or pruned: pass historical state roots and see whether they are present.
--metadata-range <FROM> <TO> Rebuild metadata for a block range only, for benchmarking without paying for a full pass. Requires a completed full pass, since it seeds from existing values at the range boundaries. Non-destructive.

Chain metadata: three ways

rskj's index is unreadable from Rust, so the canonical chain and total difficulty have to come from somewhere else. Three methods, all producing identical output:

Flag Method Time (9.2M blocks) Needs
--metadata-in-memory Sequential header scan into RAM, walk in memory, batched writes 2m 47s ~2 GB RAM
--metadata-by-walk Follow parentHash on disk from the tip 5h 0m minimal RAM
--import-index-dump <FILE> Load a dump of rskj's MapDB index ~2.5h + dump JVM, tools/dump-index

Use --metadata-in-memory. The other two exist for constrained cases: --metadata-by-walk when 2 GB of headers is unaffordable, and the index dump as an independent cross-check of the derivation.

Why the difference is so large

Following parentHash is a chain of dependent random lookups — each address is only known once the previous read returns, so nothing can be batched or prefetched. The data's location is not the problem; the traversal order is.

Reading the same headers sequentially into a map and walking that removes the constraint. Identical logic, measured 1,065 blk/s on disk against 2,089,745 blk/s in RAM — a factor of ~1,960.

Note the block header carries difficulty, its own, but no cumulative total. Total difficulty exists only as a running sum, which is why every method has to accumulate rather than copy it. That sum was never the bottleneck: the forward pass runs at 420,825 blk/s.

Measured performance

RSK mainnet, block 9,230,008. 4 vCPU, 7.6 GB RAM, network-attached SSD.

Phase Items Time Rate
unitrie 1,246,651,400 nodes 44m 3s 471,717 nodes/s
blocks 12,531,886 entries 9m 25s 22,167 blk/s
metadata (in-memory) 9,230,009 blocks 2m 47s —
— header scan 12,531,886 headers 140.3s 89,312/s
— chain walk (RAM) 9,230,009 blocks 4.42s 2,089,745 blk/s
— forward pass 9,230,009 blocks 21.9s 420,825 blk/s

Resulting database: ~154 GB.

Write-tuning progression on the trie phase:

Configuration nodes/s
Checked lookups, default writes 59,394
Unconditional writes 395,599 (6.7x)
+ WAL off, 256 MB buffers, no auto-compaction 467,398 (7.9x)

Notes and gotchas

Stop the node first. RocksDB's lock file prevents two writers, so this fails safe rather than corrupting — but the import will simply refuse to open.

Never use SIGKILL. Import writes bypass the WAL, so anything still in a memtable exists only in RAM. SIGTERM and SIGINT are caught: the import stops at the next batch boundary and flushes. SIGKILL cannot be caught, and killing an import has produced a database whose canonical-chain walk died 59 blocks from the tip, because a phase reported 12.5M blocks written while part of that was still in memory.

Compaction runs before the metadata phase, not after. That phase is read-heavy; against 735 un-merged L0 files it managed 7 MB/s at 10% CPU. The compaction is also nearly free, because the import writes in sorted key order — RocksDB satisfies it with trivial moves to level-6 rather than rewriting 153 GB.

Progress is measured by keyspace position. Both datasources are keyed by uniformly distributed hashes and RocksDB iterates in sorted order, so the leading bytes of the current key give the progress directly. rocksdb.estimate-num-keys was tried first and proved badly wrong: it sums table properties over open SST files, and the source is opened with a 256-handle table cache against 2,180 files, so it under-reported by ~8x and percentages ran past 100%.

Archival vs pruned snapshots. Use --probe-unitrie to check. An archival snapshot holds every historical state root, which means every block's pre-state is already on disk — so blocks need no re-execution to import, and re-execution for benchmarking can be parallelised, since each block's pre-state is available independently of its neighbours.