slipstream: take txs from anyone, get them into the pool's blocks

[rsantacroce]

What

This adds a new service, slipstream/, alongside payout/. Anyone can POST it a raw tx, and it gets that tx into the pool's blocks. That includes txs the network will not relay, such as BIP300/301 txs (deposits, withdrawal bundles, BMM requests), or any other consensus-valid tx.

The tx goes to the pool's own bitcoind. The enforcer's template mempool mirrors that node over ZMQ, so once the node accepts a tx it reaches the templates the proxy mines. The enforcer is unchanged, and the enforcer's own BIP300 rules still apply on the way in.

The first version of this PR used an enforcer-side private mempool (LayerTwo-Labs/cusf-enforcer-mempool#130 and LayerTwo-Labs/bip300301_enforcer#642). On the enforcer maintainer's suggestion it was replaced by this simpler path, and both of those PRs are closed.

The node

• Non-standard txs need the node to run -acceptnonstdtxn=1.
  • Core refuses that flag on mainnet: acceptnonstdtxn is not currently supported for main chain. This was checked against the drivechain-patched, ecash betanet and stock builds. The flag works on signet and regtest.
  • drynet3 reports itself as main, so non-standard txs there need a patched Core that lifts the check.
• BIP300 deposits are standard on a drivechain-patched node already, so they need no flag.
• Txs are relayed. The node broadcasts an accepted tx like any other, so another pool can mine it. mined_by_pool records whose block it was.

How it behaves

• Fee rule. A tx must pay the higher of the minimum submission rate (1 sat/vB, SLIPSTREAM_MIN_FEE_RATE) and the current mineable rate. This follows Slipstream's own rule.
  • The mineable rate is read from the template the proxy is mining: the floor while the template has room, and the fee rate of its cheapest tx once it is full.
  • Nothing can be taken back out of a mempool, so the rule runs before broadcast. testmempoolaccept gives the fee and size without broadcasting, and only a tx that passes both the node and the rule reaches sendrawtransaction.
• Every message is saved. slipstream_submissions keeps every POST exactly as it arrived, including refusals and their reasons.
• Tracking.
  • Each accepted tx moves through pending, in_template, mined (with mined_by_pool from blocks_found) and confirmed.
  • If a tx leaves the mempool unmined, the service sends it again. The node's answer decides between pending and dropped, and a dropped tx carries the node's reason.
  • A tx whose block is orphaned goes back to pending when the node restores it.
  • Every status change is appended to slipstream_events.
• info.json is served here, at GET /info.json.
  • Facts come from pool_meta: mode (the exact pool_mode), fee_bps, coinbase_tag and the addresses. They are never configured here.
  • Presentation fields come from env: POOL_NAME, POOL_CHAIN (e.g. betanet), POOL_LOGO, the public URLs and so on.
  • slipstream_url is included.

Storage

The service keeps its own slipstream.db. It never writes to shares.db, and reads it only for pool_meta and blocks_found.

Also in this PR

• Docker: Dockerfile.slipstream, a slipstream compose service bound to loopback (publish it through nginx with SLIPSTREAM_TRUST_PROXY=1), and the image in the GHCR build matrix.
• CI: the service's unit tests run in node-tests, and the end-to-end test runs in integration-test.
• Deploy templates: deploy/systemd/simplepool-slipstream.service and deploy/nginx/slipstream.conf, in the @USER@/@ROOT@ form install.sh already renders. The install steps are in slipstream/README.md. install.sh does not set slipstream up yet.
• Docs:
  • slipstream/README.md, plus short entries in the README and tests/README.md.
  • docs/simplepool.html gains a Slipstream section, with a generated sequence diagram (sequence-diagrams.py --check passes).
  • docs/simplepool.html gains a Dashboard section listing every page and endpoint, and slipstream.db joins the data model.
  • Stale parts of the page are updated: long polling, the payout worker's three modes, and the footer.

Fixed along the way

Behind nginx, the rate limit was keyed on the first X-Forwarded-For hop. The client writes that one itself, so any client could choose its own key. It is now keyed on the last hop, the one the proxy added, and the vhost template overwrites the header rather than appending to it. There is a unit test for it.

Testing

• cd slipstream && npm test: 25 unit tests against a fake bitcoind and enforcer.
  • Every status transition, including rebroadcast, dropped with the node's reason, an orphaned block, and "already in block chain".
  • The fee rule, including that a refused tx is never sent.
  • Submission logging, info.json for every mode, and the HTTP routes.
• tests/test_slipstream_regtest.sh, against patched bitcoind (acceptnonstdtxn=1) and the stock release enforcer. Passed locally, and now runs in CI:
  1. A non-standard tx (a dust output) is accepted and lands in the node's mempool.
  2. It reaches the enforcer's template, and the service reports it as in_template.
  3. generateblock mines that template. The history reads accepted, in_template, mined, confirmed.
  4. A tx at about 0.14 sat/vB is refused as fee-rate-too-low and never reaches the node's mempool.
  5. A tx replaced with bumpfee in the node's mempool is recorded as dropped, with the node's reason: insufficient fee, rejecting replacement ....
  6. Every submission is in the log, refusals included.
  7. info.json and /api/fees answer.

Not in this PR

• The dashboard /slipstream page. That is dashboard: a Slipstream page #84, stacked on this PR.
• A CHANGELOG entry. It is written at release time.