feat(pathways): pause and resume delivery from the control plane

[jbiskur]

Blocked on a dependency publish. This needs @flowcore/data-pump@0.23.0, which is tagged on GitHub but not on npm. See the blocker note at the bottom.

Summary

Adds datapumpPause and datapumpResume to the poll-based command queue, and makes the pause durable so a pathway comes back paused at the same position after a restart.

A paused pump keeps fetching, keeps its buffer, keeps its cursor and keeps reporting pulses. Only delivery to the handlers stops, so the control plane still sees a live pathway. A batch already inside a handler finishes and acknowledges.

Targeting needs no new control-plane field

All three commands share one target list, carried in the existing sourceFlowTypes:

Entry	Targets
"orders.0"	Every pump group on that flow type
"orders.0::hot"	That one pump only
empty or absent	Every pump

The composite form is already public API — it is the concurrency.byPumpGroup key and the value reset() returns. A pump group belongs to exactly one flow type, so the pair is the identity and one field carries both levels of precision.

Three gaps fixed along the way

• datapumpRestart silently dropped pump groups. It passed only sourceFlowTypes to reset(), though PumpResetFilter already accepted pumpGroups. Restart now shares the pause filter.
• stopAt was parsed and discarded. It now reaches FlowcoreDataPump.restart(state, stopAt).
• Unknown command types reported success. The handler logged a warning and the poller then posted running, so the control plane recorded a success that never happened. Unknown types and filters matching no pump now throw and report failed.

Validation

The poll response is validated with Zod instead of cast. Everything downstream acts on a live pump: a malformed position would move a cursor, a malformed target list would pause the wrong pathway.

Durability

New PathwayDeliveryStore, configured with withPathwayDeliveryStore(). The default is in memory; PostgresPathwayDeliveryStore survives a redeploy, a pod restart and a cluster leader change. The stored pause is applied before the pumps start, so a restored pathway never delivers a batch on the way up.

Test plan

• CI green
• @flowcore/data-pump bumped to ^0.23.0 (blocked, see below)
• 78 tests / 236 steps pass locally with Postgres running
• deno fmt --check, deno lint, deno check src/mod.ts clean
• 36 new steps across three new test files

New tests cover: target parsing, pause/resume by flow type and by pump group, union targeting, empty-match handling, idempotence, restart honouring pump groups, stopAt pass-through, unknown-type failure, persistence on pause and resume, a store write failure failing the command, and the Postgres store surviving a new instance.

Blocker

This PR must not merge until @flowcore/data-pump@0.23.0 is on npm. The library calls pump.pause() and pump.resume(), which do not exist in the currently resolved 0.22.1.

The publish is broken in the data-pump repo, and has been since 0.22.1 — 0.22.2, 0.22.3 and 0.23.0 are all tagged on GitHub and absent from npm:

1. The publish job pinned node 20 and ran npm install -g npm@latest, failing with EBADENGINE — fixed in fix(ci): publish to npm on node 22 data-pump#94.
2. A failed publish could not be re-run — fixed in fix(ci): allow the npm publish to be dispatched manually data-pump#96.
3. npm auth. The publish now reaches npm publish and fails with E404 Not Found - PUT https://registry.npmjs.org/@flowcore%2fdata-pump. The package exists, so a 404 on PUT means the token is expired or no longer has publish rights on the @flowcore scope. This needs a maintainer to rotate NODE_AUTH_TOKEN.