Skip to content

CASSJAVA-124: Add GRACEFUL_DISCONNECT support (CEP-59) - #2091

Open
Shanzita wants to merge 22 commits into
apache:trunkfrom
Shanzita:cep-59
Open

CASSJAVA-124: Add GRACEFUL_DISCONNECT support (CEP-59)#2091
Shanzita wants to merge 22 commits into
apache:trunkfrom
Shanzita:cep-59

Conversation

@Shanzita

@Shanzita Shanzita commented May 28, 2026

Copy link
Copy Markdown

Summary

Implements client-side support for CEP-59 (Graceful Disconnect – In-Band
Connection Draining) in the Java driver.

When a Cassandra node shuts down, it emits a GRACEFUL_DISCONNECT event to
subscribed connections. The driver:

  • Stops sending new requests on the affected connection
  • Drains in-flight requests to completion
  • Closes the whole pool to the draining node and fails over
  • Reconnects automatically via the normal reconnection policy

Design

  • Capability is negotiated per connection: every channel that intends to
    register for GRACEFUL_DISCONNECT sends OPTIONS and filters its REGISTER
    against its own SUPPORTED response. There is no driver-global capability
    flag, so mixed-version clusters (rolling upgrades) work correctly.
  • The internal GracefulDisconnectEvent is node-scoped: it means "this node
    is shutting down", regardless of whether it arrived on a pool connection or
    the control connection, and the pool drains accordingly.
  • If a server advertises the capability but rejects the REGISTER, the driver
    retries once without the event type instead of failing channel init
    (mixed-version / evolving-server tolerance).
  • advanced.connection.graceful-disconnect-enabled config option (default true).
  • graceful-disconnects (session) and pool.graceful-disconnects (node)
    counters, initialized in the Dropwizard, Micrometer and MicroProfile
    backends and incremented where events are received.

Testing

  • 3,552 core unit tests pass (new coverage: ProtocolInitHandler capability
    filtering and REGISTER-rejection degradation, node-scoped ChannelPool drain,
    metric increments, duplicate/late event tolerance)
  • New GracefulDisconnectIT: bounded 2-node ccm test that drains a node under
    steady query load and asserts the metric observed the event, queries fail
    over, and zero exceptions reach the application. Gated on Cassandra 7.0+, so
    it is skipped until a server with CEP-59 is available in CI.
  • Manually validated end-to-end against a CEP-59 server build
    (CASSANDRA-21191: [CEP-59] Implementation of In-Band Connection Draining (Graceful Disconnect) cassandra#4953) on 2-node and 3-node ccm clusters:
    nodetool drain mid-load → 0 disruptive exceptions, thousands of successful
    queries after the event, drained node goes DOWN cleanly, failover invisible
    to the application. The 3-node run exercised the event arriving on the
    control connection and a pool connection simultaneously.

Depends on: native-protocol 1.5.3-SNAPSHOT (Shanzita/native-protocol:cep-59,
the branch behind datastax/native-protocol#61; installed by install-snapshots.sh
until 1.5.3 is released)
JIRA: CASSJAVA-124

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants