Skip to content

Session pool shut down by return_connection is never replaced when cluster DOWN handling is discounted #1027

Description

@dkropachev

Problem

HostConnection.return_connection repeats, in the session pool, the pattern #847 identified in the control connection:

is_down = self.host.signal_connection_failure(connection.last_error)   # pool.py:572
...
if is_down:
    self.shutdown()
    self._session.cluster.on_down(self.host, is_host_addition=False)   # pool.py:580

The conviction policy accepting the failure is treated as a guarantee that cluster DOWN handling will follow, but Cluster.on_down() can decline.

With two sessions on one cluster, if session A still holds an open pool to the host, session B's on_down() is discounted and returns early: Session.on_down -> remove_pool never runs, and _start_reconnector never runs. Session B keeps a shut-down pool object in _pools with nothing scheduled to replace it, while the host stays up.

The already-down and already-reconnecting variants are mostly covered incidentally, because a dispatched on_down calls session.on_down() for every session.

Expected behavior

A session that shuts down its pool for a host ends up either with a working pool again or with a reconnection scheduled, regardless of whether cluster-wide DOWN handling ran.

Notes

Unlike #847 this is not a one-line contract fix -- there is no existing mechanism for "one session lost its pool while the host is still up". Options include recreating the pool, leaving it for update_created_pools to recover, or giving sessions their own reconnector. Overlaps #921 / #925 (topology-generation fencing) and #1015.

Found while reviewing #1024. The return-value contract that PR adds to Cluster.on_down() and Cluster.signal_connection_failure() is what this path would consume.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions