Skip to content

fix(hande): exit on SIGTERM while the native bot stays connected (#2535) - #2536

Merged
jinon86 merged 1 commit into
mainfrom
fix/2535-hande-sigterm-hang
Oct 26, 2026
Merged

jinon86 merged 1 commit into
mainfrom
fix/2535-hande-sigterm-hang

Conversation

@jinon86

@jinon86 jinon86 commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Fixes #2535.

Problem

A CCC_CHANNEL=hande bridge never exits on SIGTERM by itself while the native bot is connected (which is always):

  • systemd: every stop runs into TimeoutStopSec=70s → State 'stop-sigterm' timed out. Killing. / Failed with result 'timeout' (observed on soonwook, nosuk, dungae — exactly 70 s each; the Matrix bridge on the same node stops in 1 s).
  • Termux runit: no automatic KILL, so the unit sits in got TERM indefinitely with health degraded until sv force-restart (gongyung, daegyo).

Root cause

HandeTransport.run() did async with self._server: await self._stopping.wait(). SIGTERM cancels the serving task → Server.__aexit__ → unbounded Server.wait_closed(), which from Python 3.12 waits for every open connection. The native bot never hangs up on its own, and the transport.close() that would hang up lives in MatrixBot.serve's finally after the TaskGroup unwinds — so it was never reached. Deadlock until SIGKILL. (The bridge log shows nothing between SIGTERM and SIGKILL — not even Matrix frontend stopped.)

Fix

  • run() no longer uses async with server; it calls close() in its own finally — client hung up first, server wait bounded (existing 5 s), socket unlinked.
  • close() is now idempotent and cancel-safe (single shielded close task), so run()'s finally and MatrixBot.serve's finally share one shutdown.

Tests

  • New test_cancelled_run_returns_while_the_bot_stays_connected — fails on main (hangs → TimeoutError after 10 s) and passes with the fix; also asserts the bot sees EOF and the socket is removed.
  • New test_close_while_the_bot_stays_connected_ends_run.
  • Local: hande suites 62 passed; -k "hande or matrix" 837 passed; ruff check . clean; mypy (CI subset, .github/pythonpath) no issues — with the pinned .github/requirements/bridge-ci.txt toolchain.

Rollout note

After merge, nodes pick this up via the normal self-update path. The first restart onto the fix still runs the old code's stop (70 s on systemd; Termux needs sv -w 90 force-restart); later stops should complete within the drain budget. Verification plan in #2535 (acceptance criteria).

🤖 Generated with Claude Code

HandeTransport.run() awaited 'async with server', whose exit calls
Server.wait_closed() unbounded; from Python 3.12 that waits for every open
connection and the native bot never hangs up, so a SIGTERM-cancelled run()
never reached the close() in MatrixBot.serve's finally. systemd stops hit the
70s TimeoutStopSec and SIGKILL every time; runit (Termux) stayed in 'got TERM'.

run() now closes the transport in its finally (client first, bounded server
wait) and close() is idempotent/cancel-safe so both callers share one close.
Regression tests cover cancel and close with the bot still connected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@seoseo-ai seoseo-ai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved after exact-head validation and fresh operator authorization using the relay-held seoseo-ai credential.

@jinon86
jinon86 added this pull request to the merge queue Oct 26, 2026
Merged via the queue into main with commit 3eee62e Oct 26, 2026
14 checks passed
@jinon86
jinon86 deleted the fix/2535-hande-sigterm-hang branch October 26, 2026 08:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants