Skip to content

Preserve in-flight opens when clearing connection pools - #4718

Merged
mdaigle merged 13 commits into
dotnet:mainfrom
mdaigle:mdaigle-fix-clearallpools-openasync-race
Sep 23, 2026
Merged

mdaigle merged 13 commits into
dotnet:mainfrom
mdaigle:mdaigle-fix-clearallpools-openasync-race

Conversation

@mdaigle

@mdaigle mdaigle commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes #4714.

Restore the wait-handle pool's previous behavior: admitted requests can finish on a retired pool, and their connections are destroyed when returned. Remove the post-wait shutdown short-circuit and synthetic semaphore releases together. Keep the cached error and its expiry timer available to remaining waiters.

Leave clearing to the factory's explicit-clear or deferred-pruning path rather than calling Clear() inside wait-handle Shutdown(). Shutdown can run under the pool-group lock before the pool is queued for release. An inline code comment records why reclamation and disposal remain outside that path.

Remove the earlier retry exception and SqlConnection.OpenAsync retry changes. New requests still use the factory's existing replacement-pool routing. Channel-pool behavior is unchanged.

Deterministic coverage

  • Hold the first physical login behind a gate, confirm a second request is pending on the same pool, clear the pool, then release the login. Exercise Open and OpenAsync with both ClearPool and ClearAllPools against the simulated TDS server.
  • Assert exact login counts, retired-pool ownership, disposal on close, and replacement-pool selection.
  • Gate mocked physical creation to force cancellation before a pending acquisition completes and verify its connection is returned and destroyed.
  • Advance a fake clock to verify blocking-period expiry after shutdown.
  • Verify shutdown leaves idle connections untouched until the subsequent Clear() drains them.

Removed the repeated-open stress workload and wall-clock timeout threshold test. No test relies on repeated attempts to encounter the race. Bounded waits only guard against hangs.

Validation

  • All eight added acquisition/error-state regression cases failed when the removed shutdown short-circuit and error-state disposal were temporarily restored. The async public-API cases reproduced the reported pool-timeout exception.
  • The updated deferred-clear regression failed with Clear() still inside shutdown and passed after its removal.
  • With the latest changes, 365 pooling tests passed on each of net8.0 and net9.0.

Validation used installed SDK 10.0.300 from outside the checkout because pinned SDK 10.0.401 is unavailable. The deterministic public-API tests use the in-process TDS server; the customer's SQL Server workload remains unverified locally.

Deferred behavior

The late-return/clear race and saturated retired-pool waiter behavior identified in review also predate #4302 and remain outside this compatibility restoration. Factory-owned async retry is tracked in #4720. This PR does not guarantee that every admitted request succeeds after clearing.

Checklist

  • Tests added or updated
  • Public API changes documented: N/A, no API changes
  • Verified against customer repro (SQL Server unavailable locally)
  • Ensure no breaking changes introduced: restore pre-hardening wait-handle behavior

Copilot AI balanced review requested due to automatic review settings September 21, 2026 22:19
@mdaigle
mdaigle requested a review from a team as a code owner September 21, 2026 22:19
@github-project-automation github-project-automation Bot moved this to To triage in SqlClient Board Sep 21, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The retry path can leak replacement-pool capacity and allow exceptions to escape its background thread.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity

Open (1)
What changed in this PR

Fixes async opens interrupted by WaitHandleDbConnectionPool shutdown by retrying against the replacement pool.

Changes:

  • Distinguishes shutdown from pool timeout.
  • Adds replacement-pool retry logic and regression coverage.
File Description
WaitHandleDbConnectionPool.cs Retries pending async opens after shutdown.
WaitHandleDbConnectionPoolShutdownTest.cs Tests completion from a replacement pool.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@mdaigle
mdaigle marked this pull request as draft September 21, 2026 22:51
Route pending async opens on a cleared wait-handle pool back through the connection factory so they can complete from the replacement pool instead of surfacing a misleading pool timeout.

Add a regression test that parks an async pending open, clears the pool group, and verifies completion from a replacement pool.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 21, 2026 22:52
@mdaigle
mdaigle force-pushed the mdaigle-fix-clearallpools-openasync-race branch from e71bc6a to 39f4d39 Compare September 21, 2026 22:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The retry is unbounded and its public OpenAsync behavior lacks end-to-end coverage.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity · 1 Low severity

Open (2)
Resolved since last review (1)

Comment thread src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlConnection.cs Outdated
Remove shutdown interruption and retry routing. Let admitted requests finish on the retired pool and dispose their connections on return. Preserve error expiry for remaining waiters.

Cover sync and async clearing, cancellation, timeouts, and error expiry. Add a SQL Server regression workload for dotnet#4714.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 21, 2026 23:46
@mdaigle mdaigle changed the title Fix OpenAsync retry after ClearAllPools pool shutdown Preserve in-flight opens when clearing connection pools Sep 21, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

A shutdown/return race can re-vend a connection marked for disposal by pool clearing.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity · 1 Low severity

Open (2)
Resolved since last review (1)

Remove the concurrent stress workload and elapsed-time assertions. Gate physical creation and assert exact request ordering and creation counts around pool shutdown.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 22, 2026 00:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Saturated retired pools can still leave admitted waiters blocked until timeout or indefinitely.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 2 High severity · 1 Low severity

Open (3)
Previously missed (2)

In code that hasn't changed since last review

Low severity Document helper parameters and returned acquisition task

src/​Microsoft.Data.SqlClient/​tests/​UnitTests/​ConnectionPool/​WaitHandleDbConnectionPoolShutdownTest.cs:233

The repository's test documentation policy requires <param> and <returns> tags for helper methods where applicable (.github/instructions/testing.instructions.md:181-185). This new helper should document its three parameters and returned acquisition task.

This issue also appears on line 249 of the same file.

Low severity Document helper connection, async, and return values

src/​Microsoft.Data.SqlClient/​tests/​UnitTests/​SimulatedServerTests/​PoolClearDuringOpenTests.cs:132

The repository's test documentation policy requires <param> and <returns> documentation for helper methods where applicable (.github/instructions/testing.instructions.md:181-185). Please document connection, async, and the returned task.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 22, 2026 18:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Newly added test helpers lack the required parameter and return-value XML documentation.

Review effort: Balanced
Findings: 2 High severity · 1 Low severity

Open (3)
Previously missed (2)

In code that hasn't changed since last review

Low severity Document helper parameters and returned acquisition task

src/​Microsoft.Data.SqlClient/​tests/​UnitTests/​ConnectionPool/​WaitHandleDbConnectionPoolShutdownTest.cs:234

The test guide requires helper methods to document parameters and return values (.github/instructions/testing.instructions.md:181-185). Please document the pool, owner, optional completion source, and the acquisition task returned here.

This issue also appears on line 249 of the same file.

Low severity Document sync/async selector and returned task

src/​Microsoft.Data.SqlClient/​tests/​UnitTests/​SimulatedServerTests/​PoolClearDuringOpenTests.cs:135

The test guide requires helper methods to document their parameters and return values (.github/instructions/testing.instructions.md:181-185). This new helper only has a summary, so its sync/async selector and returned task contract are undocumented.

@mdaigle

mdaigle commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor Author

🤖 Moved the error-state disposal rationale inline beside the changed code. The factory-owned clearing rationale is also inline.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 22, 2026 19:17
This was referenced Oct 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Hotfix 7.1.1 PRs targeting main that should be backported to release/7.1 for 7.1.1.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

7.1.0 regression: in-flight OpenAsync fails instantly with pool "Timeout expired" when ClearAllPools()/ClearPool() runs

4 participants