Cancelled client-facing requests (search, dialectic chat/stream) could
abandon an open transaction: when a request task is cancelled, the
CancelledError can be re-delivered onto the ROLLBACK/close awaits in the
session-cleanup `finally`, so ROLLBACK never reaches Postgres and close()
never runs. Supavisor (transaction mode) does not reset the orphaned
backend on disconnect, leaving it `idle in transaction` indefinitely.
These zombies accrued ~5/hour, exhausting the connection pool and pinning
the xmin horizon.
Fix, centralized in the three session entry points (get_db, get_read_db,
tracked_db) so every subsystem inherits it:
- `_run_to_completion`: run cleanup as a detached task and await it through
asyncio.shield in a loop, so a (re-delivered) cancellation cannot
interrupt ROLLBACK/close. Survives a cancel storm.
- `_finalize_session`: rollback then always close (inner try/finally);
invalidate the connection on a broken-mid-protocol InterfaceError/
OperationalError/DBAPIError so a dead connection is not returned to the
pool; never let a cleanup error mask the original.
Note: anyio.CancelScope(shield=True) — the originally-proposed primitive —
does NOT defer a native asyncio.Task.cancel() (verified empirically under
both asyncio and anyio loops), which is how Starlette and the deriver's
uvloop cancel; hence the stdlib approach. No new dependency.
Tests: cancellation, re-delivery-during-cleanup, cancel-storm,
broken-connection->invalidate, and DI-path teardown.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
13a1497436