Both session recovery paths (the startup stale-entry repoint and the
lazy in-message recovery) rebuilt the routing entry with updated_at=now
and never consulted _should_reset, so an opt-in idle/daily session_reset
policy was silently dead across any gateway restart: a recovered session
always looked freshly active, and since every subsequent message bumps
updated_at, a session recovered stale could then never age out at all.
Fix in three parts:
- _create_entry_from_recovered_row derives updated_at from the durable
last_activity_at the finder already returns on the row (no extra DB
round-trip; the original PR added SessionDB.get_last_activity for
this, unnecessary post-#82633), falling back to created_at. An
invalid or missing started_at now maps to epoch 0 instead of now — an
invalid durable timestamp must look old, never freshly active.
reset_had_activity is set from the row's durable activity/message
signals so the continuity hint stays accurate.
- _recover_session_from_db evaluates _should_reset on the rebuilt entry:
an overdue session is durably promoted to a reset boundary
(promote_to_session_reset, falling back to end_session) and the stale
mapping is dropped instead of repointed.
- _query_recoverable_session no longer reopens the row; the
get_or_create_session recovery phase evaluates _should_reset first and
either feeds the normal auto-reset create path (reset notice,
prev_session_id continuity, durable promotion) or reopens and
publishes the recovered entry exactly as before.
Behavior is unchanged under the default session_reset mode "none":
_should_reset returns None there, so recovery still resumes every
recoverable row — only users who opted into idle/daily resets see the
policy actually applied across restarts.
Cherry-picked from #78618 and adapted to the #82633 finder.
(cherry picked from commit 31c71f7629)
find_latest_gateway_session_for_peer filtered non-recoverable rows out of
candidacy BEFORE ordering, so recovery could search behind a /new reset
boundary and resurrect an older still-open row for the same peer —
silently restoring the exact context the user reset.
Rebuilt against the #82633 finder (has-messages ranking +
COALESCE(last_activity_at, started_at) recency): the fence is expressed
as a NOT EXISTS guard inside both the exact-key and peer-fallback
queries — a candidate is rejected when an intentional boundary row
(session_reset / session_switch / idle / daily / suspended /
resume_pending_expired) for the same peer ended after the candidate's
last activity. If the conversation's most recent event is an intentional
reset, recovery returns nothing rather than reaching behind it.
Cherry-picked from #68617 and adapted to the rewritten finder.
(cherry picked from commit bb2c562a16)
Second, deeper pass over tools/gateway/hermes_cli plus first pass over
the trees wave 1 missed (acp, acp_adapter, skills, computer_use, docker,
dashboard, conformance, monitoring, secret_sources, hermes_state,
providers). Same rubric as wave 1 (AGENTS.md test policy); security,
alternation/caching invariants, issue-number regressions, and E2E kept.
Real test-quality fixes found and rooted out along the way:
- tests/tools/test_command_guards.py made real auxiliary-LLM HTTPS calls
(DEFAULT_CONFIG smart-approval leaked in) — pinned approval
mode=manual via autouse fixture: 17.4s → 0.4s.
- test_model_switch_custom_providers.py / test_user_providers_model_switch.py
silently probed live provider catalogs (~2s/test) — stubbed
cached_provider_model_ids/provider_model_ids/fetch_api_models.
- test_telegram_noise_filter.py: 15-platform copy-paste matrix over
shared gateway.run logic → 3 representative platforms (55s → 3.9s).
- test_gateway_shutdown.py: stop()'s 5s interrupt-deadline loop spun on
MagicMock agents — interrupt.side_effect now clears _running_agents
(22s → 1.0s).
- test_gateway_inactivity_timeout.py poll-harness timings shrunk 3-5x
(24s → 1.1s); test_mcp_stability.py backoff/SIGTERM-grace sleeps
patched (15.4s → 2.5s); test_async_delegation.py negative-drain wait
5s → 0.5s.
- test_telegram_init_deadline.py: loop-block margin restored to 1.0s
with rationale comment — the watchdog-dump assertion needs the loop
blocked well past deadline+grace under parallel load (flaked once in
the 40-worker verification run at a 0.2s margin).
Verification: full hermetic suite via scripts/run_tests.sh —
2,438 files, 21,718 tests passed, 0 failed, 293.9s wall.
Suite totals vs original baseline: 46,820 → 19,757 test functions
(−57.8%), wall 583.5s → 293.9s (−50%), subprocess CPU 13,564s → 11,623s.
A hard gateway crash (exit code 1) skips the graceful shutdown path, so
sessions.json is never cleared and is left pointing at sessions already
ended in state.db. On the next startup get_or_create_session() reuses
those stale entries as long as the time/policy reset checks pass — it
never consults end_reason — so every incoming message is silently routed
into a closed session, with no log or error (#52804).
SessionStore._ensure_loaded_locked() now calls a new
_prune_stale_sessions_locked() that drops any entry whose session_id has
end_reason IS NOT NULL in state.db. Idempotent, _db=None / legacy-absent
safe, DB errors non-fatal, sessions.json rewritten only when something
was pruned. Self-heals into a fresh session on the next message.
Reported and diagnosed by @terry197913 (#52808).
Hill Chitsanupong
Tranquil-Flow
Teknium
dsad
João Vitor Cunha