hermes-agent/gateway
kshitij 8855766716 fix(gateway): expire orphaned drain markers past a max-age so a leaked marker can't wedge the gateway (#85433)
The NS-570 epoch stamp clears a drain marker that survives a machine
restart — but it assumes every drain-gated action ends in a restart. When
a maintenance action completes WITHOUT recreating the container and the
writer never cancels the drain, the orphaned marker still carries the
current epoch, so the 1s drain watcher honours it forever and the gateway
bounces every inbound message with the 'draining for a maintenance
action' text (observed in the field: a Hermes Cloud instance refused all
Telegram turns for ~3 days).

The marker already records requested_at; now the readers check it. A
marker older than DRAIN_REQUEST_MAX_AGE_SECONDS (1h) reads as stale in
drain_requested() and drain_notification_suppressed(), with a loud
warning log. Leniency mirrors the epoch check: a missing or unparseable
timestamp still reads as drain-active (fail-safe toward quiescing), and
a legitimately long drain keeps a sanctioned keep-alive — re-calling
write_drain_request() refreshes requested_at.

Fixes #85433
2026-08-13 23:23:48 +05:30
..
assets
…
builtin_hooks
…
platforms fix: don't double-count session-stream turns in the shutdown drain 2026-08-13 09:34:08 -07:00
relay fix(cron): relay-fronted Slack delivery — synthetic creation-thread capture + preflight fronted-platform blindness 2026-08-13 10:46:13 -07:00
__init__.py
…
agent_cache_pressure.py fix: follow-ups for salvaged PR #80795 2026-08-07 21:02:33 +05:30
authz_mixin.py fix(plugins): isolate ownership by profile 2026-08-12 19:13:32 -07:00
cgroup_cleanup.py
…
channel_directory.py fix(gateway): run channel-directory write off the event loop 2026-08-11 20:37:06 +05:30
code_skew.py
…
config.py feat(gateway): allow selective multiplex profile serving 2026-08-10 22:48:24 -07:00
cwd_placeholder.py
…
dead_targets.py
…
delivery.py
…
delivery_ledger.py
…
display_config.py
…
drain_control.py fix(gateway): expire orphaned drain markers past a max-age so a leaked marker can't wedge the gateway (#85433) 2026-08-13 23:23:48 +05:30
hooks.py fix: handle replacement transforms after CLI streaming 2026-08-08 14:06:30 +05:30
kanban_watchers.py feat(kanban): explicit notify/wake delivery modes with faithful wake session routing 2026-08-13 10:47:40 -07:00
lifecycle_ledger.py fix(gateway): treat pid=None lifecycle sentinel as unknown ownership in mark_exited 2026-07-28 22:41:09 -07:00
memory_monitor.py
…
message_timestamps.py
…
mirror.py
…
pairing.py fix(secrets): route authz gate and pairing allowlist reads through the profile secret scope 2026-08-02 10:02:33 -07:00
platform_registry.py fix(plugins): isolate ownership by profile 2026-08-12 19:13:32 -07:00
profile_routing.py feat(gateway): allow selective multiplex profile serving 2026-08-10 22:48:24 -07:00
readiness.py
…
response_filters.py
…
restart.py fix(gateway): defer in-band restart until active turns finish (#77184) 2026-08-03 09:57:58 +05:30
restart_loop_guard.py fix(gateway): make the restart-loop breaker see slow crash cycles (#81642) 2026-08-09 21:40:23 +05:30
rich_sent_store.py
…
run.py feat(kanban): explicit notify/wake delivery modes with faithful wake session routing 2026-08-13 10:47:40 -07:00
runtime_footer.py feat(gateway): add opt-in 'latency' runtime footer field 2026-08-03 17:16:57 +05:30
scale_to_zero.py fix(gateway): scale-to-zero gateway self-suspends via flaps socket instead of relying on Fly autostop (#84295) 2026-08-12 14:45:48 +10:00
session.py feat(gateway): allow plugins to inject session messages 2026-08-12 18:25:33 -07:00
session_context.py feat(kanban): explicit notify/wake delivery modes with faithful wake session routing 2026-08-13 10:47:40 -07:00
session_stall.py feat(gateway): session activity watchdog, stall notify, compress timeout (#72424) 2026-08-02 16:16:36 -07:00
session_state.py refactor(gateway): extract the hygiene recovery gate and forward the failure reason 2026-08-06 15:04:53 +05:30
shutdown_flush.py fix(gateway): spool cap-dropped pending transcript messages instead of discarding 2026-08-09 14:08:51 -07:00
shutdown_forensics.py
…
shutdown_watchdog.py fix(gateway): detect and report unclean shutdowns via lifecycle ledger (NS-608) 2026-07-28 22:41:09 -07:00
slash_access.py
…
slash_commands.py feat(kanban): explicit notify/wake delivery modes with faithful wake session routing 2026-08-13 10:47:40 -07:00
status.py fix(gateway): classify terminal adapter connect failures + escalate long-lived retry loops (OOF-156) 2026-08-12 22:16:12 -07:00
status_phrases.py
…
sticker_cache.py
…
stream_consumer.py fix(gateway,relay): stop frozen-preview finals and dropped idle-session delegation callbacks (#82592) 2026-08-11 09:56:13 +10:00
stream_dispatch.py
…
stream_events.py
…
streaming_tts_consumer.py feat(gateway): streaming TTS adapter contract and consumer (#60671) 2026-07-28 22:31:40 -07:00
systemd_notify.py
…
turn_context.py fix(gateway): reap only the background processes an abandoned turn created 2026-08-02 14:23:55 +05:30
turn_lease.py refactor(gateway): drop dead degraded token field; de-churn salvage diff 2026-08-07 18:23:57 +05:30
wake.py
…
whatsapp_identity.py
…