- Drain the eviction plan (pop + del) before trim_memory: the batch
thread previously held every evicted agent in its local list while
gc.collect + malloc_trim ran, so the in-pass trim freed almost
nothing, the next tick re-read a still-high RSS, and the valve
over-evicted an extra batch of warm prompt caches per cycle.
- Clear _db_flush_scan_prefix in _release_evicted_agent_soft: it is a
shallow copy of the flushed transcript (stamped on every successful
flush) sharing every message dict — and pressure-evictable agents
have flushed by definition, so it pinned the multi-MB content strings
on exactly the agents the valve targets.
- Config-read failure now falls back to resolve_agent_cache_bounds({})
instead of bare AgentCacheBounds(): the dataclass default disables
the pressure pass, but an absent config section means 'auto' — a
transient read failure must not permanently switch off the OOM valve.
- protect_recent: false (YAML bool; False == 0) keeps the default MRU
protection instead of silently disabling it.
- 'No evictable session' warning now distinguishes sessions blocked on
un-flushed persistence (e.g. session DB never initialized — NFS
HERMES_HOME) from mid-turn agents, so operators can diagnose why the
valve isn't shedding instead of being pointed at running turns.
- _cgroup_limit_bytes checks the process's own cgroup (via the existing
gateway.cgroup_cleanup._own_cgroup_path) before the root files, so a
systemd unit's MemoryHigh=/MemoryMax= is detected — the root
memory.high/max read 'max' on those deployments.
- Tests: 5 new guards; drain-before-trim and scan-prefix-clear
mutation-checked (revert each fix -> its guard fails).
The per-session agent cache is capped at 128 entries with a 1h idle TTL, and
neither bound knows how many bytes it holds. Each cached agent pins
_session_messages -- the full transcript including tool output, tens of MB on
a session with 100+ tool calls -- so a gateway serving many chats keeps every
warm transcript resident: agents that took a turn inside the TTL are never
idle-swept, and the idle sweep additionally defers finalizable sessions until
they expire. RSS climbs until the cgroup throttles and SIGTERM can no longer
flush inside systemd's stop timeout.
Add the missing bound. Each session-expiry watcher tick compares the process's
anonymous RSS against a budget and, when over, sheds LRU agents through the
same soft-eviction path the cap enforcer uses, then runs malloc_trim so the
freed arenas actually return to the OS. Evicted sessions rebuild their
transcript from the persisted session on the next turn.
Three classes of session are never shed: agents mid-turn, the most recently
used ones, and any session whose transcript has not finished reaching disk
(_last_flushed_db_idx vs len(_session_messages) -- the same divergence the FTS
write-corruption guard reacts to when it preserves live history).
memory_high_mb defaults to "auto", deriving the budget from the cgroup limit
the gateway runs under, so a MemoryHigh/MemoryMax on the unit is respected
without a second number to keep in sync. The two existing bounds become
configurable alongside it under agent.agent_cache.
protect_recent is clamped to half the cache: a couple of sessions can exhaust
the budget on their own, and a fixed MRU guard would then protect everything
and leave the gateway climbing with nothing it would shed.
Fixes#80764
kshitij
HexLab98