hermes-agent/hermes_cli
Ben Barclay 5cf6e28a2f
fix(gateway): auto-start after container restart via planned-stop marker (#42675) (#43236)
* fix(gateway): auto-start after container restart via planned-stop marker

On Docker (s6-overlay), the gateway runs as a dynamically-registered s6
service. When the container stops/restarts/upgrades, s6 sends the gateway
a plain SIGTERM. The shutdown path (_stop_impl) ended with an
unconditional _update_runtime_status("stopped"), persisting
gateway_state=stopped to the volume. container_boot.py reads that on the
next boot and only auto-starts gateways whose last state was "running"
(_AUTOSTART_STATES) — so after a routine `docker compose up
--force-recreate` the gateway stays down and messaging channels silently
go dark, with no error surfaced (issue #42675).

The codebase already distinguishes intentional stops from unexpected
signals via the planned-stop marker (write_planned_stop_marker /
consume_planned_stop_marker_for_self): `hermes gateway stop`,
systemd/launchd ExecStop, and Ctrl+C write a marker before signalling,
so the handler classifies them as planned. An unmarked SIGTERM
(container/s6 restart, OOM, bare kill) is signal-initiated.

This wires that existing classification through to the state persist,
rather than adding unreliable signal-source inference:

- run.py: GatewayRunner._signal_initiated_shutdown, set in
  shutdown_signal_handler's unmarked-signal branch. In _stop_impl, a
  signal-initiated (non-restart) teardown now persists "running" instead
  of "stopped" — preserving the operator's run-intent and overwriting the
  mid-shutdown "draining" marker so _AUTOSTART_STATES matches on reboot.
  Operator stops and restarts persist "stopped" as before.

- service_manager.py: S6ServiceManager.stop() now writes the planned-stop
  marker for the supervised PID (read from s6-svstat) before `s6-svc -d`,
  so an in-container `hermes gateway stop` is correctly classified as
  intentional (parity with the systemd/launchd/host stop paths, which
  already mark). Best-effort: a marker-write failure falls back to the
  safe signal-initiated path.

Tests: shutdown persist-decision table (signal→running, operator→stopped,
restart→stopped), s6 stop marker write + svstat PID parse + failure
tolerance. The signal→running and s6-marker tests fail without the
respective source change. Verified end-to-end against a container built
from this branch: an unmarked SIGTERM to the live gateway leaves
gateway_state=running (shutdown-context log confirms signal path);
existing real container-restart suite still green.

* docs(docker): clarify gateway autostart distinguishes operator-stop from container-kill

The per-profile-supervision section described the autostart-across-restart
contract as "running gateways come back, stopped stay stopped" without
spelling out what records 'stopped'. That contract was the source of
#42675 confusion: users expected a restart to bring the gateway back and
it didn't. With the write-side fix, only an explicit `hermes gateway stop`
records 'stopped'; container/s6 restart SIGTERMs (incl. image upgrades and
unexpected exits) leave the state 'running' so the gateway auto-starts.
Make that distinction explicit in both the multi-profile and
per-profile-supervision sections.

* test(docker): real-restart autostart E2E for #42675

Adds test_live_gateway_autostarts_after_real_restart_without_manual_state_stamp:
a live s6-supervised gateway is killed by an actual `docker restart`
SIGTERM (no manual gateway_state stamp, no planned-stop marker) and must
auto-start on the next boot. Exercises the WRITE side of the fix that the
existing stamp-based tests bypass.

Verified to FAIL against an origin/main image (reconciler logs
prior_state=stopped action=registered — the #42675 bug) and PASS against
the fixed image (prior_state=running action=started).
2026-06-10 14:01:34 +10:00
..
dashboard_auth fix(dashboard-auth): warn when public_url override is silently rejected (#43214) 2026-06-10 12:14:57 +10:00
proxy
…
subcommands refactor(cli): promote 9 closure handlers to top-level + extract their parsers (god-file Phase 2 follow-up) 2026-06-07 22:56:23 -07:00
__init__.py
…
_parser.py
…
_subprocess_compat.py
…
active_sessions.py feat(sessions): add optional max session cap 2026-06-08 15:12:12 -07:00
auth.py fix(auth): add Codex OAuth accounts as distinct pool entries 2026-06-08 11:57:03 -07:00
auth_commands.py fix(auth): add Codex OAuth accounts as distinct pool entries 2026-06-08 11:57:03 -07:00
azure_detect.py
…
backup.py
…
banner.py
…
browser_connect.py
…
build_info.py
…
bundles.py
…
callbacks.py
…
checkpoints.py
…
claw.py
…
cli_agent_setup_mixin.py fix(cli): gate tool-rendering paths with tool_progress_mode, not quiet_mode 2026-06-08 11:29:53 -07:00
cli_commands_mixin.py refactor(cli): extract 32 slash-command handlers into CLICommandsMixin (god-file Phase 4) 2026-06-08 02:13:07 -07:00
cli_output.py
…
clipboard.py
…
codex_models.py
…
codex_runtime_plugin_migration.py
…
codex_runtime_switch.py
…
colors.py
…
commands.py
…
completion.py
…
config.py fix(tui): apply terminal backend config before launch 2026-06-09 00:31:27 -07:00
container_boot.py
…
copilot_auth.py
…
cron.py
…
curator.py
…
curses_ui.py
…
dashboard_register.py fix(cli): persist custom --portal-url to .env on dashboard register (#42435) 2026-06-09 13:56:33 +10:00
debug.py
…
default_soul.py
…
dep_ensure.py
…
dingtalk_auth.py
…
doctor.py fix(state.db): recover from malformed sqlite_master so hidden sessions reappear (#43149) 2026-06-09 18:49:08 -05:00
dump.py fix(auth): auto-detect OpenRouter credential from the pool, not just env (#42263) 2026-06-08 10:01:47 -07:00
env_loader.py
…
fallback_cmd.py
…
fallback_config.py
…
gateway.py fix(gateway): drop --replace from systemd unit templates (#41892) 2026-06-08 00:20:08 -07:00
gateway_windows.py
…
goals.py
…
gui_uninstall.py
…
hooks.py
…
inventory.py
…
kanban.py
…
kanban_db.py
…
kanban_decompose.py
…
kanban_diagnostics.py
…
kanban_specify.py
…
kanban_swarm.py
…
logs.py
…
main.py fix(state.db): recover from malformed sqlite_master so hidden sessions reappear (#43149) 2026-06-09 18:49:08 -05:00
managed_uv.py
…
mcp_catalog.py
…
mcp_config.py
…
mcp_picker.py
…
mcp_startup.py
…
memory_setup.py
…
middleware.py
…
migrate.py
…
model_catalog.py feat(models): seed model-catalog disk cache from checkout on update (#42614) 2026-06-08 22:31:06 -07:00
model_normalize.py
…
model_setup_flows.py refactor(cli): extract 18 model-flow wizard functions into model_setup_flows (god-file Phase 2) 2026-06-08 09:42:44 -07:00
model_switch.py
…
models.py fix(models): keep curated Anthropic aliases in /model picker (#43103) 2026-06-09 14:45:19 -07:00
nous_account.py
…
nous_subscription.py
…
oneshot.py
…
pairing.py
…
partial_compress.py
…
platforms.py
…
plugins.py
…
plugins_cmd.py feat(plugins): install from a subdirectory within a repo (#42963) 2026-06-09 13:42:51 -04:00
portal_cli.py
…
profile_describer.py
…
profile_distribution.py fix(dist): stop USER_OWNED_EXCLUDE from filtering nested directories 2026-06-07 21:50:57 -07:00
profiles.py fix(desktop): release profile backends before delete (#42613) 2026-06-09 10:52:02 -05:00
prompt_size.py
…
providers.py fix(doctor): allow vendor slugs for named custom providers 2026-06-08 15:53:09 -07:00
psutil_android.py
…
pt_input_extras.py
…
pty_bridge.py fix(pty-bridge): mark os.killpg/getpgid windows-footgun-ok (POSIX-only module) 2026-06-08 07:03:12 -07:00
relaunch.py
…
runtime_provider.py
…
secret_prompt.py
…
secrets_cli.py
…
security_advisories.py
…
security_audit.py
…
send_cmd.py
…
service_manager.py fix(gateway): auto-start after container restart via planned-stop marker (#42675) (#43236) 2026-06-10 14:01:34 +10:00
session_recap.py
…
setup.py
…
skills_config.py
…
skills_hub.py fix(skills): browse shows full catalog, not first 5000 (#41413) 2026-06-07 10:15:31 -07:00
skin_engine.py
…
slack_cli.py
…
status.py
…
stdio.py
…
telegram_managed_bot.py
…
timeouts.py
…
tips.py
…
tools_config.py
…
uninstall.py
…
voice.py
…
web_server.py fix(tui): apply terminal backend config before launch 2026-06-09 00:31:27 -07:00
webhook.py
…
win_pty_bridge.py feat(windows): enable dashboard /chat tab via ConPTY (win_pty_bridge) + tests (#42251) 2026-06-08 11:32:43 -07:00
xai_retirement.py
…