hermes-agent/tests/gateway
kshitij aaf9688519 refactor(gateway): extract the hygiene recovery gate and forward the failure reason
Follow-up to c0d974b19 (#79741). Three review findings against that commit,
none of which change the escalation behaviour it shipped.

1. The recovery decision lived inline in `_handle_message_with_agent`, a
   ~2000-line async method, so the only way to pin it was a test that read
   `inspect.getsource(...)` and asserted on substrings. AGENTS.md bans reading
   source in tests outright and names this exact situation: "if the logic lives
   inline in a god-file (gateway/run.py) and extracting it feels disruptive:
   that's the actual signal to do the extraction, not to regex around it."

   Those tests were not merely stylistically wrong, they were actively harmful.
   One asserted the substring `_new_tokens < _approx_tokens` was PRESENT -- so
   it passed while the gate had the bug that substring represents, and had to be
   edited when the gate was fixed. It failed on correct code and passed on
   broken code, in one assertion.

   Extracted `hygiene_compaction_recovered()` as a module-level pure predicate
   and replaced the three source-reading tests with eight direct unit tests.
   The extraction immediately earned itself: the new tests caught a `NameError`
   (the predicate called `compression_made_progress` while the module bound it
   under an alias) that a source-text assertion cannot see, because the symbol
   is spelled correctly in the source and only fails at runtime.

2. The gate inferred "did the transcript actually get rewritten" from a numeric
   side effect -- the degenerate "did not rotate or compact in place" path
   (#21301) reuses the pre-compression counts -- when the booleans
   `_hyg_rotated` / `_hyg_in_place` were already in scope and explicitly set
   False on that path. The predicate now takes them directly, so a future edit
   that re-estimates instead of reusing the old counts cannot silently defeat
   the escalation.

3. `_record_hygiene_cooldown` passed no `error` to
   `record_compression_failure_cooldown`, which writes `compression_failure_error`
   unconditionally -- so a hygiene failure clobbered to NULL whatever reason the
   in-conversation path had recorded, and readers then show the user "unknown
   error" (agent/manual_compression_feedback.py, gateway/slash_commands.py). The
   reason was already in hand at both call sites. Pre-existing from #74136 but
   amplified by escalation: a blank reason on a 45-minute cooldown is far more
   user-visible than on a 5-minute one.

Also: the ladder docstring described the compressor's absolute 60/300/900s
ladder while the constant is multipliers (1, 3, 9); the config docs still
described `hygiene_failure_cooldown_seconds` as a flat interval rather than the
first rung of a capped ladder; and `PersistentState.hygiene_failure_streak` now
documents that it is process-local by design -- keying on `session_key` is what
survives compaction rotation, which the persisted `compression_*_streak`
columns cannot express since they key on the rotating `session_id`. Making it
durable is a schema change, tracked on #79624 rather than smuggled in here.

Also replaces the file's hand-written `_Runner` stub with
`object.__new__(GatewayRunner)` (already the idiom elsewhere in the same file).
The stub reimplemented `_session_state` and `_peek_session_state`, so the tests
exercised copies that could drift from production; using the real class
immediately made one assertion stronger -- on a fresh runner `_sessions` does not
exist at all until something materialises it, so the reset provably did not even
create the map.

A review pass on this follow-up then caught that the CALL SITE was still
unbound: deleting the whole `if not _hyg_aborted: if
hygiene_compaction_recovered(...)` block left every ladder test green, because
the unit tests prove the predicate correct without proving it is wired in. The
merged commit had the same gap and its only cover was the banned source-reading
test. `test_session_hygiene_forces_in_place_compaction_with_bound_session_db`
now spies the reset on a genuine in-place compaction, so deleting the wiring
fails. Two earlier attempts at this test did NOT close the gap -- asserting on
streak VALUES passes either way, since the streak is 0 whether or not the gate
ran; only a positive spy assertion on a recovering run detects the deletion.

Same pass also corrected an overstatement: point (2) is hardening, not a live
bug. The degenerate path also sets `_new_count = _msg_count` and `_new_tokens =
_approx_tokens`, and `compression_made_progress(n, n, t, t)` is always False, so
the merged code already declined to reset there. A 200k-trial fuzz over the
reachable state space found zero behavioural disagreements between the merged
gate and this one. The guard's value is surviving a future edit that stops
reusing those counts.

Tests: 28 in tests/gateway/test_hygiene_failure_cooldown_ladder.py (8 new unit
tests for the predicate, 3 for reason forwarding, 3 source-reading tests
deleted). All 5 mutations caught -- including one that restores the hand-rolled
comparison and one that removes the rotated/in_place guard. Two mutations
initially SURVIVED and exposed vacuous tests of my own: the no-rewrite test used
counts the progress predicate already rejects, so it passed without binding the
guard at all; it now passes counts that read as progress on their own, proving
the guard is what rejects them. gateway hygiene + session-state + agent
compression-progress suites: 54 passed; ruff clean.

Refs #79624
2026-08-06 15:04:53 +05:30
..
platforms
relay
__init__.py
_plugin_adapter_loader.py
conftest.py
feishu_helpers.py
restart_test_helpers.py
test_7100_transient_failure_transcript.py
test_10710_auto_reset_evicts_cached_agent.py
test_13121_shutdown_inflight_transcript_flush.py
test_25107_stale_base_url_api_mode.py
test_35809_auto_reset_clean_context.py
test_35994_reset_button_deadlock.py
test_42039_duplicate_user_message.py
test_48031_model_switch_after_auto_reset.py
test_53175_cleanup_off_loop.py
test_64674_multiplex_primary_token_scope.py
test_71671_faulthandler_no_stderr.py
test_73297_memory_flush_on_reset.py
test_73771_media_resend_dedup.py
test_75349_whatsapp_multiplex_secret_scope.py
test_abandoned_turn_process_cleanup.py
test_active_session_text_merge.py
test_adapter_connect_is_reconnect_contract.py
test_adapter_startup_secret_scope.py
test_agent_cache.py
test_agents_command_delegations.py
test_aiohttp_body_caps.py
test_allowed_channels_widening.py
test_allowlist_startup_check.py
test_api_server.py
test_api_server_active_work_drain.py
test_api_server_bind_guard.py
test_api_server_jobs.py
test_api_server_media_data_urls.py
test_api_server_multimodal.py
test_api_server_multiplex_secret_scope.py
test_api_server_normalize.py
test_api_server_runs.py
test_api_server_toolset.py
test_approval_prompt_redaction.py
test_approvals_command.py
test_approve_deny_commands.py
test_async_delegation_session_binding.py
test_async_delivery_capability.py
test_async_session_db.py
test_async_session_store.py
test_audio_cache.py
test_auth_fallback.py
test_auto_continue.py
test_auto_voice_reply_format.py
test_background_command.py
test_background_process_notifications.py
test_base_auto_tts_output_format.py
test_base_topic_sessions.py
test_bluebubbles.py
test_bounded_adapter_teardown.py
test_bundles_command.py
test_busy_session_ack.py
test_busy_session_auth_bypass.py
test_buzz_adapter.py
test_buzz_websocket.py
test_cached_agent_max_iterations.py
test_cancel_background_drain.py
test_cgroup_cleanup.py
test_channel_continuity_hint.py
test_channel_directory.py
test_channel_directory_connected_only.py
test_channel_overrides.py
test_checkpoint_config.py
test_choice_picker.py
test_cjk_fts_config_bridge.py
test_clarify_active_session_bypass.py
test_clarify_progress_leak.py
test_clarify_thread_followup_not_swallowed.py
test_clean_shutdown_marker.py
test_code_fence_tracking.py
test_command_bypass_active_session.py
test_complete_path_at_filter.py
test_completion_delivery.py
test_compress_command.py
test_compress_focus.py
test_compress_plugin_engine.py
test_compress_preview.py
test_compression_concurrent_sessions.py
test_compression_deferred_soft_result.py
test_compression_failure_session_sync.py
test_compression_in_flight_check.py
test_compression_interrupt_demotion_56391.py
test_compression_progress_notices.py
test_compression_session_id_persistence.py
test_config.py
test_config_cwd_bridge.py
test_config_driven_access_policy.py
test_config_env_bridge_authority.py
test_context_ref_expansion_runtime.py
test_conversation_scope_funnel.py
test_cron_active_work_drain.py
test_cron_fire_webhook.py
test_cron_shutdown_drain.py
test_cwd_placeholder.py
test_dead_targets.py
test_debug_command.py
test_dedupe_user_turns.py
test_delegation_session_id_leak.py
test_delivery.py
test_delivery_ledger.py
test_delivery_ledger_fd_leak.py
test_delivery_ledger_producer.py
test_delivery_silence_filter.py
test_destructive_slash_always_persist_report.py
test_destructive_slash_confirm.py
test_diff_command.py
test_dingtalk.py
test_discord_allowed_channels.py
test_discord_allowed_mentions.py
test_discord_approval_mentions.py
test_discord_attachment_download.py
test_discord_bot_auth_bypass.py
test_discord_bot_filter.py
test_discord_channel_controls.py
test_discord_channel_prompts.py
test_discord_channel_skills.py
test_discord_clarify_buttons.py
test_discord_component_auth.py
test_discord_connect.py
test_discord_document_handling.py
test_discord_double_dispatch.py
test_discord_edit_message_overflow.py
test_discord_exec_approval_content.py
test_discord_fail_closed_feedback.py
test_discord_format.py
test_discord_free_response.py
test_discord_imports.py
test_discord_lazy_install_views.py
test_discord_liveness.py
test_discord_media_metadata.py
test_discord_missed_message_backfill.py
test_discord_model_picker.py
test_discord_opus.py
test_discord_pending_text_batch_shutdown.py
test_discord_plugin_setup.py
test_discord_prompt_content_siblings.py
test_discord_prompt_timeout_config.py
test_discord_race_polish.py
test_discord_reactions.py
test_discord_reply_mode.py
test_discord_roles_dm_scope.py
test_discord_send.py
test_discord_slash_auth.py
test_discord_slash_commands.py
test_discord_sync_limit.py
test_discord_system_messages.py
test_discord_thread_persistence.py
test_discord_thread_slash_expired_defer.py
test_discord_voice_mixer.py
test_display_config.py
test_dm_topics.py
test_document_cache.py
test_document_context_note.py
test_duplicate_reply_suppression.py
test_email.py
test_email_robustness.py
test_email_secret_scope.py
test_empty_model_recovery.py
test_env_flag_truthy.py
test_ephemeral_reply.py
test_escape_reasoning_fences.py
test_external_drain_control.py
test_extract_local_files.py
test_fallback_chain_reload.py
test_fallback_eviction.py
test_fast_command.py
test_feishu.py
test_feishu_approval_buttons.py
test_feishu_bot_admission.py
test_feishu_bot_auth_bypass.py
test_feishu_channel_prompts.py
test_feishu_comment.py
test_feishu_comment_rules.py
test_feishu_lazy_import.py
test_feishu_meeting_invite.py
test_feishu_onboard.py
test_feishu_sdk_executor.py
test_feishu_table_markdown.py
test_feishu_voice_message_type.py
test_fence_chunker.py
test_first_turn_session_meta_rebaseline.py
test_footer_command_mid_run.py
test_fresh_reset_skill_injection.py
test_gateway_command_dispatch_minimal.py
test_gateway_command_help.py
test_gateway_command_line_matcher.py
test_gateway_inactivity_timeout.py
test_gateway_process_exit.py
test_gateway_shutdown.py
test_gateway_silence_tokens.py
test_gateway_utf8_encoding.py
test_goal_continuation_drain.py
test_goal_max_turns_config.py
test_goal_status_notice.py
test_goal_verdict_send.py
test_google_chat.py
test_google_chat_oauth_dependencies.py
test_handoff_thread_session_key.py
test_handoff_watcher_async_db.py
test_history_media_current_turn.py
test_home_target_env_var.py
test_homeassistant.py
test_hooks.py
test_hygiene_failure_cooldown_ladder.py
test_image_input_routing_runtime.py
test_incomplete_gateway_turns.py
test_insights_unicode_flags.py
test_interactive_prompt_base.py
test_internal_event_bypass_pairing.py
test_internal_event_never_interrupts_busy_session.py
test_interrupt_key_match.py
test_irc_adapter.py
test_kanban_auto_decompose_live.py
test_kanban_notifier.py
test_kanban_notifier_apiserver_wake.py
test_kanban_notifier_watcher_dispatch_gate.py
test_kanban_notifier_zero_sub_gate.py
test_kanban_watchers_mixin.py
test_keep_typing_timeout.py
test_lifecycle_ledger.py
test_line_plugin.py
test_load_transcript_db_only.py
test_loop_exception_handler.py
test_loop_liveness_watchdog.py
test_matrix.py
test_matrix_approval_reaction_fail_closed.py
test_matrix_dm_invite_recording.py
test_matrix_exec_approval.py
test_matrix_mention.py
test_matrix_message_length.py
test_matrix_plugin_setup.py
test_matrix_project_context_isolation.py
test_matrix_recovery_key_scope.py
test_matrix_voice.py
test_mattermost.py
test_mattermost_plugin_setup.py
test_max_concurrent_sessions.py
test_max_tokens_propagation.py
test_mcp_reload_refreshes_cached_agents.py
test_media_cache.py
test_media_download_retry.py
test_media_extraction.py
test_media_metadata_contract.py
test_media_spaced_paths_and_history_dedupe.py
test_media_tag_cleanup.py
test_media_tag_formatting_variants.py
test_media_tag_separator.py
test_memory_monitor.py
test_memory_trim_housekeeping.py
test_message_deduplicator.py
test_message_timestamps.py
test_mirror.py
test_mixed_attachment_routing.py
test_moa_one_shot_restore.py
test_model_command_async_offload.py
test_model_command_context_offload.py
test_model_command_custom_providers.py
test_model_command_expensive_confirm.py
test_model_command_flat_string_config.py
test_model_picker_persist.py
test_model_switch_persistence.py
test_msgraph_webhook.py
test_multiplex_adapter_registry.py
test_multiplex_api_server_routing.py
test_multiplex_background_task_scope.py
test_multiplex_credential_isolation.py
test_multiplex_http_routing.py
test_multiplex_lifecycle.py
test_multiplex_pairing_stores.py
test_multiplex_phase0.py
test_multiplex_profile_authz.py
test_native_image_buffer_isolation.py
test_new_clears_last_resolved_model.py
test_notice_delivery.py
test_notice_rendering.py
test_ntfy_plugin.py
test_own_policy_startup_gate.py
test_pairing.py
test_pairing_allowlist_bypass.py
test_pending_drain_no_recursion.py
test_pending_drain_race.py
test_pending_event_none.py
test_per_platform_streaming_defaults.py
test_pii_redaction.py
test_plaintext_approval_routing.py
test_planned_stop_watcher.py
test_platform_base.py
test_platform_connected_checkers.py
test_platform_http_client_limits.py
test_platform_reconnect.py
test_platform_reconnect_fd_leak.py
test_platform_registry.py
test_plugin_platform_interface.py
test_post_delivery_callback_chaining.py
test_post_stream_media_delivery.py
test_pre_gateway_dispatch.py
test_priority_path_compression_demotion_56391.py
test_profile_resolution.py
test_profile_routing.py
test_prompt_tail_freeze.py
test_proxy_mode.py
test_qqbot.py
test_qqbot_credential_isolation.py
test_qqbot_scope_paths.py
test_queue_command.py
test_queue_consumption.py
test_queued_native_image_session_key.py
test_raft_adapter.py
test_readiness.py
test_reasoning_command.py
test_reasoning_config_per_model.py
test_relay_capability_surface.py
test_relay_upstream_authz.py
test_reload_skills_command.py
test_reload_skills_discord_resync.py
test_replace_child_reap.py
test_replay_entry_fields.py
test_reply_to_injection.py
test_response_filters.py
test_restart_after_turn.py
test_restart_drain.py
test_restart_notification.py
test_restart_redelivery_dedup.py
test_restart_resume_pending.py
test_restart_service_detection.py
test_resume_command.py
test_retry_replacement.py
test_retry_response.py
test_routing_save_fast_path.py
test_run_cleanup_progress.py
test_run_progress_interrupt.py
test_run_progress_topics.py
test_run_tool_media_re.py
test_runner_fatal_adapter.py
test_runner_startup_failures.py
test_running_agent_session_toggles.py
test_runtime_config_env_expansion.py
test_runtime_env_reload_config_authority.py
test_runtime_footer.py
test_safe_adapter_disconnect.py
test_scale_to_zero.py
test_scale_to_zero_watcher.py
test_send_error_classification.py
test_send_image_file.py
test_send_multiple_images.py
test_send_retry.py
test_send_voice_reply_notify.py
test_session.py
test_session_api.py
test_session_boundary_hooks.py
test_session_boundary_security_state.py
test_session_context_inheritance.py
test_session_dm_thread_seeding.py
test_session_env.py
test_session_hygiene.py
test_session_id_cache_coherence.py
test_session_info.py
test_session_list_allowed_sources.py
test_session_load_bool.py
test_session_messages_shutdown_preserve.py
test_session_model_override_credential_pool.py
test_session_model_override_persistence.py
test_session_model_override_routing.py
test_session_model_reset.py
test_session_override_thread_recovery.py
test_session_race_guard.py
test_session_reset_notify.py
test_session_split_brain_11016.py
test_session_stall_watchdog.py
test_session_state_cleanup.py
test_session_store_expiry_finalized.py
test_session_store_lock_io.py
test_session_store_prune.py
test_session_store_runtime_stale_guard.py
test_session_store_stale_prune.py
test_setup_feishu.py
test_shared_group_sender_prefix.py
test_shutdown_cache_cleanup.py
test_shutdown_flush.py
test_shutdown_forensics.py
test_shutdown_memory_provider_messages.py
test_shutdown_watchdog.py
test_signal.py
test_signal_format.py
test_signal_rate_limit.py
test_simplex_plugin.py
test_skip_context_files_wiring.py
test_slack.py
test_slack_approval_buttons.py
test_slack_block_kit.py
test_slack_block_kit_adapter.py
test_slack_bot_auth_bypass.py
test_slack_channel_session_scope.py
test_slack_channel_skills.py
test_slack_clarify_buttons.py
test_slack_cron_continuable_surface.py
test_slack_dedup_ttl.py
test_slack_download_ssrf.py
test_slack_group_dm_scope_warning.py
test_slack_ignore_other_user_mentions.py
test_slack_log_noise.py
test_slack_mention.py
test_slack_mention_humanization.py
test_slack_peer_agent_smoke.py
test_slack_plugin_action_handlers.py
test_slack_plugin_setup.py
test_slack_relay_parent_command.py
test_slack_require_mention_channels.py
test_slack_runner_ignored_channels.py
test_slack_send_retry.py
test_slack_socket_reconnect_heal.py
test_slack_status_update.py
test_slack_user_token_warning.py
test_slack_wake_external_bot_messages.py
test_slash_access.py
test_slash_access_dispatch.py
test_sms.py
test_sse_agent_cancel.py
test_sse_frame.py
test_ssl_cert_detection.py
test_ssl_certs.py
test_stacked_skill_platform_disabled.py
test_stale_confirmation_expiry.py
test_stale_finalize_suppression.py
test_stale_platform_lock_retryable.py
test_stale_self_heal_agent_cache_eviction.py
test_startup_no_eager_platform_install.py
test_startup_restart_race.py
test_status.py
test_status_command.py
test_status_phrases.py
test_steer_command.py
test_steer_fifo_overwrite.py
test_step_callback_compat.py
test_sticker_cache.py
test_stop_thread_sibling.py
test_stream_consumer.py
test_stream_consumer_draft.py
test_stream_consumer_fresh_final.py
test_stream_consumer_silence.py
test_stream_consumer_thread_routing.py
test_stream_events.py
test_streaming_tts_consumer.py
test_streaming_tts_gateway_regression.py
test_stt_config.py
test_stt_transcript_echo_config.py
test_stuck_loop.py
test_subagent_protection_30170.py
test_systemd_notify.py
test_systemd_watchdog_lifecycle.py
test_table_helpers.py
test_teams.py
test_teams_dotenv_isolation.py
test_teams_pipeline_runtime_wiring.py
test_telegram_approval_buttons.py
test_telegram_audio_vs_voice.py
test_telegram_auth_check.py
test_telegram_bot_auth_bypass.py
test_telegram_callback_auth_fail_closed.py
test_telegram_caption_merge.py
test_telegram_channel_posts.py
test_telegram_clarify_buttons.py
test_telegram_closewait_limits_31599.py
test_telegram_conflict.py
test_telegram_connect.py
test_telegram_documents.py
test_telegram_error_redaction.py
test_telegram_fallback_pool_release_71593.py
test_telegram_final_delivery.py
test_telegram_format.py
test_telegram_forum_commands.py
test_telegram_group_gating.py
test_telegram_init_deadline.py
test_telegram_long_command_batching.py
test_telegram_max_doc_bytes.py
test_telegram_media_read_timeout.py
test_telegram_mention_boundaries.py
test_telegram_model_picker.py
test_telegram_network.py
test_telegram_network_reconnect.py
test_telegram_noise_filter.py
test_telegram_overflow_partial.py
test_telegram_pending_update_probe.py
test_telegram_photo_interrupts.py
test_telegram_polling_progress.py
test_telegram_progress_edit_transient.py
test_telegram_prune_stale_topic_binding_31501.py
test_telegram_reactions.py
test_telegram_reply_mode.py
test_telegram_reply_quote.py
test_telegram_rich_messages.py
test_telegram_rich_newlines.py
test_telegram_send_draft_format.py
test_telegram_send_path_health.py
test_telegram_slash_confirm.py
test_telegram_start_polling_timeout.py
test_telegram_status_indicator.py
test_telegram_status_update.py
test_telegram_text_batch_perf.py
test_telegram_text_batching.py
test_telegram_thread_fallback.py
test_telegram_topic_mode.py
test_telegram_typing_backoff.py
test_telegram_username_chat_id.py
test_telegram_voice_caption_markdown.py
test_telegram_voice_duration.py
test_telegram_voice_v0_regressions.py
test_telegram_webhook_secret.py
test_text_batching.py
test_title_command.py
test_tool_log_mode.py
test_tool_response_drop_recovery.py
test_transcript_offset.py
test_tts_media_routing.py
test_tui_approval_redaction.py
test_turn_context.py
test_turn_lease.py
test_typing_indicator_toggle.py
test_unauthorized_dm_behavior.py
test_unavailable_skill_hint.py
test_undo_rewind_session.py
test_unknown_command.py
test_update_command.py
test_update_cron_drain.py
test_update_streaming.py
test_usage_command.py
test_verbose_command.py
test_version_command.py
test_video_context_note.py
test_vision_memory_leak.py
test_voice_command.py
test_voice_mode_platform_isolation.py
test_wake_delivery.py
test_watchdog_review_76354.py
test_weak_credential_guard.py
test_webhook_adapter.py
test_webhook_deliver_only.py
test_webhook_dynamic_routes.py
test_webhook_integration.py
test_webhook_session_close.py
test_webhook_signature_rate_limit.py
test_wecom.py
test_wecom_callback.py
test_wecom_plugin_setup.py
test_weixin.py
test_weixin_secret_scope.py
test_weixin_typing.py
test_whatsapp_allowlist_lid_resolution.py
test_whatsapp_bridge_dir_resolution.py
test_whatsapp_bridge_pidfile.py
test_whatsapp_cloud.py
test_whatsapp_cloud_allowed_users.py
test_whatsapp_connect.py
test_whatsapp_formatting.py
test_whatsapp_from_owner.py
test_whatsapp_group_gating.py
test_whatsapp_identity.py
test_whatsapp_media_path_profile.py
test_whatsapp_native_delivery.py
test_whatsapp_plugin_setup.py
test_whatsapp_reply_prefix.py
test_whatsapp_stale_bridge.py
test_whatsapp_text_batching.py
test_whatsapp_to_jid.py
test_ws_auth_retry.py
test_yolo_command.py
test_yuanbao_forwarded_heartbeat.py
test_yuanbao_media_ssrf.py