hermes-agent/tests/gateway
Teknium 31f70d1f2a
fix(ci): recover 38 failing tests on main (#17642)
CI Tests workflow has been red on main for 40+ consecutive runs. This
commit recovers every failure visible in run 25130722163 (most recent
completed run prior to this PR).

Root causes, by group:

Test-mock drift after product landed (fix: update mocks)
- test_mcp_structured_content / test_mcp_dynamic_discovery (6 tests):
  product added _rpc_lock (#02ae15222) and _schedule_tools_refresh
  (#1350d12b0) without updating sibling test files. Install a real
  asyncio.Lock inside the fake run-loop and patch at _schedule_tools_refresh.
- test_session.py: renamed normalize_whatsapp_identifier → canonical_
  whatsapp_identifier upstream; keep a local alias so the legacy tests
  keep working.
- test_run_progress_topics Slack DM test: PR #8006 made Slack default
  tool_progress=off; explicitly set it to 'all' in the test fixture so
  the progress-callback path still runs. Also read tool_progress_callback
  at call time rather than freezing it in FakeAgent.__init__ — production
  assigns it AFTER construction.
- test_tui_gateway_server session-create/close race: session.create now
  defers _start_agent_build behind a 50ms timer — wait for the build
  thread to enter _make_agent before closing, otherwise the orphan-
  cleanup path never runs.
- test_protocol session.resume: product get_messages_as_conversation now
  takes include_ancestors kwarg; accept **_kwargs in the test stub.
- test_copilot_acp_client redaction: redactor is OFF by default (snapshots
  HERMES_REDACT_SECRETS at import); patch agent.redact._REDACT_ENABLED=True
  for the duration of the test.
- test_minimax_provider: after #17171, dots in non-Anthropic model names
  stay dots even with preserve_dots=False. Assert the new invariant
  rather than the old 'broken for MiniMax' behavior.
- test_update_autostash: updater now scans `ps -A` for dashboard PIDs;
  the test's catch-all subprocess.run stub needed stdout/stderr fields.
- test_accretion_caps: read_timestamps dict is populated lazily when
  os.path.getmtime succeeds. Use .get("read_timestamps", {}) to tolerate
  CI filesystems where the stat races file creation.

Change-detector tests (fix: rewrite as structural invariants)
- test_credential_sources_registry_has_expected_steps: was a frozen set
  comparison that broke when minimax-oauth was added. Rewrite as an
  invariant check (every step has description, no dupes, core steps
  present) per AGENTS.md 'don't write change-detector tests'.

xdist ordering / test pollution (fix: reset state, use module-local patches)
- test_setup vercel: sibling test saved VERCEL_PROJECT_ID='project' to
  os.environ via save_env_value() and never cleared it. monkeypatch.delenv
  the VERCEL_* vars in the link-file test.
- test_clipboard TestIsWsl: GitHub Actions is on Azure VMs whose real
  /proc/version often contains 'microsoft'. Patching builtins.open with
  mock_open didn't reliably intercept hermes_constants.is_wsl's call in
  xdist workers that had already cached _wsl_detected=True from an
  earlier test. Patch hermes_constants.open directly and add
  teardown_method to reset the cache after each test.

Pytest-asyncio cancellation hangs (fix: bound product await with timeout)
- test_session_split_brain_11016 (3 params) + test_gateway_shutdown
  cancel-inflight: under pytest-asyncio 1.3.0, 'await task' and
  'asyncio.gather(cancelled_tasks)' can stall for 30s when the cancelled
  task's finally block awaits typing-task cleanup. Bound both with
  asyncio.wait_for(..., timeout=5.0) and asyncio.shield — the stragglers
  are released from adapter tracking and allowed to finish unwinding in
  the background. This is also a legitimate hardening: a wedged finally
  shouldn't stall the caller's dispatch or a gateway shutdown.

Orphan UI config (fix: merge tiny tab into messaging category)
- test_web_server test_no_single_field_categories: the telegram.reactions
  config field lived in its own 'telegram' schema category with no
  siblings. Fold it under 'discord' via _CATEGORY_MERGE so the dashboard
  doesn't render an orphan single-field tab.

Local verification: 38/38 originally-failing tests pass; 4044/4044
gateway tests pass; 684/684 targeted subset (all 16 touched test files)
passes.
2026-04-29 20:05:32 -07:00
..
__init__.py
…
conftest.py
…
restart_test_helpers.py
…
test_agent_cache.py feat(gateway): bust cached agent on compression/context_length config edits (#17008) 2026-04-28 06:37:42 -07:00
test_allowlist_startup_check.py
…
test_api_server.py fix(api-server): collapse tool start/lifecycle into a single SSE event 2026-04-29 08:08:16 -07:00
test_api_server_bind_guard.py
…
test_api_server_jobs.py
…
test_api_server_multimodal.py
…
test_api_server_normalize.py
…
test_api_server_runs.py feat(api_server): expose run status for external UIs (#17085) 2026-04-29 06:38:10 -07:00
test_api_server_toolset.py
…
test_approve_deny_commands.py
…
test_auth_fallback.py
…
test_auto_continue.py
…
test_background_command.py
…
test_background_process_notifications.py
…
test_base_topic_sessions.py
…
test_bluebubbles.py
…
test_busy_session_ack.py
…
test_cancel_background_drain.py
…
test_channel_directory.py
…
test_clean_shutdown_marker.py
…
test_command_bypass_active_session.py
…
test_complete_path_at_filter.py
…
test_compress_command.py fix(compression): notify users when configured aux model fails even if main-model fallback recovers (#16775) 2026-04-27 20:08:23 -07:00
test_compress_focus.py
…
test_compress_plugin_engine.py
…
test_config.py
…
test_config_cwd_bridge.py feat: add Vercel Sandbox backend 2026-04-29 07:22:33 -07:00
test_debug_command.py
…
test_delivery.py
…
test_dingtalk.py
…
test_discord_allowed_channels.py
…
test_discord_allowed_mentions.py
…
test_discord_attachment_download.py
…
test_discord_bot_auth_bypass.py
…
test_discord_bot_filter.py
…
test_discord_channel_controls.py
…
test_discord_channel_prompts.py
…
test_discord_channel_skills.py
…
test_discord_connect.py
…
test_discord_document_handling.py
…
test_discord_free_response.py
…
test_discord_imports.py
…
test_discord_media_metadata.py
…
test_discord_model_picker.py
…
test_discord_opus.py
…
test_discord_race_polish.py
…
test_discord_reactions.py
…
test_discord_reply_mode.py
…
test_discord_send.py
…
test_discord_slash_commands.py
…
test_discord_system_messages.py
…
test_discord_thread_persistence.py
…
test_display_config.py
…
test_dm_topics.py
…
test_document_cache.py
…
test_duplicate_reply_suppression.py
…
test_email.py fix(email): add required Date header to outbound mail 2026-04-27 06:41:11 -07:00
test_extract_local_files.py
…
test_fallback_eviction.py
…
test_fast_command.py fix(gateway): guard against None request_overrides in _build_api_kwargs 2026-04-28 06:57:23 -07:00
test_feishu.py
…
test_feishu_approval_buttons.py
…
test_feishu_comment.py
…
test_feishu_comment_rules.py
…
test_feishu_onboard.py
…
test_gateway_inactivity_timeout.py
…
test_gateway_shutdown.py fix(gateway,cron): close ephemeral agents + reap stale aux clients (salvage #13979) (#16598) 2026-04-27 07:41:42 -07:00
test_homeassistant.py
…
test_hooks.py
…
test_insights_unicode_flags.py
…
test_internal_event_bypass_pairing.py
…
test_interrupt_key_match.py
…
test_keep_typing_timeout.py fix(gateway): keep typing indicator alive across slow send_typing calls (#16763) 2026-04-27 19:09:32 -07:00
test_matrix.py fix: MatrixAdapter respects proxy configuration 2026-04-27 21:22:44 -07:00
test_matrix_exec_approval.py test(matrix): set user_id in approval-reaction test to bypass defensive self-drop 2026-04-27 21:22:44 -07:00
test_matrix_mention.py test(matrix): adapt outbound-mention notice test to current _send_simple_message API 2026-04-27 21:22:44 -07:00
test_matrix_voice.py
…
test_mattermost.py
…
test_media_download_retry.py
…
test_media_extraction.py
…
test_message_deduplicator.py
…
test_mirror.py
…
test_model_command_custom_providers.py
…
test_model_switch_persistence.py
…
test_pairing.py
…
test_pending_drain_race.py
…
test_pending_event_none.py
…
test_pii_redaction.py
…
test_platform_base.py fix: MatrixAdapter respects proxy configuration 2026-04-27 21:22:44 -07:00
test_platform_reconnect.py fix(gateway): isolate platform connect failures with per-platform timeout 2026-04-29 05:00:37 -07:00
test_pre_gateway_dispatch.py
…
test_proxy_mode.py
…
test_qqbot.py
…
test_queue_consumption.py
…
test_reasoning_command.py
…
test_reply_to_injection.py
…
test_restart_drain.py
…
test_restart_notification.py
…
test_restart_redelivery_dedup.py
…
test_restart_resume_pending.py fix(gateway): use transcript timestamp for auto-continue freshness 2026-04-28 05:20:35 -07:00
test_resume_command.py feat(memory): notify providers on mid-process session_id rotation (#17409) 2026-04-29 04:57:22 -07:00
test_retry_replacement.py
…
test_retry_response.py
…
test_run_progress_interrupt.py
…
test_run_progress_topics.py fix(ci): recover 38 failing tests on main (#17642) 2026-04-29 20:05:32 -07:00
test_runner_fatal_adapter.py
…
test_runner_startup_failures.py
…
test_running_agent_session_toggles.py
…
test_runtime_footer.py feat(gateway): opt-in runtime-metadata footer on final replies (#17026) 2026-04-28 06:50:04 -07:00
test_safe_adapter_disconnect.py
…
test_send_image_file.py
…
test_send_retry.py
…
test_session.py fix(ci): recover 38 failing tests on main (#17642) 2026-04-29 20:05:32 -07:00
test_session_boundary_hooks.py
…
test_session_boundary_security_state.py
…
test_session_dm_thread_seeding.py
…
test_session_env.py
…
test_session_hygiene.py feat(gateway): make hygiene hard message limit configurable (#17000) 2026-04-28 05:43:12 -07:00
test_session_info.py
…
test_session_list_allowed_sources.py
…
test_session_model_override_routing.py
…
test_session_model_reset.py
…
test_session_race_guard.py
…
test_session_reset_notify.py
…
test_session_split_brain_11016.py
…
test_session_state_cleanup.py
…
test_session_store_prune.py
…
test_setup_feishu.py
…
test_shared_group_sender_prefix.py
…
test_shutdown_cache_cleanup.py
…
test_shutdown_memory_provider_messages.py fix(gateway): pass session messages to shutdown_memory_provider (#15165) 2026-04-27 06:41:16 -07:00
test_signal.py feat(gateway/signal): native formatting, reply quotes, and reactions 2026-04-29 04:38:17 -07:00
test_signal_format.py feat(gateway/signal): native formatting, reply quotes, and reactions 2026-04-29 04:38:17 -07:00
test_slack.py
…
test_slack_approval_buttons.py
…
test_slack_channel_skills.py
…
test_slack_mention.py fix(gateway): keep env-token Slack enabled 2026-04-27 18:19:14 -07:00
test_sms.py
…
test_sse_agent_cancel.py
…
test_ssl_certs.py
…
test_status.py fix(gateway,cron): close ephemeral agents + reap stale aux clients (salvage #13979) (#16598) 2026-04-27 07:41:42 -07:00
test_status_command.py
…
test_steer_command.py
…
test_step_callback_compat.py
…
test_sticker_cache.py
…
test_stream_consumer.py fix(gateway): linearize tool-progress bubbles with content messages (#17280) 2026-04-28 22:17:33 -07:00
test_stream_consumer_fresh_final.py
…
test_stt_config.py
…
test_stuck_loop.py
…
test_telegram_approval_buttons.py
…
test_telegram_caption_merge.py
…
test_telegram_conflict.py
…
test_telegram_documents.py
…
test_telegram_format.py feat(telegram): render markdown tables as row groups 2026-04-28 05:37:50 -07:00
test_telegram_group_gating.py
…
test_telegram_mention_boundaries.py
…
test_telegram_network.py
…
test_telegram_network_reconnect.py fix(gateway): drain stale httpx polling connections on Telegram reconnect 2026-04-28 06:37:22 -07:00
test_telegram_photo_interrupts.py
…
test_telegram_reactions.py
…
test_telegram_reply_mode.py
…
test_telegram_text_batching.py
…
test_telegram_thread_fallback.py
…
test_telegram_webhook_secret.py
…
test_text_batching.py
…
test_title_command.py
…
test_transcript_offset.py
…
test_unauthorized_dm_behavior.py
…
test_unknown_command.py
…
test_update_command.py
…
test_update_streaming.py
…
test_usage_command.py
…
test_verbose_command.py
…
test_vision_memory_leak.py fix(memory): narrow scrub surface to known wrapper boundaries 2026-04-27 12:37:33 -07:00
test_voice_command.py
…
test_voice_mode_platform_isolation.py
…
test_weak_credential_guard.py
…
test_webhook_adapter.py
…
test_webhook_deliver_only.py
…
test_webhook_dynamic_routes.py
…
test_webhook_integration.py
…
test_webhook_signature_rate_limit.py
…
test_wecom.py
…
test_wecom_callback.py
…
test_weixin.py test(weixin): cover _is_stale_session_ret helper (#17228) 2026-04-29 05:44:44 -07:00
test_whatsapp_connect.py
…
test_whatsapp_formatting.py
…
test_whatsapp_group_gating.py
…
test_whatsapp_reply_prefix.py
…
test_ws_auth_retry.py
…
test_yolo_command.py
…