hermes-agent/tests/hermes_cli
Teknium 31f70d1f2a
fix(ci): recover 38 failing tests on main (#17642)
CI Tests workflow has been red on main for 40+ consecutive runs. This
commit recovers every failure visible in run 25130722163 (most recent
completed run prior to this PR).

Root causes, by group:

Test-mock drift after product landed (fix: update mocks)
- test_mcp_structured_content / test_mcp_dynamic_discovery (6 tests):
  product added _rpc_lock (#02ae15222) and _schedule_tools_refresh
  (#1350d12b0) without updating sibling test files. Install a real
  asyncio.Lock inside the fake run-loop and patch at _schedule_tools_refresh.
- test_session.py: renamed normalize_whatsapp_identifier → canonical_
  whatsapp_identifier upstream; keep a local alias so the legacy tests
  keep working.
- test_run_progress_topics Slack DM test: PR #8006 made Slack default
  tool_progress=off; explicitly set it to 'all' in the test fixture so
  the progress-callback path still runs. Also read tool_progress_callback
  at call time rather than freezing it in FakeAgent.__init__ — production
  assigns it AFTER construction.
- test_tui_gateway_server session-create/close race: session.create now
  defers _start_agent_build behind a 50ms timer — wait for the build
  thread to enter _make_agent before closing, otherwise the orphan-
  cleanup path never runs.
- test_protocol session.resume: product get_messages_as_conversation now
  takes include_ancestors kwarg; accept **_kwargs in the test stub.
- test_copilot_acp_client redaction: redactor is OFF by default (snapshots
  HERMES_REDACT_SECRETS at import); patch agent.redact._REDACT_ENABLED=True
  for the duration of the test.
- test_minimax_provider: after #17171, dots in non-Anthropic model names
  stay dots even with preserve_dots=False. Assert the new invariant
  rather than the old 'broken for MiniMax' behavior.
- test_update_autostash: updater now scans `ps -A` for dashboard PIDs;
  the test's catch-all subprocess.run stub needed stdout/stderr fields.
- test_accretion_caps: read_timestamps dict is populated lazily when
  os.path.getmtime succeeds. Use .get("read_timestamps", {}) to tolerate
  CI filesystems where the stat races file creation.

Change-detector tests (fix: rewrite as structural invariants)
- test_credential_sources_registry_has_expected_steps: was a frozen set
  comparison that broke when minimax-oauth was added. Rewrite as an
  invariant check (every step has description, no dupes, core steps
  present) per AGENTS.md 'don't write change-detector tests'.

xdist ordering / test pollution (fix: reset state, use module-local patches)
- test_setup vercel: sibling test saved VERCEL_PROJECT_ID='project' to
  os.environ via save_env_value() and never cleared it. monkeypatch.delenv
  the VERCEL_* vars in the link-file test.
- test_clipboard TestIsWsl: GitHub Actions is on Azure VMs whose real
  /proc/version often contains 'microsoft'. Patching builtins.open with
  mock_open didn't reliably intercept hermes_constants.is_wsl's call in
  xdist workers that had already cached _wsl_detected=True from an
  earlier test. Patch hermes_constants.open directly and add
  teardown_method to reset the cache after each test.

Pytest-asyncio cancellation hangs (fix: bound product await with timeout)
- test_session_split_brain_11016 (3 params) + test_gateway_shutdown
  cancel-inflight: under pytest-asyncio 1.3.0, 'await task' and
  'asyncio.gather(cancelled_tasks)' can stall for 30s when the cancelled
  task's finally block awaits typing-task cleanup. Bound both with
  asyncio.wait_for(..., timeout=5.0) and asyncio.shield — the stragglers
  are released from adapter tracking and allowed to finish unwinding in
  the background. This is also a legitimate hardening: a wedged finally
  shouldn't stall the caller's dispatch or a gateway shutdown.

Orphan UI config (fix: merge tiny tab into messaging category)
- test_web_server test_no_single_field_categories: the telegram.reactions
  config field lived in its own 'telegram' schema category with no
  siblings. Fold it under 'discord' via _CATEGORY_MERGE so the dashboard
  doesn't render an orphan single-field tab.

Local verification: 38/38 originally-failing tests pass; 4044/4044
gateway tests pass; 684/684 targeted subset (all 16 touched test files)
passes.
2026-04-29 20:05:32 -07:00
..
__init__.py
…
test_ai_gateway_models.py
…
test_anthropic_model_flow_stale_oauth.py
…
test_anthropic_oauth_flow.py
…
test_anthropic_provider_persistence.py
…
test_api_key_providers.py test(cli): cover minimax-oauth resolution, refresh, menu wiring 2026-04-29 09:53:42 -07:00
test_apply_model_switch_result_context.py
…
test_arcee_provider.py
…
test_argparse_flag_propagation.py
…
test_at_context_completion_filter.py
…
test_atomic_json_write.py
…
test_atomic_yaml_write.py
…
test_auth_codex_provider.py
…
test_auth_commands.py fix(ci): recover 38 failing tests on main (#17642) 2026-04-29 20:05:32 -07:00
test_auth_nous_provider.py
…
test_auth_provider_gate.py
…
test_auth_qwen_provider.py
…
test_auth_ssl_macos.py
…
test_aux_config.py
…
test_azure_detect.py
…
test_backup.py
…
test_banner.py
…
test_banner_git_state.py
…
test_banner_skills.py
…
test_bedrock_model_picker.py
…
test_chat_skills_flag.py
…
test_claw.py
…
test_clear_stale_base_url.py
…
test_cmd_update.py
…
test_coalesce_session_args.py
…
test_codex_cli_model_picker.py
…
test_codex_models.py
…
test_commands.py
…
test_completion.py
…
test_config.py fix(cli): prevent .env sanitizer from splitting GLM_API_KEY by LM_API_KEY suffix 2026-04-28 22:22:45 -07:00
test_config_drift.py
…
test_config_env_expansion.py
…
test_config_env_refs.py
…
test_config_validation.py
…
test_container_aware_cli.py
…
test_copilot_auth.py
…
test_copilot_catalog_oauth_fallback.py
…
test_copilot_context.py
…
test_copilot_in_model_list.py
…
test_copilot_token_exchange.py
…
test_cron.py
…
test_custom_provider_context_length.py
…
test_custom_provider_model_switch.py
…
test_debug.py
…
test_deprecated_cwd_warning.py
…
test_detect_api_mode_for_url.py
…
test_determine_api_mode_hostname.py
…
test_dingtalk_auth.py
…
test_doctor.py feat: add Vercel Sandbox backend 2026-04-29 07:22:33 -07:00
test_doctor_command_install.py
…
test_env_loader.py
…
test_env_sanitize_on_load.py
…
test_fallback_cmd.py
…
test_gateway.py
…
test_gateway_linger.py
…
test_gateway_runtime_health.py
…
test_gateway_service.py [verified] fix(gateway): accept user systemd private socket during preflight 2026-04-29 04:57:01 -07:00
test_gateway_wsl.py
…
test_gemini_free_tier_setup_block.py
…
test_gemini_provider.py
…
test_gmi_provider.py
…
test_hooks_cli.py
…
test_ignore_user_config_flags.py
…
test_image_gen_picker.py
…
test_launcher.py
…
test_logs.py
…
test_managed_installs.py
…
test_mcp_config.py
…
test_mcp_tools_config.py
…
test_memory_reset.py
…
test_model_catalog.py
…
test_model_normalize.py
…
test_model_picker_viewport.py
…
test_model_provider_persistence.py
…
test_model_switch_context_display.py
…
test_model_switch_copilot_api_mode.py
…
test_model_switch_custom_providers.py
…
test_model_switch_opencode_anthropic.py
…
test_model_switch_variant_tags.py
…
test_model_validation.py
…
test_models.py
…
test_models_dev_preferred_merge.py
…
test_non_ascii_credential.py
…
test_nous_hermes_non_agentic.py
…
test_nous_subscription.py
…
test_ollama_cloud_auth.py
…
test_ollama_cloud_provider.py
…
test_opencode_go_in_model_list.py
…
test_opencode_go_validation_fallback.py
…
test_overlay_slug_resolution.py
…
test_path_completion.py
…
test_placeholder_usage.py
…
test_plugin_cli_registration.py
…
test_plugin_scanner_recursion.py
…
test_plugins.py
…
test_plugins_cmd.py
…
test_profile_export_credentials.py
…
test_profiles.py fix(cli): exclude profiles/ from profile create --clone-all 2026-04-29 14:21:35 -07:00
test_provider_config_validation.py
…
test_pty_bridge.py
…
test_reasoning_effort_menu.py
…
test_redact_config_bridge.py
…
test_regression_16767.py
…
test_resolve_last_session.py
…
test_runtime_provider_resolution.py test(cli): cover minimax-oauth resolution, refresh, menu wiring 2026-04-29 09:53:42 -07:00
test_session_browse.py
…
test_sessions_delete.py
…
test_set_config_value.py feat: add Vercel Sandbox backend 2026-04-29 07:22:33 -07:00
test_setup.py fix(ci): recover 38 failing tests on main (#17642) 2026-04-29 20:05:32 -07:00
test_setup_agent_settings.py
…
test_setup_hermes_script.py
…
test_setup_matrix_e2ee.py
…
test_setup_model_provider.py
…
test_setup_noninteractive.py
…
test_setup_ollama_cloud_force_refresh.py
…
test_setup_openclaw_migration.py
…
test_setup_prompt_menus.py
…
test_setup_reconfigure.py
…
test_skills_config.py
…
test_skills_hub.py
…
test_skills_install_flags.py
…
test_skills_skip_confirm.py
…
test_skills_subparser.py
…
test_skin_engine.py
…
test_spotify_auth.py
…
test_status.py feat: add Vercel Sandbox backend 2026-04-29 07:22:33 -07:00
test_status_model_provider.py
…
test_subparser_routing_fallback.py
…
test_subprocess_timeouts.py
…
test_suppress_eio_on_interrupt.py
…
test_tencent_tokenhub_provider.py
…
test_terminal_menu_fallbacks.py
…
test_timeouts.py
…
test_tips.py
…
test_tool_token_estimation.py
…
test_tools_config.py
…
test_tools_disable_enable.py
…
test_tui_npm_install.py
…
test_tui_resume_flow.py fix(tui): honor launch toolsets (#17623) 2026-04-29 16:55:27 -07:00
test_update_autostash.py fix(ci): recover 38 failing tests on main (#17642) 2026-04-29 20:05:32 -07:00
test_update_check.py
…
test_update_config_clears_custom_fields.py
…
test_update_gateway_restart.py
…
test_update_hangup_protection.py
…
test_update_stale_dashboard.py fix(update,test): clarify wmic comment; switch tests to monkeypatch sys.platform 2026-04-29 06:34:13 -07:00
test_user_providers_model_switch.py fix(model_switch): dedup /model picker rows when custom provider endpoint matches a built-in (#16970) (#17511) 2026-04-29 08:11:05 -07:00
test_voice_wrapper.py
…
test_web_server.py feat: add Vercel Sandbox backend 2026-04-29 07:22:33 -07:00
test_web_server_host_header.py
…
test_web_ui_build.py
…
test_webhook_cli.py
…
test_xiaomi_provider.py
…