RomanNum3ral
/
hermes-agent
mirror of https://github.com/NousResearch/hermes-agent.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
hermes-agent/tests/tools
History
teknium1 369075dc95 feat(tools): progressive tool disclosure for MCP and plugin tools 
Adds Tool Search, a structured-tools progressive-disclosure layer that
replaces MCP and non-core plugin tools in the model-visible tools array
with three bridge tools (tool_search / tool_describe / tool_call) when
the deferrable surface would consume more than a configurable percentage
of the active model's context window. Core Hermes tools are never deferred.

Default mode is 'auto' with a 10% context threshold, so small toolsets
pay no overhead. Set tools.tool_search.enabled to 'on' to force or 'off'
to disable.

Design carefully reflects the OpenClaw production failure modes
documented in the openclaw-tool-search-report:

  - Core tools never defer (toolsets._HERMES_CORE_TOOLS). Addresses the
    'tools silently missing from isolated cron turns' regression class
    (openclaw#84141) by construction: there is no code path that can
    drop a core tool.
  - Catalog is stateless across turns — rebuilt from the live tool-defs
    list on every assembly. No session-keyed Map that can drift out of
    sync with the registry.
  - tool_call unwraps the bridge call before any hook fires, so plugin
    pre/post hooks, guardrails, approval flows, and the activity feed
    all see the underlying tool name, not the bridge (addresses
    openclaw#85588 and the verbose-mode complaint on openclaw#79823).
  - The unwrap happens in both the parallel and sequential paths of
    agent/tool_executor.py and also in handle_function_call, so direct
    callers (sandboxed code, eval harnesses) are covered too.
  - Bridge tools cannot invoke each other (recursion guard) and cannot
    invoke core tools (those must be called directly).
  - Tools mode only — no JS-sandbox code-mode. Keeps the surface small.
  - Token estimation via cheap char/4 heuristic; precision isn't needed
    for the threshold decision.

Files:
  - tools/tool_search.py — new module (BM25 retrieval, classification,
    threshold gate, bridge dispatch, unwrap helper).
  - tests/tools/test_tool_search.py — 35 tests including the OpenClaw
    #84141 regression guard.
  - model_tools.py — wires assembly into _compute_tool_definitions as the
    final step, adds skip_tool_search_assembly kwarg so the bridge can
    see the real catalog, dispatches the three bridge tools.
  - agent/tool_executor.py — unwraps tool_call in both parallel and
    sequential parsing loops so checkpointing, guardrails, plugin hooks,
    and tool-progress callbacks all observe the underlying tool name.
  - hermes_cli/config.py — DEFAULT_CONFIG['tools']['tool_search'] block.
  - website/docs/user-guide/features/tool-search.md — user docs.

Validation:
  - 35/35 new tests pass.
  - Existing tool/registry/model_tools/config/coercion/executor tests
    (82 + 74 + small adjacents) green.
  - Live E2E: 20 fake MCP tools registered, get_tool_definitions returns
    3 bridges, tool_search returns top 3 hits, tool_describe returns
    full schema, tool_call dispatches to the real underlying handler
    and the underlying result is what the model sees.
  - Reserved-name recursion guard verified live.
  - Core-tool refusal via tool_call verified live.
 		2026-05-29 02:04:12 -07:00
..
__init__.py
…
conftest.py
…
test_accretion_caps.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_ansi_strip.py
…
test_approval.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_approval_heartbeat.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_approval_plugin_hooks.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_base_environment.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_browser_camofox.py fix(browser): rewrite Camofox Docker loopback URLs (#25541) 2026-05-29 15:43:55 +10:00
test_browser_camofox_persistence.py
…
test_browser_camofox_state.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_browser_cdp_override.py
…
test_browser_cdp_tool.py
…
test_browser_chromium_check.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_browser_cleanup.py
…
test_browser_cloud_fallback.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_browser_cloud_provider_cache.py
…
test_browser_console.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_browser_content_none_guard.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_browser_eval_supervisor_path.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_browser_hardening.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_browser_homebrew_paths.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_browser_hybrid_routing.py
…
test_browser_lightpanda.py
…
test_browser_orphan_reaper.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_browser_secret_exfil.py
…
test_browser_ssrf_local.py
…
test_browser_supervisor.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_browser_supervisor_healthcheck.py
…
test_budget_config.py
…
test_checkpoint_manager.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_clarify_gateway.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_clarify_tool.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_clipboard.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_code_execution.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_code_execution_modes.py
…
test_code_execution_windows_env.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_command_guards.py fix: restore side-effect imports in test files (test_kanban_tools, test_command_guards) 2026-05-28 22:26:25 -07:00
test_computer_use.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_computer_use_capture_routing.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_computer_use_vision_routing.py
…
test_config_null_guard.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_credential_files.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_credential_pool_env_fallback.py
…
test_cron_approval_mode.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_cron_prompt_injection.py
…
test_cronjob_tools.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_cross_profile_guard.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_daytona_environment.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_debug_helpers.py
…
test_delegate.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_delegate_composite_toolsets.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_delegate_subagent_timeout_diagnostic.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_delegate_toolset_scope.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_discord_tool.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_docker_environment.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_docker_find.py
…
test_docker_orphan_reaper_integration.py fix(docker): startup orphan reaper for crashed-process containers 2026-05-29 11:49:54 +10:00
test_dockerfile_node_modules_perms.py
…
test_dockerfile_pid1_reaping.py
…
test_env_passthrough.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_env_probe.py feat(prompt): universal task-completion guidance + local Python toolchain probe (#34340) 2026-05-28 22:26:09 -07:00
test_feishu_tools.py
…
test_file_operations.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_file_operations_edge_cases.py
…
test_file_ops_cwd_tracking.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_file_read_guards.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_file_staleness.py
…
test_file_state_registry.py
…
test_file_sync.py
…
test_file_sync_back.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_file_sync_perf.py
…
test_file_tools.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_file_tools_container_config.py
…
test_file_tools_live.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_file_write_safety.py
…
test_force_dangerous_override.py
…
test_fuzzy_match.py fix(patch): widen new_string \t/\r unescape to all match strategies (#33733) 2026-05-28 03:27:20 -07:00
test_hardline_blocklist.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_heartbeat_stale_thresholds.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_hidden_dir_filter.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_homeassistant_tool.py
…
test_image_generation.py
…
test_image_generation_env.py
…
test_image_generation_plugin_dispatch.py
…
test_init_session_cwd_respect.py
…
test_interrupt.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_kanban_codex_lane_skill.py
…
test_kanban_tools.py fix: restore side-effect imports in test files (test_kanban_tools, test_command_guards) 2026-05-28 22:26:25 -07:00
test_lazy_deps.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_line_ending_preservation.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_llm_content_none_guard.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_local_background_child_hang.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_local_env_blocklist.py fix(security): narrow Bedrock subprocess strip to inference bearer token only 2026-05-29 01:48:08 -07:00
test_local_env_cwd_recovery.py
…
test_local_env_windows_msys.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_local_interrupt_cleanup.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_local_shell_init.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_local_tempdir.py
…
test_managed_browserbase_and_modal.py feat(auth) normalise the way in which we check whether a user has free/paid access to nous portal so we can expose behaviour and error messages accordingly. 2026-05-28 00:19:31 -07:00
test_managed_media_gateways.py feat(auth) normalise the way in which we check whether a user has free/paid access to nous portal so we can expose behaviour and error messages accordingly. 2026-05-28 00:19:31 -07:00
test_managed_modal_environment.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_managed_tool_gateway.py
…
test_mcp_cancelled_error_propagation.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_mcp_circuit_breaker.py
…
test_mcp_client_cert.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_mcp_dynamic_discovery.py
…
test_mcp_empty_error_message.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_mcp_image_content.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_mcp_invalid_url.py
…
test_mcp_oauth.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_mcp_oauth_bidirectional.py
…
test_mcp_oauth_cold_load_expiry.py
…
test_mcp_oauth_integration.py
…
test_mcp_oauth_manager.py
…
test_mcp_oauth_metadata.py
…
test_mcp_probe.py
…
test_mcp_reconnect_signal.py
…
test_mcp_sse_transport.py
…
test_mcp_stability.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_mcp_structured_content.py
…
test_mcp_tool.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_mcp_tool_401_handling.py
…
test_mcp_tool_issue_948.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_mcp_tool_session_expired.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_mcp_utility_capability_gating.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_memory_tool.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_memory_tool_import_fallback.py
…
test_memory_tool_schema.py
…
test_microsoft_graph_auth.py
…
test_microsoft_graph_client.py
…
test_mixture_of_agents_tool.py
…
test_modal_bulk_upload.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_modal_sandbox_fixes.py
…
test_modal_snapshot_isolation.py
…
test_notify_on_complete.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_osv_check.py
…
test_parse_env_var.py
…
test_patch_failure_tracking.py
…
test_patch_parser.py
…
test_pr_6656_regressions.py
…
test_process_registry.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_read_loop_detection.py
…
test_registry.py
…
test_resolve_path.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_schema_sanitizer.py
…
test_search_hidden_dirs.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_send_message_missing_platforms.py
…
test_send_message_telegram_proxy.py
…
test_send_message_tool.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_session_search.py
…
test_shared_container_task_id.py
…
test_signal_media.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_singularity_preflight.py
…
test_skill_env_passthrough.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_skill_improvements.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_skill_manager_tool.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_skill_provenance.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_skill_size_limits.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_skill_usage.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_skill_view_path_check.py
…
test_skill_view_traversal.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_skills_ast_audit.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_skills_guard.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_skills_hub.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_skills_hub_browse_sh.py
…
test_skills_hub_clawhub.py fix(skills): pull full ClawHub catalog into the skills index (200 → 20k+) (#33748) 2026-05-28 01:42:19 -07:00
test_skills_sync.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_skills_tool.py
…
test_slash_confirm.py
…
test_spotify_client.py
…
test_ssh_bulk_upload.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_ssh_environment.py
…
test_stage2_hook_puid_pgid.py fix(docker): accept PUID/PGID as aliases for HERMES_UID/HERMES_GID (#25872) (#34401) 2026-05-29 16:07:15 +10:00
test_symlink_prefix_confusion.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_sync_back_backends.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_terminal_compound_background.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_terminal_config_env_sync.py fix(docker): startup orphan reaper for crashed-process containers 2026-05-29 11:49:54 +10:00
test_terminal_exit_semantics.py
…
test_terminal_foreground_timeout_cap.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_terminal_none_command_guard.py
…
test_terminal_output_transform_hook.py
…
test_terminal_requirements.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_terminal_task_cwd.py
…
test_terminal_timeout_output.py
…
test_terminal_tool.py
…
test_terminal_tool_pty_fallback.py
…
test_terminal_tool_requirements.py
…
test_threaded_process_handle.py
…
test_threat_patterns.py
…
test_tirith_security.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_todo_tool.py
…
test_tool_backend_helpers.py fix(tools): guard Path.home() against PermissionError in has_direct_modal_credentials (#33528) 2026-05-29 13:35:39 +10:00
test_tool_output_limits.py
…
test_tool_result_storage.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_tool_search.py feat(tools): progressive tool disclosure for MCP and plugin tools 2026-05-29 02:04:12 -07:00
test_transcription.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_transcription_command_providers.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_transcription_dotenv_fallback.py
…
test_transcription_plugin_dispatch.py
…
test_transcription_tools.py
…
test_tts_command_providers.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_tts_dotenv_fallback.py
…
test_tts_gemini.py
…
test_tts_kittentts.py
…
test_tts_max_text_length.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_tts_mistral.py
…
test_tts_opus_routing.py
…
test_tts_path_traversal.py
…
test_tts_piper.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_tts_plugin_dispatch.py
…
test_tts_speed.py
…
test_tts_xai_speech_tags.py
…
test_url_safety.py
…
test_video_analyze.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_video_generation_dispatch.py
…
test_video_generation_dynamic_schema.py
…
test_video_generation_tool_surface_matrix.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_vision_native_fast_path.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_vision_tools.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_voice_cli_integration.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_voice_mode.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_watch_patterns.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_web_providers.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_web_providers_brave_free.py chore(web): remove web_crawl tool + provider crawl plumbing (#33824) 2026-05-28 04:52:42 -07:00
test_web_providers_ddgs.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_web_providers_searxng.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_web_providers_xai.py chore(web): remove web_crawl tool + provider crawl plumbing (#33824) 2026-05-28 04:52:42 -07:00
test_web_tools_config.py
…
test_web_tools_tavily.py chore(web): remove web_crawl tool + provider crawl plumbing (#33824) 2026-05-28 04:52:42 -07:00
test_website_policy.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_windows_compat.py
…
test_windows_native_support.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_write_deny.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00
test_x_search_tool.py
…
test_yolo_mode.py
…
test_zombie_process_cleanup.py chore: prune unused imports and duplicate import redefinitions 2026-05-28 22:26:25 -07:00