hermes-agent/tests/run_agent
Teknium 1d4218be56
feat(review): active-update bias, loaded-skill-first, support-file variants (#17213)
The background skill-review prompts (_SKILL_REVIEW_PROMPT and the **Skills**
half of _COMBINED_REVIEW_PROMPT) steered the reviewer toward passive
behavior — most passes concluded 'Nothing to save.' even when the session
produced real lessons. User-preference corrections (style, format,
legibility, verbosity) were especially lost: they were read as memory
signals only, so skills never carried the fix.

This rewrite changes the stance:

- **Active-update bias.** The reviewer now treats inaction as a missed
  learning opportunity. 'Nothing to save.' remains an explicit escape
  but is no longer framed as the most-common outcome.

- **User-preference corrections are first-class skill signals.** Style,
  tone, format, legibility, verbosity complaints — and the actual
  phrasings users use ('stop doing X', 'this is too verbose', 'I hate
  when you Y', 'remember this') — now warrant patching the skill that
  governs the task, not just writing to memory.

- **Loaded-skill-first preference order.** When a skill was loaded via
  /skill-name or skill_view during the session, the reviewer patches
  THAT one first. It was in play; it's the right place.

- **Four-step ladder: patch-loaded → patch-umbrella → support-file →
  create.** Support files are explicitly enumerated as three kinds:
    * references/<topic>.md — session-specific detail OR condensed
      knowledge banks (quoted research, API docs excerpts, domain notes)
    * templates/<name>.<ext> — starter files to copy and modify
    * scripts/<name>.<ext>  — statically re-runnable actions

- **Name-veto for CREATE.** New skill names MUST be class-level — no PR
  numbers, error strings, codenames, library-alone names, or session
  artifacts ('fix-X / debug-Y / audit-Z-today'). If the proposed name
  only fits today's task, fall back to one of the patch/support-file
  options.

- **Memory scope clarified.** 'who the user is and what the current
  situation and state of your operations are' — MEMORY.md is
  situational/state, USER.md is identity/preferences.

- **Curator handoff.** Reviewer flags overlap; the background curator
  handles consolidation at scale. Single-session reviewer doesn't
  attempt umbrella-rebalancing.

Tests: tests/run_agent/test_review_prompt_class_first.py upgraded to
assert the new behavioral contracts (active bias, user-correction
signals, loaded-skill-first, support-file kinds, name-veto, memory
framing, curator handoff). 17 tests, all pass.

Co-authored-by: teknium1 <teknium@users.noreply.github.com>
2026-04-28 21:11:48 -07:00
..
__init__.py
…
conftest.py
…
test_413_compression.py
…
test_860_dedup.py
…
test_1630_context_overflow_loop.py
…
test_agent_guardrails.py
…
test_agent_loop.py
…
test_agent_loop_tool_calling.py
…
test_agent_loop_vllm.py
…
test_anthropic_error_handling.py
…
test_anthropic_prompt_cache_policy.py
…
test_anthropic_third_party_oauth_guard.py
…
test_anthropic_truncation_continuation.py
…
test_api_max_retries_config.py
…
test_async_httpx_del_neuter.py fix(copilot): send vision header for Copilot vision requests 2026-04-27 08:35:50 -07:00
test_background_review.py fix(approval): close remaining prompt_toolkit deadlock vectors (#15216) 2026-04-27 06:42:32 -07:00
test_background_review_summary.py
…
test_background_review_toolset_restriction.py fix(agent): restrict background review agent to memory and skills toolsets 2026-04-27 06:41:23 -07:00
test_compress_focus_plugin_fallback.py
…
test_compression_boundary.py
…
test_compression_boundary_hook.py
…
test_compression_feasibility.py
…
test_compression_persistence.py
…
test_compression_trigger_excludes_reasoning.py
…
test_compressor_fallback_update.py
…
test_concurrent_interrupt.py
…
test_context_token_tracking.py
…
test_copilot_native_vision_headers.py fix(copilot): mark native image requests as vision 2026-04-27 08:35:50 -07:00
test_create_openai_client_kwargs_isolation.py
…
test_create_openai_client_proxy_env.py
…
test_create_openai_client_reuse.py
…
test_deepseek_reasoning_content_echo.py
…
test_dict_tool_call_args.py
…
test_exit_cleanup_interrupt.py
…
test_fallback_model.py
…
test_image_shrink_recovery.py
…
test_interactive_interrupt.py
…
test_interrupt_propagation.py
…
test_invalid_context_length_warning.py
…
test_jsondecodeerror_retryable.py
…
test_long_context_tier_429.py
…
test_memory_provider_init.py
…
test_memory_sync_interrupted.py
…
test_openai_client_lifecycle.py
…
test_percentage_clamp.py
…
test_plugin_context_engine_init.py
…
test_primary_runtime_restore.py
…
test_provider_attribution_headers.py
…
test_provider_fallback.py
…
test_provider_parity.py fix(gateway): guard against None request_overrides in _build_api_kwargs 2026-04-28 06:57:23 -07:00
test_real_interrupt_subagent.py
…
test_redirect_stdout_issue.py
…
test_repair_tool_call_arguments.py
…
test_repair_tool_call_name.py
…
test_review_prompt_class_first.py feat(review): active-update bias, loaded-skill-first, support-file variants (#17213) 2026-04-28 21:11:48 -07:00
test_run_agent.py fix(agent): persist streamed reasoning_content on assistant turns (#16844) (#16892) 2026-04-28 01:19:18 -07:00
test_run_agent_codex_responses.py fix(memory): drop scrub from interim commentary + final response 2026-04-27 12:37:33 -07:00
test_run_agent_multimodal_prologue.py
…
test_sequential_chats_live.py
…
test_session_meta_filtering.py
…
test_session_reset_fix.py
…
test_steer.py
…
test_stream_interrupt_retry.py
…
test_streaming.py fix(copilot-acp): disable streaming path for CopilotACPClient 2026-04-28 11:33:07 -07:00
test_streaming_tool_call_repair.py
…
test_strict_api_validation.py
…
test_strip_reasoning_tags_cli.py
…
test_switch_model_context.py
…
test_switch_model_fallback_prune.py
…
test_thinking_only_sanitizer.py fix(agent): drop thinking-only assistant turns before provider call (#16959) 2026-04-28 03:50:51 -07:00
test_token_persistence_non_cli.py
…
test_tool_arg_coercion.py fix(mcp): preserve nullable schema coercion 2026-04-28 04:58:03 -07:00
test_tool_call_args_sanitizer.py
…
test_unicode_ascii_codec.py
…
test_vision_aware_preprocessing.py
…