Second, deeper pass over tools/gateway/hermes_cli plus first pass over
the trees wave 1 missed (acp, acp_adapter, skills, computer_use, docker,
dashboard, conformance, monitoring, secret_sources, hermes_state,
providers). Same rubric as wave 1 (AGENTS.md test policy); security,
alternation/caching invariants, issue-number regressions, and E2E kept.
Real test-quality fixes found and rooted out along the way:
- tests/tools/test_command_guards.py made real auxiliary-LLM HTTPS calls
(DEFAULT_CONFIG smart-approval leaked in) — pinned approval
mode=manual via autouse fixture: 17.4s → 0.4s.
- test_model_switch_custom_providers.py / test_user_providers_model_switch.py
silently probed live provider catalogs (~2s/test) — stubbed
cached_provider_model_ids/provider_model_ids/fetch_api_models.
- test_telegram_noise_filter.py: 15-platform copy-paste matrix over
shared gateway.run logic → 3 representative platforms (55s → 3.9s).
- test_gateway_shutdown.py: stop()'s 5s interrupt-deadline loop spun on
MagicMock agents — interrupt.side_effect now clears _running_agents
(22s → 1.0s).
- test_gateway_inactivity_timeout.py poll-harness timings shrunk 3-5x
(24s → 1.1s); test_mcp_stability.py backoff/SIGTERM-grace sleeps
patched (15.4s → 2.5s); test_async_delegation.py negative-drain wait
5s → 0.5s.
- test_telegram_init_deadline.py: loop-block margin restored to 1.0s
with rationale comment — the watchdog-dump assertion needs the loop
blocked well past deadline+grace under parallel load (flaked once in
the 40-worker verification run at a 0.2s margin).
Verification: full hermetic suite via scripts/run_tests.sh —
2,438 files, 21,718 tests passed, 0 failed, 293.9s wall.
Suite totals vs original baseline: 46,820 → 19,757 test functions
(−57.8%), wall 583.5s → 293.9s (−50%), subprocess CPU 13,564s → 11,623s.
`hermes prompt-size` reported skills as one <available_skills> block total
and tools as one json-bytes total, so there was no way to see which
installed skill or toolset actually dominates the fixed prompt budget.
Add two additive breakdowns to compute_prompt_breakdown (hermes_cli/
prompt_size.py):
- toolsets_breakdown: each resolved tool is attributed to its single
canonical registry toolset (registry.get_tool_to_toolset_map), summed by
group. Fully attributable — the grand total equals the existing
tools.json_bytes minus JSON array framing (2*count bytes).
- skills_breakdown: parsed from the rendered <available_skills> block, one
entry per skill with two honest, distinct numbers — index_line_bytes (the
always-on cost of listing the skill) and skill_md_bytes (on-disk SKILL.md
size, the real read cost paid only on skill_view). Sorted largest-first
by read cost.
render_breakdown prints both as sorted "Toolsets by size" / "Skills by
size" tables (skills capped at 20; --json carries them all). All existing
keys and output are unchanged.
Runs fully offline (dummy credentials, no network). Tests cover shapes,
largest-first ordering, per-tool attribution reconciling to the total,
namespaced-name parsing, and unmapped-skill handling.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Regression tests from PR #51586: the inspection agent must receive the
platform-resolved enabled_toolsets and agent.disabled_toolsets, and a
Blank Slate profile's prompt-size must count exactly the 6 file/terminal
tool schemas.
Adds a 'hermes prompt-size' command that reports the fixed prompt budget
for a fresh session: system prompt total, skills index, memory, user
profile, prompt tiers, and tool-schema JSON bytes. Runs offline (dummy
credentials force the direct-construction path, no network call).
Lets users see which block dominates their per-call payload — the skills
index is often the largest single block when many skills are installed
(issue #34667). Zero model-tool footprint: it's a top-level CLI
subcommand, not an agent tool.
--platform <name> simulates a channel's platform hint; --json emits a
machine-readable breakdown.
Closes#34667
Teknium
joelbrilliant
dodo-reach