Azure Foundry's OpenAI-compatible Responses surface rejects the post-tool
follow-up payload with HTTP 400 `invalid_payload` when a replayed encrypted
`reasoning` item is sent alongside `function_call` / `function_call_output`.
The initial function-call request and ordinary multi-turn continuity are both
accepted, so the failure only appears after the first tool executes.
Detect the Foundry endpoint in `ResponsesApiTransport.build_kwargs` and drop
only the encrypted reasoning replay on that follow-up turn, leaving
function_call / function_call_output continuity intact.
Salvage of #59981, rebuilt on current main. Same root cause and fix direction
as the original, which was correct; this version resolves three defects:
- No `chat_completion_helpers.py` change. main already forwards `provider`
and `base_url` to the Responses transport, so the original's re-added
arguments produced `SyntaxError: keyword argument repeated: provider` on
merge. Dropping the hunk removed the syntax error and the conflict.
- Host matching uses `utils.base_url_host_matches`, not a substring test.
`".services.ai.azure.com" in base_url` also matches URLs carrying the
domain in a path or query segment, which would silently disable reasoning
replay on an unrelated provider.
- The post-tool predicate tests the trailing messages, not the whole history.
Scanning for any tool call plus any tool result made it sticky: one tool
call early in a conversation suppressed reasoning on every later turn.
- Tool calls pair on `call_id` as well as `id`. Responses histories carry the
function call id in `call_id` while `id` holds the response item id
(`fc_...`). Identity is resolved via the converter's own
`_split_responses_tool_id`, covering composite `"call_x|fc_y"` ids and bare
`fc_` ids on both sides of the pairing.
Tests: 27 cases across the transport and the live `build_api_kwargs` bridge,
including six parametrized tool-call id shapes, non-Foundry host lookalikes,
the sticky-history guard, parallel tool results, and an unpaired tool result.
Each guard was confirmed to catch its defect by reverting the fix.
Verified with `scripts/run_tests.sh tests/agent/ tests/run_agent/`:
532 files, 5602 tests passed, 0 failed.
Not verified against a live Azure Foundry endpoint — no credentials. The
original HTTP 400 reproduction and post-fix Foundry Project / Azure Container
Apps harness runs are @AshuJoshi's, from #59981. This change is verified at
the payload-construction layer only.
Closes#59981.
Co-authored-by: Ashu Joshi <AshuJoshi@users.noreply.github.com>
- web_server CONFIG_SCHEMA: fold the one-field models_dev category
(models_dev.url) into the agent tab via _CATEGORY_MERGE, matching the
established pattern for single-field categories (slice 7,
test_no_single_field_categories).
- image_routing._lookup_supports_vision: pass allow_network=True to
get_model_capabilities. The vision-capability lookup runs when an
image actually needs routing (not per conversation turn), and the
#31179 text-only-main guard depends on catalog data — with the new
allow_network=False default a cold cache returned 'unknown', which
falls back to attempting the call and reintroduced the #31179
failure shape (slice 8, test_text_only_main_skipped_when_no_
aggregator). This preserves that path's historical
network-on-cold-cache behavior; the fetch stays 4h-TTL cached and
backoff-limited.
- Cold force_refresh (fresh CLI process, e.g. hermes config refresh)
now hydrates the memory cache from disk before fetching, so the
conditional GET actually fires on the flow the feature was built for
instead of silently re-downloading the full ~2 MB registry
(empirically probed: If-None-Match sent, 304 serves disk data).
- Conditional-GET decision is passed in explicitly
(_fetch_models_dev_from_network(conditional=...)) by callers holding
the fetch lock, removing the hidden read of module globals inside
the fetch; the background worker now fetches INSIDE the lock,
symmetric with foreground (true singleflight — no concurrent
double-download, no fetching against mid-commit etag state).
- Corrupt disk cache is QUARANTINED (renamed to .json.corrupt) rather
than left in place: rejection becomes a one-time event instead of a
re-read + re-parse + warning + unlink on every hot-path call while
offline (probed: 1 warning across 5 calls, was 5).
- Dropped the dead _DEFAULT_MODELS_DEV_URL constant; module and
function docstrings updated to match the servable-cache conditional
semantics.
- Conditional GET now requires a servable in-memory registry: an
If-None-Match sent while holding no cache invited a 304 against
nothing, permanently serving {} with a blocking foreground fetch on
every call (the exact #35838 class this PR fixes). Empirically
repro'd and verified fixed (corrupt cache + stale sidecar: was 3
calls -> {} forever; now 1 unconditional fetch -> real data).
- ETag persists atomically WITH the cache body via
_commit_registry -> _save_disk_cache(data, etag), wiring up the
previously-dead etag param; the sidecar can no longer get ahead of
the registry it vouches for. _save_etag now uses
utils.atomic_write_text (unique tempnames + fsync) instead of a
hand-rolled fixed-name .tmp replace.
- Corrupt/unreadable disk cache clears the ETag sidecar so the
refetch is unconditional; _confirm_cache_not_modified keeps a
defense-in-depth guard (clear sidecar + arm backoff) should a 304
ever land on an empty registry.
- allow_network=True paths use the zero-arg fetch_models_dev() call
shape at all sites (was 1 of 5) — ~46 test sites monkeypatch it
with zero-arg lambdas; the unconditional kwarg broke
test_xiaomi_provider (verified fail->pass).
- _get_models_dev_url falls back to the MODELS_DEV_URL module global
(not the constant) so existing patch sites keep working.
- Tests: replaced two mock-riddled corrupt-cache tests with real
tmp_path file tests; added regression tests for the 304/empty-cache
loop, sidecar clearing, and conditional-GET gating.
Harden the models.dev catalog refresh path (#35838) with three missing
pieces:
1. ETag conditional GET — every network request sends If-None-Match
with the last-known ETag (persisted alongside the cache file). A 304
Not Modified re-confirms the existing cache without re-downloading
the full ~2 MB registry. This makes the 4-hour TTL effectively free
to maintain.
2. No-network-on-hot-paths invariant — allow_network=False is now the
default for every query function called on the conversation hot path:
get_model_capabilities, get_model_info, lookup_models_dev_context,
_get_provider_models. These are called during vision routing, image
routing, cost-guard checks, and context-length resolution on every
turn — they must never block on the network. Interactive flows
(model picker, model switch) explicitly pass allow_network=True.
3. Mirror URL override — models_dev.url in config.yaml lets deployments
point at a self-hosted mirror without code changes. Follows the same
pattern as model_catalog.url.
Additional hardening:
- Cache TTL bumped from 1h to 4h (ETag makes refresh cheap)
- Corrupt/empty disk cache is rejected with a warning instead of being
served as {} and silently breaking provider/model resolution
- _validate_registry() guards against non-dict and empty-dict payloads
Fixes#35838
- Single-source the included note as _INCLUDED_NOTE and attach it at
BOTH status='included' sites (the zero-amount pricing-entry branch
previously returned the same status with no note).
- Docstring/comment precision on format_cost_label: the fallback
triggers on 4dp ROUNDING to 0.0000 (banker's rounding includes the
exact $0.00005 boundary), not truncation; note why the rendered-label
guard beats a naive Decimal threshold.
- Tests: replaced a dead assertion with the exact-boundary case
($0.00005), fixed an overclaiming comment, aligned the terminal
cost column.
- Insights formatters now route aggregate estimated cost through the
shared format_cost_label() instead of hardcoded 2dp — a sub-cent
aggregate (one cheap DeepSeek session, ~$0.0046) no longer renders
'Estimated: ~$0.00', the exact bug class this PR fixes (#79220).
- format_cost_label: positive amounts below $0.00005 render '~$<0.0001'
instead of the zero-looking '~$0.0000' 4dp truncation artifact.
- Renamed _format_cost_label -> format_cost_label (now a cross-module
shared helper).
- Tests: renamed test_gateway_format_hides_cost ->
test_gateway_format_hides_cache_details and
test_no_cost_section_when_all_zero ->
test_unknown_bucket_shown_for_costless_session (names contradicted
behavior); restored a real assertion in the custom-models test that
had been weakened to a comment; added sub-cent-aggregate and 4dp-floor
contract tests (mutation-checked).
Three cost-display honesty fixes:
1. Sub-cent cost label rendering (#79220) — _format_cost_label() scales
precision to magnitude: zero renders as '$0.00', sub-cent (< $0.01)
renders at 4 decimal places (e.g. '~$0.0046'), normal costs keep 2dp.
This fixes the bug where DeepSeek per-turn costs of $0.004640 rendered
as '~$0.00' despite amount_usd carrying full Decimal precision.
2. Cost bucket surfacing (#77223) — insights format_terminal and
format_gateway now display three cost buckets: estimated (with dollar
figure), included (session count, labeled 'subscription — no provider
invoice'), and unknown (session count, labeled 'no pricing data').
Previously, included and unknown sessions silently collapsed to $0 in
the aggregate view, hiding 315 of 473 sessions in the reporter's DB.
3. Subscription-included cost notes — estimate_usage_cost now attaches a
'subscription-included; no provider invoice for usage' note to
CostResult for subscription-included routes (openai-codex), so
consumers can distinguish 'free because subscription' from 'free
because $0 pricing'.
Fixes#79220Fixes#77223
- Deleted the id(cfg)-keyed _OVERRIDE_CACHE layer: id() is unique only
among live objects, so a config reload could serve stale overrides
forever when CPython reuses the freed dict's address. The upstream
load_config_readonly is already (mtime,size)-cached (~1 stat/hit), so
the local layer was redundant state with a correctness risk.
- _override_to_catalog_shape returns (patch, vision) instead of
smuggling an in-band _vision_override sentinel key through the merged
dict; removed the two dead call-site pops.
- _find_model_entry gains the :cloud/-cloud suffix fallback that
lookup_models_dev_context already had, so 'catalog hit' means the
same thing to every consumer — a suffix-keyed model (kimi-k2.6:cloud)
now counts as KNOWN and keeps its catalog capabilities instead of
being displaced by a fill-gap _default (mutation-checked contract
test added).
- get_model_info's unknown-model override path seeds the same safe
defaults as get_model_capabilities (200K ctx, tools on, 8192 out),
so a partial override no longer yields ctx=0/tools-off on that path
(contract test added); the DEFAULT_CONFIG defaults claim is now true
for both paths.
- Activated the previously-dead _MODELS_DEV_TO_PROVIDER reverse map
(lazily built, many-to-one aware) and used it in
_provider_override_section instead of a per-call linear scan.
Review follow-ups on the model_overrides feature:
- ONE canonical override schema everywhere. get_model_info previously
merged the override dict raw into the models.dev catalog shape
({**raw, **override}), so the documented context_window/supports_*
keys silently did nothing on that path (cost guard, inventory) while
working in capabilities/context paths — same config key, two
incompatible schemas. Overrides are now translated into the catalog
shape at the get_model_info boundary (_override_to_catalog_shape),
and sub-dicts (limit, modalities) are MERGED, not clobbered — an
override setting only context_window no longer wipes the catalog's
limit.output.
- _default is now a FILL-GAP default, not an override: it applies only
to models the catalog does not know (the #8731/#84482 self-unblock
path) and never displaces catalog data. A
_default: {context_window: 128000} can no longer clamp every model
of a provider. Explicit per-provider+model entries keep their
win-over-catalog semantics.
- Early-chain _override_context_window (model_metadata step 0b) is
explicit-only, so a _default can never preempt custom_providers
per-model settings or live probes; fill-gap defaults apply at the
lookup_models_dev_context catalog-miss boundary (step 5f) instead.
This fixes the precedence inversion where a provider/global _default
silently overrode an explicit per-endpoint per-model context_length.
- Provider keys accept BOTH id spaces (Hermes id and models.dev id:
copilot/github-copilot both work) and model ids match
case-insensitively, mirroring catalog lookup.
- Malformed override values (context_window: '512k') log a one-shot
warning instead of being silently swallowed.
- DEFAULT_CONFIG comment: removed the false family/dated-snapshot
inheritance claim, documented the recognized field list, fill-gap
semantics, and the id-space rule.
- Tests: rewritten for the new contracts (fill-gap invariants,
dual-id-space keys, sub-dict merge preservation, one-shot warning);
added a real-config-yaml e2e plumbing test (mutation-checked: fails
when the config key wiring is broken).
Add a unified model_overrides config section that lets users manually
declare context_window, max_output_tokens, capabilities, cost, and
family for any provider+model — winning over models.dev, OpenRouter, and
hardcoded defaults.
Resolution order (first hit wins):
1. model_overrides.<provider>.<model_id> (per-provider+model)
2. model_overrides.<provider>._default (per-provider default)
3. model_overrides._default (global default)
4. Normal catalog resolution
Key subtlety: an unknown model id (not in the
catalog) derives base metadata from sensible defaults before patching,
so overriding a model the catalog doesn't know yet is the supported
self-unblock path. This is exactly the #84482 scenario (Upstage
solar-pro4/syn-pro wrong context) and the #8731 scenario (custom/local
models with manual capability declaration).
Wired into:
- get_model_capabilities() — patches capability fields; unknown models
get safe defaults (tools on, vision/reasoning off) before patching
- lookup_models_dev_context() — context_window override, checked before
catalog lookup so it works even for providers not in PROVIDER_TO_MODELS_DEV
- get_model_info() — merges override dict onto catalog entry (shallow
merge); for unknown models, the override is the sole source of metadata
- get_model_context_length() — step 0b in the resolution pipeline,
before custom_providers (0c) and before any network probe
Config example:
model_overrides:
upstage:
solar-pro4:
context_window: 524288
syn-pro:
context_window: 65536
custom:my-local-vllm:
my-llava-model:
context_window: 8192
supports_vision: true
supports_reasoning: false
supports_tools: true
_default:
context_window: 128000
Fixes#8731Fixes#84482
Refs #47247
Maintainer fixup on the #79787 salvage:
- An explicit fb.api_mode of "chat_completions" was silently overridden
by the codex_responses / bedrock re-detection pass (which only skipped
re-detection when the pre-computed mode was non-default). Track
explicitness in fb_api_mode_explicit and gate the whole re-detection
block on it.
- Replace the locals().get('fb_api_mode') dead-code hack with clean code
(fb_api_mode is always bound at that point).
- Restore the post-resolve /anthropic + api.anthropic.com host check for
named custom providers whose base_url comes from config rather than
the fallback entry (#32243, #49247), which the PR's restructure dropped.
- Add regression tests: explicit api_mode honored (incl. explicit
chat_completions not overridden), /anthropic-hint fallback detected
pre-rewrite, api_mode forwarded to resolve_provider_client, plain
fallback unchanged.
Fallback activation determined api_mode from the POST-rewrite client
base_url, losing the Anthropic wire signal for /anthropic endpoints
routed through provider 'custom', and never honored an explicit
fb.api_mode config field. Pre-compute fb_api_mode from the ORIGINAL
fallback base_url hint (before _to_openai_base_url rewriting), honor
the explicit api_mode config field, check provider name before the
base_url gate, and pass api_mode into resolve_provider_client at the
fallback call site.
Salvaged from PR #79787 (chat_completion_helpers.py hunks; the
auxiliary_client.py hunk is redundant with #85466's wrap_base fix).
- /model switch now refreshes agent._custom_providers from the config
loaded during the switch before re-evaluating cache policy — a
prompt_caching flag added to config.yaml after session start was
invisible to a mid-session switch (policy read the stale init-time
snapshot while context_length resolution used the live list).
- Production-path test: real config.yaml in the modern providers: dict
shape through the real loader chain, exercising the init-order fallback
(no _custom_providers attr) for both the fable opt-in and the opus
explicit opt-out.
- Pin operator kill-switch precedence: _cache_disabled (prompt_caching.
cache_ttl falsy) beats an explicit per-model prompt_caching: true.
- Log (debug) instead of silently swallowing capability-lookup failures in
anthropic_prompt_cache_policy — a swallowed failure would otherwise
downgrade an explicit prompt_caching: true to (False, False) with zero
trace. Matches the sibling MoA branch's logger.debug style.
- Use load_config_readonly() for the None-fallback in
get_custom_provider_model_capability: the helper only reads, and the
fallback fires on the blank-stub paths (agent init before
_custom_providers is assigned, MoA/auxiliary destination planning), so
skip the ~135us defensive deepcopy per call.
- Add route-isolation regression tests at both levels (config helper +
agent policy): a prompt_caching declaration for one provider route must
never apply to another route with the same model name. Mutation-checked:
both tests fail when the URL match is disabled.
Substring matching over the whole URL let a path containing
'api.minimax' false-positive an Anthropic-only gateway into the
/anthropic→/v1 rewrite. Parse the host and match exact-domain /
subdomain suffixes (plus the api.minimax.* prefix family) instead.
`_try_anthropic()` applies the configured `model.base_url` only when
`_is_anthropic_compatible_host()` trusts it, but that check accepted only the
literal `api.anthropic.com` host. Anthropic-compatible gateways that expose the
native Messages protocol under a `/anthropic` path suffix (MiniMax, Zhipu GLM,
LiteLLM-style relays, self-hosted proxies) were rejected, so every auxiliary
call (title generation, memory extraction, vision, reflection) and the
`provider: anthropic` fallback chain discarded the configured base_url and fell
back to `https://api.anthropic.com`. That diverges from the primary path, which
already trusts the `/anthropic` suffix via
`runtime_provider._detect_api_mode_for_url`, and fails outright when the gateway
(not Anthropic) holds the credentials.
Accept `/anthropic` and `/anthropic/v1` suffixed URLs in
`_is_anthropic_compatible_host()`, matching the primary-path convention and
`_wrap_if_needed`. A bare non-Anthropic base_url (e.g. `openrouter.ai/api/v1`
left on `provider: anthropic`) still returns False, preserving the #52608 guard.
- _submit_background and _prefetch_provider: replace unreadable
(lambda inner: (lambda: ctx.run(inner)))(fn) with functools.partial(ctx.run, fn)
- from_env(): set config_path=resolve_config_path() so bound_config_path()
doesn't re-resolve from ContextVar on daemon threads (the exact bug
the PR fixes for from_global_config)
Review follow-ups for salvaged PR #83525.
MemoryManager dispatches provider sync_turn/queue_prefetch work on a
single-worker executor and hot prefetch on a plain thread. Neither
carried the caller's contextvars, so in multi-profile processes the
provider work ran outside the profile's ContextVar-scoped HERMES_HOME
override — any ambient resolution inside a provider landed on the
default profile.
Wrap the submitted callable and the prefetch thread target with
contextvars.copy_context().run, mirroring the gateway's
_run_in_executor_with_context pattern. Provider-agnostic: benefits
every external memory provider, not just Honcho.
save_context_length() and _invalidate_cached_context_length() did an
unguarded read-modify-write into $HERMES_HOME/context_length_cache.yaml.
The plain `open(path, "w")` truncates the file before the dump runs. If
the process is killed mid-dump, the file is left empty or partial. The
next _load_context_cache() swallows the YAML error and returns {} —
silently wiping every persisted context length. A concurrent process
reading between truncate and dump-complete also sees a torn file.
After the cache is lost, every model re-probes the network, and when a
probe fails it falls back to the generic 256K default — so a user on a
1M-window model ends up with a wrong, short context window.
Hermes routinely runs several processes against one shared $HERMES_HOME
(a cron agent plus an interactive session, multiple gateway sessions),
so this is hit in normal use.
Switch both writers to the existing utils.atomic_yaml_write helper
(temp file + fsync + os.replace, symlink- and mode-preserving). The real
file is only ever swapped from a fully written temp file, so an
interrupted write leaves the previous cache intact and readers never see
a partial file. Matches the atomic-write pattern already used for
auth.json, config.yaml, and other persisted state.
Makes the persistent model context-length cache write crash-safe. The
old non-atomic write could truncate or wipe the entire cache on an
interrupted or concurrent write, which then forces models onto the wrong
fallback context window. The fix routes both cache writers through the
repo's atomic temp-file + os.replace helper.
N/A
- [x] 🐛 Bug fix (non-breaking change that fixes an issue)
- [ ] ✨ New feature (non-breaking change that adds functionality)
- [ ] 🔒 Security fix
- [ ] 📝 Documentation update
- [ ] ✅ Tests (adding or improving test coverage)
- [ ] ♻️ Refactor (no behavior change)
- [ ] 🎯 New skill (bundled or hub)
- `agent/model_metadata.py`: `save_context_length()` and
`_invalidate_cached_context_length()` now write via
`utils.atomic_yaml_write` instead of a truncating `open(path, "w")`.
Added the `atomic_yaml_write` import.
- `tests/agent/test_model_metadata.py`: added
`test_write_failure_leaves_existing_cache_intact` — simulates a crash
during the atomic swap and asserts the existing cache survives
byte-for-byte with no stray temp file.
1. `pytest tests/agent/test_model_metadata.py -q` — 98 pass, including
the new crash-safety test.
2. The new test seeds a valid cache, forces the swap step to raise, and
confirms the file is not truncated and no `.cache_*.tmp` is left.
3. `ruff check agent/model_metadata.py` passes.
- [x] I've read the Contributing Guide
- [x] My commit messages follow Conventional Commits (`fix(scope):`, etc.)
- [x] I searched for existing PRs to make sure this isn't a duplicate
- [x] My PR contains **only** changes related to this fix
- [x] I've run the affected tests (`pytest tests/agent/test_model_metadata.py -q`) and they pass
- [x] I've added tests for my changes
- [x] I've tested on my platform: macOS 15 (Darwin 25.5)
- [x] I've updated relevant documentation (README, `docs/`, docstrings) — or N/A
- [x] I've updated `cli-config.yaml.example` if I added/changed config keys — or N/A
- [x] I've updated `CONTRIBUTING.md` or `AGENTS.md` if I changed architecture or workflows — or N/A
- [x] I've considered cross-platform impact (Windows, macOS) — the helper uses os.replace, which is atomic on both
- [x] I've updated tool descriptions/schemas if I changed tool behavior — or N/A
Reapply the non-positive context-length guards onto the post-history-replacement
mainline without carrying any stale branch history. save_context_length() now
refuses to persist length <= 0 (keeping upstream's normalized _context_cache_key),
and get_model_context_length() drops non-positive cache hits at the head of the
invalidation chain (Codex/Kimi/MiniMax/Grok branches become elif) so a poisoned
entry re-resolves instead of short-circuiting to 0.
Refresh of PR #25812; original head d62ed5eb92f057d8c707ba937b44f168f2df0677.
An empty/blank model id reaching get_model_context_length() can't be
meaningfully resolved — and it's worse than a miss: the endpoint
metadata fuzzy matcher ('model in key or key in model') is vacuously
true for "", so it matches an ARBITRARY catalog entry from the live
/v1/models response and returns whatever context length that entry
happens to have, persisting it under a junk '@<base_url>' cache key.
This started failing CI on main when the Nous portal catalog changed:
tests/run_agent/test_primary_runtime_restore.py constructs agents with
model='' against the live portal URL, the arbitrary match now lands on
a 32K entry, and init_agent raises the 64K-floor ValueError
(test_allowed_for_nous_anthropic_messages, red on every PR's slice).
Guard early: a blank model id falls back to DEFAULT_FALLBACK_CONTEXT
immediately, before any cache write or network probe.
Salvaged from #65515 by @whirmill (rebased onto current main; the
guard now sits after the malformed-base_url normalization added since,
and carries an explanatory comment for the fuzzy-match footgun).
Fixes the red slice on #85444, #85452 and every other open PR.
Co-authored-by: whirmill <5079591+whirmill@users.noreply.github.com>
1. Use open_credentialed_url() instead of bare urlopen() in
templates.py apply_template() and probe_existing_customization().
Both send Authorization: Bearer headers; bare urlopen forwards
credentials on cross-origin redirects. The codebase has
open_credentialed_url() in hermes_cli/urllib_security.py that
strips credentials on cross-origin redirects — used by 4 other
modules.
2. Guard unavailable_reason() with the dedup set check before
calling it. The gateway builds a fresh AIAgent per message, so
without this guard unavailable_reason() (which calls _load_config()
→ stat + file read + JSON parse, and _check_local_runtime() →
importlib probes) runs on every gateway turn for an unavailable
provider, even though the warning is deduped after the first.
3. Move INDICATOR_GLYPH from Hindsight's eye emoji to a generic
brain (🧠) in core (agent/memory_provider.py). Hindsight overrides
with its own _HINDSIGHT_GLYPH (👁️) in recall_status() and
_emit_saving_indicator(). Other memory providers no longer inherit
Hindsight's brand mark as the default glyph.
Bundles previously-separate Hindsight/memory PRs into a single review surface:
- opt-in synchronous recall (recall_sync) — recall the injected memory in-turn instead of next-turn prefetch (#5820)
- actionable error when local_embedded runtime is missing — tells the user which package to install (#7718)
- default retain_source to 'hermes' so every stored memory self-identifies its provenance
- offer a starter memory template during hermes memory setup, plus warn before overwriting an already-configured bank
- warn when a configured memory provider reports unavailable (#2765)
- deterministic 'recalled N memories' recall indicator — Hermes itself emits a status line when auto-recall injects memory
- 'saving to memory' retain indicator — emitted the moment a turn is dispatched to the writer
Authored by @benfrank241 (ben.bartholomew@vectorize.io).
Salvaged from PR #74379.
notify_session_compacted closed the old session scope immediately on a
legacy rotating compaction. A compaction can complete while a turn is
still live on the old session; closing then pops the session scope under
the live turn scope, violating the stack's LIFO order — the exact
invariant the rest of the segmentation feature protects.
Now: when the old session has an active turn, set close_pending instead;
that turn's end_turn consumes the flag after its own turn scope pops and
it unregisters from the active-turn table. Sabotage-verified: the new
test fails without the fix.
Continuous gateway sessions keep the Relay session scope open for days;
close-driven export means the session root span and out-of-turn marks
never export until /new or idle-end, and a crash loses the open segment
entirely.
Opt-in segmentation (both defaults OFF => scope lifecycle byte-identical
to today):
gateway.telemetry.session_segments.on_compaction: false
gateway.telemetry.session_segments.max_turns: 0
Rotation closes the current session scope and pushes the next segment
(same session_id attribute, plus hermes.session.segment=N and
segment_reason=compaction|max_turns) ONLY at a turn boundary in
begin_turn — never mid-turn (scope stack is LIFO). Compaction completion
just flags rotate_pending (observer semantics, nothing on the compaction
critical path); legacy rotating compaction closes the orphaned old
session scope so its segment exports. Both native calls ride the
existing bounded scope-op executor: a wedged rotation costs one segment
span, never the agent. Segment bookkeeping advances even on native
failure so a degraded rotation cannot retry every turn.
The diff-apply salvage introduced stale-base revert hunks — the PR was 1246
commits behind main, and its diff for conversation_loop.py and moa_loop.py
silently dropped symbols added after the PR's base (e.g.
_CODEX_ACK_CONTINUATION_NUDGE, _INTERRUPT_SCAFFOLD_MARKER, cache_ttl plumbing,
finalize_turn import, _restore_user_after_reference_handoff).
Restored both files to origin/main and re-applied only the PR's additive
changes: _moa_reference_metrics_for_hook, _system_prompt_for_hooks, the
system_prompt= and moa_references= hook kwargs, _last_reference_metrics
attribute and accessors, and the slot_metrics population in the fan-out path.
Fixes CI ImportError: cannot import name '_CODEX_ACK_CONTINUATION_NUDGE' from
'agent.conversation_loop'.
Follow-up fixes from /hermes-pr-review + /simplify-code on PR #83437:
1. Replace _redact_secrets with agent.redact.redact_sensitive_text(force=True)
— the plugin's 11-pattern list was a strict subset of the 50+ patterns in
agent/redact.py. Secrets like Stripe keys, Google API keys, GitLab tokens,
HuggingFace tokens, DB connection strings, and Telegram bot tokens would
all leak through the plugin's list but are caught by the existing redactor.
Added pk-lf- (Langfuse public key) to _PREFIX_PATTERNS in agent/redact.py.
2. Remove dead 'not isinstance(client, object)' check in on_session_finalize —
always False for any Python value.
3. Fix MoAClient.last_reference_metrics() to call the public
self.chat.completions.last_reference_metrics() instead of reaching into
the private _last_reference_metrics attribute via getattr.
4. Deduplicate _coerce_request_messages call in on_pre_llm_request — pass
pre_coerced=input_messages to _messages_for_langfuse_input to avoid
double-coercion + double _capture_content serialization per API request.
5. Add HERMES_LANGFUSE_CAPTURE to OPTIONAL_ENV_VARS in hermes_cli/config.py
for consistency with the other HERMES_LANGFUSE_* env vars.
6. Fix test_sanitized_mode_redacts_secrets test data — the old samples
('sk-abc...1234', 'sk-ant...1234', 'Authorization: Bearer ***') were too
short to match the regex thresholds and never actually tested redaction.
Updated to realistic-length secrets and changed assertions to check that
the output differs from input (redact_sensitive_text masks rather than
inserting the literal string 'REDACTED').
Salvaged from PR #83437 by @erosika, with adopted fixes from @bgodlin (#81054),
@aldoeliacim (#82332), @nftpoetrist (#42326), @rodboev (#39653), @FnExpress
(#64292, supersedes #32175 by @db-aeon), @Per0-1 (#61166), @NaMinhyeok (#64797),
and @liuhao1024 (#43130).
Widens the bundled Langfuse plugin from 6 to 11 hooks and fixes two
attribution bugs. Also adopts shutdown/atexit lifecycle fixes and composes
8 prior community PRs with interaction-fix follow-ups.
Model attribution: on_pre_llm_request and on_post_llm_call now prefer the
wire value (request body model, response model) over the agent attribute,
which goes stale after /model switch or provider fallback.
Cost total: both cost paths now send a summed total alongside the per-type
breakdown, since Langfuse does not derive calculatedTotalCost from
cost_details keys. Subscription-included routes send no cost keys at all.
New coverage: api_request_error closes failed generations with ERROR level;
on_session_finalize/on_session_end close dangling traces for tool-only and
interrupted turns; subagent_start/subagent_stop trace delegated children as
spans; MoA advisor fan-out emits one generation per advisor priced at the
advisor's own model.
Capture modes: HERMES_LANGFUSE_CAPTURE=metadata|sanitized|full (default
sanitized). Sanitized mode redacts secret patterns before truncation.
Adopted lifecycle fixes: shutdown client at session finalize when
reason=shutdown (not on session rotation); atexit finalizer ends open root
spans for short-lived processes; root context manager exited to prevent
interpreter-teardown TypeError; TOCTOU on _get_langfuse() fixed with lock;
reasoning_content surfaced in traces; system prompt included in generation
input for Anthropic/Codex/Bedrock; SDK v3 update_trace replaces set_trace_io.
Closes#29482, #43129, #72661.
Supersedes #81054, #82332, #42326, #39653, #64292, #32175, #61166, #64797, #43130.
Partially addresses #67544 (capture modes + secret redaction; user_id remains open).
The custom + explicit_base_url branch of resolve_provider_client()
unconditionally rewrote a trailing /anthropic to /v1 via
_to_openai_base_url(), even when api_mode was anthropic_messages. The
Anthropic wrapper then never saw the real /anthropic path, so auxiliary
tasks (title generation, compression, vision, web_extract,
session_search) hit .../v1/chat/completions on a Messages-only endpoint
and failed.
Guard the wrap base on api_mode: for anthropic_messages, pass the raw
/anthropic base to _wrap_if_needed (which builds the Anthropic wrapper),
while the plain OpenAI client keeps the /v1-rewritten base so the
OpenAI-wire fallback (used when the anthropic SDK is unavailable) never
lands on /anthropic/chat/completions.
Refs #16254
Replaces the per-model _model_name_suggests_grok_4_3/_grok_4_6/
_minimax_m3 stale-cache predicates with one generic
_stale_pre_catalog_cache_entry() guard driven by
_PRE_CATALOG_STALE_KEYS. A cached context length is dropped when the
model resolves (longest-key-first, same as step 8) to a listed catalog
key and the cached value is at or below what the old resolution path
could have produced (largest shorter matching catch-all, or the 256K
fallback).
Also covers qwen3.6-plus, grok-4-fast, and grok-4.20 (the models
PR #37684 requested guards for), absorbing that PR.
_model_name_suggests_minimax_m3 is kept for its two non-cache callers
(models.dev underreport guard, cache-control gating in
agent_runtime_helpers).
docs.x.ai (2026-08-12): grok-4.6 is the flagship, 500K context.
Live GET /v1/models lists grok-4.6 at context_length 500000
(no grok-4.6-latest alias).
#84661 landed the catalog. Main already lists native grok-4.6
on the xAI picker. This is only the leftover cache guard
(same pattern as grok-4.3): pre-catalog builds persisted the
grok-4 catch-all (256K).
'' is a substring of every catalog key, so _resolve_endpoint_context_length
with an empty model name "matched" whatever the endpoint listed first —
on the Nous portal that is currently a 32K embedding model, which poisoned
the resolved context length and made AIAgent init fail the 64K minimum.
This is what turned tests/run_agent/test_primary_runtime_restore.py::
TestTryRecoverPrimaryTransport::test_allowed_for_nous_anthropic_messages
red on every PR (CI slice 7/12) after the portal catalog reordered.
Single-model endpoints still resolve with an empty name (unambiguous);
non-empty names keep the substring fuzzy match.
Applies the batch-disposition SALVAGE conditions from #64231: the hook id
moves to the taxonomy transform-family name, and run-all-then-pick-first
dispatch now logs a runtime warning when a valid-but-losing classification
is skipped (the #64714 skipped-transform rule). Chaining semantics are
stated explicitly at the VALID_HOOKS entry, the dispatch helper docstring,
and the hooks.md catalog row and detail section.
Rebased onto current main, where the OpenRouter tool-use 404 is now
handled natively (the bundled demo's exact reason to exist), so the demo
plugin is removed per the standalone-repo policy and every test now uses
a synthetic unclaimed error (fake provider, neutral message, no status
code) that no present or future built-in rule can claim.
classify_api_error is now explicitly Python-plugin-only: VALID_HOOKS
doubles as the shell-hook allow-list, but the shell response parser has
no channel for the classification directive, so shell registrations are
refused at config parse with a warning instead of being silently
ignored (new SHELL_UNSUPPORTED_HOOKS set + regression test).
The hook is documented in the hooks reference as the third
behavior-changing hook, with the full kwargs contract, return shape,
and the Python-only note.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PnMCvi2vXqfs996AjVeF2F
Adds a plugin seam at the top of agent/error_classifier.classify_api_error()
(step 0, before the built-in pipeline) so model-provider plugins can classify
their provider's error quirks without patching core:
- New "classify_api_error" entry in VALID_HOOKS. Callbacks receive the parsed
error context (provider, model, status_code, error_type, error_code,
error_message, error_body, error, approx_tokens, context_length,
num_messages), self-scope on `provider`, and return None to pass or a dict
{"reason": "<FailoverReason name>", ...optional recovery-hint overrides}.
- get_plugin_error_classification() helper mirrors
get_pre_tool_call_block_message(): first valid result wins, invalid dicts
and unknown reasons are skipped, callback exceptions are isolated — a
broken plugin can never break classification. Zero behavior change when no
plugin claims the error (all 179 existing classifier tests pass untouched).
- Bundled reference plugin `openrouter-tool-use-404` (opt-in, like all
bundled standalone plugins) re-implements PR #58451: OpenRouter's
"No endpoints found that support tool use" 404 carries no
_MODEL_NOT_FOUND_PATTERNS signal, so it classifies as unknown/retryable
and the retry loop burns 3-5 attempts on a deterministic rejection.
The plugin classifies it as model_not_found (retryable=False,
should_fallback=True) so the fast-fallback path fires immediately —
demonstrating a waiting core PR converted to a publishable plugin.
Motivation: ~10 open PRs are single-provider error-classification patches
(#58451, #58355, #58502, #58474, #58366, ...). This hook turns that whole
class of contribution into plugin territory.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FWMcB7RPSYUpsXDfBgwjzM
delegate_task gains a control plane: action='list' / 'steer' / 'stop'
let the parent agent see, redirect, and early-stop its own running
subagents mid-flight — the model-facing counterpart of the TUI's
delegation.pause / subagent.interrupt / subagent.steer RPCs.
- action='list': live children of this conversation's spawn tree
(ids, goal, status, running_seconds, accepting_steer, live
transcript path). Ownership is enforced via a _delegate_parent_ref
weakref chain stamped at child build time, so a conversation can
only control its own descendants, never a sibling tree.
- action='steer': queues text into a running child via the existing
steer_subagent() registry path (delivered at the child's next tool
boundary; missed steers surface as missed_steer in the completion).
- action='stop': interrupt_subagent() — child stops at its next
iteration boundary, partial result still re-enters as a completion.
- Spawn dispatch response now includes subagent_ids + control hint.
- Control actions run synchronously (never backgrounded) and bypass
the spawn pause gate and depth limit; they also never consume the
per-turn subagent spawn cap, and remain usable once the cap is hit
(that is when stop matters most).
- Small-model robustness (found live with gpt-5.4-mini on Nous
Portal): tasks=[] alongside goal no longer trips the "Batch mode
requires at least 2 tasks" gate — treated as single-goal.
- CLI display: control calls render as "steer sa-…" / "list" instead
of an empty goal.
Live-tested E2E on Nous Portal (fable-5 + gpt-5.4-mini): full
spawn→list→steer→stop cycle, plus a steer-efficacy run where the
child acked the steer mid-essay and switched topics before finishing.
A type="compaction" item is the server-side stand-in for history that has
already been pruned, and it rides the same codex_reasoning_items sidecar as
per-turn reasoning. e00965a7e taught compaction pruning to filter that
sidecar instead of popping it so checkpoints survive on every retained
message.
The thinking-only sanitizer reaches the same sidecar from the other
direction and asks a coarser question: does any item have type ==
"reasoning"? A commentary turn carrying both a reasoning item and a
checkpoint answers yes, so the whole message is dropped from the wire copy
and the only copy of the checkpoint goes with it. The request then carries
neither the compacted history nor the checkpoint standing in for it.
Extract has_compaction_checkpoint() into agent/native_compaction.py — the
module that owns the concept, and where merge_interim_reasoning_items()
already spelled the same predicate inline — and consult it before the
thinking-only verdict. A reasoning-only carrier is still dropped.
Two reliability gaps from #82777:
1. Rejection matcher required only a field-name mention, so a transient
5xx/timeout whose body echoed the request (which contains
context_management) permanently downgraded native compaction for the
session. Now requires rejection language (unknown/unsupported/invalid/...)
alongside the field name, and when a parsed HTTP status is available,
400 specifically — non-400 statuses never match. Message-only transports
(no status attribute) keep working unchanged.
2. compression.codex_responses_native was coerced with bool(), so the
strings "false"/"off" enabled the feature. Now uses the shared
utils.is_truthy_value helper.
Conversation-loop call site passes api_error.status_code through.
Sabotage-verified: reverting the matcher to field-name-only fails the new
echo and non-400 tests.
Simplify-pass follow-ups on the salvage stack (all guard tests re-run,
mutation-checked):
1. conversation_loop.py: moved `_preflight_compression_blocked = False`
from 9 per-site copies into the restart_with_rebuilt_messages handler
(its single consumer). Besides removing the 9 duplicated blocks, this
fixes a 10th pre-existing retry-loop site (content-filter stall
failover, #32421) that set the flag and broke WITHOUT clearing the
preflight block — a content-filter failover previously restarted with
preflight compression still blocked against the fallback's smaller
window, the same #84733 bug class. The outer-loop empty-response site
keeps its own clear (it never passes through the handler). New AST
guard test_restart_handler_clears_preflight_block pins the hoisted
clear (mutation-checked).
2. agent_runtime_helpers.py: extracted _raw_cache_ttl_from_config() —
prompt_caching_disabled_from_config and configured_cache_ttl were
verbatim copies of the same config read. Added VALID_CACHE_TTLS.
3. prompt_caching.py: added is_qwen_model() next to
ALIBABA_FAMILY_PROVIDERS; effective_cache_ttl and
anthropic_prompt_cache_policy now share both the family set and the
qwen predicate — neither can desync.
4. Guard-test hardening: assert every _try_activate_fallback reference
is a direct `if agent._try_activate_fallback(...):` site, so a future
`activated = ...` form can't silently escape the restart-discipline
guard.
Follow-ups on the salvaged #84782 (webtecnica):
1. conversation_loop.py: the empty-response fallback site sits directly
in the OUTER iteration loop, not the retry loop. The salvaged commit's
`break` there exited the conversation loop and ended the turn without
ever calling the just-activated fallback (caught by CI:
test_empty_response_triggers_fallback_provider). Restored `continue`
(which already re-runs the pre-API preflight at the top of the next
outer iteration) while keeping the `_preflight_compression_blocked`
reset. The other 9 sites are inside the retry loop, where `break` to
the restart_with_rebuilt_messages handler is correct.
2. test_prompt_cache_ttl_propagation.py: made the AST guard loop-aware —
retry-loop sites must break, outer-loop sites must continue (the old
assertion pinned the bug in (1)). Mutation-checked both directions.
3. test_failover_identity.py: added `model` to the SimpleNamespace agent
fixture — _redecorate_prompt_cache_for_provider now reads agent.model
for the per-destination TTL clamp (2 CI failures).
4. prompt_caching.py / agent_runtime_helpers.py: single source of truth
for the alibaba-family provider set — ALIBABA_FAMILY_PROVIDERS lives
in prompt_caching and anthropic_prompt_cache_policy imports it, so the
cache-policy opt-in and the TTL clamp can never desync.
5. auxiliary_client.py: threaded the configured tier into
_replan_synchronous_cache_sections via new configured_cache_ttl()
(no live agent on that path) — the aux half of #84733's report also
stopped regressing 1h to 5m. Guarded by
TestAuxFallbackReplanThreadsTtl (mutation-checked).
6. Dropped the redundant `or "5m"` at the two threaded call sites —
effective_cache_ttl already resolves None to "5m", and the `or`
masked the cache-disabled (None) semantics.
OpenCode identifies clients by request headers, the same way OpenRouter
does. Our opencode-zen and opencode-go profiles never set any, so every
request went out with the OpenAI SDK default "OpenAI/Python x.y.z"
User-Agent and OpenCode had no way to tell the traffic was Hermes Agent.
Two changes:
- Add HTTP-Referer, X-Title, and a HermesAgent User-Agent to both
OpenCode profiles through profile.default_headers, the same path
Fireworks uses. This covers chat_completions, codex_responses,
auxiliary clients, model switches, and the models catalog fetch.
- Merge the same headers in build_anthropic_client for opencode.ai
base URLs. The Anthropic Messages route (Claude on Zen, MiniMax and
Qwen on Go) builds its client there and never sees profile headers.
Verified against the live Go relay with a real key. Both wire formats
return HTTP 200 and the requests now carry X-Title "Hermes Agent",
HTTP-Referer, and User-Agent HermesAgent/0.20.0.
Live verification (gpt-5.6 @ api.openai.com) proved the Responses server
renders NOTHING placed before a replayed compaction checkpoint: a fact
stated in a pre-checkpoint input item is invisible to the model, while the
same item after the checkpoint recalls perfectly. Hermes was replaying the
full pre-checkpoint transcript anyway — dead upload weight, and worse, every
plaintext user ask from before the boundary silently vanished from the
model's view, surviving only inside the opaque server summary. That is the
goal-drift failure mode reported against native compaction sessions.
Codex CLI never hits this because it rebuilds history client-side after
compaction, retaining user messages verbatim under a token budget. This
change is the wire-level equivalent: when a replayed checkpoint is present,
_chat_messages_to_responses_input restructures the input as
[newest checkpoint run] + [retained pre-checkpoint user messages,
newest-first within a 64K-token budget] + [post-checkpoint tail]
Histories without a checkpoint are returned unchanged, so non-native
sessions see a byte-identical wire.
New desktop_ui tool: the agent proposes an MCP server (install/enable/
authorize + a one-line reason) and blocks on mcp.setup.request until the
renderer's consent card answers mcp.setup.respond with the outcome
(installed/enabled/authorized/declined/unanswered/error). Same lifecycle
as clarify: 10-min timeout, allow_expired late answers, tool lifecycle
events forced on so the card mounts even with tool progress off. Desktop
prompt hint steers the model to the tool instead of hand-editing config;
every other surface keeps the schema out and is pointed at hermes mcp
install.
Ben Barclay
kshitij
Teknium
sjungwon03
Xie
fangliquanflq
686f6c61
iso2kx
Erosika
luoxiao6645
sasquatch9818
Omar Baradei
whirmill
Ben
Victor Kyriazakos
Nikita Barkov
Julientalbot
webdevtodayjason
Drexuxux
webtecnica
Adolanium
Brooklyn Nicholson