Run the expensive-model warning for explicit startup `-m` / `--provider`
overrides before the chat loop starts, and fail closed for non-interactive
invocations that select an expensive or known-confusing model.
Also classify Nous paid-model 404s that say credits are required as billing
exhaustion so they fail fast with billing guidance.
Tested:
- scripts/run_tests.sh tests/hermes_cli/test_cli_startup_model_cost_guard.py tests/hermes_cli/test_model_cost_guard.py tests/agent/test_error_classifier.py -- --tb=short -q
Regression tests salvaged from PR #70522 by @JoaoMarcos44. The Qwen flat
cached_tokens behavior is provided by the shared top-level fallback from
PR #66105 (@mehmetkr-31); the codex cache_write_tokens read landed in the
previous commit.
Salvaged from PR #85702 by @JoaoMarcos44, composed onto the mapping-safe
_usage_get reads (PR #74591 by @RelaxJonh) and the flat cached_tokens /
Anthropic-name fallbacks (PRs #66105, #52571):
- cache-write precedence in the chat_completions branch:
details.cache_write_tokens > details.cache_creation_input_tokens >
usage.cache_creation_input_tokens > usage.cache_write_tokens
- codex_responses branch reads details.cache_write_tokens (GPT-5.6+
documented name) with cache_creation_tokens fallback (from PR #70522)
- _usage_count(): clamp malformed negative counters to 0
- all reads in every branch are mapping-safe via _usage_get
When the Responses API returns usage as a plain dict (e.g. from a
middleware or proxy that deserialises JSON to dict instead of a typed
SDK object), normalize_usage() used getattr() exclusively, which
silently returned 0 for every field on a dict.
Add _usage_get() helper that reads via .get() for dicts and getattr()
for attribute-style objects. All accessor sites in normalize_usage()
now use this helper, so token counts and cost are correct regardless
of the usage object's type.
Regression tests: two new tests feed the same payload as both a dict
and a SimpleNamespace through the codex_responses and
chat_completions branches, asserting identical output and non-zero
values.
Kimi/Moonshot's native API (api.moonshot.cn / .ai) reports context-cache hits
as a top-level ``usage.cached_tokens``. The chat-completions branch of
normalize_usage() walks a fallback chain of
prompt_tokens_details.cached_tokens -> cache_read_input_tokens ->
prompt_cache_hit_tokens; none of those names match, so direct Kimi sessions
normalized to cache_read_tokens=0. The hits were invisible in accounting and
the cached prefix was billed at the full input rate.
Appended as the last link in that chain, so it only fills a genuine zero and
cannot override a provider that reports the nested OpenAI shape or DeepSeek's
prompt_cache_hit_tokens.
Rebuilt on current main rather than rebased — the branch was ~3400 commits
behind. The DeepSeek half of the original branch is dropped: 03c0b00f4
(#65678) landed prompt_cache_hit_tokens on main, so this is Kimi-only as the
review asked. The scripts/release.py addition to the frozen LEGACY_AUTHOR_MAP
is dropped too; contributors/emails/mehmet.kar@std.yildiz.edu.tr already
exists on main.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Azure Foundry's OpenAI-compatible Responses surface rejects the post-tool
follow-up payload with HTTP 400 `invalid_payload` when a replayed encrypted
`reasoning` item is sent alongside `function_call` / `function_call_output`.
The initial function-call request and ordinary multi-turn continuity are both
accepted, so the failure only appears after the first tool executes.
Detect the Foundry endpoint in `ResponsesApiTransport.build_kwargs` and drop
only the encrypted reasoning replay on that follow-up turn, leaving
function_call / function_call_output continuity intact.
Salvage of #59981, rebuilt on current main. Same root cause and fix direction
as the original, which was correct; this version resolves three defects:
- No `chat_completion_helpers.py` change. main already forwards `provider`
and `base_url` to the Responses transport, so the original's re-added
arguments produced `SyntaxError: keyword argument repeated: provider` on
merge. Dropping the hunk removed the syntax error and the conflict.
- Host matching uses `utils.base_url_host_matches`, not a substring test.
`".services.ai.azure.com" in base_url` also matches URLs carrying the
domain in a path or query segment, which would silently disable reasoning
replay on an unrelated provider.
- The post-tool predicate tests the trailing messages, not the whole history.
Scanning for any tool call plus any tool result made it sticky: one tool
call early in a conversation suppressed reasoning on every later turn.
- Tool calls pair on `call_id` as well as `id`. Responses histories carry the
function call id in `call_id` while `id` holds the response item id
(`fc_...`). Identity is resolved via the converter's own
`_split_responses_tool_id`, covering composite `"call_x|fc_y"` ids and bare
`fc_` ids on both sides of the pairing.
Tests: 27 cases across the transport and the live `build_api_kwargs` bridge,
including six parametrized tool-call id shapes, non-Foundry host lookalikes,
the sticky-history guard, parallel tool results, and an unpaired tool result.
Each guard was confirmed to catch its defect by reverting the fix.
Verified with `scripts/run_tests.sh tests/agent/ tests/run_agent/`:
532 files, 5602 tests passed, 0 failed.
Not verified against a live Azure Foundry endpoint — no credentials. The
original HTTP 400 reproduction and post-fix Foundry Project / Azure Container
Apps harness runs are @AshuJoshi's, from #59981. This change is verified at
the payload-construction layer only.
Closes#59981.
Co-authored-by: Ashu Joshi <AshuJoshi@users.noreply.github.com>
- Cold force_refresh (fresh CLI process, e.g. hermes config refresh)
now hydrates the memory cache from disk before fetching, so the
conditional GET actually fires on the flow the feature was built for
instead of silently re-downloading the full ~2 MB registry
(empirically probed: If-None-Match sent, 304 serves disk data).
- Conditional-GET decision is passed in explicitly
(_fetch_models_dev_from_network(conditional=...)) by callers holding
the fetch lock, removing the hidden read of module globals inside
the fetch; the background worker now fetches INSIDE the lock,
symmetric with foreground (true singleflight — no concurrent
double-download, no fetching against mid-commit etag state).
- Corrupt disk cache is QUARANTINED (renamed to .json.corrupt) rather
than left in place: rejection becomes a one-time event instead of a
re-read + re-parse + warning + unlink on every hot-path call while
offline (probed: 1 warning across 5 calls, was 5).
- Dropped the dead _DEFAULT_MODELS_DEV_URL constant; module and
function docstrings updated to match the servable-cache conditional
semantics.
- Conditional GET now requires a servable in-memory registry: an
If-None-Match sent while holding no cache invited a 304 against
nothing, permanently serving {} with a blocking foreground fetch on
every call (the exact #35838 class this PR fixes). Empirically
repro'd and verified fixed (corrupt cache + stale sidecar: was 3
calls -> {} forever; now 1 unconditional fetch -> real data).
- ETag persists atomically WITH the cache body via
_commit_registry -> _save_disk_cache(data, etag), wiring up the
previously-dead etag param; the sidecar can no longer get ahead of
the registry it vouches for. _save_etag now uses
utils.atomic_write_text (unique tempnames + fsync) instead of a
hand-rolled fixed-name .tmp replace.
- Corrupt/unreadable disk cache clears the ETag sidecar so the
refetch is unconditional; _confirm_cache_not_modified keeps a
defense-in-depth guard (clear sidecar + arm backoff) should a 304
ever land on an empty registry.
- allow_network=True paths use the zero-arg fetch_models_dev() call
shape at all sites (was 1 of 5) — ~46 test sites monkeypatch it
with zero-arg lambdas; the unconditional kwarg broke
test_xiaomi_provider (verified fail->pass).
- _get_models_dev_url falls back to the MODELS_DEV_URL module global
(not the constant) so existing patch sites keep working.
- Tests: replaced two mock-riddled corrupt-cache tests with real
tmp_path file tests; added regression tests for the 304/empty-cache
loop, sidecar clearing, and conditional-GET gating.
Harden the models.dev catalog refresh path (#35838) with three missing
pieces:
1. ETag conditional GET — every network request sends If-None-Match
with the last-known ETag (persisted alongside the cache file). A 304
Not Modified re-confirms the existing cache without re-downloading
the full ~2 MB registry. This makes the 4-hour TTL effectively free
to maintain.
2. No-network-on-hot-paths invariant — allow_network=False is now the
default for every query function called on the conversation hot path:
get_model_capabilities, get_model_info, lookup_models_dev_context,
_get_provider_models. These are called during vision routing, image
routing, cost-guard checks, and context-length resolution on every
turn — they must never block on the network. Interactive flows
(model picker, model switch) explicitly pass allow_network=True.
3. Mirror URL override — models_dev.url in config.yaml lets deployments
point at a self-hosted mirror without code changes. Follows the same
pattern as model_catalog.url.
Additional hardening:
- Cache TTL bumped from 1h to 4h (ETag makes refresh cheap)
- Corrupt/empty disk cache is rejected with a warning instead of being
served as {} and silently breaking provider/model resolution
- _validate_registry() guards against non-dict and empty-dict payloads
Fixes#35838
- Single-source the included note as _INCLUDED_NOTE and attach it at
BOTH status='included' sites (the zero-amount pricing-entry branch
previously returned the same status with no note).
- Docstring/comment precision on format_cost_label: the fallback
triggers on 4dp ROUNDING to 0.0000 (banker's rounding includes the
exact $0.00005 boundary), not truncation; note why the rendered-label
guard beats a naive Decimal threshold.
- Tests: replaced a dead assertion with the exact-boundary case
($0.00005), fixed an overclaiming comment, aligned the terminal
cost column.
- Insights formatters now route aggregate estimated cost through the
shared format_cost_label() instead of hardcoded 2dp — a sub-cent
aggregate (one cheap DeepSeek session, ~$0.0046) no longer renders
'Estimated: ~$0.00', the exact bug class this PR fixes (#79220).
- format_cost_label: positive amounts below $0.00005 render '~$<0.0001'
instead of the zero-looking '~$0.0000' 4dp truncation artifact.
- Renamed _format_cost_label -> format_cost_label (now a cross-module
shared helper).
- Tests: renamed test_gateway_format_hides_cost ->
test_gateway_format_hides_cache_details and
test_no_cost_section_when_all_zero ->
test_unknown_bucket_shown_for_costless_session (names contradicted
behavior); restored a real assertion in the custom-models test that
had been weakened to a comment; added sub-cent-aggregate and 4dp-floor
contract tests (mutation-checked).
Three cost-display honesty fixes:
1. Sub-cent cost label rendering (#79220) — _format_cost_label() scales
precision to magnitude: zero renders as '$0.00', sub-cent (< $0.01)
renders at 4 decimal places (e.g. '~$0.0046'), normal costs keep 2dp.
This fixes the bug where DeepSeek per-turn costs of $0.004640 rendered
as '~$0.00' despite amount_usd carrying full Decimal precision.
2. Cost bucket surfacing (#77223) — insights format_terminal and
format_gateway now display three cost buckets: estimated (with dollar
figure), included (session count, labeled 'subscription — no provider
invoice'), and unknown (session count, labeled 'no pricing data').
Previously, included and unknown sessions silently collapsed to $0 in
the aggregate view, hiding 315 of 473 sessions in the reporter's DB.
3. Subscription-included cost notes — estimate_usage_cost now attaches a
'subscription-included; no provider invoice for usage' note to
CostResult for subscription-included routes (openai-codex), so
consumers can distinguish 'free because subscription' from 'free
because $0 pricing'.
Fixes#79220Fixes#77223
- Deleted the id(cfg)-keyed _OVERRIDE_CACHE layer: id() is unique only
among live objects, so a config reload could serve stale overrides
forever when CPython reuses the freed dict's address. The upstream
load_config_readonly is already (mtime,size)-cached (~1 stat/hit), so
the local layer was redundant state with a correctness risk.
- _override_to_catalog_shape returns (patch, vision) instead of
smuggling an in-band _vision_override sentinel key through the merged
dict; removed the two dead call-site pops.
- _find_model_entry gains the :cloud/-cloud suffix fallback that
lookup_models_dev_context already had, so 'catalog hit' means the
same thing to every consumer — a suffix-keyed model (kimi-k2.6:cloud)
now counts as KNOWN and keeps its catalog capabilities instead of
being displaced by a fill-gap _default (mutation-checked contract
test added).
- get_model_info's unknown-model override path seeds the same safe
defaults as get_model_capabilities (200K ctx, tools on, 8192 out),
so a partial override no longer yields ctx=0/tools-off on that path
(contract test added); the DEFAULT_CONFIG defaults claim is now true
for both paths.
- Activated the previously-dead _MODELS_DEV_TO_PROVIDER reverse map
(lazily built, many-to-one aware) and used it in
_provider_override_section instead of a per-call linear scan.
Review follow-ups on the model_overrides feature:
- ONE canonical override schema everywhere. get_model_info previously
merged the override dict raw into the models.dev catalog shape
({**raw, **override}), so the documented context_window/supports_*
keys silently did nothing on that path (cost guard, inventory) while
working in capabilities/context paths — same config key, two
incompatible schemas. Overrides are now translated into the catalog
shape at the get_model_info boundary (_override_to_catalog_shape),
and sub-dicts (limit, modalities) are MERGED, not clobbered — an
override setting only context_window no longer wipes the catalog's
limit.output.
- _default is now a FILL-GAP default, not an override: it applies only
to models the catalog does not know (the #8731/#84482 self-unblock
path) and never displaces catalog data. A
_default: {context_window: 128000} can no longer clamp every model
of a provider. Explicit per-provider+model entries keep their
win-over-catalog semantics.
- Early-chain _override_context_window (model_metadata step 0b) is
explicit-only, so a _default can never preempt custom_providers
per-model settings or live probes; fill-gap defaults apply at the
lookup_models_dev_context catalog-miss boundary (step 5f) instead.
This fixes the precedence inversion where a provider/global _default
silently overrode an explicit per-endpoint per-model context_length.
- Provider keys accept BOTH id spaces (Hermes id and models.dev id:
copilot/github-copilot both work) and model ids match
case-insensitively, mirroring catalog lookup.
- Malformed override values (context_window: '512k') log a one-shot
warning instead of being silently swallowed.
- DEFAULT_CONFIG comment: removed the false family/dated-snapshot
inheritance claim, documented the recognized field list, fill-gap
semantics, and the id-space rule.
- Tests: rewritten for the new contracts (fill-gap invariants,
dual-id-space keys, sub-dict merge preservation, one-shot warning);
added a real-config-yaml e2e plumbing test (mutation-checked: fails
when the config key wiring is broken).
Add a unified model_overrides config section that lets users manually
declare context_window, max_output_tokens, capabilities, cost, and
family for any provider+model — winning over models.dev, OpenRouter, and
hardcoded defaults.
Resolution order (first hit wins):
1. model_overrides.<provider>.<model_id> (per-provider+model)
2. model_overrides.<provider>._default (per-provider default)
3. model_overrides._default (global default)
4. Normal catalog resolution
Key subtlety: an unknown model id (not in the
catalog) derives base metadata from sensible defaults before patching,
so overriding a model the catalog doesn't know yet is the supported
self-unblock path. This is exactly the #84482 scenario (Upstage
solar-pro4/syn-pro wrong context) and the #8731 scenario (custom/local
models with manual capability declaration).
Wired into:
- get_model_capabilities() — patches capability fields; unknown models
get safe defaults (tools on, vision/reasoning off) before patching
- lookup_models_dev_context() — context_window override, checked before
catalog lookup so it works even for providers not in PROVIDER_TO_MODELS_DEV
- get_model_info() — merges override dict onto catalog entry (shallow
merge); for unknown models, the override is the sole source of metadata
- get_model_context_length() — step 0b in the resolution pipeline,
before custom_providers (0c) and before any network probe
Config example:
model_overrides:
upstage:
solar-pro4:
context_window: 524288
syn-pro:
context_window: 65536
custom:my-local-vllm:
my-llava-model:
context_window: 8192
supports_vision: true
supports_reasoning: false
supports_tools: true
_default:
context_window: 128000
Fixes#8731Fixes#84482
Refs #47247
Unknown hosts (e.g. gateway.example.com) no longer get /anthropic→/v1;
use a real dual-surface MiniMax base for the rewrite assertions and add
a case proving Anthropic-only gateways keep their path on the OpenAI wire.
Substring matching over the whole URL let a path containing
'api.minimax' false-positive an Anthropic-only gateway into the
/anthropic→/v1 rewrite. Parse the host and match exact-domain /
subdomain suffixes (plus the api.minimax.* prefix family) instead.
`_try_anthropic()` applies the configured `model.base_url` only when
`_is_anthropic_compatible_host()` trusts it, but that check accepted only the
literal `api.anthropic.com` host. Anthropic-compatible gateways that expose the
native Messages protocol under a `/anthropic` path suffix (MiniMax, Zhipu GLM,
LiteLLM-style relays, self-hosted proxies) were rejected, so every auxiliary
call (title generation, memory extraction, vision, reflection) and the
`provider: anthropic` fallback chain discarded the configured base_url and fell
back to `https://api.anthropic.com`. That diverges from the primary path, which
already trusts the `/anthropic` suffix via
`runtime_provider._detect_api_mode_for_url`, and fails outright when the gateway
(not Anthropic) holds the credentials.
Accept `/anthropic` and `/anthropic/v1` suffixed URLs in
`_is_anthropic_compatible_host()`, matching the primary-path convention and
`_wrap_if_needed`. A bare non-Anthropic base_url (e.g. `openrouter.ai/api/v1`
left on `provider: anthropic`) still returns False, preserving the #52608 guard.
save_context_length() and _invalidate_cached_context_length() did an
unguarded read-modify-write into $HERMES_HOME/context_length_cache.yaml.
The plain `open(path, "w")` truncates the file before the dump runs. If
the process is killed mid-dump, the file is left empty or partial. The
next _load_context_cache() swallows the YAML error and returns {} —
silently wiping every persisted context length. A concurrent process
reading between truncate and dump-complete also sees a torn file.
After the cache is lost, every model re-probes the network, and when a
probe fails it falls back to the generic 256K default — so a user on a
1M-window model ends up with a wrong, short context window.
Hermes routinely runs several processes against one shared $HERMES_HOME
(a cron agent plus an interactive session, multiple gateway sessions),
so this is hit in normal use.
Switch both writers to the existing utils.atomic_yaml_write helper
(temp file + fsync + os.replace, symlink- and mode-preserving). The real
file is only ever swapped from a fully written temp file, so an
interrupted write leaves the previous cache intact and readers never see
a partial file. Matches the atomic-write pattern already used for
auth.json, config.yaml, and other persisted state.
Makes the persistent model context-length cache write crash-safe. The
old non-atomic write could truncate or wipe the entire cache on an
interrupted or concurrent write, which then forces models onto the wrong
fallback context window. The fix routes both cache writers through the
repo's atomic temp-file + os.replace helper.
N/A
- [x] 🐛 Bug fix (non-breaking change that fixes an issue)
- [ ] ✨ New feature (non-breaking change that adds functionality)
- [ ] 🔒 Security fix
- [ ] 📝 Documentation update
- [ ] ✅ Tests (adding or improving test coverage)
- [ ] ♻️ Refactor (no behavior change)
- [ ] 🎯 New skill (bundled or hub)
- `agent/model_metadata.py`: `save_context_length()` and
`_invalidate_cached_context_length()` now write via
`utils.atomic_yaml_write` instead of a truncating `open(path, "w")`.
Added the `atomic_yaml_write` import.
- `tests/agent/test_model_metadata.py`: added
`test_write_failure_leaves_existing_cache_intact` — simulates a crash
during the atomic swap and asserts the existing cache survives
byte-for-byte with no stray temp file.
1. `pytest tests/agent/test_model_metadata.py -q` — 98 pass, including
the new crash-safety test.
2. The new test seeds a valid cache, forces the swap step to raise, and
confirms the file is not truncated and no `.cache_*.tmp` is left.
3. `ruff check agent/model_metadata.py` passes.
- [x] I've read the Contributing Guide
- [x] My commit messages follow Conventional Commits (`fix(scope):`, etc.)
- [x] I searched for existing PRs to make sure this isn't a duplicate
- [x] My PR contains **only** changes related to this fix
- [x] I've run the affected tests (`pytest tests/agent/test_model_metadata.py -q`) and they pass
- [x] I've added tests for my changes
- [x] I've tested on my platform: macOS 15 (Darwin 25.5)
- [x] I've updated relevant documentation (README, `docs/`, docstrings) — or N/A
- [x] I've updated `cli-config.yaml.example` if I added/changed config keys — or N/A
- [x] I've updated `CONTRIBUTING.md` or `AGENTS.md` if I changed architecture or workflows — or N/A
- [x] I've considered cross-platform impact (Windows, macOS) — the helper uses os.replace, which is atomic on both
- [x] I've updated tool descriptions/schemas if I changed tool behavior — or N/A
An empty/blank model id reaching get_model_context_length() can't be
meaningfully resolved — and it's worse than a miss: the endpoint
metadata fuzzy matcher ('model in key or key in model') is vacuously
true for "", so it matches an ARBITRARY catalog entry from the live
/v1/models response and returns whatever context length that entry
happens to have, persisting it under a junk '@<base_url>' cache key.
This started failing CI on main when the Nous portal catalog changed:
tests/run_agent/test_primary_runtime_restore.py constructs agents with
model='' against the live portal URL, the arbitrary match now lands on
a 32K entry, and init_agent raises the 64K-floor ValueError
(test_allowed_for_nous_anthropic_messages, red on every PR's slice).
Guard early: a blank model id falls back to DEFAULT_FALLBACK_CONTEXT
immediately, before any cache write or network probe.
Salvaged from #65515 by @whirmill (rebased onto current main; the
guard now sits after the malformed-base_url normalization added since,
and carries an explanatory comment for the fuzzy-match footgun).
Fixes the red slice on #85444, #85452 and every other open PR.
Co-authored-by: whirmill <5079591+whirmill@users.noreply.github.com>
1. Use open_credentialed_url() instead of bare urlopen() in
templates.py apply_template() and probe_existing_customization().
Both send Authorization: Bearer headers; bare urlopen forwards
credentials on cross-origin redirects. The codebase has
open_credentialed_url() in hermes_cli/urllib_security.py that
strips credentials on cross-origin redirects — used by 4 other
modules.
2. Guard unavailable_reason() with the dedup set check before
calling it. The gateway builds a fresh AIAgent per message, so
without this guard unavailable_reason() (which calls _load_config()
→ stat + file read + JSON parse, and _check_local_runtime() →
importlib probes) runs on every gateway turn for an unavailable
provider, even though the warning is deduped after the first.
3. Move INDICATOR_GLYPH from Hindsight's eye emoji to a generic
brain (🧠) in core (agent/memory_provider.py). Hindsight overrides
with its own _HINDSIGHT_GLYPH (👁️) in recall_status() and
_emit_saving_indicator(). Other memory providers no longer inherit
Hindsight's brand mark as the default glyph.
Bundles previously-separate Hindsight/memory PRs into a single review surface:
- opt-in synchronous recall (recall_sync) — recall the injected memory in-turn instead of next-turn prefetch (#5820)
- actionable error when local_embedded runtime is missing — tells the user which package to install (#7718)
- default retain_source to 'hermes' so every stored memory self-identifies its provenance
- offer a starter memory template during hermes memory setup, plus warn before overwriting an already-configured bank
- warn when a configured memory provider reports unavailable (#2765)
- deterministic 'recalled N memories' recall indicator — Hermes itself emits a status line when auto-recall injects memory
- 'saving to memory' retain indicator — emitted the moment a turn is dispatched to the writer
Authored by @benfrank241 (ben.bartholomew@vectorize.io).
Salvaged from PR #74379.
notify_session_compacted closed the old session scope immediately on a
legacy rotating compaction. A compaction can complete while a turn is
still live on the old session; closing then pops the session scope under
the live turn scope, violating the stack's LIFO order — the exact
invariant the rest of the segmentation feature protects.
Now: when the old session has an active turn, set close_pending instead;
that turn's end_turn consumes the flag after its own turn scope pops and
it unregisters from the active-turn table. Sabotage-verified: the new
test fails without the fix.
Continuous gateway sessions keep the Relay session scope open for days;
close-driven export means the session root span and out-of-turn marks
never export until /new or idle-end, and a crash loses the open segment
entirely.
Opt-in segmentation (both defaults OFF => scope lifecycle byte-identical
to today):
gateway.telemetry.session_segments.on_compaction: false
gateway.telemetry.session_segments.max_turns: 0
Rotation closes the current session scope and pushes the next segment
(same session_id attribute, plus hermes.session.segment=N and
segment_reason=compaction|max_turns) ONLY at a turn boundary in
begin_turn — never mid-turn (scope stack is LIFO). Compaction completion
just flags rotate_pending (observer semantics, nothing on the compaction
critical path); legacy rotating compaction closes the orphaned old
session scope so its segment exports. Both native calls ride the
existing bounded scope-op executor: a wedged rotation costs one segment
span, never the agent. Segment bookkeeping advances even on native
failure so a degraded rotation cannot retry every turn.
Salvaged from PR #83437 by @erosika, with adopted fixes from @bgodlin (#81054),
@aldoeliacim (#82332), @nftpoetrist (#42326), @rodboev (#39653), @FnExpress
(#64292, supersedes #32175 by @db-aeon), @Per0-1 (#61166), @NaMinhyeok (#64797),
and @liuhao1024 (#43130).
Widens the bundled Langfuse plugin from 6 to 11 hooks and fixes two
attribution bugs. Also adopts shutdown/atexit lifecycle fixes and composes
8 prior community PRs with interaction-fix follow-ups.
Model attribution: on_pre_llm_request and on_post_llm_call now prefer the
wire value (request body model, response model) over the agent attribute,
which goes stale after /model switch or provider fallback.
Cost total: both cost paths now send a summed total alongside the per-type
breakdown, since Langfuse does not derive calculatedTotalCost from
cost_details keys. Subscription-included routes send no cost keys at all.
New coverage: api_request_error closes failed generations with ERROR level;
on_session_finalize/on_session_end close dangling traces for tool-only and
interrupted turns; subagent_start/subagent_stop trace delegated children as
spans; MoA advisor fan-out emits one generation per advisor priced at the
advisor's own model.
Capture modes: HERMES_LANGFUSE_CAPTURE=metadata|sanitized|full (default
sanitized). Sanitized mode redacts secret patterns before truncation.
Adopted lifecycle fixes: shutdown client at session finalize when
reason=shutdown (not on session rotation); atexit finalizer ends open root
spans for short-lived processes; root context manager exited to prevent
interpreter-teardown TypeError; TOCTOU on _get_langfuse() fixed with lock;
reasoning_content surfaced in traces; system prompt included in generation
input for Anthropic/Codex/Bedrock; SDK v3 update_trace replaces set_trace_io.
Closes#29482, #43129, #72661.
Supersedes #81054, #82332, #42326, #39653, #64292, #32175, #61166, #64797, #43130.
Partially addresses #67544 (capture modes + secret redaction; user_id remains open).
The custom + explicit_base_url branch of resolve_provider_client()
unconditionally rewrote a trailing /anthropic to /v1 via
_to_openai_base_url(), even when api_mode was anthropic_messages. The
Anthropic wrapper then never saw the real /anthropic path, so auxiliary
tasks (title generation, compression, vision, web_extract,
session_search) hit .../v1/chat/completions on a Messages-only endpoint
and failed.
Guard the wrap base on api_mode: for anthropic_messages, pass the raw
/anthropic base to _wrap_if_needed (which builds the Anthropic wrapper),
while the plain OpenAI client keeps the /v1-rewritten base so the
OpenAI-wire fallback (used when the anthropic SDK is unavailable) never
lands on /anthropic/chat/completions.
Refs #16254
Replaces the per-model _model_name_suggests_grok_4_3/_grok_4_6/
_minimax_m3 stale-cache predicates with one generic
_stale_pre_catalog_cache_entry() guard driven by
_PRE_CATALOG_STALE_KEYS. A cached context length is dropped when the
model resolves (longest-key-first, same as step 8) to a listed catalog
key and the cached value is at or below what the old resolution path
could have produced (largest shorter matching catch-all, or the 256K
fallback).
Also covers qwen3.6-plus, grok-4-fast, and grok-4.20 (the models
PR #37684 requested guards for), absorbing that PR.
_model_name_suggests_minimax_m3 is kept for its two non-cache callers
(models.dev underreport guard, cache-control gating in
agent_runtime_helpers).
docs.x.ai (2026-08-12): grok-4.6 is the flagship, 500K context.
Live GET /v1/models lists grok-4.6 at context_length 500000
(no grok-4.6-latest alias).
#84661 landed the catalog. Main already lists native grok-4.6
on the xAI picker. This is only the leftover cache guard
(same pattern as grok-4.3): pre-catalog builds persisted the
grok-4 catch-all (256K).
'' is a substring of every catalog key, so _resolve_endpoint_context_length
with an empty model name "matched" whatever the endpoint listed first —
on the Nous portal that is currently a 32K embedding model, which poisoned
the resolved context length and made AIAgent init fail the 64K minimum.
This is what turned tests/run_agent/test_primary_runtime_restore.py::
TestTryRecoverPrimaryTransport::test_allowed_for_nous_anthropic_messages
red on every PR (CI slice 7/12) after the portal catalog reordered.
Single-model endpoints still resolve with an empty name (unambiguous);
non-empty names keep the substring fuzzy match.
Applies the batch-disposition SALVAGE conditions from #64231: the hook id
moves to the taxonomy transform-family name, and run-all-then-pick-first
dispatch now logs a runtime warning when a valid-but-losing classification
is skipped (the #64714 skipped-transform rule). Chaining semantics are
stated explicitly at the VALID_HOOKS entry, the dispatch helper docstring,
and the hooks.md catalog row and detail section.
Rebased onto current main, where the OpenRouter tool-use 404 is now
handled natively (the bundled demo's exact reason to exist), so the demo
plugin is removed per the standalone-repo policy and every test now uses
a synthetic unclaimed error (fake provider, neutral message, no status
code) that no present or future built-in rule can claim.
classify_api_error is now explicitly Python-plugin-only: VALID_HOOKS
doubles as the shell-hook allow-list, but the shell response parser has
no channel for the classification directive, so shell registrations are
refused at config parse with a warning instead of being silently
ignored (new SHELL_UNSUPPORTED_HOOKS set + regression test).
The hook is documented in the hooks reference as the third
behavior-changing hook, with the full kwargs contract, return shape,
and the Python-only note.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PnMCvi2vXqfs996AjVeF2F
Simplify-pass follow-ups on the salvage stack (all guard tests re-run,
mutation-checked):
1. conversation_loop.py: moved `_preflight_compression_blocked = False`
from 9 per-site copies into the restart_with_rebuilt_messages handler
(its single consumer). Besides removing the 9 duplicated blocks, this
fixes a 10th pre-existing retry-loop site (content-filter stall
failover, #32421) that set the flag and broke WITHOUT clearing the
preflight block — a content-filter failover previously restarted with
preflight compression still blocked against the fallback's smaller
window, the same #84733 bug class. The outer-loop empty-response site
keeps its own clear (it never passes through the handler). New AST
guard test_restart_handler_clears_preflight_block pins the hoisted
clear (mutation-checked).
2. agent_runtime_helpers.py: extracted _raw_cache_ttl_from_config() —
prompt_caching_disabled_from_config and configured_cache_ttl were
verbatim copies of the same config read. Added VALID_CACHE_TTLS.
3. prompt_caching.py: added is_qwen_model() next to
ALIBABA_FAMILY_PROVIDERS; effective_cache_ttl and
anthropic_prompt_cache_policy now share both the family set and the
qwen predicate — neither can desync.
4. Guard-test hardening: assert every _try_activate_fallback reference
is a direct `if agent._try_activate_fallback(...):` site, so a future
`activated = ...` form can't silently escape the restart-discipline
guard.
Follow-ups on the salvaged #84782 (webtecnica):
1. conversation_loop.py: the empty-response fallback site sits directly
in the OUTER iteration loop, not the retry loop. The salvaged commit's
`break` there exited the conversation loop and ended the turn without
ever calling the just-activated fallback (caught by CI:
test_empty_response_triggers_fallback_provider). Restored `continue`
(which already re-runs the pre-API preflight at the top of the next
outer iteration) while keeping the `_preflight_compression_blocked`
reset. The other 9 sites are inside the retry loop, where `break` to
the restart_with_rebuilt_messages handler is correct.
2. test_prompt_cache_ttl_propagation.py: made the AST guard loop-aware —
retry-loop sites must break, outer-loop sites must continue (the old
assertion pinned the bug in (1)). Mutation-checked both directions.
3. test_failover_identity.py: added `model` to the SimpleNamespace agent
fixture — _redecorate_prompt_cache_for_provider now reads agent.model
for the per-destination TTL clamp (2 CI failures).
4. prompt_caching.py / agent_runtime_helpers.py: single source of truth
for the alibaba-family provider set — ALIBABA_FAMILY_PROVIDERS lives
in prompt_caching and anthropic_prompt_cache_policy imports it, so the
cache-policy opt-in and the TTL clamp can never desync.
5. auxiliary_client.py: threaded the configured tier into
_replan_synchronous_cache_sections via new configured_cache_ttl()
(no live agent on that path) — the aux half of #84733's report also
stopped regressing 1h to 5m. Guarded by
TestAuxFallbackReplanThreadsTtl (mutation-checked).
6. Dropped the redundant `or "5m"` at the two threaded call sites —
effective_cache_ttl already resolves None to "5m", and the `or`
masked the cache-disabled (None) semantics.
OpenCode identifies clients by request headers, the same way OpenRouter
does. Our opencode-zen and opencode-go profiles never set any, so every
request went out with the OpenAI SDK default "OpenAI/Python x.y.z"
User-Agent and OpenCode had no way to tell the traffic was Hermes Agent.
Two changes:
- Add HTTP-Referer, X-Title, and a HermesAgent User-Agent to both
OpenCode profiles through profile.default_headers, the same path
Fireworks uses. This covers chat_completions, codex_responses,
auxiliary clients, model switches, and the models catalog fetch.
- Merge the same headers in build_anthropic_client for opencode.ai
base URLs. The Anthropic Messages route (Claude on Zen, MiniMax and
Qwen on Go) builds its client there and never sees profile headers.
Verified against the live Go relay with a real key. Both wire formats
return HTTP 200 and the requests now carry X-Title "Hermes Agent",
HTTP-Referer, and User-Agent HermesAgent/0.20.0.
Address rewinds/edits via SQLite messages.id (truncate_before_row_id)
instead of shifting user ordinals. Resolve against in-memory stamps,
then durable session history when live turns drop _row_id; refuse
unknown durable targets with 4018 (no ordinal fallback) and 4030 on
ordinal/row_id mismatch. Stamp _row_id on insert, load row ids on
resume paths, send rowId from Desktop, filter renderer-synthetic ids,
and stop silently resending failed targeted edits without truncation.
Add production-shaped SessionDB tests for resolve and fail-closed paths.
Fixes#82959
Adds a 'council' synthesis style to MoA (per preset via synthesis_style,
one-shot via the new /council command on CLI + gateway). Reference models
answer independently; the aggregator chairs the deliberation and produces
a user-facing report of consensus, per-model disagreements (with the
differing assumptions behind them), unique contributions, and a
recommendation with an explicit confidence level.
Inspired by Perplexity's Model Council rollout to Perplexity Computer
(changelog 08/04/26): pick a board of 2-8 models, run them independently,
synthesize where they agree/disagree and what each uniquely surfaces.
Port from openai/codex#37527: Terminate timed-out hook process trees.
A shell hook that forked helpers (scanners, watchers, "cmd &") and then hit
its timeout left those descendants running forever — subprocess.run() only
kills the direct child. Worse, descendants holding the inherited pipe write
ends could stall run()'s post-kill communicate() drain.
- agent/shell_hooks.py _spawn(): spawn hooks in their own process group on
POSIX (process_group=0, Python >=3.11); on timeout/error, reap the whole
tree via the shared kill_process_tree() helper, then drain bounded (1s).
Hooks that complete in time keep their descendants, so intentionally
detached helpers survive successful runs (mirrors codex semantics).
- hermes_cli/_subprocess_compat.py: rename _kill_git_process_tree ->
kill_process_tree (it was never git-specific; taskkill /T /F on Windows,
ownership-gated os.killpg on POSIX). Backward-compat alias retained.
- tests/agent/test_shell_hooks_tree_kill.py: real-subprocess regression
tests (descendant killed on timeout, preserved on success, own-group
spawn, fast-path contract, fail-open). Sabotage-verified: reverting the
process_group spawn fails exactly the two new behavior tests.
Gap proven live on main first: a forking hook timed out at 2s and its
descendant survived; same probe against this branch shows it reaped.
CI caught the file hanging AFTER '6 passed in 4.32s' until the runner's
300s SIGKILL. Two defects, same class the PR fixes:
1. The executor-refused (interpreter shutdown) fallback ran the native
call UNBOUNDED on the calling thread — a wedged pipeline would block
process exit forever. Now runs on a bounded daemon exit-thread with
the same timeout/abandon semantics as the executor lane.
2. The wedge tests left daemon workers parked on Event.wait() and live
sessions registered on the atexit shutdown hook; exit re-ran the
wedged pops (bounded, 10s each) and the per-file runner timed out.
Autouse teardown now releases every wedge and drains each runtime.
Canonical runner: 4.4s (was 300s file-timeout kill). Bare pytest was a
false green for this class — it exits before atexit replay cost shows.
The NeMo Relay native binding's scope.pop/push are synchronous and
unbounded ('returns after the scope is closed successfully'). When the
native pipeline cannot make progress, the session coordinator's turn and
session finalization block forever inside run_conversation: delegated
children finish their turns but never return, and delegation batches die
on the stall watchdog. Proven live 2026-08-10 on the staging fleet — a
falsification probe (plugin disabled, identical config) completed the
same delegation batch that wedged with the plugin active.
Bound every scope lifecycle operation that gates turn/session completion
(session push, turn push, turn pop, logical-LLM pops, session pop,
subscriber flush) by running the native call on a shared
DaemonThreadPoolExecutor and honoring a 10s result timeout. On breach a
TimeoutError propagates into each call site's existing exception
handling — warn, retain the unclosed-prefix diagnostics, continue — so
the worst case is one lost span, never a blocked agent. timeout=None
preserves byte-identical synchronous behavior for all other callers, and
interpreter-shutdown paths fall back to the synchronous call so the
atexit flush still exports.
Observability must never block the product.
Closes#26193
Adds ContextReferenceProvider ABC so plugins can register custom
@-prefixes (e.g. @issue:ENG-123) with autocomplete and expansion.
Plugin output flows through existing token-limit guards. Zero
breaking changes.
lkz-de
fangliquanflq
joaomarcos
RelaxJonh
mehmetkr-31
Ben Barclay
kshitij
Xie
Simone Marzola
Teknium
686f6c61
iso2kx
luoxiao6645
sasquatch9818
whirmill
Ben
Victor Kyriazakos
Nikita Barkov
Julientalbot
webdevtodayjason
webtecnica
Adolanium
StanleyStetson
Daniel Magro
Hermes Agent
zccyman