Add HERMES_JOURNAL_MODE env / database.journal_mode config for
virtiofs/NFS/SMB where WAL is not crash-safe. Route 5 bypass openers
through apply_wal_with_fallback so a single setting covers every .db
(#68545).
Telegram clients split messages above 4096 chars into multiple updates. A
long '/queue <prompt>' paste arrives as a COMMAND chunk near the limit plus
plain TEXT continuation chunk(s). _handle_command dispatched the command
chunk immediately, so the continuation landed as a separate plain message
that interrupted the running agent instead of being queued.
Near-limit (>= _SPLIT_THRESHOLD) command chunks now route through the same
text-batching pipeline used for split plain-text messages, merging the
continuation before dispatch. Short commands (/stop, /approve, ...) keep the
immediate path and are never delayed.
The compile_mention_patterns promotion moved the 'patterns is None ->
return []' short-circuit into the shared helper, which meant the
telegram/dingtalk wrappers now evaluated self.name (via log_prefix=)
even on the no-patterns path. On main that path returned before
touching any adapter attributes; tests construct bare adapters via
object.__new__ that lack .platform, so TestTelegramGuestMentionGating
failed with AttributeError. Restore the early return in both wrappers
for exact behavior parity with main.
- base.on_processing_complete implements the opt-in remove-ack/add-outcome
flow driven by _OK_EMOJI/_FAIL_EMOJI class attrs and the
_add_reaction(chat_id, message_id, emoji)/_remove_reaction(chat_id,
message_id) primitive shape; default stays a no-op.
- photon drops its override (exact behavioral match).
- slack/discord/feishu/matrix/telegram/google_chat keep overrides: divergent
primitive signatures (team_id routing, raw message objects, reaction-id
handles, replace-semantics setMessageReaction) or extra state protocols.
The disease: ~15 scattered raw yaml.safe_load(config.yaml) reads that
silently miss managed-scope overlay, ${ENV_VAR} expansion, profile-aware
pathing, and root-model normalization. Every new config feature needed an
N-site sweep (incident chain 9cbcc0c9c8 → 732293cf87 → b0e47a98f9 →
1928aa0443). This commit assigns every raw read to an owner and adds a
lint-guard test so the class cannot regrow.
New primitive (additive-only change to hermes_cli/config.py):
read_user_config_raw(path=None) — reads the user file EXACTLY as
written; docstring states it is ONLY legal for write-back round-trips
and raw-file diagnostics. Behavioral reads must use
load_config()/load_config_readonly().
BEHAVIOR FIXES (class-a sites migrated to a canonical loader — these
previously read values that could DIFFER from the effective config):
gateway/run.py _try_resolve_fallback_provider → _load_gateway_runtime_config
keys: fallback_providers/fallback_model (provider, model, base_url,
api_key). Drift fixed: a managed-pinned fallback chain was ignored;
an api_key of "${OPENROUTER_API_KEY}" reached the resolver unexpanded.
gateway/run.py GatewayRunner._load_provider_routing → same loader
key: provider_routing. Drift fixed: managed-pinned routing prefs and
${VAR} templates were ignored.
gateway/run.py GatewayRunner._load_fallback_model → same loader
keys: fallback chain. Same drift as above.
gateway/run.py GatewayRunner._refresh_fallback_model
keeps the raw primitive (its last-known-good-on-parse-failure contract
forbids the fail-open loader, which returns {} on a torn write) but now
applies managed overlay + env expansion inline. Drift fixed: chain
edits under managed scope / env templates were previously frozen out.
tui_gateway/server.py _load_cfg (72 behavioral call sites)
now = raw read + managed overlay (pre-existing) + NEW ${VAR} expansion,
split from a new _load_cfg_raw() write-back primitive. Drift fixed:
e.g. custom_prompt: "hello ${VAR}", agent.system_prompt, model,
api_key/base_url templates reached sessions unexpanded. DEFAULT_CONFIG
is deliberately NOT merged (callers treat missing keys as unset;
`_load_cfg() == {}` sentinels and _save_cfg round-trips depend on it).
tui_gateway/server.py _profile_configured_cwd
keys: terminal.cwd of a NON-launch profile. Drift fixed: managed
overlay + ${VAR} expansion now apply (load_config() would resolve the
wrong profile's home, so the raw primitive + inline pipeline is used).
plugins/platforms/telegram/adapter.py _reload_dm_topics_from_config
→ load_config_readonly(). keys: platforms.telegram.extra.dm_topics.
Drift fixed: managed overlay + profile-aware pathing + expansion.
plugins/memory/holographic _load_plugin_config → load_config_readonly().
keys: plugins.hermes-memory-store.*. Same drift class.
WRITE-BACK ROUND-TRIPS (class-b: stay raw BY DESIGN via read_user_config_raw;
merging defaults/overlay would pollute the saved user file):
gateway/slash_commands.py: model persist x2, _save_gateway_config_key,
memory/skills write_approval toggles
gateway/platforms/yuanbao.py auto-sethome
tui_gateway/server.py _write_config_key + all cfg→_save_cfg blocks
(reasoning show/hide/full/clamp, details_mode[.section], prompt)
→ new _load_cfg_raw()
plugins/memory/holographic save_config
RAW-FILE DIAGNOSTICS + presence-sensitive bridges (class-c: stay raw,
now via the shared primitive with an explanatory comment):
hermes_cli/doctor.py x5 (model validation, stale-root-keys, .env drift,
deprecation sweep, memory-provider probe — the latter two keep their
inline managed overlay where they had one)
gateway/run.py _bridge_max_turns_from_config and the module-level
TERMINAL_*/HERMES_* env bridge (bridging merged defaults would export
all of DEFAULT_CONFIG into the environment; both keep their inline
overlay + expansion)
hermes_cli/send_cmd.py env bridge (same presence-sensitivity)
hermes_cli/gateway.py multiplex-conflict probe (reads the DEFAULT root's
config, not the active profile's — load_config is the wrong owner)
hermes_cli/profiles.py / hermes_cli/web_server.py / tools/wake_word.py
multi-profile reads (load_config targets only the ACTIVE profile home)
cron/jobs.py _resolve_default_model_snapshot and cron/scheduler.py
run_job config read keep their existing inline overlay+expansion but
now share the primitive (their fail-open + last-value semantics and
the deliberate no-defaults merge are preserved exactly).
Failure-semantics audit: every migrated site preserves its exact previous
behavior on missing file ({} / early return) and parse failure (raise into
the caller's existing except, warn, last-known-good, or fail-open) —
read_user_config_raw intentionally mirrors bare open()+safe_load semantics
(raises on parse errors, {} only on FileNotFoundError/non-dict root).
Guard: tests/hermes_cli/test_config_read_guard.py scans the tree for
yaml.safe_load within 6 lines of a 'config.yaml' reference outside an
explicit ALLOWLIST (hermes_cli/config.py, gateway/config.py, gateway/run.py
fallback path, hermes_cli/managed_scope.py which reads the MANAGED file,
gateway/readiness.py parse-health probe) and fails on new offenders.
E2E: tests/hermes_cli/test_config_loader_e2e.py runs a subprocess with a
temp HERMES_HOME (config.yaml containing ${E2E_PROMPT_SUFFIX}) plus a
HERMES_MANAGED_DIR overlay pinning agent.reasoning_effort, asserting
tui _load_cfg resolves "hello world"/"high" while _load_cfg_raw +
_save_cfg round-trip the template and user value verbatim with no
managed/default leakage.
Three GUI Capabilities-tab defects reported on Windows:
1. Browser rows stuck on 'Setup required' after a successful setup run.
Root causes, all in the readiness probe (not the installer):
- _has_agent_browser() never searched the Hermes-managed Node dir
(%LOCALAPPDATA%/hermes/node / $HERMES_HOME/node/bin) where the
Windows install lands, and probed node_modules/.bin/agent-browser
as the extensionless POSIX shim, which fails exec on Windows
(WinError 193) — now resolved via PATHEXT-aware shutil.which
against both rungs, mirroring _find_agent_browser().
- Cloud rows (Nous Subscription Browser Use, Browserbase, Browser
Use, Firecrawl) declared post_setup: agent_browser, whose
readiness gate requires a LOCAL Chromium build the cloud never
uses — switched to the cloud-scoped 'browserbase' hook (CLI-only).
- _agent_browser_installed() could read browser_tool's stale cached
'Chromium missing' result from before the install ran in the
spawned post-setup process — cache now dropped before probing so
the pill flips to Ready right after a successful run.
2. No way to tell which backend is active, and clicking a row to read
its details silently rewrote config. Row click now only
expands/collapses; activation is an explicit 'Use this backend'
button, the active row carries an 'Active' pill, and the expanded
active row says 'This is your active backend'.
3. OpenAI TTS showed one model and one voice. The options were always
defined but rendered through a native <datalist>, which filters by
the field's current value — a field already set to a valid option
suggested only itself. Replaced with a real combobox (Input +
dropdown) that lists every option, and voice suggestions now track
the selected model per the OpenAI TTS docs: tts-1/tts-1-hd = 9
voices, gpt-4o-mini-tts = 13 (adds ballad, verse, marin, cedar).
LINE plugin.yaml plus line/wecom-callback/msgraph-webhook/
whatsapp-cloud/teams user-guide pages still documented the old
IPv4-only 0.0.0.0 defaults; update to the dual-stack unset default.
Telegram webhook env docs live in the adapter docstring (updated with
the code change); its plugin.yaml has no webhook host entry.
Same class of bug as the LINE adapter (NS-603): defaulting the webhook
bind to "0.0.0.0" (or hardcoding it) binds IPv4 ONLY, so the listener
is unreachable over IPv6-only private networks such as Fly.io 6PN.
- wecom callback_adapter: DEFAULT_HOST None; config.py env seed no
longer forces 0.0.0.0 when WECOM_CALLBACK_HOST is unset.
- msgraph_webhook: DEFAULT_HOST None; the allowed_source_cidrs
requirement still fires for the all-interfaces default (host=None is
treated as network-accessible).
- whatsapp_cloud: DEFAULT_WEBHOOK_HOST None.
- teams: hardcoded 0.0.0.0 TCPSite bind → _DEFAULT_HOST=None with new
TEAMS_HOST / extra.host override (mirrors LINE_HOST pattern).
- telegram: hardcoded listen="0.0.0.0" → default "" (tornado
bind_sockets opens one socket per address family; verified against
PTB 22.6/tornado) with new TELEGRAM_WEBHOOK_HOST / extra.webhook_host
override.
Explicit host overrides everywhere are preserved; empty/unset collapses
to the dual-stack default. "::" remains a bad substitute on
bindv6only=1 hosts (see LINE adapter comment).
The LINE adapter's webhook server defaulted to host="0.0.0.0", which
binds IPv4 ONLY. On IPv6-only private networks — notably Fly.io 6PN,
where the hosted edge router reverse-proxies LINE ingest to
<app>.internal:8646 over an fdaa: IPv6 address — nothing is listening
on the dialed address: connection refused → customer-visible 502 when
LINE's console verifies the webhook (NS-603).
This is the same bug the generic webhook adapter fixed in d542894ad;
the LINE adapter was never updated to match. Fix mirrors that commit:
- DEFAULT_HOST = None → asyncio/aiohttp create_server binds one socket
per address family (v4 + v6), regardless of the bindv6only sysctl.
"::" is NOT a valid substitute — Fly machines set bindv6only=1, so
it would yield an IPv6-only socket and break IPv4 loopback probes.
- Empty-string host collapses to None; LINE_HOST/extra.host still pin
a specific bind address.
- reuse_address=False scoped to macOS only (BSD wildcard-socket
traffic-splitting footgun), mirroring 9420ad946.
- The three outbound-media guards compared webhook_host == "0.0.0.0"
by string equality; extracted to _missing_public_url() which treats
None/0.0.0.0/::/"" as "no fetchable hostname" so the LINE_PUBLIC_URL
requirement still fires under the new default. _media_url() falls
back to 127.0.0.1 instead of interpolating 'None' into URLs.
Tests: dual-stack default decision table (default None, empty→None,
pinned preserved, LINE_HOST override), behavioural both-families bind
proof via runner.addresses, and the media public-URL guard matrix.
88 passed, ruff clean.
Companion router fix (hermes-agent-router) routes /webhooks/line and
/line/webhook|/line/media to :8646 over 6PN; both are needed for
end-to-end hosted LINE delivery.
Fixes NS-603
resolve_sidecar_dir() probes the filesystem (touch/unlink) and can mirror
sidecar files to HERMES_HOME. Doing that as a module-import side effect
meant plugin discovery, `hermes --help`, and test collection all paid a
filesystem probe (and possibly a mirror copy) just for importing the
photon adapter or CLI.
Convert _SIDECAR_DIR/_NPM_ERROR_LOG in adapter.py and cli.py to lazy
cached accessors (_sidecar_dir()/_npm_error_log()); resolution now
happens on first actual use. Existing tests that monkeypatch the
_SIDECAR_DIR module global keep working — the accessors honor a
non-None value. Adds a regression test proving import performs no
resolution.
The Photon iMessage sidecar needs node_modules under
plugins/platforms/photon/sidecar/, but hosted/managed images keep the
whole install tree under an immutable /opt/hermes — every install and
self-heal path (setup CLI, stale-deps reinstall, cold install) died on
EROFS, and hosted users have no shell to work around it.
Three-layer fix, mirroring the WhatsApp bridge resolver pattern:
1. Bake the deps into the image. The Dockerfile now runs npm ci for the
sidecar in the layer-cached dependency stage (deterministic installs
from the committed lockfile; the postinstall spectrum-ts patch runs
at build time). Hosted happy path needs no runtime install at all.
2. New sidecar_paths.resolve_sidecar_dir() decides where the sidecar
runs from: PHOTON_SIDECAR_DIR override > writable source dir (dev
installs, unchanged) > read-only dir with baked fresh deps (managed
image) > mirror to $HERMES_HOME/photon/sidecar (writable data
volume) when deps are missing or stale in a read-only tree. The
mirror refreshes changed source files on image updates while
keeping node_modules, so the existing lockfile-staleness self-heal
works there.
3. connect() can now cold-install: _start_sidecar() runs the bounded
npm ci bootstrap when node_modules is missing instead of raising
immediately, and check_requirements() reports available when a
self-install is possible (npm present + writable resolved dir) so
the gateway actually creates the adapter on hosted instances. A
failed bootstrap still raises the actionable error, which connect()
surfaces as the retryable SIDECAR_FAILED fatal state on the
dashboard.
Tests: resolver decision table (env override, in-place, mirror,
refresh, fail-open), cold-install lifecycle paths, and a Dockerfile
contract test guarding the baked-deps + no-chown invariants.
Fixes NS-606.
Widen NS-605 to the two remaining direct-install sites in the hindsight
plugin, which still shelled out to 'uv pip install --python
sys.executable' and therefore failed (EROFS/EACCES) on immutable hosted
images with sealed venvs, and lost packages on redeploy:
- post_setup dependency install (~L835): install_specs() with ok /
blocked-reason / stderr handling matching honcho/mem0.
- initialize()-time hindsight-client auto-upgrade (~L1240):
install_specs(); blocked installs log the gate reason with the manual
command instead of a raw subprocess error, and init proceeds.
Audited every other memory plugin (supermemory, byterover, holographic,
openviking, retaindb) for direct pip/uv install subprocess calls: none
remain — their deps flow through plugin.yaml pip_dependencies or
lazy_deps.ensure().
Tests: TestClientAutoUpgradeRoutesThroughLazyDeps — upgrade goes through
install_specs with the exact spec (regression guard asserts no
subprocess.run), blocked upgrade is non-fatal and surfaces the gate
reason. Updated TestPostSetupEnvEncoding stubs to the new install path.
Installing a memory provider (Honcho, mem0, hindsight, ...) from the
dashboard Plugins page failed on hosted deployments with a permission
error: the setup endpoint shelled out to
`uv pip install --python sys.executable`, which targets the sealed
read-only venv under /opt/hermes (immutable hosted image, NS-579/#49113).
The correct mechanism already exists: tools/lazy_deps.py redirects
installs to the writable durable target on the data volume
(HERMES_LAZY_INSTALL_TARGET=/opt/data/lazy-packages) when the venv is
sealed (HERMES_DISABLE_LAZY_INSTALLS=1), appends the target to the END
of sys.path (core venv always wins collisions), and constrains shared
deps to core-venv versions. The dashboard installer simply never used
it.
Fix:
- tools/lazy_deps.py: new public install_specs() — installs arbitrary
manifest-declared pip specs through the same environment routing as
ensure(): venv-scoped by default, durable-target on sealed images,
refused with an actionable reason when gated off (config kill switch
or sealed venv without a target — never surfaces raw EROFS/EACCES).
Specs are validated with _spec_is_safe(); post-install it invalidates
import/metadata caches so availability rechecks in the same process
see the new packages without a restart. Never raises.
- hermes_cli/web_server.py: _install_memory_provider_pip_dependencies
now calls install_specs() instead of building its own uv/pip
subprocess. Blocked installs surface the gate reason in the setup
results; the response's status block reflects post-install
availability (stale 'missing deps' state clears immediately).
- hermes_cli/memory_setup.py, plugins/memory/honcho/cli.py,
plugins/memory/mem0/_setup.py: CLI setup wizards routed through
install_specs() too — same sealed-venv failure mode, same fix.
No hosted setup path writes to /opt/hermes anymore; provider discovery
and installation now use the same environment (sys.path activation is
shared with the lazy-install bootstrap in hermes_bootstrap).
Tests:
- tests/tools/test_lazy_deps.py: TestInstallSpecs — gating matrix
(sealed+no-target blocked with immutable-deployment reason, config
kill switch, sealed+target proceeds), spec-safety rejection before
any subprocess, venv-scoped vs --target command display, failure
stderr passthrough, never-raises contract.
- tests/hermes_cli/test_web_server.py: setup endpoint routes pip
through lazy_deps (regression guard asserts no direct 'pip install'
subprocess), blocked-reason surfacing, same-response availability
recheck clears stale missing state.
Fixes NS-605 (Plain T-1111).
Maintainer rework of #45580 (issue #54036) on top of the contributor's
cherry-pick, which targeted spectrum-ts 3.1.0 while main pins 8.0.0:
Sidecar (primary detection, new):
- stream-staleness.mjs: pure decision rules, executable under node.
* classifyProbeRejection: only a not-found-shaped rejection of the
synthetic-id read counts as a completed round-trip (ALIVE); any other
rejection is INCONCLUSIVE — never alive. The original /probe treated
ANY rejection as alive, which was too loose.
* shouldProbe: probe only after 10+ min of stream silence (configurable
via PHOTON_STREAM_SILENCE_PROBE_MS; <=0 disables) with a cooldown.
* isZombieSuspect: zombie only on silence past threshold AND a
probe-proven live channel. Silence alone NEVER degrades (shared lines
can be quiet for hours); inconclusive probes NEVER degrade (network
may be down — the iterator will throw and the re-subscribe loop
recovers on its own).
- index.mjs: track last inbound-iterator yield (noteInboundYield), run a
30s watchdog tick, and on a confirmed zombie feed markStreamDegraded ->
the existing exit-75 restart path. /healthz gains a stream.staleness
block (silentForMs, threshold, lastProbeOutcome, zombieSuspected).
/probe reworked to strict semantics: 200 only on a proven round-trip,
503 with outcome hung|inconclusive otherwise.
Adapter (second layer, reworked):
- _probe_once returns tri-state alive|hung|inconclusive; only a hung
sidecar HTTP call counts toward the respawn counter — inconclusive
resets nothing and triggers nothing.
- default probe_interval_seconds 60 -> 600 (conservative; avoid restart
storms on quiet lines).
- _monitor_sidecar_health surfaces zombieSuspected from /healthz as a
warning; the fatal UPSTREAM_STREAM_DEGRADED path is unchanged and fires
when the sidecar escalates.
Tests: test_zombie_stream_watchdog.py executes the real node decision
module and drives the adapter against mocked /healthz responses;
test_presence_watchdog.py updated for the tri-state probe.
Also adds contributor mappings for nickkarhan (#53283) and vaibhavjnf
(#45580).
Follow-up to the URL markdown fix: extract the /send builder decision into
sidecar/send-format.mjs and rewrite test_url_send_path.py to execute the real
module under node (format+text in -> chosen builder out) instead of regex-
grepping index.mjs source, which is a banned test pattern in this repo.
spectrum-ts's live-stream consumer (consumeLive in spectrum-ts 3.x) only
reconnects when its inbound async iterator throws or ends. A half-open
("zombie") gRPC socket — where the TCP connection stays ESTABLISHED but the
peer is gone (NAT idle-timeout, network blip, laptop sleep) — makes the
iterator hang forever: no error, no end. The SDK exposes no gRPC keepalive
knob (createClient takes only {address, tls, token}; grpc.keepalive_time_ms
defaults to -1 = pings off), so the inbound stream silently dies and stays
dead until the gateway is restarted. Symptom: the agent's iMessage line goes
"online but deaf" — Photon's cloud-side fallback answers users with "the agent
isn't online right now" and inbound never reaches the gateway.
Fix, entirely in the code we own (no SDK fork):
- Sidecar gains a POST /probe endpoint that drives a cheap unary read
(space.getMessage on a synthetic id) over the SAME gRPC channel the inbound
stream uses. A live channel round-trips in ms (server returns not-found,
which is success for liveness); a zombie hangs. It sends nothing to any user
and creates no chat (space.get is local in shared/dedicated mode; only the
message read touches the wire).
- The adapter runs a presence watchdog: it probes on an interval, skips the
probe when natural inbound traffic already proved liveness within the
window, and after N consecutive failed probes respawns the sidecar — a fresh
Spectrum() re-subscribes the stream and re-registers presence. Successful
probes double as application-level keepalive, helping prevent the zombie
from forming at all. Respawn is lock-guarded against double-spawn and the
watchdog is torn down cleanly on disconnect.
Behavioural settings live in config.yaml (extra), bridged to env per the
.env-is-secrets-only convention:
probe_interval_seconds (60), probe_timeout_seconds (10),
probe_max_failures (3). A non-positive interval disables the watchdog.
Tests: tests/plugins/platforms/photon/test_presence_watchdog.py covers config
resolution, the disable switch, probe alive/dead(500)/timeout/no-client, the
core N-failures->one-respawn detection, success-resets-failures, stop-then-
start respawn ordering, and lock-guarding — all without spawning Node or
hitting the network.
Contributed by Vaibhav Sharma (X: @vabbyshabby).
imessage.effect.message crashed the sidecar at import against SDK stubs/
builds lacking the effect surface (caught by the patch-failure health test).
Optional-chain with {} fallback; /send-effect rejects cleanly instead.
Follow-up to the #48194 pick: it was written before #43665 landed and
re-added its own /send-poll sidecar route and poll import. Collapse the
duplicates:
- keep #43665's /send-poll route (>=2 trimmed string options) as the
single sidecar implementation; drop #48194's variant
- drop the duplicated poll import in the sidecar destructure
- make adapter.send_poll() a thin wrapper over _sidecar_send_poll(), the
one /send-poll client (shared with the poll-backed clarify path), and
align its validation to the sidecar's >=2-options contract
The `clarify` tool's multiple-choice prompts flattened to a numbered text
list on Photon/iMessage, even though iMessage has a native poll bubble and
spectrum-ts already exposes it via the `poll()` content builder. Two gaps
caused the flattening:
* Outbound: the sidecar only had `/send` (text); there was no way to send
a poll, so the base adapter's numbered-text fallback was used.
* Inbound: `normalizeContent()` handled only text/attachment/voice, so a
poll vote (`poll_option`) was dropped on the floor ("[Photon content
type not handled: poll_option]") and never resolved the clarify.
Fix, end to end:
* Sidecar: import `poll` from spectrum-ts; add a `/send-poll` route
(`space.send(poll(title, ...options))`); serialize inbound `poll_option`
(the vote: chosen title + selected bool) and `poll` content in
`normalizeContent()`.
* Adapter: override `send_clarify` — for choices, send a native poll via
`_sidecar_send_poll` and call `mark_awaiting_text` so the gateway's
existing pending-clarify text-intercept resolves the answer; open-ended
clarifies keep the plain-text path. Inbound `poll_option` selections are
dispatched as a plain-text MessageEvent carrying the chosen option
(deselections / empty votes are dropped). If the poll send fails (an
older sidecar without `/send-poll`, or a send error) it falls back to the
numbered-text clarify, so nothing regresses on a half-upgraded restart.
No new model tool, no new env var, no core change — the capability lives at
the platform edge. The poll vote reuses the existing clarify text-intercept
resolution path, so no new gateway resolution mechanism is introduced.
Tests: tests/plugins/platforms/photon/test_poll_clarify.py — inbound vote ->
choice text, deselection/empty-vote dropped, send_clarify sends a poll +
enables text-capture, open-ended stays text, and poll-failure falls back to
the text list. Full photon suite green.
Contributed by Vaibhav Sharma (X: @vabbyshabby).
Maintainer follow-up to the #51193 salvage:
- _send_with_retry: permanent classes (auth_or_config, target_not_allowed)
now short-circuit BEFORE the unconditional plain-text fallback resend,
including when a retry attempt surfaces one — no more double-sends of
permanently-failing requests.
- sidecar classifySidecarError: new structured code target_not_allowed for
Spectrum's 'Target not allowed for this project' AuthenticationError
(shared/free-tier lines cannot initiate outbound sends to new targets).
Classification applies to every handler sharing the catch-all
serverError path (/send, /send-attachment, /react, /typing, ...).
- _standalone_send now parses the structured error body too (it reads
sidecar responses independently of _sidecar_call) and returns
error_class/retryable alongside the message.
- target_not_allowed maps to a canonical user-facing message in both
paths; raw upstream error text never leaks through the structured code.
Closes the actionable halves of #50971, #51897, #52794.
Photon's Node sidecar intentionally hides raw handler exceptions, but the Python adapter still needs a safe failure class and retryability bit so delivery retries do not collapse into an opaque generic 500.
Constraint: Sidecar responses must not leak raw stack traces or private exception text
Rejected: Retry every internal sidecar error | masks permanent auth/config failures
Confidence: high
Scope-risk: narrow
Directive: Keep sidecar error text generic; extend safe error classes instead of exposing raw SDK failures
Tested: uv run --with pytest-timeout pytest tests/plugins/platforms/photon/test_overflow_recovery.py -q
Tested: uv run --with pytest-timeout pytest tests/plugins/platforms/photon -q
Tested: uv run ruff check plugins/platforms/photon/adapter.py tests/plugins/platforms/photon/test_overflow_recovery.py
Tested: python3 -m py_compile plugins/platforms/photon/adapter.py tests/plugins/platforms/photon/test_overflow_recovery.py
Tested: node --check plugins/platforms/photon/sidecar/index.mjs
Tested: git diff --check
Tested: python3 scripts/check-windows-footguns.py --diff origin/main
Not-tested: Live Photon/Spectrum delivery against a real iMessage account
Related: #50971
`PhotonAdapter._start_sidecar` is `async`, but it ran the Spectrum
mixed-attachment patch script with a bare `subprocess.run(...)`: it spawns
node and *waits* for it, with `timeout=10`. Executed inline that holds the
shared gateway event loop for the whole window, so no other platform's
messages, heartbeats, or sessions are serviced until it returns.
The same function already establishes this exact invariant twenty lines
above, where the stale-dependency reinstall hops to a worker thread:
# Runs off the event loop so a cold install can't freeze every other
# platform's traffic.
if _sidecar_deps_stale():
await asyncio.to_thread(_reinstall_sidecar_deps)
The patch spawn never got the same treatment. It is not startup-only
either — `_start_sidecar` is called from `connect()`, which takes
`is_reconnect`, so an ordinary Photon reconnect (network blip, sidecar
death) re-runs it and stalls a live gateway that is actively serving
Discord/Telegram/Slack traffic.
Dispatch it via `asyncio.to_thread` like its sibling. Same off-the-loop
class as the inbound-image decision (#66688) and the cron-fire verifier.
Adds a regression test asserting the spawn executes on a worker thread
rather than the loop thread.
`_reap_stale_sidecar` is `async`, but it identified the processes holding
the sidecar port with two blocking helpers called inline:
* `_find_listener_pids` -> `subprocess.run(["lsof", ...], timeout=5.0)`
* `_pid_is_sidecar` -> `subprocess.run(["ps", ...], timeout=5.0)`, once
per candidate pid
so the inspection can hold the shared gateway loop for 5 + 5·N seconds
while nothing else on it is serviced. It only runs once the /healthz probe
finds something already listening — the orphaned-sidecar recovery path —
and `_reap_stale_sidecar` is awaited from `_start_sidecar`, which runs on
every reconnect (`connect(is_reconnect=True)`). The stall therefore lands
on a live gateway that is still serving every other platform, right when a
crashed sidecar has already left an orphan behind.
Move the whole inspection to one `asyncio.to_thread` hop (one hop rather
than N+1 round trips). The reaping semantics are untouched: SIGTERM for
verified orphans, SIGKILL escalation, and both foreign-listener
RuntimeErrors behave exactly as before.
Same off-the-loop class as the inbound-image decision (#66688) and the
cron-fire verifier.
Adds a regression test asserting both the lsof lookup and the per-pid ps
check execute on a worker thread rather than the loop thread.
Follow-up widening of the #73170 pattern: apply the same
asyncio.current_task() guard used for the health task (and now the
supervisor task in _stop_sidecar) to the inbound-task cancel path in
disconnect(), so an inbound task that triggers disconnect() cannot
cancel-and-await itself.
Fixes#73159.
When the Photon sidecar exits unexpectedly, _supervise_sidecar()
(running as self._sidecar_supervisor_task) correctly detects
SIDECAR_CRASHED and calls self._notify_fatal_error(). The Gateway's
fatal-error handler answers that by calling adapter.disconnect(),
which calls _stop_sidecar() -- from INSIDE the very task that's
currently executing this whole chain.
_stop_sidecar()'s cleanup unconditionally cancelled
self._sidecar_supervisor_task. Cancelling the currently-running task
raises CancelledError at its own next await point (inside
_notify_fatal_error() or _stop_sidecar() itself). Since
asyncio.CancelledError inherits from BaseException (not Exception),
the Gateway's `except Exception` guards around the fatal-error handler
don't catch it -- the handler aborts before ever reaching the
"queue platform for background reconnection" step. Photon then stays
permanently in `retrying` state until the whole process is manually
restarted, even though detection worked correctly and the underlying
transient upstream outage had long since recovered.
Fix: in _stop_sidecar()'s cleanup, check whether
self._sidecar_supervisor_task is asyncio.current_task() before
cancelling it. A task cannot legally cancel itself in any useful way
anyway (the cancellation only takes effect at its own next await,
which is exactly the corruption described above) -- when we're inside
the supervisor's own call stack, it's already in the process of
finishing on its own once _notify_fatal_error() returns, so skip the
cancel and just clear the reference.
This mirrors an existing precedent in the same file: disconnect()
already guards its OTHER task-cancellation (self._sidecar_health_task)
the same way (`if task is not asyncio.current_task()`), just not the
supervisor task in _stop_sidecar().
Per the issue's own note that existing tests mock out
_notify_fatal_error() entirely (so this integration chain was never
exercised), added two tests that drive the REAL chain: one runs
_supervise_sidecar() as an actual asyncio task with a real
_notify_fatal_error() that calls the real disconnect() -> _stop_sidecar(),
confirming the task completes without CancelledError and that the
post-disconnect reconnect-queue step actually executes; a second
confirms the OTHER call path (external cleanup, a different task)
still correctly cancels a running supervisor exactly as before.
Reverting only the adapter.py fix (keeping the new test) reproduces
the exact CancelledError from the bug report, confirming this is a
genuine regression test.
2 new tests pass; 113/113 in the full tests/plugins/platforms/photon/
directory (no regression to sidecar lifecycle, overflow recovery, or
health-monitoring behavior).
Follow-up to #69112. That PR hardened the shared gateway dispatch path in
gateway/run.py against the *caller* being cancelled. A second, self-referential
cancellation specific to PhotonAdapter sits one layer underneath it and survived
that fix.
PhotonAdapter is the only platform adapter that awaits _notify_fatal_error()
inline, on the same task that detected the fault. Both _monitor_sidecar_health
and _supervise_sidecar run as self._sidecar_health_task /
self._sidecar_supervisor_task, and the notification routes into
GatewayRunner._handle_adapter_fatal_error_impl, which tears the adapter down via
_safe_adapter_disconnect -> disconnect(). disconnect() then cancels
self._sidecar_health_task and awaits it -- which, when the health task is what
raised the notification, means disconnect() cancels its own caller several
plain-await frames up.
disconnect()'s `task is not asyncio.current_task()` guard does not catch this.
The current task where that guard evaluates is the wrapper
_await_adapter_cleanup_with_timeout creates around disconnect() via
asyncio.ensure_future, not the health task further up the chain, so the guard
passes and the cancel lands.
CancelledError stopped subclassing Exception in Python 3.8, so the
`except Exception` that wrapped the inline notify call never saw it. The health
task died silently mid-handoff: no log line, no "exception never retrieved"
warning (cancellation is normal asyncio), and no retry. The platform stayed
stranded until the gateway was restarted by hand.
Fix: dispatch the notification onto a new task, the same pattern
DiscordAdapter._handle_bot_task_done already uses for this reason. disconnect()
can then cancel the health/supervisor task freely without that cancellation
reaching the code still running the handoff, so the handoff always reaches the
reconnect queue. Both Photon fatal call sites are converted: the health-poll
path (observed wedging) and the sidecar-crash path (same shape, not yet
observed). gateway/run.py is untouched.
Observed twice on a self-hosted gateway, ~4h38m and ~52min of silent inbound
outage, both cleared only by a manual restart, both post-dating #69112's merge.
In each case the fatal log line appears and `queued for background reconnection`
never does.
Tests: new tests/plugins/platforms/photon/test_fatal_notify_self_cancel.py
covers the self-cancellation (fails with CancelledError without this change),
that the dispatch does not block its caller, that a failing notification warns
rather than raising, and a source guard against reintroducing either inline
await. Two assertions in test_overflow_recovery.py that drove these coroutines
directly now drain pending tasks before asserting delivery, since the
notification is deliberately no longer awaited inline.
Prepared with agent assistance and reviewed before submission.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Consolidates the native-transport half of PR #73636 by @ScaleLeanChris
onto the merged adapter: persistent NIP-42-authenticated Nostr WebSocket
subscription as the default inbound path (transport=auto|websocket|poll),
kind-44100 membership events for live DM discovery, since-timestamp
resume on reconnect with bounded exponential backoff, and automatic
fallback to CLI polling when the WS can't be established. Events route
through the same _handle_event() pipeline as the poll loop, so de-dupe,
mention gating, p-tag DM latching, and allow-lists behave identically on
both transports. Outbound stays on the CLI (one-shot sends never race a
WS auth handshake — his design).
E2E verified against a real in-process websockets relay: NIP-42
challenge -> signed kind-22242 AUTH (event id re-derived server-side) ->
REQ subscription -> EVENT dispatch -> clean disconnect.
Co-authored-by: ScaleLeanChris <chris@scalelean.com>
Extracted verbatim from PR #73636 by @ScaleLeanChris — nsec/hex key
decoding, secp256k1 point math, BIP-340 Schnorr signing, and NIP-42
AUTH event construction with optional NIP-OA owner attestation tag.
Two independent cross-PR collisions red on main (slice 8/8):
1. fd4f756492 (salvaged from stale #54514) added an early 'drop U+FFFC
placeholder' return at the top of _dispatch_inbound — written before
the deferred-wait handler (6b91b50c6e/afab7ed46e) existed further
down the same function. The early return shadowed it: _pending_fffc
never populated, no attachment-timeout tracking, 4 tests red. Remove
the duplicate block; the deferred handler already drops the
placeholder AND tracks/cancels/warns.
2. 9cf2046081 tightened sidecar_deps_installed() to require
node_modules/spectrum-ts, but test_runtime_record's _patch_spawn
fixture still created only bare node_modules/ — 2 tests red. Mirror
a real completed install.
tests/plugins/platforms/photon: 164/164 after; 158/164 before.
The sidecar auth token is generated at spawn (secrets.token_hex) and
existed only in the gateway process memory + sidecar child env, so
_standalone_send from cron subprocesses, hermes send, or the dashboard
structurally could not authenticate (#69960).
The adapter now writes <hermes-home>/runtime/photon-sidecar.json
({port, token, pid}, 0600, atomic tempfile+os.replace) once the sidecar
passes its /healthz readiness check, and deletes it in _stop_sidecar,
on every startup-failure path, and at disconnect so a stale record
never outlives a dead sidecar. _standalone_send falls back to the
record when PHOTON_SIDECAR_TOKEN is unset, validating the recorded pid
is alive first; a stale record yields a clear 'gateway appears to be
down' error. Docs note the gateway-must-be-running requirement and the
Photon-side shared-line initiation policy (#51897).
All Photon sidecar HTTP requests target 127.0.0.1 — they should
never be routed through a system HTTP proxy. When trust_env=True
(the default), httpx picks up macOS system proxy settings and
routes localhost requests through the proxy. If the proxy returns
a spurious response (e.g. 502), _reap_stale_sidecar() interprets
it as 'port in use by a non-sidecar process' and refuses to start,
yielding: 'pids: unknown, not a Photon sidecar'.
Set trust_env=False on all five httpx.AsyncClient call sites in
the Photon adapter so localhost sidecar communication bypasses
the system proxy entirely.
Addresses teknium1 review on #50983:
- cli.py's `hermes photon status` and adapter.py's _start_sidecar() still
used the old node_modules/-existence check while check_requirements()
had moved to a spectrum-ts content check. Extracted the check into a
shared sidecar_deps_installed(), used by all three, so an
empty/partial node_modules/ (aborted npm install) is rejected
consistently instead of only in check_requirements().
- _install_sidecar()'s success-path _NPM_ERROR_LOG.unlink() only caught
FileNotFoundError, so a PermissionError/OSError on a locked file would
propagate. Broadened to OSError.
- npm stderr was truncated only when read back in check_requirements();
an unbounded stderr was written to disk on every failed install. Now
truncated to _NPM_ERROR_LOG_MAX_CHARS before write_text().
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Problema
--------
Quando o npm install do sidecar Photon falhava, o Hermes descartava toda
evidencia e continuava normalmente - deixando o adapter de iMessage
silenciosamente ausente, sem nenhuma mensagem de erro acionavel.
Tres falhas independentes formavam o caminho de falha silenciosa:
1. check_requirements() sem logging
Cada branch de return False retornava sem emitir nenhum log. O core em
platform_registry.py so consome o bool de check_fn() e loga uma mensagem
generica com o install_hint - sem acesso ao motivo real da falha.
if not HTTPX_AVAILABLE: return False # sem log
if not shutil.which(node): return False # sem log
if not node_modules.exists: return False # sem log
2. node_modules/ parcialmente criado passava o guard (Risk 2)
npm cria node_modules/ antes de abortar em ENOSPC, timeout de rede ou
EACCES. O diretorio existia, check_requirements() retornava True (falso
positivo), o adapter era registrado, e o crash acontecia em runtime com
um erro de modulo ausente aparentemente nao relacionado ao setup.
3. stderr do npm descartado (Risk 3)
subprocess.run sem stderr=PIPE. O output de erro aparecia no terminal
durante o setup e sumia depois - diagnostico impossivel em CI/CD, Docker,
VPS headless, e qualquer reinstalacao posterior.
Correcoes
---------
adapter.py - check_requirements() agora loga por branch:
- httpx ausente -> logger.warning com nome do pacote
- node nao no PATH -> logger.warning com nome do binario e env var
- spectrum-ts ausente -> logger.debug com path do sidecar + ultimo erro npm
(DEBUG nao WARNING: estado normal pre-setup; check_fn() e chamado de
5 hot paths do core incluindo polling do /api/status)
adapter.py - content check em vez de existence check (Risk 2):
antes: if not (_SIDECAR_DIR / node_modules).exists()
depois: if not (_SIDECAR_DIR / node_modules / spectrum-ts).exists()
spectrum-ts e a unica dependencia do package.json. Checar sua presenca
garante que instalacao parcial/abortada e detectada no boot do gateway,
nao na primeira mensagem recebida via gRPC.
cli.py - stderr capturado e persistido (Risk 3):
subprocess.run passa agora stderr=subprocess.PIPE, text=True em ambas as
chamadas (npm ci e npm install fallback). O stderr capturado e:
- impresso em sys.stderr imediatamente (output visivel no terminal)
- persistido em _NPM_ERROR_LOG = sidecar/.photon-npm-error.log se
returncode != 0, limitado a 300 chars
- apagado de _NPM_ERROR_LOG se returncode == 0 (evita erro stale)
check_requirements() le _NPM_ERROR_LOG quando spectrum-ts esta ausente e
inclui o conteudo no DEBUG log - o erro do npm sobrevive ao terminal, ao
restart do gateway e a reinicializacao da maquina.
sidecar/.gitignore - adicionado node_modules/ e .photon-npm-error.log.
Isolamento - sem impacto no core:
- check_fn() continua retornando apenas bool; core nao e modificado
- Logging usa namespace plugins.platforms.photon.adapter, isolado de
gateway.* e hermes_cli.*
- Cada plugin tem seu proprio check_requirements() independente
- OSError no write/read de _NPM_ERROR_LOG e silenciado - nunca propaga
Testes - 24/24 passando:
test_check_requirements_risks.py (7 testes):
WARNING emitido quando httpx ausente
WARNING emitido quando node nao no PATH
DEBUG emitido (nao WARNING) quando spectrum-ts ausente, com path
node_modules/ vazio agora retorna False (Risk 2 resolvido)
_NPM_ERROR_LOG escrito no stderr do npm em falha
_NPM_ERROR_LOG apagado apos npm bem-sucedido
erro npm aparece no DEBUG log quando node_modules ausente
test_npm_error_log_regression.py (9 testes - vetores de falha da solucao):
return code contrato intacto (0 em sucesso, nao-zero em falha)
OSError no write do log silenciado, exit code ainda propagado
OSError no read do log silenciado, check_requirements() retorna False
stderr vazio nao cria arquivo de log
proc.stderr=None nao lanca AttributeError
log stale apagado apos reinstall bem-sucedido
DEBUG emitido mesmo sem log de erro (setup pela primeira vez)
Closes#50981
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
hermes photon setup unconditionally called regenerate_project_secret()
on every re-run, invalidating the credential held by a running sidecar
and causing all outbound sends to fail with AuthenticationError.
Now validates existing credentials via a lightweight list_users call
before deciding to regenerate. Only rotates when no credentials exist
or the existing ones are invalid, and warns the user to restart the
gateway when rotation occurs.
Fixes#50755
Maintainer follow-up to the #72763 salvage: check_photon_token_valid()
now delegates to the existing validate_photon_token() (session lookup +
/api/projects/) instead of a bespoke get-session-only probe, since the
device flow can mint tokens that pass the session check but fail the
project APIs that setup actually uses. Semantics preserved: definitive
auth rejection = stale, transient errors = probably-valid.
Two related bugs in hermes photon setup / gateway_setup:
Bug 1 -- stale token reused (401)
----------------------------------
_cmd_setup reused an existing dashboard token without validation.
The device token has a short TTL (~3-4 days observed); reusing a
stale token caused every management API call (find_project_by_name,
regenerate_project_secret, etc.) to fail with 401. The operator
saw confusing "spectrum provisioning failed: 401" errors.
Fix: check GET /api/auth/get-session before using the stored token.
On 401/403, clear the stale token with clear_photon_token() and
fall back to a fresh device-login flow automatically.
Bug 2 -- channel left disabled after successful setup
-----------------------------------------------------
After all five provisioning steps completed, config.yaml still had
photon.enabled: false, so the gateway never loaded the Photon
adapter. Every inbound iMessage hit Photon's offline auto-responder
without the operator being notified.
Fix: call write_platform_config_field('photon', 'enabled', True,
raw=True) as a final setup step so the gateway picks up the freshly
configured channel on its next start.
New public API in auth.py:
- clear_photon_token() -- discard stored token from auth.json
- check_photon_token_valid(token) -- lightweight session-check test
References: #72763
store_photon_token/store_project_credentials/store_user_numbers read-modify-
wrote auth.json without hermes_cli/auth.py's _auth_store_lock(), the
cross-process flock every other writer of that file (credential_pool
refresh, model_switch, fallback_cmd, nous_portal adapter, etc.) already
holds. A concurrent write from either side during the unlocked
load-mutate-save window silently drops the other side's update.
Review follow-up: if os.fdopen() raised before taking ownership of the
descriptor returned by os.open(), the cleanup handler unlinked the temp
file but leaked the fd. Close it explicitly on that path, mirroring the
credential-writer cleanup from #62837.
Strengthen the tests so the old writer could not pass them: an os.open
spy asserts O_CREAT | O_EXCL and an explicit 0o600 mode (the final-mode
check alone was also satisfied by the post-write chmod), and a forced
fdopen-failure test asserts the raw fd is closed and no temp file is
left behind.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
_save_auth() wrote the bearer token with tmp.open('w') — created at
process umask (typically 0o644) — and only chmod'ed to 0o600 after the
write, leaving a window where the token sat world-readable. The temp
name was also fixed and predictable (auth.json.tmp), so it could be
pre-planted (symlink attack).
Create the temp file with os.open(O_WRONLY|O_CREAT|O_EXCL, 0o600) and a
per-process random suffix, fsync before the atomic replace, and clean
the temp file up on failure. Mirrors hermes_cli/auth.py:_save_auth_store
(#19673, #21148), which hardened the same pattern in the core writer.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
gemini-2.5-flash shuts down Oct 16 2026 (Google deprecation schedule)
and gemini-3-flash-preview is superseded. Update every hardcoded
default to the current GA flash model:
- gemini_native_adapter: probe_gemini_tier + _create_chat_completion
default params; free-tier guidance de-pinned from a specific model's
RPD number so it doesn't stale again
- auxiliary_client: gemini/kilocode fallback aux models,
_OPENROUTER_MODEL, _NOUS_MODEL -> google/gemini-3.6-flash
(verified live on both OpenRouter and Nous portal /models)
- provider plugins: kilocode + vertex default_aux_model
- hindsight memory plugin: gemini provider default
- setup wizard gemini list: 3-flash-preview -> 3.6-flash (matches the
curated picker catalog)
- tests: aux-client assertions that pinned the old default literal now
reference the _NOUS_MODEL constant, so the next default bump can't
break them (change-detector cleanup)
Fixes#32360.
Photon (iMessage) has no real edit API for already-sent messages. When
streaming completes, the gateway attempts to edit the message to remove
the streaming cursor (▉). Without edit support, this cursor gets stuck
in the final message, corrupting Unicode characters.
This change sets SUPPORTS_MESSAGE_EDITING=False on PhotonAdapter, which
causes the gateway to suppress the streaming cursor entirely for this
platform (via _effective_cursor in gateway/run.py). This prevents the
stale tofu square (▉) from appearing in streamed iMessage responses.
Fixes#49253
Follow-up on the salvaged #71610 commits:
- acquire/release a scoped lock on relay_url+pubkey in connect/disconnect
(IRC pattern) so two profiles can't drive one Buzz identity — duplicate
replies and split de-dupe state; +2 tests
- negative-cache _resolve_user_name failures so a profile-less pubkey
doesn't re-hit 'users get' every poll sweep (flagged by @jethac on the PR)
- register user-guide/messaging/buzz in website/sidebars.ts (page was
unreachable — the #63359 trap)
On hosted relays `buzz dms list` reliably returns [] even when DM
conversations exist, so DMs leaked in via `channels list` (as entries
named "DM" with an empty description) and were seeded chat_type="group".
That put them behind the channel mention gate: "@Chip /whoami" worked
but an un-mentioned DM was silently dropped.
Classify from the Nostr tags of real traffic instead: a message another
user sends in a DM carries a structural ["p", <own pubkey>] tag even
when the text never mentions the agent, while in a real channel a
p-tag-to-self only ever accompanies a visible @mention (typed mention
or reply). A group conversation therefore latches to chat_type="dm" on
the first kind-9 event that is p-tagged to self WITHOUT a visible
mention in the content — guarded by channels-list metadata so a real
community channel (real name / non-empty description) is never
reclassified by a reply or mention that p-tags the agent.
- latch during history seeding too, so a leaked DM bypasses the
mention gate from the very first poll after connect
- keep `dms list` as a best-effort source, and scan `channels list`
as a fallback so DM conversations opened mid-run still get watched
- strip a leading @mention in DMs as well, so "@Chip /whoami" keeps
firing as a slash command after the conversation reclassifies
Refs #68871
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Channel mention-gating was hardcoded on. Add a configurable require_mention
(default True, preserving current behavior). When False, the agent responds to
every message in a watched channel, not only when @mentioned; DMs always
dispatch. Read from config.yaml gateway.platforms.buzz.extra.require_mention
with BUZZ_REQUIRE_MENTION env override, bridged via apply_yaml_config_fn like
the other settings. A leading mention is still stripped when present.
Refs #68871
Channel messages address the agent with a leading @mention (e.g. '@Chip
/whoami'). The adapter passed the raw content through, so the gateway's
is_command() check (text.lstrip().startswith('/')) never matched and slash
commands were routed as plain chat. Strip a leading mention (name, npub, or
hex form) before dispatch in channels, mirroring the Discord adapter. Also
cleans normal prompts ('@Chip what's up?' -> 'what's up?'). DMs are untouched.
Verified live: '@Chip /whoami' -> '/whoami' after connect populates identity.
Refs #68871
The Buzz adapter's check_requirements() reads config from env only, so a
config.yaml-only setup (relay URL in gateway.platforms.buzz.extra) failed the
check_fn gate and was silently skipped at startup. Add an apply_yaml_config_fn
hook that bridges buzz.extra -> BUZZ_* env vars, mirroring the Slack/Telegram
pattern; BUZZ_PRIVATE_KEY stays a .env secret.
Also fix send_reaction() to use buzz-cli's real flags (--event <id> --emoji),
replacing the non-existent --channel/--message-id flags that would have failed
on every message. Verified live against the hosted relay (accepted:true).
Refs #68871
Plugin-path adapter (zero core changes) connecting Hermes to a Buzz
community relay via the buzz CLI binary (JSON in/out, arg-list exec,
key passed via env only). Inbound uses a poll loop with per-channel
high-water marks seeded from newest (no history replay), event-id
de-dupe, self-echo suppression by pubkey, and mention gating in
channels (DMs always dispatch). Registers env_enablement, cron
home-channel delivery, and an out-of-process standalone sender,
mirroring the IRC plugin.
Verified against a live relay: connect -> send -> poll -> MessageEvent
round-trip, self-echo suppressed, clean disconnect.
Known limitation: polled inbound (default 4s); a websocket transport
(buzz-ws-client) is a future optimization.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Move U+FFFC placeholder detection before _record_last_inbound() so the
placeholder message id is not recorded as the reaction target
- Cancel pending U+FFFC tasks in disconnect() to prevent task leaks
- Check mimeType audio/x-caf in addition to filename for CAF→VOICE promotion,
fixing unnamed attachments that default to '(unnamed)'
- Add 7 Photon adapter tests: CAF named/unnamed promotion, U+FFFC no-dispatch,
U+FFFC not recorded as last inbound, U+FFFC+attachment cancel, timeout fire,
disconnect cleanup
- Add 6 transcription tests: ffmpeg conversion, afconvert fallback, all-fail,
CAF→WAV before Groq, conversion failure error, local provider skip
iMessage's gRPC cloud push fires before CloudKit syncs the attachment
metadata, emitting a U+FFFC placeholder as text. Instead of dispatching
a spurious message, start a non-blocking 15s wait. If the real attachment
arrives, cancel the timeout and process normally. If not, log a warning.
spectrum-ts classifies all inbound attachments as type 'attachment',
never 'voice'. Without this, .caf voice notes are routed as AUDIO or
DOCUMENT and bypass the STT pipeline.
The msg_type_str == "richText" branch reset msg_type to PHOTO/TEXT after
the rich-text item scan had already promoted it (e.g. a native voice item
→ VOICE), dropping voice notes from the auto-STT path. Only re-derive when
the scan left the type at TEXT.
Ports the root-cause analysis from PR #38276 (stale, targeted the deleted
gateway/platforms/dingtalk.py) onto the live plugin adapter, with
regression tests.
Refs #38211#38219#38276
- Promote EXT_MAP to a module-level constant for reuse
- Classify DingTalk image messages and image file attachments as PHOTO
- Extract DingTalk card/interactiveCard document link content with defensive parsing
- Handle None, empty, JSON, and plain-string card content fields
- Add coverage: 8 card + 4 interactiveCard + 5 _extract_media = 17 test cases
- Remove a shadowed duplicate TestExtractText class (pytest collected only the later one)
3 fixes for the dingtalk-platform plugin (plugins/platforms/dingtalk/adapter.py):
1. _extract_text: parse ASR recognition text from voice messages via
extensions['content']['recognition'], fall back to fileName for
file messages ("[文件] xxx")
2. _extract_media: handle audio/file/image msgtype_str with correct
MIME mapping; exclude audio from media_urls to preserve DingTalk's
built-in ASR result (avoids failed whisper re-transcription)
3. _resolve_media_codes: parse downloadCode from extensions['content']
for file/image message types
All tested with live DingTalk group chat: voice → recognition text,
PDF/Markdown file → filename + download.
Lark's native "audio" msg_type is an in-app voice recording — uploaded
audio files arrive as "file"/"media". But _resolve_normalized_message_type
resolved the "audio" preferred type to MessageType.AUDIO, which the gateway
treats as a non-transcribed file attachment (run.py: AUDIO -> audio_file_paths,
"never STT"; VOICE -> audio_paths, "always STT"). Result: a Feishu voice
note reached the agent as an untranscribable audio attachment and was
silently ignored — the user's spoken message never became text.
Every other platform that receives native voice notes (Telegram, Discord,
Slack, WhatsApp, Signal, Matrix, WeChat, WeCom, DingTalk, QQ, BlueBubbles,
Mattermost, Yuanbao) classifies them as MessageType.VOICE. Feishu was the
only one classifying them as AUDIO. This is the follow-up to #28993, which
added native voice-note transcription for Discord + DingTalk but did not
cover Feishu.
Return MessageType.VOICE for the "audio" branch. The branch is reached only
for Lark's top-level audio msg_type (set in the normalizer; file uploads map
to "document"), so VOICE is unconditionally correct here — no risk of
auto-transcribing an uploaded music/audio file.
- plugins/platforms/feishu/adapter.py: _resolve_normalized_message_type audio
branch returns VOICE instead of resolving to AUDIO via mime.
- tests/gateway/test_feishu.py: test_extract_audio_message_downloads_and_caches
asserted the old AUDIO behavior on a fixture literally named voice.ogg —
updated to expect VOICE (the corrected classification).
- tests/gateway/test_feishu_voice_message_type.py: new focused regression
tests (audio->VOICE with and without mime; photo/document/text unaffected).
Rebased onto current main: the Feishu platform moved from the single-file
gateway/platforms/feishu.py into the plugins/platforms/feishu/ package; the
fix applies to the same _resolve_normalized_message_type logic at its new home.
Verified the new voice tests fail when the branch resolves to AUDIO and pass
with VOICE, while the photo/document/text cases are unaffected either way.
Note: classification is mock-tested here; the downstream STT pipeline is the
shared, already-proven path (#28993). End-to-end verification on a live
Feishu account would be a welcome confirmation.
Refs #28993 (sibling-gap: Feishu was the platform left uncovered).
Keep one owner for PATH/local-prefix ffmpeg discovery: ffmpeg_utils now
delegates to tools.transcription_tools._find_ffmpeg_binary and only adds
the Discord-specific FFMPEG_PATH override and Windows winget fallback on
top (follow-up to PR #60627 by @LauraGPT, fixes#60624).
Discord's voice socket needs a brief warm-up before receiving clients
actually hear audio; the first ~100-200ms is lost, clipping the first
word/syllable of TTS playback. Prepend a configurable lead of silence to
speech on both playback paths:
- Mixer path: new _lead_silence_bytes() helper prepends PCM silence
(BYTES_PER_MS constant added to voice_mixer.py) before play_speech, on
both the reply and the pre-tool ack.
- Legacy FFmpegPCMAudio path: apply -af adelay=<ms>:all=1.
Tunable via discord.voice_fx.lead_silence_ms (default 200, 0 disables).
Fixes#66827
VoiceMixer duck-typed the discord.AudioSource interface (is_opus, read,
cleanup) but never inherited from it. discord.py's vc.play() does an
isinstance check and rejects non-AudioSource objects, causing the voice
fx mixer to silently fail with:
"Voice mixer failed to start: source must be an AudioSource not
VoiceMixer"
Added missing `import discord` and changed the class definition from
`class VoiceMixer:` to `class VoiceMixer(discord.AudioSource):`.
- Wire adapter._voice_input_callback at connect and reconnect so voice
transcription is forwarded without requiring /voice join (#60623).
- Add optional text_channel_id and source params to DiscordAdapter
.join_voice_channel() so automatic/programmatic voice joins can
establish the text-channel binding needed by _handle_voice_channel_input.
- Add TestVoiceInputCallbackWiring: asserts callback wiring on startup
and reconnect for Discord adapters with voice attributes.
Closes the #32029 gap: TelegramAdapter.send_voice passed captions raw with
no parse_mode, so auto-TTS captions (which carry the agent's markdown
reply) showed literal *asterisks*, backticks and [links](...). Captions
are now formatted to MarkdownV2 via the adapter's format_message when the
formatted text fits Telegram's 1024-char caption cap, with fallback to the
plain truncated caption when formatting overflows or the Bot API rejects
the entities (mirrors the text-send markdown fallback ladder).
Fixes#32029
Salvaged from PR #53157 (@LLQWQ). Three changes for native Feishu voice
bubble delivery:
1. Include audio duration in the file-upload body when uploading opus
files — Feishu renders 0:00 bubbles without it. Duration is extracted
by parsing the OGG container's last granule position (pure Python,
no ffprobe dependency).
2. Thread routing fallback: Feishu's create-message API rejects
msg_type='audio' with receive_id_type='thread_id' (error 99992402);
retry via the reply API against the thread's last message, then fall
back to chat_id routing.
3. (dropped) tools/tts_tool.py want_opus hunk — superseded by main's
OPUS_VOICE_PLATFORMS, which already includes feishu. The PR's stray
scripts/release.py hunk was also dropped (frozen AUTHOR_MAP policy;
mapping added under contributors/emails/ instead).
Fixes#45557
Refs #18831#16524
Salvaged from PR #18861 (@shellybotmoyer). Switch the central Opus
transcoders from CBR 64k (-vbr off) to VBR 48k with -application voip and
-compression_level 10. The old CBR settings produced lower-quality speech
audio and occasionally failed to trigger Telegram's native voice-bubble
rendering. Applied to _ffmpeg_transcode_to_opus (the single transcoder
behind _convert_to_opus and _repair_ogg_container), the Gemini WAV→OGG
path, and the Matrix adapter boundary transcoder added in this branch.
Fixes#18818
Salvaged from PR #68063 (@malaiwah). MatrixAdapter.send_voice now transcodes
any non-Ogg audio to Ogg/Opus at the adapter boundary (best-effort — the
original file is sent unchanged when ffmpeg is unavailable), so MSC3245
voice bubbles render even when a caller hands the adapter MP3/WAV audio.
_matrix_voice_metadata_for_file probing now runs via asyncio.to_thread so
ffprobe/ffmpeg subprocess timeouts can't stall the adapter event loop.
The PR's tools/tts_tool.py want_opus hunk was dropped: main's
OPUS_VOICE_PLATFORMS set (PR #73072) already includes matrix; the Matrix
opus-routing test is kept.
Refs #14841
Salvaged from PR #68063 (base commit, runner-path hunks superseded by the
platform-aware OPUS_VOICE_PLATFORMS fix in this branch). Element and other
Matrix clients render voice bubbles more reliably when m.audio events carry
duration and waveform metadata; probe both best-effort via ffprobe/ffmpeg.
The native Gemini provider profile's default_aux_model and the curated
model picker catalog were still pinned to gemini-3.5-flash, a stale
generation now superseded by gemini-3.6-flash (documented GA). Bump
both so the auxiliary-task default and the picker stay in sync with
the current model.
Contract test asserts the durable lockstep invariant only
(default_aux_model is a member of _PROVIDER_MODELS["gemini"]) rather
than pinning either side to a frozen model-name string, so it doesn't
need updating on the next model-generation bump.
- Change await self._send_read_receipt to asyncio.create_task to avoid
blocking message dispatch on slow bridge responses (matches BlueBubbles
pattern). Up to 5s per-message latency eliminated.
- Update tests: assert_called_once_with instead of assert_awaited_once_with
since the receipt is now scheduled, not directly awaited.
- Add send_read_receipts documentation to whatsapp.md following the
BlueBubbles docs pattern.
Direct tool HTTP calls already identified as Hermes-Agent, but the main
OpenAI-SDK chat path still sent OpenAI/Python. Set Hermes-Agent/<ver> for
api.x.ai clients (xai + xai-oauth) so normal text traffic is attributed correctly.
The per-IP httpx transports were built once in __init__ and never torn
down. A connect that reached ESTABLISHED and was then closed by the peer
left its socket in CLOSE_WAIT inside the pool, and the failure path only
logged and continued — so the poisoned pool was retained and leaked one
descriptor per retry.
With DNS for api.telegram.org failing, every poll fell through to the
seed IP and leaked another fd every ~2.5s. The bot gateway reached 177
CLOSE_WAIT sockets against launchd's 256 soft limit and wedged: accept()
on the gateway port, config reads and DNS resolution all failed with
EMFILE, which in turn made the primary path fail and fed the loop.
Build fallback transports lazily and discard them on a retryable connect
failure, and bound every pool at 8 connections (httpx defaults to 100,
so two seed IPs plus primary could alone exceed the fd ceiling).
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
_check_local_runtime() only imported 'hindsight' and
'hindsight_embed.daemon_embed_manager', a strictly weaker import surface
than the embedded daemon actually needs: the daemon imports
sentence_transformers at startup (embeddings + reranker). When the
embedding stack is broken (e.g. a dependency conflict on a shared package
like huggingface-hub), the daemon can never start, yet is_available() and
'hermes memory status' still report Hindsight as available — every
retain/recall then fails silently.
Import sentence_transformers in the same probe so local/local_embedded
availability goes red with the real ImportError as the reason, letting
the agent degrade loudly instead of silently dropping memory. Local path
only; cloud mode is untouched and no network or model download is
triggered by the import.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MN8RMDLwxCfFxwtADoEJJf
Stop offering deepseek-chat/reasoner in the static catalog and point
fallback/aux defaults at the permanent v4 IDs. Keep retired aliases in
a detection-only map so /model deepseek-chat still resolves to deepseek.
MEDIA_DELIVERY_EXTS in gateway/platforms/base.py omitted .3gp, causing
MEDIA: tags with .3gp files to leak as plain text instead of being
extracted for native video delivery. _VIDEO_EXTS in
tools/send_message_tool.py and _MIGRATION_VIDEO_EXTS in the Feishu
adapter omitted .webm, causing .webm files to be classified as
documents instead of video on Telegram and other platforms.
Both extensions are already present in every gateway-side _VIDEO_EXTS
definition (run.py, kanban_watchers.py, weixin.py, base.py local).
Closes#71621, Closes#71603
Surface a user-visible delivery notice when non-streaming media dispatch
gets success=False after MEDIA tags were stripped, and validate forum
starter-message attachments the same way as direct channel sends (#66797).
Outbound MEDIA video/document tags on the Discord non-streaming path
were extracted (stripped from the visible text) but never delivered:
no attachment, no error, and no dispatch log line (#66797). The
open-handle discord.File plus singular file= form could race Discord's
multipart encoder after an earlier image batch on the same channel and
return a successful message carrying zero attachments.
Fix _send_file_attachment (used by send_video/send_document/
send_image_file) to:
- use a path-based discord.File via the plural files=[...] kwarg, the
same pattern as the working send_multiple_images batch path;
- pre-flight os.path.isfile and return a File not found result instead
of raising deep inside discord.File;
- fail loud when Discord accepts the message but attaches nothing, so
the dispatch loop surfaces a warning rather than a silent drop;
- add INFO dispatch logs (base non-image MEDIA fan-out, video send, and
file attachment) so a MEDIA:.mp4 cannot vanish without a trace.
Tests (tests/gateway/test_discord_send.py):
- path-based files=[...] kwarg used, singular file= unset;
- fail-loud when the returned message has no attachments;
- missing file fails fast without resolving the channel;
- forum-parent delivery routes through create_thread with files=[...];
- end-to-end image+video response routes the mp4 to send_video while
images still batch.
Fixes#66797
The identity-refresh TTL used 0.0 as the "never checked" sentinel and
compared it against time.monotonic(). That epoch is arbitrary and starts
near zero on a freshly-booted host, so on CI runners and containers
`monotonic() - 0.0` was itself below the TTL — "never checked" read as
"checked just now" and the first identity refresh was suppressed for the
first 5 minutes of uptime. The stale-handle recovery therefore did nothing
on exactly the machines most likely to be freshly booted.
Invisible on a long-lived dev box (uptime >> TTL); caught by CI.
- Sentinel is now None, meaning never checked and always stale.
- Both TTL comparison sites route through _bot_identity_is_fresh().
- Regression test pins the invariant under a faked 12s-uptime clock.
Verified by re-running the recheck tests against a simulated 3s-uptime
host: green with the fix, and restoring the 0.0 sentinel reproduces the
exact CI failure locally.
Renaming a Telegram bot's @username in BotFather silently stopped the
gateway from answering in groups.
PTB caches getMe() in Bot._bot_user and only rewrites it inside get_me(),
so after a rename the adapter kept comparing mentions against the OLD
handle. The exclusive-mention gate then saw the new @handle, failed to
match itself, and concluded the message was addressed to a different bot
— dropping it before the reply and wake-word fallbacks could run. Native
replies to the bot were discarded too. Polling mode recovered on the next
90s heartbeat; webhook mode never calls get_me() again, so it stayed dead
until restart.
Separately, the bot-handle pattern assumed every bot username ends in
"bot". Collectible (Fragment) usernames can be assigned to bots and drop
that suffix (@jarvis, @pic), so such a bot could not recognise its own
handle in the entity-less fallback and was suppressed by any message that
also named another bot.
- Route every mention comparison through _current_bot_username(), which
prefers the last observed handle over PTB's cache.
- Learn the live handle from inbound updates: Telegram stamps the current
username on our own messages and on reply_to_message. Guarded by user
id, so another account's handle is never adopted.
- Re-check identity out of band (TTL-bounded, one getMe per 5 min) when
the exclusive gate is about to drop a message — the exact stale-handle
symptom — so the mistake self-corrects instead of persisting.
- Refresh identity in webhook mode via a dedicated low-frequency loop,
cancelled on the same teardown fence as the heartbeat.
- Match our own handle by identity rather than shape. Foreign handles keep
the deliberate "...bot" narrowing so human @handles still never act as
routing hints (the intent behind ce4d857021).
Validation: 11 regression tests; sabotage runs confirm each behavioral
test fails with the fix reverted. 157 tests green across the Telegram
gating, reconnect, and topic-mode suites.
The sidebar strip and the Channels page could contradict each other on
the same page load — "Gateway running" next to "The gateway is not
running." /api/status and /api/messaging/platforms each open-coded their
own liveness ladder: status probed GATEWAY_HEALTH_URL and scoped its
PID/state reads to the requested profile, messaging did neither and used
the uncached raw PID probe.
Three deployments hit the split: a cross-container gateway (no local PID,
only the health probe can see it), a profile-scoped dashboard (messaging
borrowed a DIFFERENT profile's runtime state, reporting a false
"connected" that hides a real outage — #71211), and a launch-service
managed gateway with no PID file.
Adds resolve_gateway_liveness() in gateway/status.py as the single ladder
(cached PID -> HTTP health probe -> runtime-status PID with
expected_home) and routes both endpoints, /api/messaging/platforms/{id}/test,
and the kanban dispatcher-presence probe through it. Probe callables are
injectable so the existing monkeypatch seams keep working, and
GatewayLiveness.probe_error distinguishes "down" from "couldn't tell" so
the kanban warning keeps failing OPEN instead of crying wolf.
Closes#71211.
Auxiliary LLM calls routed through OpenRouter now carry the `session_id`
sticky routing key, so they pin to the same provider endpoint as the
conversation they belong to instead of routing independently.
`OpenRouterProfile.build_extra_body` sourced the key only from its explicit
argument. Auxiliary call sites — compression, title generation, vision,
web_extract, session_search, MoA slots — funnel through
`agent.auxiliary_client`, which has no session handle and never passes one.
Those calls sent NO sticky key at all, so OpenRouter fell back to hashing
the opening messages and each aux call could land on a different provider
than the conversation it served. Per OpenRouter's prompt-caching docs a
present `session_id` is used directly as the routing key and activates
stickiness on the first successful request rather than only after a cache
hit, so the missing key also delayed pinning for the main loop.
Resolve it from the ambient conversation contextvar first, explicit argument
as fallback — the same resolution the Nous Portal profile uses (f2f4df064d)
and the same one `nous_portal_tags` already used for the `conversation=` tag.
The ambient value is the session-lineage ROOT, so the key also stays stable
for installs that opt out of the default `compression.in_place: true` and
across delegate-subagent trees.
The xAI `x-grok-conv-id` cache-affinity header in `build_api_kwargs_extras`
had the identical gap and is fixed the same way; it remains model-gated to
`x-ai/grok-*` / `xai/grok-*`.
Fixes#70820. Credit to @webtecnica, who reported it and submitted #70883
first; that PR threaded `session_id` through `call_llm` to reach the same
goal. This takes the ambient-contextvar route instead because none of the
34 `call_llm` call sites currently pass a `session_id`, so the parameter
alone would not have changed any live request.
Tests: 2 cases in tests/providers/test_provider_profiles.py, both verified
to fail against the pre-fix behavior via a sabotage run. E2E-validated on a
real AIAgent driving the out-of-turn compaction path.
The Nous Portal profile publishes a top-level `session_id` that the Portal
uses as its sticky routing key, pinning a conversation to one upstream
endpoint so explicit Anthropic `cache_control` breakpoints stay warm
(those caches are instance-local, so a reroute cold-writes instead of reads).
`NousProfile.build_extra_body` sourced that key only from the explicit
`session_id` argument. Auxiliary call sites — compression, title generation,
vision, web_extract, session_search, MoA slots — funnel through
`agent.auxiliary_client`, which has no session handle and never passes one.
They carried the `conversation=` tag but NO sticky key at all, so each one
routed independently of the conversation it belonged to.
Resolve the key the way `nous_portal_tags` already resolves that tag:
ambient lineage-ROOT contextvar first, explicit argument as fallback. Fixes
every aux call site at once with no per-call-site plumbing.
Also publish the root in the `_compress_context` forwarder when nothing is
ambient. Out-of-turn entry points (`/compact`, the gateway `/compress`
command and its hygiene sweep, partial head compression) call it outside
`run_conversation`'s scope, so the summarizer's aux call ran untagged and
unkeyed; the caller's value is restored in `finally` so a compaction never
leaks its tag.
Scope note: this does NOT keep the compaction turn's own prompt cache warm.
Compaction replaces the history with a summary and rebuilds the system
prompt, so that request is a cold write on any endpoint — what a stable key
buys is the turns AFTER compaction reading the cache it wrote. Under the
default `compression.in_place: true` (#38763) the session id does not rotate
at compaction, so for the main loop the ambient root and the explicit
argument already agree; the ambient resolution additionally holds the key
stable for installs that opt back into rotating compaction and across
delegate-subagent trees.
Salvaged from #71747. Subsumes the aux-call half of #70883 (@webtecnica),
which threaded `session_id` through `call_llm` to close the same gap.
Tests: 4 cases in tests/agent/test_portal_tags.py; both fix halves verified
to fail against the pre-fix behavior via sabotage runs.
The Codex image backend rejected our own request shape for every account, and
we then translated that rejection into "Image generation is not enabled for the
current Codex account. Switch the image provider to OpenAI API key, FAL, or
xAI." — telling every affected user to abandon a provider that had never
actually been tried. That message is why this reads as a setup failure rather
than a bug: the wire error was replaced with a confident, wrong diagnosis.
Removes the classifier and its exception, so any HTTP failure surfaces
verbatim. The paired request-shape fix (previous commit) is what makes the
400 stop happening; this commit makes the next one diagnosable.
Also fixes error-body truncation: bodies were head-truncated at 500 chars, and
Codex error payloads can carry hundreds of bytes of leading metadata, so the
user got a wall of padding and no message. _summarize_error_body() prefers the
parsed error.message and falls back to a truncated raw body.
Docs: drop the unqualified image-to-image claim for the Codex backend and note
that the hosted tool call cannot be forced, so it is best-effort.
Verified E2E against a local fake Codex backend: success path writes a real PNG
with no tool_choice on the wire; the 400 path now returns api_error carrying
"Tool choice 'image_generation' not found in 'tools' parameter" (148 chars)
instead of the entitlement message. Sabotage run confirms all 4 regression
tests fail when the old behavior is restored.
Refs #19505, #49008, #31335.
The chatgpt.com/backend-api/codex backend 400s on every tool_choice
shape for the hosted image_generation tool — it looks up tool_choice as
a function name and never recognizes hosted-tool entries. Removing the
field from _build_responses_payload() lets the host model decide; the
instructions field nudges it toward the tool.
Salvaged from PR #19979 (originally targeted the old
client.responses.stream call, which no longer exists on upstream/main;
the live request now flows through _build_responses_payload + httpx in
_collect_image_b64).
Pair conditional approval buttons into rows of two so the full Allow Once /
Session / Always / Deny set stays readable instead of one truncated 4x1 row.
Follow-up hardening for the salvaged #69240 readiness gate (#67498):
- _start_polling_once now returns its (generation, progress_event) pair
so the strict cold-start gate binds to exactly the generation it
started, instead of re-reading self._polling_progress_event which a
concurrent recovery task may have replaced with a newer generation's
event (the G1/G2 race flagged in the #69240 review).
- Strict cold start no longer schedules background polling recovery: a
polling error during the readiness wait is captured by a strict
callback and fails the connect attempt immediately with a loud
OSError, so GatewayRunner disposes the partial adapter and retries
with a fresh one — no more waiting out the full readiness deadline on
a generation that already errored, and no G2-on-partial-app healing.
- After readiness is proven the strict callback delegates every later
polling error to the real background-recovery callback, preserving
the existing degraded/reconnect semantics for the polling lifetime.
- The readiness-timeout error message now states the deadline and that
the gateway will retry with a fresh adapter (loud failure, not a
silent wait).
- Regression tests: current-generation progress connects; a polling
error during strict cold start fails fast without scheduling
background recovery (the #67498 idle-threads shape); stale-generation
progress is rejected.
Progresses #67498
Use wall deadlines for deleteWebhook and start_polling, then fail cold startup unless getUpdates proves progress. This lets the gateway discard partial PTB state and retry with a fresh adapter.\n\nRefs #67498
The cherry-pick auto-resolution + AST sweep applied plain utf-8 at three
.env reader sites where the salvaged PRs (#62617, #62123) deliberately
use utf-8-sig — a Notepad BOM must not hide/duplicate the first key.
Restore the contract (tests pin it).
AST-driven pass over every Path.read_text()/write_text() without an
explicit encoding= across non-test code: 71 sites in 34 files
(skills_hub, hermes_cli/main+profiles+service_manager+container_boot,
mem0/hindsight/honcho plugins, achievements dashboard, release/CI
scripts, productivity+comfyui skill helpers, agent/*). Verified zero
positional-encoding collisions before insertion; per-file compile()
check after.
Adds a check-windows-footguns rule flagging bare single-line
read_text/write_text (multi-line forms stay covered by the AST guard
test from #38985). Together with the salvaged contributor commits this
retires the ~169-site bare file-I/O class (#37423's long tail).
Follow-up to review feedback:
- mem0 _prompt_api_key read .env with the locale default, so a Notepad
BOM hid the first key from the masked current-value lookup; read it
with utf-8-sig + errors=replace like the canonical readers in
hermes_cli/config.py.
- hindsight _load_simple_env used plain utf-8; it also parses the Hermes
.env during post_setup, where a BOM stuck to the first key. Switch to
utf-8-sig + errors=replace.
- Add hindsight regressions: BOM key matching in _load_simple_env and in
the cloud post_setup writer, plus non-ASCII round-trip preservation,
and a mem0 regression for the BOM'd masked-key lookup. The BOM tests
fail without the fix on any platform.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The mem0 and hindsight memory-provider setup routines round-trip the
user's ~/.hermes/.env: they read existing lines, update the keys they
manage, and rewrite the whole file preserving every other line verbatim.
Both used env_path.read_text() / write_text() with no encoding.
read_text()/write_text() with no encoding fall back to the system locale
(cp1252/GBK on Windows), so on a non-UTF-8 host the preserved lines get
mangled or the call crashes on any non-ASCII value, and — because the
reader never strips a BOM — a Notepad-edited .env makes the first key
fail the in-place match and get duplicated instead of updated.
Match the canonical .env readers in hermes_cli/config.py: read with
encoding='utf-8-sig' (BOM-tolerant) and write with encoding='utf-8'.
mem0/_setup.py already pins utf-8 for mem0.json, so this just aligns the
.env path in the same file. Fixes both memory plugins in one class fix.
Adds regression tests: a BOM'd .env updates the first key in place
(locale-independent, fails without the fix) and non-ASCII existing lines
survive the round-trip.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
On Windows with CJK locales (e.g. Chinese/GBK), pathlib.Path.read_text()
defaults to the system encoding instead of UTF-8, causing UnicodeDecodeError
when reading .env or .json config files that contain non-ASCII characters.
Explicitly pass encoding='utf-8' to all read_text() and write_text() calls
in the hindsight memory provider plugin.
Path.read_text() without an explicit encoding uses the platform's
default encoding. On Windows this is typically cp1252 or mbcs, which
causes UnicodeDecodeError or silent data corruption when reading
UTF-8 content (JSON files, user text, config with non-ASCII chars).
This is the read-side companion to the write_text() encoding fix.
Fixed the most critical locations that read JSON data, user content,
and config files across 14 files with 31 call sites.
Pattern: .read_text() → .read_text(encoding='utf-8')
json.loads(path.read_text()) → json.loads(path.read_text(encoding='utf-8'))
Follow-up to the salvaged #38985: guard the 4 bare read_text/write_text
sites its allowlist missed (google_chat thread-count store + oauth JSON)
and add whatsapp/google_chat to the AST guard test's file list.
- Strip the salvaged commit's inline encoding kwargs where main had since
gained its own (process_registry, local env, cua doctor, gateway,
commands, gateway_windows — the latter keeps its locale-aware
_schtasks_encoding() from #38186)
- Revert encoding kwargs mistakenly applied to non-subprocess APIs
(exa get_contents, tempfile.mkstemp in webhook.py)
- Guard the ddgs worker Popen (new on main since #55339)
- Update two kwarg-snapshot test assertions for the new kwargs
AST-driven pass over every subprocess.run/Popen/check_output/check_call/call
with text=True (or universal_newlines=True) and no explicit encoding=:
append encoding='utf-8', errors='replace' at the kwarg site. 136 call
sites across 28 files (cli.py, hermes_cli/main.py, tools_config.py,
environments, computer_use, gateway, scripts, skills helpers, agent/*).
Together with the salvaged #55339/#60741 commits this closes out issue
#53428's bug class; the salvaged #60751 linter rule in
check-windows-footguns.py now enforces it repo-wide (verified: 807 files
scanned, zero findings).
On Windows with Chinese locale (GBK), subprocess.run(text=True) without
explicit encoding causes UnicodeDecodeError crashes. This fix adds
encoding='utf-8', errors='replace' to all subprocess.run() and
subprocess.Popen() calls that use text=True across 76 non-test Python files.
Fixes#53428 (master tracker for Windows GBK locale crash).
Note: credential_pool.py and electron changes excluded per reviewer request —
those will be submitted as separate focused PRs.
The Telegram adapter's connect retry loop could silently stall after
'Connecting to Telegram (attempt 1/8)...' with the event loop permanently
parked in select() — all threads idle, no attempt 2/8 ever scheduled.
Root cause analysis:
- The retry loop reused the same Application object across all
8 attempts. After a failed initialize() the app could be in a partially-
initialized state (closed httpx transports from ,
or flag set before the hang) causing subsequent calls
to silently skip real initialization.
- CancelledError (a BaseException, not an Exception) propagated silently
through all except handlers with no logging — the task driving the retry
loop could exit without any trace.
- No total watchdog bound existed for the entire retry loop; only per-attempt
timeouts via _await_with_thread_deadline. If the loop itself stalled
between attempts (between-attempt sleep, cleanup, or scheduling), there
was no timeout to catch it.
Fixes:
1. **Total watchdog deadline**: Compute a total deadline for the entire
connect loop (8 attempts × init_timeout + 120s margin). Before each
attempt, check the wall clock; if exceeded, raise OSError immediately
instead of attempting another initialize().
2. **Fresh Application per retry**: On each failed attempt, rebuild
via and re-register all handlers. The old
app is best-effort shutdown with . This ensures
each retry starts with a clean slate — no stale transports, no stale
flag, no leaked state from the previous attempt.
3. **BaseException logging + propagation**: Added
(placed LAST after all other handlers) to log CancelledError and other
non-Exception signals before propagating. Previously these exited the
retry loop silently with no log message.
4. ** block for app rebuild**: The clause runs after
every failed attempt that isn't the last, rebuilding the app and
discarding the old one regardless of which exception class caused the
failure.
RFC 8628 §3.2 makes the device-authorization `interval` optional with a
client-side default of 5 seconds. request_device_code required it, so a
compliant AS that omitted it hit the malformed-response path and the flow
could never complete. Fall back to 5s and cover it with a regression test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds a device authorization grant flow alongside the existing loopback
OAuth flow, so `hermes setup` can connect to Honcho cloud from SSH and
other no-browser environments.
- oauth.py: new HTTP seams — _http_post_form_status (non-raising, since
RFC 8628 polling reads the OAuth error off a 400) and _http_get_json
for the RFC 8414 metadata probe
- oauth_flow.py: DeviceCode, request_device_code, poll_for_token with
slow_down backoff (+5s, capped at 60s) bounded by expires_in, typed
errors (AccessDenied, DeviceCodeExpired, AuthorizationTimeout), and
supports_device_login (fail-closed metadata gate); device flow ends in
the same install_grant tail as loopback so refresh/status work
unchanged
- oauth_flow.py: loopback callback now serves a "sign-in was not
completed" page on consent cancel instead of the success page
- cli.py: cloud menu offers oauth / device / apikey; the device option
only appears when the host advertises the grant, and becomes the
default when no browser is detected
- 18 new tests covering the full flow against a local fake AS, backoff
schedule, error mapping, deadline bound, metadata gate, and wizard
branches
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Follow-up hardening on the salvaged _ensure_client() (#21130 fix):
- Failed-config cooldown: after a refresh attempt fails for a given
resolved config, skip re-probing for 30s. Previously every provider
access against a down endpoint paid a 3s health probe under
_client_refresh_lock and emitted a warning (2+ per turn, some on
user-facing threads: prefetch, tool calls, session end). Retries
still happen after the cooldown or immediately when config changes,
and the log message now says so instead of the false 'disabled until
config changes'.
- Atomic connection snapshot: _conn_snapshot (5-tuple, single
assignment) is published only after a health check passes.
_new_client() and on_memory_write's writer read it as one load, so
background writers can no longer observe a torn mix of old/new
identity fields mid-refresh or target an endpoint that never passed
health. Field writes in _ensure_client_locked keep tracking the
attempted config for the unchanged-config dedupe.
- _env_refresh_enabled moves to the top of initialize(): an exception
mid-initialize (swallowed by MemoryManager) can no longer leave the
provider silently stuck in never-refresh mode.
- _search_prefetch_context reuses _new_client() and degrades to ''
on construction failure instead of propagating.
Mutation-checked: neutering the cooldown or publishing the snapshot on
failed health makes the new regression tests fail.
Avoid spawning multiple local OpenViking server processes while a runtime autostart waiter is already active. Remote endpoints still retry on later accesses because they do not install a local waiter.
Route refreshed unreachable local OpenViking configs through the existing runtime recovery path so /reload can attach to a locally starting server instead of disabling memory until restart.
(cherry picked from commit 040e18ad90)
The _needs_trusted_identity_retry method was hard-coding specific
server-side error strings to detect when a request failed due to
missing X-OpenViking-Account / X-OpenViking-User headers. Each new
server-side error variant required another string added to the client.
Replace the string enumeration with a structural match: the error
message mentions one of the tenant headers AND the HTTP status is 400.
This covers all current error variants:
- "Trusted mode requests must include X-OpenViking-Account and User"
- "ROOT requests to tenant-scoped APIs must include X-OpenViking-Account"
- "Trusted mode requests must include X-OpenViking-Account."
- "Trusted mode requests must include X-OpenViking-User."
The 400 status guard avoids false-positives on 403 errors such as
"USER API keys cannot override X-OpenViking-User", which must not
trigger a retry.
All 176 existing tests pass.
(cherry picked from commit 5a24d6766c)
`OpenVikingMemoryProvider.shutdown()` joins in-flight writers, deferred-commit
threads, and prefetch threads, but not `_runtime_start_thread` — the tracked
`daemon=True` waiter that runs `_finish_runtime_openviking_start`, which blocks
on network health probes (`_wait_for_openviking_health` polling + a
`_VikingClient.health()` request).
If the local OpenViking runtime is slow or unreachable, that waiter can still
be blocked in network I/O at interpreter exit. CPython then forcibly kills it
during `Py_FinalizeEx` (`PyThread_exit_thread` -> `__pthread_unwind` ->
`abort()`), producing SIGABRT (exit 134) with no traceback — the same daemon-
thread-at-exit failure class fixed for the Honcho provider.
Fix:
- `shutdown()` now joins `_runtime_start_thread` (timeout-bounded) alongside the
other tracked threads.
- `_wait_for_openviking_health()` gains a `should_stop` callback; the waiter
passes `lambda: self._shutting_down` so the poll loop bails out promptly once
`shutdown()` flips the flag, instead of lingering up to the 60s autostart
timeout and timing out the join (which would leave the thread alive).
- Add tests/plugins/memory/test_openviking_shutdown.py covering the short-circuit
and the shutdown-joins-runtime-thread behaviour.
(cherry picked from commit 5471ec7021)
initialize() snapshots OPENVIKING_* into the provider once, so /reload
(which only updates os.environ) leaves viking_* tools running against
stale auth — users have to restart hermes to pick up keys added to
~/.hermes/.env after startup.
Add _ensure_client(), which re-resolves the connection settings via the
same _resolve_connection_settings/_load_hermes_openviking_config path
initialize() uses and rebuilds + health-checks the client only when an
OPENVIKING_* value actually changed; otherwise it reuses the cached
client so the hot path stays at one dict comparison with no network
calls. Every `if not self._client:` guard in system_prompt_block,
queue_prefetch, sync_turn, on_session_end, on_memory_write and
handle_tool_call now goes through it.
Refreshing is gated behind a flag set at the end of initialize() so the
baseline is established before any env re-resolution happens — callers
that wire up a client directly keep the existing client untouched.
Refs #21130
(cherry picked from commit b694d21b7c4ff0330df6051e12dc8991f7ea10a6)
`_write_env_vars` in the OpenViking memory provider interpolates each
secret straight into a `KEY=VALUE` line, but the values only ever pass
through `_clean_config_value`, whose `value.strip()` trims surrounding
whitespace and leaves internal CR/LF intact. Because the file is strictly
line-oriented and is re-read via `read_text().splitlines()`, a value that
carries an embedded newline spills onto a second physical line, and the
tail is re-parsed as an independent `KEY=VALUE` entry on the next round
trip. A secret pasted with a trailing record (e.g. an `OPENVIKING_API_KEY`
copied with an extra line) therefore injects an arbitrary additional
variable into the persisted credentials file and silently corrupts it.
The fix neutralizes the line terminators at the single chokepoint where
values reach the file. A small `_env_line_safe` helper strips `\r`, `\n`,
and the NUL byte from each value, and both write sites in `_write_env_vars`
(the existing-key update branch and the appended-key branch) route through
it, so a value can only ever occupy the single line it is written on.
## What does this PR do?
Hardens the OpenViking memory provider's `.env` writer so a malformed or
pasted secret value can no longer break out of its `KEY=VALUE` line and
inject a rogue variable into the profile-scoped credentials file.
## Related Issue
N/A
## Type of Change
- [x] 🐛 Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `plugins/memory/openviking/__init__.py`: add `_env_line_safe()` which
removes `\r`, `\n`, and `\x00` from a value, and apply it to both the
updated-key and appended-key write branches in `_write_env_vars()`.
- `tests/plugins/memory/test_openviking_provider.py`: add two regression
tests covering a fresh write and an in-place key update with embedded
CR/LF, asserting no injected line survives the read-back.
## How to Test
1. Run the targeted tests:
`pytest tests/plugins/memory/test_openviking_provider.py -k env_writer -q`
2. Reverting the `_env_line_safe` sanitization makes
`test_openviking_env_writer_strips_embedded_newlines_in_values` and
`test_openviking_env_writer_strips_newlines_when_updating_existing_key`
fail with a rogue `INJECTED_KEY=`/`ROGUE=1` line appearing in the file,
confirming the tests pin the bug.
3. `ruff check plugins/memory/openviking/__init__.py` and
`python scripts/check-windows-footguns.py plugins/memory/openviking/__init__.py`
both pass.
## Checklist
### Code
- [x] I've read the Contributing Guide
- [x] My commit messages follow Conventional Commits
- [x] I searched for existing PRs to make sure this isn't a duplicate
- [x] My PR contains only changes related to this fix
- [x] I've run the relevant tests and they pass
- [x] I've added tests for my changes (required for bug fixes)
- [x] I've tested on my platform: macOS 15 (Darwin 25.5)
### Documentation & Housekeeping
- [x] I've updated relevant documentation (docstrings) — or N/A
- [x] I've updated `cli-config.yaml.example` if I added/changed config keys — N/A
- [x] I've updated `CONTRIBUTING.md` or `AGENTS.md` if I changed architecture or workflows — N/A
- [x] I've considered cross-platform impact (strips CR as well as LF) — done
- [x] I've updated tool descriptions/schemas if I changed tool behavior — N/A
(cherry picked from commit f29dd2df84)
Two related Slack delivery fixes for send_message text sends:
- Route Slack text delivery through _send_via_adapter so the live
in-process gateway adapter (multi-workspace aware, channel→client
mapping, adapter-side gates) is preferred, with the plugin's
_standalone_send as the out-of-process fallback — matching how the
media path already behaves.
- _standalone_send: SLACK_BOT_TOKEN can be a comma-separated list in
multi-workspace installs and slack_tokens.json carries OAuth
per-workspace tokens; the standalone Web-API path used to send the
literal comma-joined string, which Slack rejects as invalid_auth.
Try each token individually, retrying on token-scoped errors
(invalid_auth / not_in_channel / channel_not_found …) and stopping on
terminal ones. User-DM resolution (U…/W… targets) also tries each
token.
Adapted from #47547 by @replygirl — the original patched the legacy
tools/send_message_tool.py::_send_slack helper, which moved to the
Slack plugin's _standalone_send in #41112.
Salvaged from #47547
Previously, Slack sessions and the channel directory stored raw channel
IDs (e.g. C0ATFHY907L) as chat_name, making it impossible for operators
to identify channels in send_message listings or session data.
Changes:
- Add _channel_name_cache and _resolve_channel_name() to SlackAdapter
that calls conversations.info (channels) or users.info (DMs) with
in-memory caching to avoid repeated API calls
- Use resolved names in _handle_slack_message() build_source() calls
- Use cached names in _seed_assistant_thread_session()
- Enrich session-sourced entries in channel_directory._build_slack()
by cross-referencing API results and falling back to conversations.info
+ users.info for DMs and private channels not in the bot's scope
Fixes: channel directory and session origins now show readable names
like 'general' or 'John Doe' instead of 'C0xxx' / 'D0xxx'.
Follow-up to the #51899 pick, folding in the config-bridge half of the
competing PR #46925 (@bhanusharma, earliest submitter for the
ignored-channel gate):
- _apply_yaml_config: translate config.yaml slack.ignored_channels into
SLACK_IGNORED_CHANNELS (list or CSV), env-var-wins like every other
bridged Slack key.
- SlackAdapter._slack_ignored_channels / gateway.run's
_slack_ignored_channels_from_gateway_config: fall back to the
SLACK_IGNORED_CHANNELS env var when PlatformConfig.extra carries no
value, so top-level slack: blocks (which flow through the env bridge,
not extra) are honored at both the adapter and runner gates.
- conftest: force-clear SLACK_ALLOWED_CHANNELS / SLACK_IGNORED_CHANNELS /
SLACK_DISABLE_DMS between tests (config-loader side-effect leak class).
- Tests: env-bridge translation + precedence in test_config.py, env
fallback + extra-wins in test_slack_runner_ignored_channels.py.
Credit: #46925 by @bhanusharma proposed the same gate with the YAML→env
bridge; #51899 (picked as the base) carries the wider outbound/runner
coverage. Closes#46925 as consolidated here with first-submitter credit.
Hermes already identifies itself on outbound calls to model providers
and monitoring endpoints — see hermes_cli/models.py and
plugins/model-providers/gmi/__init__.py
("Attribution so GMI can identify traffic from Hermes Agent"). The
Slack adapter is the one outbound surface that doesn't carry the
same identifier, so HermesAgent-driven Slack traffic is
indistinguishable from any other Bolt-Python app at the Slack
platform layer.
This sets `user_agent_prefix=f"HermesAgent/{_HERMES_VERSION}"` on the
AsyncWebClient instances constructed in gateway/platforms/slack.py
and threads them through AsyncApp via its `client` kwarg. Both
kwargs are first-class in slack-sdk and slack-bolt; no new
dependencies. Resulting header looks like:
HermesAgent/<version> Python/3.x slackclient/3.x ...
No behavioral change for users — the Slack API ignores User-Agent
semantically; it lands in logs and analytics. Reversible.
Tests in tests/gateway/test_slack.py:
- TestSlackUserAgent pins the prefix shape and runs connect()
end-to-end (multi-token config) to assert every AsyncWebClient
carries the prefix and AsyncApp receives the pre-built client.
- TestSlackProxyBehavior fakes updated to tolerate the new kwargs
via **_kwargs so they don't break on future passthroughs.
.get("key", {}) only applies the default when the key is ABSENT.
When the key exists with value None (null in JSON), .get() returns
None and the subsequent .get() raises AttributeError.
Fix: replace .get("key", {}).get(...) with (.get("key") or {}).get(...)
which handles both missing keys AND None values.
8 instances across 6 files:
- gateway/run.py: tool_call function name check
- acp_adapter/server.py: tool name/description extraction
- gateway/platforms/qqbot/onboard.py: API response task_id
- gateway/platforms/yuanbao.py: message content parsing (x2)
- gateway/platforms/slack.py: block text extraction (x2)
- tui_gateway/server.py: error message extraction
Blank (or whitespace-only) answers to the home-channel prompt in the
interactive setup wizards previously left any previously saved
*_HOME_CHANNEL / *_HOME_ROOM env value in place, so operators could not
clear a stale home channel by re-running setup. Strip the prompt input
and call remove_env_value() on blank answers across the Discord, Slack,
Feishu, Matrix, Mattermost, WeCom and WhatsApp plugin setup wizards,
with per-adapter wizard tests covering set/clear/whitespace flows.
Squash of the three commits from PR #58421 (setup-wizard fix, matrix/
wecom extension, and 6-adapter test coverage) — one commit per
contributor on this salvage branch.
Fixes#12423
Salvaged from #58421
The C13 log-noise test (merged while this branch was in flight) builds a
bare SlackAdapter without __init__; the new _channel_teams ambiguity map
crashed it. getattr-guard, same pattern as the sibling defensive inits.
Salvaged from #59742 (downloads half only). Main already resolves the
event-level team id from the channel→workspace cache before downloads
(the file-EVENT half was covered by #30456), but when neither the event
nor the cache knows the workspace, both download helpers silently fell
back to the PRIMARY workspace token — Slack then returns an HTML login
page instead of file bytes for any non-primary workspace file.
Slack private file URLs embed the owning workspace id
(files-pri/<TEAM_ID>-<FILE_ID>/...), so _resolve_download_token now
prefers: explicit team_id -> URL-embedded team id -> primary token.
Both _download_slack_file and _download_slack_file_bytes route through
it.
Deferred from #59742 (out of this correctness cluster's scope): the
channel→team disk persistence, the pre-send conversations.info probe
loop, and the thread-file ingestion feature.
Reapplied from #59742 by @benjamin2026-dot onto the current adapter
(original targeted the pre-#30456 download sites).
Widen #48476's fence guarantees to the two splitters that still emitted
fence-broken chunks:
* GatewayStreamConsumer._split_text_chunks (fallback final send): close
the orphaned ``` at each chunk boundary and reopen it — with the
original language tag — on the next chunk, mirroring
BasePlatformAdapter.truncate_message's contract. Headroom is reserved
so balanced chunks stay within the platform limit.
* Slack block_kit._split_text (3000-char section chunking): same
close/reopen balancing for mrkdwn section text carrying fences.
With these, every chunk boundary — non-streaming send
(truncate_message), streaming overflow (_truncate_for_stream via
adapter.truncate_message per #45938), fallback final
(_split_text_chunks), final-send balance (ensure_closed_code_fences),
and Block Kit section splits — delivers fence-balanced chunks.
Regression tests probe each path with fenced fixtures, assert per-chunk
balance, limit compliance, language-tag reopening, and prose passthrough.