Photon's Node sidecar intentionally hides raw handler exceptions, but the Python adapter still needs a safe failure class and retryability bit so delivery retries do not collapse into an opaque generic 500.
Constraint: Sidecar responses must not leak raw stack traces or private exception text
Rejected: Retry every internal sidecar error | masks permanent auth/config failures
Confidence: high
Scope-risk: narrow
Directive: Keep sidecar error text generic; extend safe error classes instead of exposing raw SDK failures
Tested: uv run --with pytest-timeout pytest tests/plugins/platforms/photon/test_overflow_recovery.py -q
Tested: uv run --with pytest-timeout pytest tests/plugins/platforms/photon -q
Tested: uv run ruff check plugins/platforms/photon/adapter.py tests/plugins/platforms/photon/test_overflow_recovery.py
Tested: python3 -m py_compile plugins/platforms/photon/adapter.py tests/plugins/platforms/photon/test_overflow_recovery.py
Tested: node --check plugins/platforms/photon/sidecar/index.mjs
Tested: git diff --check
Tested: python3 scripts/check-windows-footguns.py --diff origin/main
Not-tested: Live Photon/Spectrum delivery against a real iMessage account
Related: #50971
`PhotonAdapter._start_sidecar` is `async`, but it ran the Spectrum
mixed-attachment patch script with a bare `subprocess.run(...)`: it spawns
node and *waits* for it, with `timeout=10`. Executed inline that holds the
shared gateway event loop for the whole window, so no other platform's
messages, heartbeats, or sessions are serviced until it returns.
The same function already establishes this exact invariant twenty lines
above, where the stale-dependency reinstall hops to a worker thread:
# Runs off the event loop so a cold install can't freeze every other
# platform's traffic.
if _sidecar_deps_stale():
await asyncio.to_thread(_reinstall_sidecar_deps)
The patch spawn never got the same treatment. It is not startup-only
either — `_start_sidecar` is called from `connect()`, which takes
`is_reconnect`, so an ordinary Photon reconnect (network blip, sidecar
death) re-runs it and stalls a live gateway that is actively serving
Discord/Telegram/Slack traffic.
Dispatch it via `asyncio.to_thread` like its sibling. Same off-the-loop
class as the inbound-image decision (#66688) and the cron-fire verifier.
Adds a regression test asserting the spawn executes on a worker thread
rather than the loop thread.
`_reap_stale_sidecar` is `async`, but it identified the processes holding
the sidecar port with two blocking helpers called inline:
* `_find_listener_pids` -> `subprocess.run(["lsof", ...], timeout=5.0)`
* `_pid_is_sidecar` -> `subprocess.run(["ps", ...], timeout=5.0)`, once
per candidate pid
so the inspection can hold the shared gateway loop for 5 + 5·N seconds
while nothing else on it is serviced. It only runs once the /healthz probe
finds something already listening — the orphaned-sidecar recovery path —
and `_reap_stale_sidecar` is awaited from `_start_sidecar`, which runs on
every reconnect (`connect(is_reconnect=True)`). The stall therefore lands
on a live gateway that is still serving every other platform, right when a
crashed sidecar has already left an orphan behind.
Move the whole inspection to one `asyncio.to_thread` hop (one hop rather
than N+1 round trips). The reaping semantics are untouched: SIGTERM for
verified orphans, SIGKILL escalation, and both foreign-listener
RuntimeErrors behave exactly as before.
Same off-the-loop class as the inbound-image decision (#66688) and the
cron-fire verifier.
Adds a regression test asserting both the lsof lookup and the per-pid ps
check execute on a worker thread rather than the loop thread.
Follow-up widening of the #73170 pattern: apply the same
asyncio.current_task() guard used for the health task (and now the
supervisor task in _stop_sidecar) to the inbound-task cancel path in
disconnect(), so an inbound task that triggers disconnect() cannot
cancel-and-await itself.
Fixes#73159.
When the Photon sidecar exits unexpectedly, _supervise_sidecar()
(running as self._sidecar_supervisor_task) correctly detects
SIDECAR_CRASHED and calls self._notify_fatal_error(). The Gateway's
fatal-error handler answers that by calling adapter.disconnect(),
which calls _stop_sidecar() -- from INSIDE the very task that's
currently executing this whole chain.
_stop_sidecar()'s cleanup unconditionally cancelled
self._sidecar_supervisor_task. Cancelling the currently-running task
raises CancelledError at its own next await point (inside
_notify_fatal_error() or _stop_sidecar() itself). Since
asyncio.CancelledError inherits from BaseException (not Exception),
the Gateway's `except Exception` guards around the fatal-error handler
don't catch it -- the handler aborts before ever reaching the
"queue platform for background reconnection" step. Photon then stays
permanently in `retrying` state until the whole process is manually
restarted, even though detection worked correctly and the underlying
transient upstream outage had long since recovered.
Fix: in _stop_sidecar()'s cleanup, check whether
self._sidecar_supervisor_task is asyncio.current_task() before
cancelling it. A task cannot legally cancel itself in any useful way
anyway (the cancellation only takes effect at its own next await,
which is exactly the corruption described above) -- when we're inside
the supervisor's own call stack, it's already in the process of
finishing on its own once _notify_fatal_error() returns, so skip the
cancel and just clear the reference.
This mirrors an existing precedent in the same file: disconnect()
already guards its OTHER task-cancellation (self._sidecar_health_task)
the same way (`if task is not asyncio.current_task()`), just not the
supervisor task in _stop_sidecar().
Per the issue's own note that existing tests mock out
_notify_fatal_error() entirely (so this integration chain was never
exercised), added two tests that drive the REAL chain: one runs
_supervise_sidecar() as an actual asyncio task with a real
_notify_fatal_error() that calls the real disconnect() -> _stop_sidecar(),
confirming the task completes without CancelledError and that the
post-disconnect reconnect-queue step actually executes; a second
confirms the OTHER call path (external cleanup, a different task)
still correctly cancels a running supervisor exactly as before.
Reverting only the adapter.py fix (keeping the new test) reproduces
the exact CancelledError from the bug report, confirming this is a
genuine regression test.
2 new tests pass; 113/113 in the full tests/plugins/platforms/photon/
directory (no regression to sidecar lifecycle, overflow recovery, or
health-monitoring behavior).
Follow-up to #69112. That PR hardened the shared gateway dispatch path in
gateway/run.py against the *caller* being cancelled. A second, self-referential
cancellation specific to PhotonAdapter sits one layer underneath it and survived
that fix.
PhotonAdapter is the only platform adapter that awaits _notify_fatal_error()
inline, on the same task that detected the fault. Both _monitor_sidecar_health
and _supervise_sidecar run as self._sidecar_health_task /
self._sidecar_supervisor_task, and the notification routes into
GatewayRunner._handle_adapter_fatal_error_impl, which tears the adapter down via
_safe_adapter_disconnect -> disconnect(). disconnect() then cancels
self._sidecar_health_task and awaits it -- which, when the health task is what
raised the notification, means disconnect() cancels its own caller several
plain-await frames up.
disconnect()'s `task is not asyncio.current_task()` guard does not catch this.
The current task where that guard evaluates is the wrapper
_await_adapter_cleanup_with_timeout creates around disconnect() via
asyncio.ensure_future, not the health task further up the chain, so the guard
passes and the cancel lands.
CancelledError stopped subclassing Exception in Python 3.8, so the
`except Exception` that wrapped the inline notify call never saw it. The health
task died silently mid-handoff: no log line, no "exception never retrieved"
warning (cancellation is normal asyncio), and no retry. The platform stayed
stranded until the gateway was restarted by hand.
Fix: dispatch the notification onto a new task, the same pattern
DiscordAdapter._handle_bot_task_done already uses for this reason. disconnect()
can then cancel the health/supervisor task freely without that cancellation
reaching the code still running the handoff, so the handoff always reaches the
reconnect queue. Both Photon fatal call sites are converted: the health-poll
path (observed wedging) and the sidecar-crash path (same shape, not yet
observed). gateway/run.py is untouched.
Observed twice on a self-hosted gateway, ~4h38m and ~52min of silent inbound
outage, both cleared only by a manual restart, both post-dating #69112's merge.
In each case the fatal log line appears and `queued for background reconnection`
never does.
Tests: new tests/plugins/platforms/photon/test_fatal_notify_self_cancel.py
covers the self-cancellation (fails with CancelledError without this change),
that the dispatch does not block its caller, that a failing notification warns
rather than raising, and a source guard against reintroducing either inline
await. Two assertions in test_overflow_recovery.py that drove these coroutines
directly now drain pending tasks before asserting delivery, since the
notification is deliberately no longer awaited inline.
Prepared with agent assistance and reviewed before submission.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Consolidates the native-transport half of PR #73636 by @ScaleLeanChris
onto the merged adapter: persistent NIP-42-authenticated Nostr WebSocket
subscription as the default inbound path (transport=auto|websocket|poll),
kind-44100 membership events for live DM discovery, since-timestamp
resume on reconnect with bounded exponential backoff, and automatic
fallback to CLI polling when the WS can't be established. Events route
through the same _handle_event() pipeline as the poll loop, so de-dupe,
mention gating, p-tag DM latching, and allow-lists behave identically on
both transports. Outbound stays on the CLI (one-shot sends never race a
WS auth handshake — his design).
E2E verified against a real in-process websockets relay: NIP-42
challenge -> signed kind-22242 AUTH (event id re-derived server-side) ->
REQ subscription -> EVENT dispatch -> clean disconnect.
Co-authored-by: ScaleLeanChris <chris@scalelean.com>
Extracted verbatim from PR #73636 by @ScaleLeanChris — nsec/hex key
decoding, secp256k1 point math, BIP-340 Schnorr signing, and NIP-42
AUTH event construction with optional NIP-OA owner attestation tag.
Two independent cross-PR collisions red on main (slice 8/8):
1. fd4f756492 (salvaged from stale #54514) added an early 'drop U+FFFC
placeholder' return at the top of _dispatch_inbound — written before
the deferred-wait handler (6b91b50c6e/afab7ed46e) existed further
down the same function. The early return shadowed it: _pending_fffc
never populated, no attachment-timeout tracking, 4 tests red. Remove
the duplicate block; the deferred handler already drops the
placeholder AND tracks/cancels/warns.
2. 9cf2046081 tightened sidecar_deps_installed() to require
node_modules/spectrum-ts, but test_runtime_record's _patch_spawn
fixture still created only bare node_modules/ — 2 tests red. Mirror
a real completed install.
tests/plugins/platforms/photon: 164/164 after; 158/164 before.
The sidecar auth token is generated at spawn (secrets.token_hex) and
existed only in the gateway process memory + sidecar child env, so
_standalone_send from cron subprocesses, hermes send, or the dashboard
structurally could not authenticate (#69960).
The adapter now writes <hermes-home>/runtime/photon-sidecar.json
({port, token, pid}, 0600, atomic tempfile+os.replace) once the sidecar
passes its /healthz readiness check, and deletes it in _stop_sidecar,
on every startup-failure path, and at disconnect so a stale record
never outlives a dead sidecar. _standalone_send falls back to the
record when PHOTON_SIDECAR_TOKEN is unset, validating the recorded pid
is alive first; a stale record yields a clear 'gateway appears to be
down' error. Docs note the gateway-must-be-running requirement and the
Photon-side shared-line initiation policy (#51897).
All Photon sidecar HTTP requests target 127.0.0.1 — they should
never be routed through a system HTTP proxy. When trust_env=True
(the default), httpx picks up macOS system proxy settings and
routes localhost requests through the proxy. If the proxy returns
a spurious response (e.g. 502), _reap_stale_sidecar() interprets
it as 'port in use by a non-sidecar process' and refuses to start,
yielding: 'pids: unknown, not a Photon sidecar'.
Set trust_env=False on all five httpx.AsyncClient call sites in
the Photon adapter so localhost sidecar communication bypasses
the system proxy entirely.
Addresses teknium1 review on #50983:
- cli.py's `hermes photon status` and adapter.py's _start_sidecar() still
used the old node_modules/-existence check while check_requirements()
had moved to a spectrum-ts content check. Extracted the check into a
shared sidecar_deps_installed(), used by all three, so an
empty/partial node_modules/ (aborted npm install) is rejected
consistently instead of only in check_requirements().
- _install_sidecar()'s success-path _NPM_ERROR_LOG.unlink() only caught
FileNotFoundError, so a PermissionError/OSError on a locked file would
propagate. Broadened to OSError.
- npm stderr was truncated only when read back in check_requirements();
an unbounded stderr was written to disk on every failed install. Now
truncated to _NPM_ERROR_LOG_MAX_CHARS before write_text().
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Problema
--------
Quando o npm install do sidecar Photon falhava, o Hermes descartava toda
evidencia e continuava normalmente - deixando o adapter de iMessage
silenciosamente ausente, sem nenhuma mensagem de erro acionavel.
Tres falhas independentes formavam o caminho de falha silenciosa:
1. check_requirements() sem logging
Cada branch de return False retornava sem emitir nenhum log. O core em
platform_registry.py so consome o bool de check_fn() e loga uma mensagem
generica com o install_hint - sem acesso ao motivo real da falha.
if not HTTPX_AVAILABLE: return False # sem log
if not shutil.which(node): return False # sem log
if not node_modules.exists: return False # sem log
2. node_modules/ parcialmente criado passava o guard (Risk 2)
npm cria node_modules/ antes de abortar em ENOSPC, timeout de rede ou
EACCES. O diretorio existia, check_requirements() retornava True (falso
positivo), o adapter era registrado, e o crash acontecia em runtime com
um erro de modulo ausente aparentemente nao relacionado ao setup.
3. stderr do npm descartado (Risk 3)
subprocess.run sem stderr=PIPE. O output de erro aparecia no terminal
durante o setup e sumia depois - diagnostico impossivel em CI/CD, Docker,
VPS headless, e qualquer reinstalacao posterior.
Correcoes
---------
adapter.py - check_requirements() agora loga por branch:
- httpx ausente -> logger.warning com nome do pacote
- node nao no PATH -> logger.warning com nome do binario e env var
- spectrum-ts ausente -> logger.debug com path do sidecar + ultimo erro npm
(DEBUG nao WARNING: estado normal pre-setup; check_fn() e chamado de
5 hot paths do core incluindo polling do /api/status)
adapter.py - content check em vez de existence check (Risk 2):
antes: if not (_SIDECAR_DIR / node_modules).exists()
depois: if not (_SIDECAR_DIR / node_modules / spectrum-ts).exists()
spectrum-ts e a unica dependencia do package.json. Checar sua presenca
garante que instalacao parcial/abortada e detectada no boot do gateway,
nao na primeira mensagem recebida via gRPC.
cli.py - stderr capturado e persistido (Risk 3):
subprocess.run passa agora stderr=subprocess.PIPE, text=True em ambas as
chamadas (npm ci e npm install fallback). O stderr capturado e:
- impresso em sys.stderr imediatamente (output visivel no terminal)
- persistido em _NPM_ERROR_LOG = sidecar/.photon-npm-error.log se
returncode != 0, limitado a 300 chars
- apagado de _NPM_ERROR_LOG se returncode == 0 (evita erro stale)
check_requirements() le _NPM_ERROR_LOG quando spectrum-ts esta ausente e
inclui o conteudo no DEBUG log - o erro do npm sobrevive ao terminal, ao
restart do gateway e a reinicializacao da maquina.
sidecar/.gitignore - adicionado node_modules/ e .photon-npm-error.log.
Isolamento - sem impacto no core:
- check_fn() continua retornando apenas bool; core nao e modificado
- Logging usa namespace plugins.platforms.photon.adapter, isolado de
gateway.* e hermes_cli.*
- Cada plugin tem seu proprio check_requirements() independente
- OSError no write/read de _NPM_ERROR_LOG e silenciado - nunca propaga
Testes - 24/24 passando:
test_check_requirements_risks.py (7 testes):
WARNING emitido quando httpx ausente
WARNING emitido quando node nao no PATH
DEBUG emitido (nao WARNING) quando spectrum-ts ausente, com path
node_modules/ vazio agora retorna False (Risk 2 resolvido)
_NPM_ERROR_LOG escrito no stderr do npm em falha
_NPM_ERROR_LOG apagado apos npm bem-sucedido
erro npm aparece no DEBUG log quando node_modules ausente
test_npm_error_log_regression.py (9 testes - vetores de falha da solucao):
return code contrato intacto (0 em sucesso, nao-zero em falha)
OSError no write do log silenciado, exit code ainda propagado
OSError no read do log silenciado, check_requirements() retorna False
stderr vazio nao cria arquivo de log
proc.stderr=None nao lanca AttributeError
log stale apagado apos reinstall bem-sucedido
DEBUG emitido mesmo sem log de erro (setup pela primeira vez)
Closes#50981
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
hermes photon setup unconditionally called regenerate_project_secret()
on every re-run, invalidating the credential held by a running sidecar
and causing all outbound sends to fail with AuthenticationError.
Now validates existing credentials via a lightweight list_users call
before deciding to regenerate. Only rotates when no credentials exist
or the existing ones are invalid, and warns the user to restart the
gateway when rotation occurs.
Fixes#50755
Maintainer follow-up to the #72763 salvage: check_photon_token_valid()
now delegates to the existing validate_photon_token() (session lookup +
/api/projects/) instead of a bespoke get-session-only probe, since the
device flow can mint tokens that pass the session check but fail the
project APIs that setup actually uses. Semantics preserved: definitive
auth rejection = stale, transient errors = probably-valid.
Two related bugs in hermes photon setup / gateway_setup:
Bug 1 -- stale token reused (401)
----------------------------------
_cmd_setup reused an existing dashboard token without validation.
The device token has a short TTL (~3-4 days observed); reusing a
stale token caused every management API call (find_project_by_name,
regenerate_project_secret, etc.) to fail with 401. The operator
saw confusing "spectrum provisioning failed: 401" errors.
Fix: check GET /api/auth/get-session before using the stored token.
On 401/403, clear the stale token with clear_photon_token() and
fall back to a fresh device-login flow automatically.
Bug 2 -- channel left disabled after successful setup
-----------------------------------------------------
After all five provisioning steps completed, config.yaml still had
photon.enabled: false, so the gateway never loaded the Photon
adapter. Every inbound iMessage hit Photon's offline auto-responder
without the operator being notified.
Fix: call write_platform_config_field('photon', 'enabled', True,
raw=True) as a final setup step so the gateway picks up the freshly
configured channel on its next start.
New public API in auth.py:
- clear_photon_token() -- discard stored token from auth.json
- check_photon_token_valid(token) -- lightweight session-check test
References: #72763
store_photon_token/store_project_credentials/store_user_numbers read-modify-
wrote auth.json without hermes_cli/auth.py's _auth_store_lock(), the
cross-process flock every other writer of that file (credential_pool
refresh, model_switch, fallback_cmd, nous_portal adapter, etc.) already
holds. A concurrent write from either side during the unlocked
load-mutate-save window silently drops the other side's update.
Review follow-up: if os.fdopen() raised before taking ownership of the
descriptor returned by os.open(), the cleanup handler unlinked the temp
file but leaked the fd. Close it explicitly on that path, mirroring the
credential-writer cleanup from #62837.
Strengthen the tests so the old writer could not pass them: an os.open
spy asserts O_CREAT | O_EXCL and an explicit 0o600 mode (the final-mode
check alone was also satisfied by the post-write chmod), and a forced
fdopen-failure test asserts the raw fd is closed and no temp file is
left behind.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
_save_auth() wrote the bearer token with tmp.open('w') — created at
process umask (typically 0o644) — and only chmod'ed to 0o600 after the
write, leaving a window where the token sat world-readable. The temp
name was also fixed and predictable (auth.json.tmp), so it could be
pre-planted (symlink attack).
Create the temp file with os.open(O_WRONLY|O_CREAT|O_EXCL, 0o600) and a
per-process random suffix, fsync before the atomic replace, and clean
the temp file up on failure. Mirrors hermes_cli/auth.py:_save_auth_store
(#19673, #21148), which hardened the same pattern in the core writer.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
gemini-2.5-flash shuts down Oct 16 2026 (Google deprecation schedule)
and gemini-3-flash-preview is superseded. Update every hardcoded
default to the current GA flash model:
- gemini_native_adapter: probe_gemini_tier + _create_chat_completion
default params; free-tier guidance de-pinned from a specific model's
RPD number so it doesn't stale again
- auxiliary_client: gemini/kilocode fallback aux models,
_OPENROUTER_MODEL, _NOUS_MODEL -> google/gemini-3.6-flash
(verified live on both OpenRouter and Nous portal /models)
- provider plugins: kilocode + vertex default_aux_model
- hindsight memory plugin: gemini provider default
- setup wizard gemini list: 3-flash-preview -> 3.6-flash (matches the
curated picker catalog)
- tests: aux-client assertions that pinned the old default literal now
reference the _NOUS_MODEL constant, so the next default bump can't
break them (change-detector cleanup)
Fixes#32360.
Photon (iMessage) has no real edit API for already-sent messages. When
streaming completes, the gateway attempts to edit the message to remove
the streaming cursor (▉). Without edit support, this cursor gets stuck
in the final message, corrupting Unicode characters.
This change sets SUPPORTS_MESSAGE_EDITING=False on PhotonAdapter, which
causes the gateway to suppress the streaming cursor entirely for this
platform (via _effective_cursor in gateway/run.py). This prevents the
stale tofu square (▉) from appearing in streamed iMessage responses.
Fixes#49253
Follow-up on the salvaged #71610 commits:
- acquire/release a scoped lock on relay_url+pubkey in connect/disconnect
(IRC pattern) so two profiles can't drive one Buzz identity — duplicate
replies and split de-dupe state; +2 tests
- negative-cache _resolve_user_name failures so a profile-less pubkey
doesn't re-hit 'users get' every poll sweep (flagged by @jethac on the PR)
- register user-guide/messaging/buzz in website/sidebars.ts (page was
unreachable — the #63359 trap)
On hosted relays `buzz dms list` reliably returns [] even when DM
conversations exist, so DMs leaked in via `channels list` (as entries
named "DM" with an empty description) and were seeded chat_type="group".
That put them behind the channel mention gate: "@Chip /whoami" worked
but an un-mentioned DM was silently dropped.
Classify from the Nostr tags of real traffic instead: a message another
user sends in a DM carries a structural ["p", <own pubkey>] tag even
when the text never mentions the agent, while in a real channel a
p-tag-to-self only ever accompanies a visible @mention (typed mention
or reply). A group conversation therefore latches to chat_type="dm" on
the first kind-9 event that is p-tagged to self WITHOUT a visible
mention in the content — guarded by channels-list metadata so a real
community channel (real name / non-empty description) is never
reclassified by a reply or mention that p-tags the agent.
- latch during history seeding too, so a leaked DM bypasses the
mention gate from the very first poll after connect
- keep `dms list` as a best-effort source, and scan `channels list`
as a fallback so DM conversations opened mid-run still get watched
- strip a leading @mention in DMs as well, so "@Chip /whoami" keeps
firing as a slash command after the conversation reclassifies
Refs #68871
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Channel mention-gating was hardcoded on. Add a configurable require_mention
(default True, preserving current behavior). When False, the agent responds to
every message in a watched channel, not only when @mentioned; DMs always
dispatch. Read from config.yaml gateway.platforms.buzz.extra.require_mention
with BUZZ_REQUIRE_MENTION env override, bridged via apply_yaml_config_fn like
the other settings. A leading mention is still stripped when present.
Refs #68871
Channel messages address the agent with a leading @mention (e.g. '@Chip
/whoami'). The adapter passed the raw content through, so the gateway's
is_command() check (text.lstrip().startswith('/')) never matched and slash
commands were routed as plain chat. Strip a leading mention (name, npub, or
hex form) before dispatch in channels, mirroring the Discord adapter. Also
cleans normal prompts ('@Chip what's up?' -> 'what's up?'). DMs are untouched.
Verified live: '@Chip /whoami' -> '/whoami' after connect populates identity.
Refs #68871
The Buzz adapter's check_requirements() reads config from env only, so a
config.yaml-only setup (relay URL in gateway.platforms.buzz.extra) failed the
check_fn gate and was silently skipped at startup. Add an apply_yaml_config_fn
hook that bridges buzz.extra -> BUZZ_* env vars, mirroring the Slack/Telegram
pattern; BUZZ_PRIVATE_KEY stays a .env secret.
Also fix send_reaction() to use buzz-cli's real flags (--event <id> --emoji),
replacing the non-existent --channel/--message-id flags that would have failed
on every message. Verified live against the hosted relay (accepted:true).
Refs #68871
Plugin-path adapter (zero core changes) connecting Hermes to a Buzz
community relay via the buzz CLI binary (JSON in/out, arg-list exec,
key passed via env only). Inbound uses a poll loop with per-channel
high-water marks seeded from newest (no history replay), event-id
de-dupe, self-echo suppression by pubkey, and mention gating in
channels (DMs always dispatch). Registers env_enablement, cron
home-channel delivery, and an out-of-process standalone sender,
mirroring the IRC plugin.
Verified against a live relay: connect -> send -> poll -> MessageEvent
round-trip, self-echo suppressed, clean disconnect.
Known limitation: polled inbound (default 4s); a websocket transport
(buzz-ws-client) is a future optimization.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Move U+FFFC placeholder detection before _record_last_inbound() so the
placeholder message id is not recorded as the reaction target
- Cancel pending U+FFFC tasks in disconnect() to prevent task leaks
- Check mimeType audio/x-caf in addition to filename for CAF→VOICE promotion,
fixing unnamed attachments that default to '(unnamed)'
- Add 7 Photon adapter tests: CAF named/unnamed promotion, U+FFFC no-dispatch,
U+FFFC not recorded as last inbound, U+FFFC+attachment cancel, timeout fire,
disconnect cleanup
- Add 6 transcription tests: ffmpeg conversion, afconvert fallback, all-fail,
CAF→WAV before Groq, conversion failure error, local provider skip
iMessage's gRPC cloud push fires before CloudKit syncs the attachment
metadata, emitting a U+FFFC placeholder as text. Instead of dispatching
a spurious message, start a non-blocking 15s wait. If the real attachment
arrives, cancel the timeout and process normally. If not, log a warning.
spectrum-ts classifies all inbound attachments as type 'attachment',
never 'voice'. Without this, .caf voice notes are routed as AUDIO or
DOCUMENT and bypass the STT pipeline.
The msg_type_str == "richText" branch reset msg_type to PHOTO/TEXT after
the rich-text item scan had already promoted it (e.g. a native voice item
→ VOICE), dropping voice notes from the auto-STT path. Only re-derive when
the scan left the type at TEXT.
Ports the root-cause analysis from PR #38276 (stale, targeted the deleted
gateway/platforms/dingtalk.py) onto the live plugin adapter, with
regression tests.
Refs #38211#38219#38276
- Promote EXT_MAP to a module-level constant for reuse
- Classify DingTalk image messages and image file attachments as PHOTO
- Extract DingTalk card/interactiveCard document link content with defensive parsing
- Handle None, empty, JSON, and plain-string card content fields
- Add coverage: 8 card + 4 interactiveCard + 5 _extract_media = 17 test cases
- Remove a shadowed duplicate TestExtractText class (pytest collected only the later one)
3 fixes for the dingtalk-platform plugin (plugins/platforms/dingtalk/adapter.py):
1. _extract_text: parse ASR recognition text from voice messages via
extensions['content']['recognition'], fall back to fileName for
file messages ("[文件] xxx")
2. _extract_media: handle audio/file/image msgtype_str with correct
MIME mapping; exclude audio from media_urls to preserve DingTalk's
built-in ASR result (avoids failed whisper re-transcription)
3. _resolve_media_codes: parse downloadCode from extensions['content']
for file/image message types
All tested with live DingTalk group chat: voice → recognition text,
PDF/Markdown file → filename + download.
Lark's native "audio" msg_type is an in-app voice recording — uploaded
audio files arrive as "file"/"media". But _resolve_normalized_message_type
resolved the "audio" preferred type to MessageType.AUDIO, which the gateway
treats as a non-transcribed file attachment (run.py: AUDIO -> audio_file_paths,
"never STT"; VOICE -> audio_paths, "always STT"). Result: a Feishu voice
note reached the agent as an untranscribable audio attachment and was
silently ignored — the user's spoken message never became text.
Every other platform that receives native voice notes (Telegram, Discord,
Slack, WhatsApp, Signal, Matrix, WeChat, WeCom, DingTalk, QQ, BlueBubbles,
Mattermost, Yuanbao) classifies them as MessageType.VOICE. Feishu was the
only one classifying them as AUDIO. This is the follow-up to #28993, which
added native voice-note transcription for Discord + DingTalk but did not
cover Feishu.
Return MessageType.VOICE for the "audio" branch. The branch is reached only
for Lark's top-level audio msg_type (set in the normalizer; file uploads map
to "document"), so VOICE is unconditionally correct here — no risk of
auto-transcribing an uploaded music/audio file.
- plugins/platforms/feishu/adapter.py: _resolve_normalized_message_type audio
branch returns VOICE instead of resolving to AUDIO via mime.
- tests/gateway/test_feishu.py: test_extract_audio_message_downloads_and_caches
asserted the old AUDIO behavior on a fixture literally named voice.ogg —
updated to expect VOICE (the corrected classification).
- tests/gateway/test_feishu_voice_message_type.py: new focused regression
tests (audio->VOICE with and without mime; photo/document/text unaffected).
Rebased onto current main: the Feishu platform moved from the single-file
gateway/platforms/feishu.py into the plugins/platforms/feishu/ package; the
fix applies to the same _resolve_normalized_message_type logic at its new home.
Verified the new voice tests fail when the branch resolves to AUDIO and pass
with VOICE, while the photo/document/text cases are unaffected either way.
Note: classification is mock-tested here; the downstream STT pipeline is the
shared, already-proven path (#28993). End-to-end verification on a live
Feishu account would be a welcome confirmation.
Refs #28993 (sibling-gap: Feishu was the platform left uncovered).
Keep one owner for PATH/local-prefix ffmpeg discovery: ffmpeg_utils now
delegates to tools.transcription_tools._find_ffmpeg_binary and only adds
the Discord-specific FFMPEG_PATH override and Windows winget fallback on
top (follow-up to PR #60627 by @LauraGPT, fixes#60624).
Discord's voice socket needs a brief warm-up before receiving clients
actually hear audio; the first ~100-200ms is lost, clipping the first
word/syllable of TTS playback. Prepend a configurable lead of silence to
speech on both playback paths:
- Mixer path: new _lead_silence_bytes() helper prepends PCM silence
(BYTES_PER_MS constant added to voice_mixer.py) before play_speech, on
both the reply and the pre-tool ack.
- Legacy FFmpegPCMAudio path: apply -af adelay=<ms>:all=1.
Tunable via discord.voice_fx.lead_silence_ms (default 200, 0 disables).
Fixes#66827
VoiceMixer duck-typed the discord.AudioSource interface (is_opus, read,
cleanup) but never inherited from it. discord.py's vc.play() does an
isinstance check and rejects non-AudioSource objects, causing the voice
fx mixer to silently fail with:
"Voice mixer failed to start: source must be an AudioSource not
VoiceMixer"
Added missing `import discord` and changed the class definition from
`class VoiceMixer:` to `class VoiceMixer(discord.AudioSource):`.
- Wire adapter._voice_input_callback at connect and reconnect so voice
transcription is forwarded without requiring /voice join (#60623).
- Add optional text_channel_id and source params to DiscordAdapter
.join_voice_channel() so automatic/programmatic voice joins can
establish the text-channel binding needed by _handle_voice_channel_input.
- Add TestVoiceInputCallbackWiring: asserts callback wiring on startup
and reconnect for Discord adapters with voice attributes.
Closes the #32029 gap: TelegramAdapter.send_voice passed captions raw with
no parse_mode, so auto-TTS captions (which carry the agent's markdown
reply) showed literal *asterisks*, backticks and [links](...). Captions
are now formatted to MarkdownV2 via the adapter's format_message when the
formatted text fits Telegram's 1024-char caption cap, with fallback to the
plain truncated caption when formatting overflows or the Bot API rejects
the entities (mirrors the text-send markdown fallback ladder).
Fixes#32029
Salvaged from PR #53157 (@LLQWQ). Three changes for native Feishu voice
bubble delivery:
1. Include audio duration in the file-upload body when uploading opus
files — Feishu renders 0:00 bubbles without it. Duration is extracted
by parsing the OGG container's last granule position (pure Python,
no ffprobe dependency).
2. Thread routing fallback: Feishu's create-message API rejects
msg_type='audio' with receive_id_type='thread_id' (error 99992402);
retry via the reply API against the thread's last message, then fall
back to chat_id routing.
3. (dropped) tools/tts_tool.py want_opus hunk — superseded by main's
OPUS_VOICE_PLATFORMS, which already includes feishu. The PR's stray
scripts/release.py hunk was also dropped (frozen AUTHOR_MAP policy;
mapping added under contributors/emails/ instead).
Fixes#45557
Refs #18831#16524
Salvaged from PR #18861 (@shellybotmoyer). Switch the central Opus
transcoders from CBR 64k (-vbr off) to VBR 48k with -application voip and
-compression_level 10. The old CBR settings produced lower-quality speech
audio and occasionally failed to trigger Telegram's native voice-bubble
rendering. Applied to _ffmpeg_transcode_to_opus (the single transcoder
behind _convert_to_opus and _repair_ogg_container), the Gemini WAV→OGG
path, and the Matrix adapter boundary transcoder added in this branch.
Fixes#18818
Salvaged from PR #68063 (@malaiwah). MatrixAdapter.send_voice now transcodes
any non-Ogg audio to Ogg/Opus at the adapter boundary (best-effort — the
original file is sent unchanged when ffmpeg is unavailable), so MSC3245
voice bubbles render even when a caller hands the adapter MP3/WAV audio.
_matrix_voice_metadata_for_file probing now runs via asyncio.to_thread so
ffprobe/ffmpeg subprocess timeouts can't stall the adapter event loop.
The PR's tools/tts_tool.py want_opus hunk was dropped: main's
OPUS_VOICE_PLATFORMS set (PR #73072) already includes matrix; the Matrix
opus-routing test is kept.
Refs #14841
Salvaged from PR #68063 (base commit, runner-path hunks superseded by the
platform-aware OPUS_VOICE_PLATFORMS fix in this branch). Element and other
Matrix clients render voice bubbles more reliably when m.audio events carry
duration and waveform metadata; probe both best-effort via ffprobe/ffmpeg.
The native Gemini provider profile's default_aux_model and the curated
model picker catalog were still pinned to gemini-3.5-flash, a stale
generation now superseded by gemini-3.6-flash (documented GA). Bump
both so the auxiliary-task default and the picker stay in sync with
the current model.
Contract test asserts the durable lockstep invariant only
(default_aux_model is a member of _PROVIDER_MODELS["gemini"]) rather
than pinning either side to a frozen model-name string, so it doesn't
need updating on the next model-generation bump.
- Change await self._send_read_receipt to asyncio.create_task to avoid
blocking message dispatch on slow bridge responses (matches BlueBubbles
pattern). Up to 5s per-message latency eliminated.
- Update tests: assert_called_once_with instead of assert_awaited_once_with
since the receipt is now scheduled, not directly awaited.
- Add send_read_receipts documentation to whatsapp.md following the
BlueBubbles docs pattern.
Direct tool HTTP calls already identified as Hermes-Agent, but the main
OpenAI-SDK chat path still sent OpenAI/Python. Set Hermes-Agent/<ver> for
api.x.ai clients (xai + xai-oauth) so normal text traffic is attributed correctly.
The per-IP httpx transports were built once in __init__ and never torn
down. A connect that reached ESTABLISHED and was then closed by the peer
left its socket in CLOSE_WAIT inside the pool, and the failure path only
logged and continued — so the poisoned pool was retained and leaked one
descriptor per retry.
With DNS for api.telegram.org failing, every poll fell through to the
seed IP and leaked another fd every ~2.5s. The bot gateway reached 177
CLOSE_WAIT sockets against launchd's 256 soft limit and wedged: accept()
on the gateway port, config reads and DNS resolution all failed with
EMFILE, which in turn made the primary path fail and fed the loop.
Build fallback transports lazily and discard them on a retryable connect
failure, and bound every pool at 8 connections (httpx defaults to 100,
so two seed IPs plus primary could alone exceed the fd ceiling).
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
_check_local_runtime() only imported 'hindsight' and
'hindsight_embed.daemon_embed_manager', a strictly weaker import surface
than the embedded daemon actually needs: the daemon imports
sentence_transformers at startup (embeddings + reranker). When the
embedding stack is broken (e.g. a dependency conflict on a shared package
like huggingface-hub), the daemon can never start, yet is_available() and
'hermes memory status' still report Hindsight as available — every
retain/recall then fails silently.
Import sentence_transformers in the same probe so local/local_embedded
availability goes red with the real ImportError as the reason, letting
the agent degrade loudly instead of silently dropping memory. Local path
only; cloud mode is untouched and no network or model download is
triggered by the import.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MN8RMDLwxCfFxwtADoEJJf
Stop offering deepseek-chat/reasoner in the static catalog and point
fallback/aux defaults at the permanent v4 IDs. Keep retired aliases in
a detection-only map so /model deepseek-chat still resolves to deepseek.
MEDIA_DELIVERY_EXTS in gateway/platforms/base.py omitted .3gp, causing
MEDIA: tags with .3gp files to leak as plain text instead of being
extracted for native video delivery. _VIDEO_EXTS in
tools/send_message_tool.py and _MIGRATION_VIDEO_EXTS in the Feishu
adapter omitted .webm, causing .webm files to be classified as
documents instead of video on Telegram and other platforms.
Both extensions are already present in every gateway-side _VIDEO_EXTS
definition (run.py, kanban_watchers.py, weixin.py, base.py local).
Closes#71621, Closes#71603
Surface a user-visible delivery notice when non-streaming media dispatch
gets success=False after MEDIA tags were stripped, and validate forum
starter-message attachments the same way as direct channel sends (#66797).
Outbound MEDIA video/document tags on the Discord non-streaming path
were extracted (stripped from the visible text) but never delivered:
no attachment, no error, and no dispatch log line (#66797). The
open-handle discord.File plus singular file= form could race Discord's
multipart encoder after an earlier image batch on the same channel and
return a successful message carrying zero attachments.
Fix _send_file_attachment (used by send_video/send_document/
send_image_file) to:
- use a path-based discord.File via the plural files=[...] kwarg, the
same pattern as the working send_multiple_images batch path;
- pre-flight os.path.isfile and return a File not found result instead
of raising deep inside discord.File;
- fail loud when Discord accepts the message but attaches nothing, so
the dispatch loop surfaces a warning rather than a silent drop;
- add INFO dispatch logs (base non-image MEDIA fan-out, video send, and
file attachment) so a MEDIA:.mp4 cannot vanish without a trace.
Tests (tests/gateway/test_discord_send.py):
- path-based files=[...] kwarg used, singular file= unset;
- fail-loud when the returned message has no attachments;
- missing file fails fast without resolving the channel;
- forum-parent delivery routes through create_thread with files=[...];
- end-to-end image+video response routes the mp4 to send_video while
images still batch.
Fixes#66797
The identity-refresh TTL used 0.0 as the "never checked" sentinel and
compared it against time.monotonic(). That epoch is arbitrary and starts
near zero on a freshly-booted host, so on CI runners and containers
`monotonic() - 0.0` was itself below the TTL — "never checked" read as
"checked just now" and the first identity refresh was suppressed for the
first 5 minutes of uptime. The stale-handle recovery therefore did nothing
on exactly the machines most likely to be freshly booted.
Invisible on a long-lived dev box (uptime >> TTL); caught by CI.
- Sentinel is now None, meaning never checked and always stale.
- Both TTL comparison sites route through _bot_identity_is_fresh().
- Regression test pins the invariant under a faked 12s-uptime clock.
Verified by re-running the recheck tests against a simulated 3s-uptime
host: green with the fix, and restoring the 0.0 sentinel reproduces the
exact CI failure locally.
Renaming a Telegram bot's @username in BotFather silently stopped the
gateway from answering in groups.
PTB caches getMe() in Bot._bot_user and only rewrites it inside get_me(),
so after a rename the adapter kept comparing mentions against the OLD
handle. The exclusive-mention gate then saw the new @handle, failed to
match itself, and concluded the message was addressed to a different bot
— dropping it before the reply and wake-word fallbacks could run. Native
replies to the bot were discarded too. Polling mode recovered on the next
90s heartbeat; webhook mode never calls get_me() again, so it stayed dead
until restart.
Separately, the bot-handle pattern assumed every bot username ends in
"bot". Collectible (Fragment) usernames can be assigned to bots and drop
that suffix (@jarvis, @pic), so such a bot could not recognise its own
handle in the entity-less fallback and was suppressed by any message that
also named another bot.
- Route every mention comparison through _current_bot_username(), which
prefers the last observed handle over PTB's cache.
- Learn the live handle from inbound updates: Telegram stamps the current
username on our own messages and on reply_to_message. Guarded by user
id, so another account's handle is never adopted.
- Re-check identity out of band (TTL-bounded, one getMe per 5 min) when
the exclusive gate is about to drop a message — the exact stale-handle
symptom — so the mistake self-corrects instead of persisting.
- Refresh identity in webhook mode via a dedicated low-frequency loop,
cancelled on the same teardown fence as the heartbeat.
- Match our own handle by identity rather than shape. Foreign handles keep
the deliberate "...bot" narrowing so human @handles still never act as
routing hints (the intent behind ce4d857021).
Validation: 11 regression tests; sabotage runs confirm each behavioral
test fails with the fix reverted. 157 tests green across the Telegram
gating, reconnect, and topic-mode suites.
The sidebar strip and the Channels page could contradict each other on
the same page load — "Gateway running" next to "The gateway is not
running." /api/status and /api/messaging/platforms each open-coded their
own liveness ladder: status probed GATEWAY_HEALTH_URL and scoped its
PID/state reads to the requested profile, messaging did neither and used
the uncached raw PID probe.
Three deployments hit the split: a cross-container gateway (no local PID,
only the health probe can see it), a profile-scoped dashboard (messaging
borrowed a DIFFERENT profile's runtime state, reporting a false
"connected" that hides a real outage — #71211), and a launch-service
managed gateway with no PID file.
Adds resolve_gateway_liveness() in gateway/status.py as the single ladder
(cached PID -> HTTP health probe -> runtime-status PID with
expected_home) and routes both endpoints, /api/messaging/platforms/{id}/test,
and the kanban dispatcher-presence probe through it. Probe callables are
injectable so the existing monkeypatch seams keep working, and
GatewayLiveness.probe_error distinguishes "down" from "couldn't tell" so
the kanban warning keeps failing OPEN instead of crying wolf.
Closes#71211.
Auxiliary LLM calls routed through OpenRouter now carry the `session_id`
sticky routing key, so they pin to the same provider endpoint as the
conversation they belong to instead of routing independently.
`OpenRouterProfile.build_extra_body` sourced the key only from its explicit
argument. Auxiliary call sites — compression, title generation, vision,
web_extract, session_search, MoA slots — funnel through
`agent.auxiliary_client`, which has no session handle and never passes one.
Those calls sent NO sticky key at all, so OpenRouter fell back to hashing
the opening messages and each aux call could land on a different provider
than the conversation it served. Per OpenRouter's prompt-caching docs a
present `session_id` is used directly as the routing key and activates
stickiness on the first successful request rather than only after a cache
hit, so the missing key also delayed pinning for the main loop.
Resolve it from the ambient conversation contextvar first, explicit argument
as fallback — the same resolution the Nous Portal profile uses (f2f4df064d)
and the same one `nous_portal_tags` already used for the `conversation=` tag.
The ambient value is the session-lineage ROOT, so the key also stays stable
for installs that opt out of the default `compression.in_place: true` and
across delegate-subagent trees.
The xAI `x-grok-conv-id` cache-affinity header in `build_api_kwargs_extras`
had the identical gap and is fixed the same way; it remains model-gated to
`x-ai/grok-*` / `xai/grok-*`.
Fixes#70820. Credit to @webtecnica, who reported it and submitted #70883
first; that PR threaded `session_id` through `call_llm` to reach the same
goal. This takes the ambient-contextvar route instead because none of the
34 `call_llm` call sites currently pass a `session_id`, so the parameter
alone would not have changed any live request.
Tests: 2 cases in tests/providers/test_provider_profiles.py, both verified
to fail against the pre-fix behavior via a sabotage run. E2E-validated on a
real AIAgent driving the out-of-turn compaction path.
The Nous Portal profile publishes a top-level `session_id` that the Portal
uses as its sticky routing key, pinning a conversation to one upstream
endpoint so explicit Anthropic `cache_control` breakpoints stay warm
(those caches are instance-local, so a reroute cold-writes instead of reads).
`NousProfile.build_extra_body` sourced that key only from the explicit
`session_id` argument. Auxiliary call sites — compression, title generation,
vision, web_extract, session_search, MoA slots — funnel through
`agent.auxiliary_client`, which has no session handle and never passes one.
They carried the `conversation=` tag but NO sticky key at all, so each one
routed independently of the conversation it belonged to.
Resolve the key the way `nous_portal_tags` already resolves that tag:
ambient lineage-ROOT contextvar first, explicit argument as fallback. Fixes
every aux call site at once with no per-call-site plumbing.
Also publish the root in the `_compress_context` forwarder when nothing is
ambient. Out-of-turn entry points (`/compact`, the gateway `/compress`
command and its hygiene sweep, partial head compression) call it outside
`run_conversation`'s scope, so the summarizer's aux call ran untagged and
unkeyed; the caller's value is restored in `finally` so a compaction never
leaks its tag.
Scope note: this does NOT keep the compaction turn's own prompt cache warm.
Compaction replaces the history with a summary and rebuilds the system
prompt, so that request is a cold write on any endpoint — what a stable key
buys is the turns AFTER compaction reading the cache it wrote. Under the
default `compression.in_place: true` (#38763) the session id does not rotate
at compaction, so for the main loop the ambient root and the explicit
argument already agree; the ambient resolution additionally holds the key
stable for installs that opt back into rotating compaction and across
delegate-subagent trees.
Salvaged from #71747. Subsumes the aux-call half of #70883 (@webtecnica),
which threaded `session_id` through `call_llm` to close the same gap.
Tests: 4 cases in tests/agent/test_portal_tags.py; both fix halves verified
to fail against the pre-fix behavior via sabotage runs.
The Codex image backend rejected our own request shape for every account, and
we then translated that rejection into "Image generation is not enabled for the
current Codex account. Switch the image provider to OpenAI API key, FAL, or
xAI." — telling every affected user to abandon a provider that had never
actually been tried. That message is why this reads as a setup failure rather
than a bug: the wire error was replaced with a confident, wrong diagnosis.
Removes the classifier and its exception, so any HTTP failure surfaces
verbatim. The paired request-shape fix (previous commit) is what makes the
400 stop happening; this commit makes the next one diagnosable.
Also fixes error-body truncation: bodies were head-truncated at 500 chars, and
Codex error payloads can carry hundreds of bytes of leading metadata, so the
user got a wall of padding and no message. _summarize_error_body() prefers the
parsed error.message and falls back to a truncated raw body.
Docs: drop the unqualified image-to-image claim for the Codex backend and note
that the hosted tool call cannot be forced, so it is best-effort.
Verified E2E against a local fake Codex backend: success path writes a real PNG
with no tool_choice on the wire; the 400 path now returns api_error carrying
"Tool choice 'image_generation' not found in 'tools' parameter" (148 chars)
instead of the entitlement message. Sabotage run confirms all 4 regression
tests fail when the old behavior is restored.
Refs #19505, #49008, #31335.
The chatgpt.com/backend-api/codex backend 400s on every tool_choice
shape for the hosted image_generation tool — it looks up tool_choice as
a function name and never recognizes hosted-tool entries. Removing the
field from _build_responses_payload() lets the host model decide; the
instructions field nudges it toward the tool.
Salvaged from PR #19979 (originally targeted the old
client.responses.stream call, which no longer exists on upstream/main;
the live request now flows through _build_responses_payload + httpx in
_collect_image_b64).
Pair conditional approval buttons into rows of two so the full Allow Once /
Session / Always / Deny set stays readable instead of one truncated 4x1 row.
Follow-up hardening for the salvaged #69240 readiness gate (#67498):
- _start_polling_once now returns its (generation, progress_event) pair
so the strict cold-start gate binds to exactly the generation it
started, instead of re-reading self._polling_progress_event which a
concurrent recovery task may have replaced with a newer generation's
event (the G1/G2 race flagged in the #69240 review).
- Strict cold start no longer schedules background polling recovery: a
polling error during the readiness wait is captured by a strict
callback and fails the connect attempt immediately with a loud
OSError, so GatewayRunner disposes the partial adapter and retries
with a fresh one — no more waiting out the full readiness deadline on
a generation that already errored, and no G2-on-partial-app healing.
- After readiness is proven the strict callback delegates every later
polling error to the real background-recovery callback, preserving
the existing degraded/reconnect semantics for the polling lifetime.
- The readiness-timeout error message now states the deadline and that
the gateway will retry with a fresh adapter (loud failure, not a
silent wait).
- Regression tests: current-generation progress connects; a polling
error during strict cold start fails fast without scheduling
background recovery (the #67498 idle-threads shape); stale-generation
progress is rejected.
Progresses #67498
Use wall deadlines for deleteWebhook and start_polling, then fail cold startup unless getUpdates proves progress. This lets the gateway discard partial PTB state and retry with a fresh adapter.\n\nRefs #67498
The cherry-pick auto-resolution + AST sweep applied plain utf-8 at three
.env reader sites where the salvaged PRs (#62617, #62123) deliberately
use utf-8-sig — a Notepad BOM must not hide/duplicate the first key.
Restore the contract (tests pin it).
AST-driven pass over every Path.read_text()/write_text() without an
explicit encoding= across non-test code: 71 sites in 34 files
(skills_hub, hermes_cli/main+profiles+service_manager+container_boot,
mem0/hindsight/honcho plugins, achievements dashboard, release/CI
scripts, productivity+comfyui skill helpers, agent/*). Verified zero
positional-encoding collisions before insertion; per-file compile()
check after.
Adds a check-windows-footguns rule flagging bare single-line
read_text/write_text (multi-line forms stay covered by the AST guard
test from #38985). Together with the salvaged contributor commits this
retires the ~169-site bare file-I/O class (#37423's long tail).
Follow-up to review feedback:
- mem0 _prompt_api_key read .env with the locale default, so a Notepad
BOM hid the first key from the masked current-value lookup; read it
with utf-8-sig + errors=replace like the canonical readers in
hermes_cli/config.py.
- hindsight _load_simple_env used plain utf-8; it also parses the Hermes
.env during post_setup, where a BOM stuck to the first key. Switch to
utf-8-sig + errors=replace.
- Add hindsight regressions: BOM key matching in _load_simple_env and in
the cloud post_setup writer, plus non-ASCII round-trip preservation,
and a mem0 regression for the BOM'd masked-key lookup. The BOM tests
fail without the fix on any platform.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The mem0 and hindsight memory-provider setup routines round-trip the
user's ~/.hermes/.env: they read existing lines, update the keys they
manage, and rewrite the whole file preserving every other line verbatim.
Both used env_path.read_text() / write_text() with no encoding.
read_text()/write_text() with no encoding fall back to the system locale
(cp1252/GBK on Windows), so on a non-UTF-8 host the preserved lines get
mangled or the call crashes on any non-ASCII value, and — because the
reader never strips a BOM — a Notepad-edited .env makes the first key
fail the in-place match and get duplicated instead of updated.
Match the canonical .env readers in hermes_cli/config.py: read with
encoding='utf-8-sig' (BOM-tolerant) and write with encoding='utf-8'.
mem0/_setup.py already pins utf-8 for mem0.json, so this just aligns the
.env path in the same file. Fixes both memory plugins in one class fix.
Adds regression tests: a BOM'd .env updates the first key in place
(locale-independent, fails without the fix) and non-ASCII existing lines
survive the round-trip.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
On Windows with CJK locales (e.g. Chinese/GBK), pathlib.Path.read_text()
defaults to the system encoding instead of UTF-8, causing UnicodeDecodeError
when reading .env or .json config files that contain non-ASCII characters.
Explicitly pass encoding='utf-8' to all read_text() and write_text() calls
in the hindsight memory provider plugin.
Path.read_text() without an explicit encoding uses the platform's
default encoding. On Windows this is typically cp1252 or mbcs, which
causes UnicodeDecodeError or silent data corruption when reading
UTF-8 content (JSON files, user text, config with non-ASCII chars).
This is the read-side companion to the write_text() encoding fix.
Fixed the most critical locations that read JSON data, user content,
and config files across 14 files with 31 call sites.
Pattern: .read_text() → .read_text(encoding='utf-8')
json.loads(path.read_text()) → json.loads(path.read_text(encoding='utf-8'))
Follow-up to the salvaged #38985: guard the 4 bare read_text/write_text
sites its allowlist missed (google_chat thread-count store + oauth JSON)
and add whatsapp/google_chat to the AST guard test's file list.
- Strip the salvaged commit's inline encoding kwargs where main had since
gained its own (process_registry, local env, cua doctor, gateway,
commands, gateway_windows — the latter keeps its locale-aware
_schtasks_encoding() from #38186)
- Revert encoding kwargs mistakenly applied to non-subprocess APIs
(exa get_contents, tempfile.mkstemp in webhook.py)
- Guard the ddgs worker Popen (new on main since #55339)
- Update two kwarg-snapshot test assertions for the new kwargs
AST-driven pass over every subprocess.run/Popen/check_output/check_call/call
with text=True (or universal_newlines=True) and no explicit encoding=:
append encoding='utf-8', errors='replace' at the kwarg site. 136 call
sites across 28 files (cli.py, hermes_cli/main.py, tools_config.py,
environments, computer_use, gateway, scripts, skills helpers, agent/*).
Together with the salvaged #55339/#60741 commits this closes out issue
#53428's bug class; the salvaged #60751 linter rule in
check-windows-footguns.py now enforces it repo-wide (verified: 807 files
scanned, zero findings).
On Windows with Chinese locale (GBK), subprocess.run(text=True) without
explicit encoding causes UnicodeDecodeError crashes. This fix adds
encoding='utf-8', errors='replace' to all subprocess.run() and
subprocess.Popen() calls that use text=True across 76 non-test Python files.
Fixes#53428 (master tracker for Windows GBK locale crash).
Note: credential_pool.py and electron changes excluded per reviewer request —
those will be submitted as separate focused PRs.
The Telegram adapter's connect retry loop could silently stall after
'Connecting to Telegram (attempt 1/8)...' with the event loop permanently
parked in select() — all threads idle, no attempt 2/8 ever scheduled.
Root cause analysis:
- The retry loop reused the same Application object across all
8 attempts. After a failed initialize() the app could be in a partially-
initialized state (closed httpx transports from ,
or flag set before the hang) causing subsequent calls
to silently skip real initialization.
- CancelledError (a BaseException, not an Exception) propagated silently
through all except handlers with no logging — the task driving the retry
loop could exit without any trace.
- No total watchdog bound existed for the entire retry loop; only per-attempt
timeouts via _await_with_thread_deadline. If the loop itself stalled
between attempts (between-attempt sleep, cleanup, or scheduling), there
was no timeout to catch it.
Fixes:
1. **Total watchdog deadline**: Compute a total deadline for the entire
connect loop (8 attempts × init_timeout + 120s margin). Before each
attempt, check the wall clock; if exceeded, raise OSError immediately
instead of attempting another initialize().
2. **Fresh Application per retry**: On each failed attempt, rebuild
via and re-register all handlers. The old
app is best-effort shutdown with . This ensures
each retry starts with a clean slate — no stale transports, no stale
flag, no leaked state from the previous attempt.
3. **BaseException logging + propagation**: Added
(placed LAST after all other handlers) to log CancelledError and other
non-Exception signals before propagating. Previously these exited the
retry loop silently with no log message.
4. ** block for app rebuild**: The clause runs after
every failed attempt that isn't the last, rebuilding the app and
discarding the old one regardless of which exception class caused the
failure.
RFC 8628 §3.2 makes the device-authorization `interval` optional with a
client-side default of 5 seconds. request_device_code required it, so a
compliant AS that omitted it hit the malformed-response path and the flow
could never complete. Fall back to 5s and cover it with a regression test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds a device authorization grant flow alongside the existing loopback
OAuth flow, so `hermes setup` can connect to Honcho cloud from SSH and
other no-browser environments.
- oauth.py: new HTTP seams — _http_post_form_status (non-raising, since
RFC 8628 polling reads the OAuth error off a 400) and _http_get_json
for the RFC 8414 metadata probe
- oauth_flow.py: DeviceCode, request_device_code, poll_for_token with
slow_down backoff (+5s, capped at 60s) bounded by expires_in, typed
errors (AccessDenied, DeviceCodeExpired, AuthorizationTimeout), and
supports_device_login (fail-closed metadata gate); device flow ends in
the same install_grant tail as loopback so refresh/status work
unchanged
- oauth_flow.py: loopback callback now serves a "sign-in was not
completed" page on consent cancel instead of the success page
- cli.py: cloud menu offers oauth / device / apikey; the device option
only appears when the host advertises the grant, and becomes the
default when no browser is detected
- 18 new tests covering the full flow against a local fake AS, backoff
schedule, error mapping, deadline bound, metadata gate, and wizard
branches
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Follow-up hardening on the salvaged _ensure_client() (#21130 fix):
- Failed-config cooldown: after a refresh attempt fails for a given
resolved config, skip re-probing for 30s. Previously every provider
access against a down endpoint paid a 3s health probe under
_client_refresh_lock and emitted a warning (2+ per turn, some on
user-facing threads: prefetch, tool calls, session end). Retries
still happen after the cooldown or immediately when config changes,
and the log message now says so instead of the false 'disabled until
config changes'.
- Atomic connection snapshot: _conn_snapshot (5-tuple, single
assignment) is published only after a health check passes.
_new_client() and on_memory_write's writer read it as one load, so
background writers can no longer observe a torn mix of old/new
identity fields mid-refresh or target an endpoint that never passed
health. Field writes in _ensure_client_locked keep tracking the
attempted config for the unchanged-config dedupe.
- _env_refresh_enabled moves to the top of initialize(): an exception
mid-initialize (swallowed by MemoryManager) can no longer leave the
provider silently stuck in never-refresh mode.
- _search_prefetch_context reuses _new_client() and degrades to ''
on construction failure instead of propagating.
Mutation-checked: neutering the cooldown or publishing the snapshot on
failed health makes the new regression tests fail.