Use database.journal_mode as the sole non-secret operator setting, preserve the vulnerable-SQLite safety gate and existing WAL databases, validate explicit DELETE results, document the active config path, and cover real SQLite openers with behavioral tests.
Add HERMES_JOURNAL_MODE env / database.journal_mode config for
virtiofs/NFS/SMB where WAL is not crash-safe. Route 5 bypass openers
through apply_wal_with_fallback so a single setting covers every .db
(#68545).
Telegram clients split messages above 4096 chars into multiple updates. A
long '/queue <prompt>' paste arrives as a COMMAND chunk near the limit plus
plain TEXT continuation chunk(s). _handle_command dispatched the command
chunk immediately, so the continuation landed as a separate plain message
that interrupted the running agent instead of being queued.
Near-limit (>= _SPLIT_THRESHOLD) command chunks now route through the same
text-batching pipeline used for split plain-text messages, merging the
continuation before dispatch. Short commands (/stop, /approve, ...) keep the
immediate path and are never delayed.
The compile_mention_patterns promotion moved the 'patterns is None ->
return []' short-circuit into the shared helper, which meant the
telegram/dingtalk wrappers now evaluated self.name (via log_prefix=)
even on the no-patterns path. On main that path returned before
touching any adapter attributes; tests construct bare adapters via
object.__new__ that lack .platform, so TestTelegramGuestMentionGating
failed with AttributeError. Restore the early return in both wrappers
for exact behavior parity with main.
- base.on_processing_complete implements the opt-in remove-ack/add-outcome
flow driven by _OK_EMOJI/_FAIL_EMOJI class attrs and the
_add_reaction(chat_id, message_id, emoji)/_remove_reaction(chat_id,
message_id) primitive shape; default stays a no-op.
- photon drops its override (exact behavioral match).
- slack/discord/feishu/matrix/telegram/google_chat keep overrides: divergent
primitive signatures (team_id routing, raw message objects, reaction-id
handles, replace-semantics setMessageReaction) or extra state protocols.
The disease: ~15 scattered raw yaml.safe_load(config.yaml) reads that
silently miss managed-scope overlay, ${ENV_VAR} expansion, profile-aware
pathing, and root-model normalization. Every new config feature needed an
N-site sweep (incident chain 9cbcc0c9c8 → 732293cf87 → b0e47a98f9 →
1928aa0443). This commit assigns every raw read to an owner and adds a
lint-guard test so the class cannot regrow.
New primitive (additive-only change to hermes_cli/config.py):
read_user_config_raw(path=None) — reads the user file EXACTLY as
written; docstring states it is ONLY legal for write-back round-trips
and raw-file diagnostics. Behavioral reads must use
load_config()/load_config_readonly().
BEHAVIOR FIXES (class-a sites migrated to a canonical loader — these
previously read values that could DIFFER from the effective config):
gateway/run.py _try_resolve_fallback_provider → _load_gateway_runtime_config
keys: fallback_providers/fallback_model (provider, model, base_url,
api_key). Drift fixed: a managed-pinned fallback chain was ignored;
an api_key of "${OPENROUTER_API_KEY}" reached the resolver unexpanded.
gateway/run.py GatewayRunner._load_provider_routing → same loader
key: provider_routing. Drift fixed: managed-pinned routing prefs and
${VAR} templates were ignored.
gateway/run.py GatewayRunner._load_fallback_model → same loader
keys: fallback chain. Same drift as above.
gateway/run.py GatewayRunner._refresh_fallback_model
keeps the raw primitive (its last-known-good-on-parse-failure contract
forbids the fail-open loader, which returns {} on a torn write) but now
applies managed overlay + env expansion inline. Drift fixed: chain
edits under managed scope / env templates were previously frozen out.
tui_gateway/server.py _load_cfg (72 behavioral call sites)
now = raw read + managed overlay (pre-existing) + NEW ${VAR} expansion,
split from a new _load_cfg_raw() write-back primitive. Drift fixed:
e.g. custom_prompt: "hello ${VAR}", agent.system_prompt, model,
api_key/base_url templates reached sessions unexpanded. DEFAULT_CONFIG
is deliberately NOT merged (callers treat missing keys as unset;
`_load_cfg() == {}` sentinels and _save_cfg round-trips depend on it).
tui_gateway/server.py _profile_configured_cwd
keys: terminal.cwd of a NON-launch profile. Drift fixed: managed
overlay + ${VAR} expansion now apply (load_config() would resolve the
wrong profile's home, so the raw primitive + inline pipeline is used).
plugins/platforms/telegram/adapter.py _reload_dm_topics_from_config
→ load_config_readonly(). keys: platforms.telegram.extra.dm_topics.
Drift fixed: managed overlay + profile-aware pathing + expansion.
plugins/memory/holographic _load_plugin_config → load_config_readonly().
keys: plugins.hermes-memory-store.*. Same drift class.
WRITE-BACK ROUND-TRIPS (class-b: stay raw BY DESIGN via read_user_config_raw;
merging defaults/overlay would pollute the saved user file):
gateway/slash_commands.py: model persist x2, _save_gateway_config_key,
memory/skills write_approval toggles
gateway/platforms/yuanbao.py auto-sethome
tui_gateway/server.py _write_config_key + all cfg→_save_cfg blocks
(reasoning show/hide/full/clamp, details_mode[.section], prompt)
→ new _load_cfg_raw()
plugins/memory/holographic save_config
RAW-FILE DIAGNOSTICS + presence-sensitive bridges (class-c: stay raw,
now via the shared primitive with an explanatory comment):
hermes_cli/doctor.py x5 (model validation, stale-root-keys, .env drift,
deprecation sweep, memory-provider probe — the latter two keep their
inline managed overlay where they had one)
gateway/run.py _bridge_max_turns_from_config and the module-level
TERMINAL_*/HERMES_* env bridge (bridging merged defaults would export
all of DEFAULT_CONFIG into the environment; both keep their inline
overlay + expansion)
hermes_cli/send_cmd.py env bridge (same presence-sensitivity)
hermes_cli/gateway.py multiplex-conflict probe (reads the DEFAULT root's
config, not the active profile's — load_config is the wrong owner)
hermes_cli/profiles.py / hermes_cli/web_server.py / tools/wake_word.py
multi-profile reads (load_config targets only the ACTIVE profile home)
cron/jobs.py _resolve_default_model_snapshot and cron/scheduler.py
run_job config read keep their existing inline overlay+expansion but
now share the primitive (their fail-open + last-value semantics and
the deliberate no-defaults merge are preserved exactly).
Failure-semantics audit: every migrated site preserves its exact previous
behavior on missing file ({} / early return) and parse failure (raise into
the caller's existing except, warn, last-known-good, or fail-open) —
read_user_config_raw intentionally mirrors bare open()+safe_load semantics
(raises on parse errors, {} only on FileNotFoundError/non-dict root).
Guard: tests/hermes_cli/test_config_read_guard.py scans the tree for
yaml.safe_load within 6 lines of a 'config.yaml' reference outside an
explicit ALLOWLIST (hermes_cli/config.py, gateway/config.py, gateway/run.py
fallback path, hermes_cli/managed_scope.py which reads the MANAGED file,
gateway/readiness.py parse-health probe) and fails on new offenders.
E2E: tests/hermes_cli/test_config_loader_e2e.py runs a subprocess with a
temp HERMES_HOME (config.yaml containing ${E2E_PROMPT_SUFFIX}) plus a
HERMES_MANAGED_DIR overlay pinning agent.reasoning_effort, asserting
tui _load_cfg resolves "hello world"/"high" while _load_cfg_raw +
_save_cfg round-trip the template and user value verbatim with no
managed/default leakage.
LINE plugin.yaml plus line/wecom-callback/msgraph-webhook/
whatsapp-cloud/teams user-guide pages still documented the old
IPv4-only 0.0.0.0 defaults; update to the dual-stack unset default.
Telegram webhook env docs live in the adapter docstring (updated with
the code change); its plugin.yaml has no webhook host entry.
Same class of bug as the LINE adapter (NS-603): defaulting the webhook
bind to "0.0.0.0" (or hardcoding it) binds IPv4 ONLY, so the listener
is unreachable over IPv6-only private networks such as Fly.io 6PN.
- wecom callback_adapter: DEFAULT_HOST None; config.py env seed no
longer forces 0.0.0.0 when WECOM_CALLBACK_HOST is unset.
- msgraph_webhook: DEFAULT_HOST None; the allowed_source_cidrs
requirement still fires for the all-interfaces default (host=None is
treated as network-accessible).
- whatsapp_cloud: DEFAULT_WEBHOOK_HOST None.
- teams: hardcoded 0.0.0.0 TCPSite bind → _DEFAULT_HOST=None with new
TEAMS_HOST / extra.host override (mirrors LINE_HOST pattern).
- telegram: hardcoded listen="0.0.0.0" → default "" (tornado
bind_sockets opens one socket per address family; verified against
PTB 22.6/tornado) with new TELEGRAM_WEBHOOK_HOST / extra.webhook_host
override.
Explicit host overrides everywhere are preserved; empty/unset collapses
to the dual-stack default. "::" remains a bad substitute on
bindv6only=1 hosts (see LINE adapter comment).
The LINE adapter's webhook server defaulted to host="0.0.0.0", which
binds IPv4 ONLY. On IPv6-only private networks — notably Fly.io 6PN,
where the hosted edge router reverse-proxies LINE ingest to
<app>.internal:8646 over an fdaa: IPv6 address — nothing is listening
on the dialed address: connection refused → customer-visible 502 when
LINE's console verifies the webhook (NS-603).
This is the same bug the generic webhook adapter fixed in d542894ad;
the LINE adapter was never updated to match. Fix mirrors that commit:
- DEFAULT_HOST = None → asyncio/aiohttp create_server binds one socket
per address family (v4 + v6), regardless of the bindv6only sysctl.
"::" is NOT a valid substitute — Fly machines set bindv6only=1, so
it would yield an IPv6-only socket and break IPv4 loopback probes.
- Empty-string host collapses to None; LINE_HOST/extra.host still pin
a specific bind address.
- reuse_address=False scoped to macOS only (BSD wildcard-socket
traffic-splitting footgun), mirroring 9420ad946.
- The three outbound-media guards compared webhook_host == "0.0.0.0"
by string equality; extracted to _missing_public_url() which treats
None/0.0.0.0/::/"" as "no fetchable hostname" so the LINE_PUBLIC_URL
requirement still fires under the new default. _media_url() falls
back to 127.0.0.1 instead of interpolating 'None' into URLs.
Tests: dual-stack default decision table (default None, empty→None,
pinned preserved, LINE_HOST override), behavioural both-families bind
proof via runner.addresses, and the media public-URL guard matrix.
88 passed, ruff clean.
Companion router fix (hermes-agent-router) routes /webhooks/line and
/line/webhook|/line/media to :8646 over 6PN; both are needed for
end-to-end hosted LINE delivery.
Fixes NS-603
resolve_sidecar_dir() probes the filesystem (touch/unlink) and can mirror
sidecar files to HERMES_HOME. Doing that as a module-import side effect
meant plugin discovery, `hermes --help`, and test collection all paid a
filesystem probe (and possibly a mirror copy) just for importing the
photon adapter or CLI.
Convert _SIDECAR_DIR/_NPM_ERROR_LOG in adapter.py and cli.py to lazy
cached accessors (_sidecar_dir()/_npm_error_log()); resolution now
happens on first actual use. Existing tests that monkeypatch the
_SIDECAR_DIR module global keep working — the accessors honor a
non-None value. Adds a regression test proving import performs no
resolution.
The Photon iMessage sidecar needs node_modules under
plugins/platforms/photon/sidecar/, but hosted/managed images keep the
whole install tree under an immutable /opt/hermes — every install and
self-heal path (setup CLI, stale-deps reinstall, cold install) died on
EROFS, and hosted users have no shell to work around it.
Three-layer fix, mirroring the WhatsApp bridge resolver pattern:
1. Bake the deps into the image. The Dockerfile now runs npm ci for the
sidecar in the layer-cached dependency stage (deterministic installs
from the committed lockfile; the postinstall spectrum-ts patch runs
at build time). Hosted happy path needs no runtime install at all.
2. New sidecar_paths.resolve_sidecar_dir() decides where the sidecar
runs from: PHOTON_SIDECAR_DIR override > writable source dir (dev
installs, unchanged) > read-only dir with baked fresh deps (managed
image) > mirror to $HERMES_HOME/photon/sidecar (writable data
volume) when deps are missing or stale in a read-only tree. The
mirror refreshes changed source files on image updates while
keeping node_modules, so the existing lockfile-staleness self-heal
works there.
3. connect() can now cold-install: _start_sidecar() runs the bounded
npm ci bootstrap when node_modules is missing instead of raising
immediately, and check_requirements() reports available when a
self-install is possible (npm present + writable resolved dir) so
the gateway actually creates the adapter on hosted instances. A
failed bootstrap still raises the actionable error, which connect()
surfaces as the retryable SIDECAR_FAILED fatal state on the
dashboard.
Tests: resolver decision table (env override, in-place, mirror,
refresh, fail-open), cold-install lifecycle paths, and a Dockerfile
contract test guarding the baked-deps + no-chown invariants.
Fixes NS-606.
Maintainer rework of #45580 (issue #54036) on top of the contributor's
cherry-pick, which targeted spectrum-ts 3.1.0 while main pins 8.0.0:
Sidecar (primary detection, new):
- stream-staleness.mjs: pure decision rules, executable under node.
* classifyProbeRejection: only a not-found-shaped rejection of the
synthetic-id read counts as a completed round-trip (ALIVE); any other
rejection is INCONCLUSIVE — never alive. The original /probe treated
ANY rejection as alive, which was too loose.
* shouldProbe: probe only after 10+ min of stream silence (configurable
via PHOTON_STREAM_SILENCE_PROBE_MS; <=0 disables) with a cooldown.
* isZombieSuspect: zombie only on silence past threshold AND a
probe-proven live channel. Silence alone NEVER degrades (shared lines
can be quiet for hours); inconclusive probes NEVER degrade (network
may be down — the iterator will throw and the re-subscribe loop
recovers on its own).
- index.mjs: track last inbound-iterator yield (noteInboundYield), run a
30s watchdog tick, and on a confirmed zombie feed markStreamDegraded ->
the existing exit-75 restart path. /healthz gains a stream.staleness
block (silentForMs, threshold, lastProbeOutcome, zombieSuspected).
/probe reworked to strict semantics: 200 only on a proven round-trip,
503 with outcome hung|inconclusive otherwise.
Adapter (second layer, reworked):
- _probe_once returns tri-state alive|hung|inconclusive; only a hung
sidecar HTTP call counts toward the respawn counter — inconclusive
resets nothing and triggers nothing.
- default probe_interval_seconds 60 -> 600 (conservative; avoid restart
storms on quiet lines).
- _monitor_sidecar_health surfaces zombieSuspected from /healthz as a
warning; the fatal UPSTREAM_STREAM_DEGRADED path is unchanged and fires
when the sidecar escalates.
Tests: test_zombie_stream_watchdog.py executes the real node decision
module and drives the adapter against mocked /healthz responses;
test_presence_watchdog.py updated for the tri-state probe.
Also adds contributor mappings for nickkarhan (#53283) and vaibhavjnf
(#45580).
Follow-up to the URL markdown fix: extract the /send builder decision into
sidecar/send-format.mjs and rewrite test_url_send_path.py to execute the real
module under node (format+text in -> chosen builder out) instead of regex-
grepping index.mjs source, which is a banned test pattern in this repo.
spectrum-ts's live-stream consumer (consumeLive in spectrum-ts 3.x) only
reconnects when its inbound async iterator throws or ends. A half-open
("zombie") gRPC socket — where the TCP connection stays ESTABLISHED but the
peer is gone (NAT idle-timeout, network blip, laptop sleep) — makes the
iterator hang forever: no error, no end. The SDK exposes no gRPC keepalive
knob (createClient takes only {address, tls, token}; grpc.keepalive_time_ms
defaults to -1 = pings off), so the inbound stream silently dies and stays
dead until the gateway is restarted. Symptom: the agent's iMessage line goes
"online but deaf" — Photon's cloud-side fallback answers users with "the agent
isn't online right now" and inbound never reaches the gateway.
Fix, entirely in the code we own (no SDK fork):
- Sidecar gains a POST /probe endpoint that drives a cheap unary read
(space.getMessage on a synthetic id) over the SAME gRPC channel the inbound
stream uses. A live channel round-trips in ms (server returns not-found,
which is success for liveness); a zombie hangs. It sends nothing to any user
and creates no chat (space.get is local in shared/dedicated mode; only the
message read touches the wire).
- The adapter runs a presence watchdog: it probes on an interval, skips the
probe when natural inbound traffic already proved liveness within the
window, and after N consecutive failed probes respawns the sidecar — a fresh
Spectrum() re-subscribes the stream and re-registers presence. Successful
probes double as application-level keepalive, helping prevent the zombie
from forming at all. Respawn is lock-guarded against double-spawn and the
watchdog is torn down cleanly on disconnect.
Behavioural settings live in config.yaml (extra), bridged to env per the
.env-is-secrets-only convention:
probe_interval_seconds (60), probe_timeout_seconds (10),
probe_max_failures (3). A non-positive interval disables the watchdog.
Tests: tests/plugins/platforms/photon/test_presence_watchdog.py covers config
resolution, the disable switch, probe alive/dead(500)/timeout/no-client, the
core N-failures->one-respawn detection, success-resets-failures, stop-then-
start respawn ordering, and lock-guarding — all without spawning Node or
hitting the network.
Contributed by Vaibhav Sharma (X: @vabbyshabby).
imessage.effect.message crashed the sidecar at import against SDK stubs/
builds lacking the effect surface (caught by the patch-failure health test).
Optional-chain with {} fallback; /send-effect rejects cleanly instead.
Follow-up to the #48194 pick: it was written before #43665 landed and
re-added its own /send-poll sidecar route and poll import. Collapse the
duplicates:
- keep #43665's /send-poll route (>=2 trimmed string options) as the
single sidecar implementation; drop #48194's variant
- drop the duplicated poll import in the sidecar destructure
- make adapter.send_poll() a thin wrapper over _sidecar_send_poll(), the
one /send-poll client (shared with the poll-backed clarify path), and
align its validation to the sidecar's >=2-options contract
The `clarify` tool's multiple-choice prompts flattened to a numbered text
list on Photon/iMessage, even though iMessage has a native poll bubble and
spectrum-ts already exposes it via the `poll()` content builder. Two gaps
caused the flattening:
* Outbound: the sidecar only had `/send` (text); there was no way to send
a poll, so the base adapter's numbered-text fallback was used.
* Inbound: `normalizeContent()` handled only text/attachment/voice, so a
poll vote (`poll_option`) was dropped on the floor ("[Photon content
type not handled: poll_option]") and never resolved the clarify.
Fix, end to end:
* Sidecar: import `poll` from spectrum-ts; add a `/send-poll` route
(`space.send(poll(title, ...options))`); serialize inbound `poll_option`
(the vote: chosen title + selected bool) and `poll` content in
`normalizeContent()`.
* Adapter: override `send_clarify` — for choices, send a native poll via
`_sidecar_send_poll` and call `mark_awaiting_text` so the gateway's
existing pending-clarify text-intercept resolves the answer; open-ended
clarifies keep the plain-text path. Inbound `poll_option` selections are
dispatched as a plain-text MessageEvent carrying the chosen option
(deselections / empty votes are dropped). If the poll send fails (an
older sidecar without `/send-poll`, or a send error) it falls back to the
numbered-text clarify, so nothing regresses on a half-upgraded restart.
No new model tool, no new env var, no core change — the capability lives at
the platform edge. The poll vote reuses the existing clarify text-intercept
resolution path, so no new gateway resolution mechanism is introduced.
Tests: tests/plugins/platforms/photon/test_poll_clarify.py — inbound vote ->
choice text, deselection/empty-vote dropped, send_clarify sends a poll +
enables text-capture, open-ended stays text, and poll-failure falls back to
the text list. Full photon suite green.
Contributed by Vaibhav Sharma (X: @vabbyshabby).
Maintainer follow-up to the #51193 salvage:
- _send_with_retry: permanent classes (auth_or_config, target_not_allowed)
now short-circuit BEFORE the unconditional plain-text fallback resend,
including when a retry attempt surfaces one — no more double-sends of
permanently-failing requests.
- sidecar classifySidecarError: new structured code target_not_allowed for
Spectrum's 'Target not allowed for this project' AuthenticationError
(shared/free-tier lines cannot initiate outbound sends to new targets).
Classification applies to every handler sharing the catch-all
serverError path (/send, /send-attachment, /react, /typing, ...).
- _standalone_send now parses the structured error body too (it reads
sidecar responses independently of _sidecar_call) and returns
error_class/retryable alongside the message.
- target_not_allowed maps to a canonical user-facing message in both
paths; raw upstream error text never leaks through the structured code.
Closes the actionable halves of #50971, #51897, #52794.
Photon's Node sidecar intentionally hides raw handler exceptions, but the Python adapter still needs a safe failure class and retryability bit so delivery retries do not collapse into an opaque generic 500.
Constraint: Sidecar responses must not leak raw stack traces or private exception text
Rejected: Retry every internal sidecar error | masks permanent auth/config failures
Confidence: high
Scope-risk: narrow
Directive: Keep sidecar error text generic; extend safe error classes instead of exposing raw SDK failures
Tested: uv run --with pytest-timeout pytest tests/plugins/platforms/photon/test_overflow_recovery.py -q
Tested: uv run --with pytest-timeout pytest tests/plugins/platforms/photon -q
Tested: uv run ruff check plugins/platforms/photon/adapter.py tests/plugins/platforms/photon/test_overflow_recovery.py
Tested: python3 -m py_compile plugins/platforms/photon/adapter.py tests/plugins/platforms/photon/test_overflow_recovery.py
Tested: node --check plugins/platforms/photon/sidecar/index.mjs
Tested: git diff --check
Tested: python3 scripts/check-windows-footguns.py --diff origin/main
Not-tested: Live Photon/Spectrum delivery against a real iMessage account
Related: #50971
`PhotonAdapter._start_sidecar` is `async`, but it ran the Spectrum
mixed-attachment patch script with a bare `subprocess.run(...)`: it spawns
node and *waits* for it, with `timeout=10`. Executed inline that holds the
shared gateway event loop for the whole window, so no other platform's
messages, heartbeats, or sessions are serviced until it returns.
The same function already establishes this exact invariant twenty lines
above, where the stale-dependency reinstall hops to a worker thread:
# Runs off the event loop so a cold install can't freeze every other
# platform's traffic.
if _sidecar_deps_stale():
await asyncio.to_thread(_reinstall_sidecar_deps)
The patch spawn never got the same treatment. It is not startup-only
either — `_start_sidecar` is called from `connect()`, which takes
`is_reconnect`, so an ordinary Photon reconnect (network blip, sidecar
death) re-runs it and stalls a live gateway that is actively serving
Discord/Telegram/Slack traffic.
Dispatch it via `asyncio.to_thread` like its sibling. Same off-the-loop
class as the inbound-image decision (#66688) and the cron-fire verifier.
Adds a regression test asserting the spawn executes on a worker thread
rather than the loop thread.
`_reap_stale_sidecar` is `async`, but it identified the processes holding
the sidecar port with two blocking helpers called inline:
* `_find_listener_pids` -> `subprocess.run(["lsof", ...], timeout=5.0)`
* `_pid_is_sidecar` -> `subprocess.run(["ps", ...], timeout=5.0)`, once
per candidate pid
so the inspection can hold the shared gateway loop for 5 + 5·N seconds
while nothing else on it is serviced. It only runs once the /healthz probe
finds something already listening — the orphaned-sidecar recovery path —
and `_reap_stale_sidecar` is awaited from `_start_sidecar`, which runs on
every reconnect (`connect(is_reconnect=True)`). The stall therefore lands
on a live gateway that is still serving every other platform, right when a
crashed sidecar has already left an orphan behind.
Move the whole inspection to one `asyncio.to_thread` hop (one hop rather
than N+1 round trips). The reaping semantics are untouched: SIGTERM for
verified orphans, SIGKILL escalation, and both foreign-listener
RuntimeErrors behave exactly as before.
Same off-the-loop class as the inbound-image decision (#66688) and the
cron-fire verifier.
Adds a regression test asserting both the lsof lookup and the per-pid ps
check execute on a worker thread rather than the loop thread.
Follow-up widening of the #73170 pattern: apply the same
asyncio.current_task() guard used for the health task (and now the
supervisor task in _stop_sidecar) to the inbound-task cancel path in
disconnect(), so an inbound task that triggers disconnect() cannot
cancel-and-await itself.
Fixes#73159.
When the Photon sidecar exits unexpectedly, _supervise_sidecar()
(running as self._sidecar_supervisor_task) correctly detects
SIDECAR_CRASHED and calls self._notify_fatal_error(). The Gateway's
fatal-error handler answers that by calling adapter.disconnect(),
which calls _stop_sidecar() -- from INSIDE the very task that's
currently executing this whole chain.
_stop_sidecar()'s cleanup unconditionally cancelled
self._sidecar_supervisor_task. Cancelling the currently-running task
raises CancelledError at its own next await point (inside
_notify_fatal_error() or _stop_sidecar() itself). Since
asyncio.CancelledError inherits from BaseException (not Exception),
the Gateway's `except Exception` guards around the fatal-error handler
don't catch it -- the handler aborts before ever reaching the
"queue platform for background reconnection" step. Photon then stays
permanently in `retrying` state until the whole process is manually
restarted, even though detection worked correctly and the underlying
transient upstream outage had long since recovered.
Fix: in _stop_sidecar()'s cleanup, check whether
self._sidecar_supervisor_task is asyncio.current_task() before
cancelling it. A task cannot legally cancel itself in any useful way
anyway (the cancellation only takes effect at its own next await,
which is exactly the corruption described above) -- when we're inside
the supervisor's own call stack, it's already in the process of
finishing on its own once _notify_fatal_error() returns, so skip the
cancel and just clear the reference.
This mirrors an existing precedent in the same file: disconnect()
already guards its OTHER task-cancellation (self._sidecar_health_task)
the same way (`if task is not asyncio.current_task()`), just not the
supervisor task in _stop_sidecar().
Per the issue's own note that existing tests mock out
_notify_fatal_error() entirely (so this integration chain was never
exercised), added two tests that drive the REAL chain: one runs
_supervise_sidecar() as an actual asyncio task with a real
_notify_fatal_error() that calls the real disconnect() -> _stop_sidecar(),
confirming the task completes without CancelledError and that the
post-disconnect reconnect-queue step actually executes; a second
confirms the OTHER call path (external cleanup, a different task)
still correctly cancels a running supervisor exactly as before.
Reverting only the adapter.py fix (keeping the new test) reproduces
the exact CancelledError from the bug report, confirming this is a
genuine regression test.
2 new tests pass; 113/113 in the full tests/plugins/platforms/photon/
directory (no regression to sidecar lifecycle, overflow recovery, or
health-monitoring behavior).
Follow-up to #69112. That PR hardened the shared gateway dispatch path in
gateway/run.py against the *caller* being cancelled. A second, self-referential
cancellation specific to PhotonAdapter sits one layer underneath it and survived
that fix.
PhotonAdapter is the only platform adapter that awaits _notify_fatal_error()
inline, on the same task that detected the fault. Both _monitor_sidecar_health
and _supervise_sidecar run as self._sidecar_health_task /
self._sidecar_supervisor_task, and the notification routes into
GatewayRunner._handle_adapter_fatal_error_impl, which tears the adapter down via
_safe_adapter_disconnect -> disconnect(). disconnect() then cancels
self._sidecar_health_task and awaits it -- which, when the health task is what
raised the notification, means disconnect() cancels its own caller several
plain-await frames up.
disconnect()'s `task is not asyncio.current_task()` guard does not catch this.
The current task where that guard evaluates is the wrapper
_await_adapter_cleanup_with_timeout creates around disconnect() via
asyncio.ensure_future, not the health task further up the chain, so the guard
passes and the cancel lands.
CancelledError stopped subclassing Exception in Python 3.8, so the
`except Exception` that wrapped the inline notify call never saw it. The health
task died silently mid-handoff: no log line, no "exception never retrieved"
warning (cancellation is normal asyncio), and no retry. The platform stayed
stranded until the gateway was restarted by hand.
Fix: dispatch the notification onto a new task, the same pattern
DiscordAdapter._handle_bot_task_done already uses for this reason. disconnect()
can then cancel the health/supervisor task freely without that cancellation
reaching the code still running the handoff, so the handoff always reaches the
reconnect queue. Both Photon fatal call sites are converted: the health-poll
path (observed wedging) and the sidecar-crash path (same shape, not yet
observed). gateway/run.py is untouched.
Observed twice on a self-hosted gateway, ~4h38m and ~52min of silent inbound
outage, both cleared only by a manual restart, both post-dating #69112's merge.
In each case the fatal log line appears and `queued for background reconnection`
never does.
Tests: new tests/plugins/platforms/photon/test_fatal_notify_self_cancel.py
covers the self-cancellation (fails with CancelledError without this change),
that the dispatch does not block its caller, that a failing notification warns
rather than raising, and a source guard against reintroducing either inline
await. Two assertions in test_overflow_recovery.py that drove these coroutines
directly now drain pending tasks before asserting delivery, since the
notification is deliberately no longer awaited inline.
Prepared with agent assistance and reviewed before submission.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Consolidates the native-transport half of PR #73636 by @ScaleLeanChris
onto the merged adapter: persistent NIP-42-authenticated Nostr WebSocket
subscription as the default inbound path (transport=auto|websocket|poll),
kind-44100 membership events for live DM discovery, since-timestamp
resume on reconnect with bounded exponential backoff, and automatic
fallback to CLI polling when the WS can't be established. Events route
through the same _handle_event() pipeline as the poll loop, so de-dupe,
mention gating, p-tag DM latching, and allow-lists behave identically on
both transports. Outbound stays on the CLI (one-shot sends never race a
WS auth handshake — his design).
E2E verified against a real in-process websockets relay: NIP-42
challenge -> signed kind-22242 AUTH (event id re-derived server-side) ->
REQ subscription -> EVENT dispatch -> clean disconnect.
Co-authored-by: ScaleLeanChris <chris@scalelean.com>
Extracted verbatim from PR #73636 by @ScaleLeanChris — nsec/hex key
decoding, secp256k1 point math, BIP-340 Schnorr signing, and NIP-42
AUTH event construction with optional NIP-OA owner attestation tag.
Two independent cross-PR collisions red on main (slice 8/8):
1. fd4f756492 (salvaged from stale #54514) added an early 'drop U+FFFC
placeholder' return at the top of _dispatch_inbound — written before
the deferred-wait handler (6b91b50c6e/afab7ed46e) existed further
down the same function. The early return shadowed it: _pending_fffc
never populated, no attachment-timeout tracking, 4 tests red. Remove
the duplicate block; the deferred handler already drops the
placeholder AND tracks/cancels/warns.
2. 9cf2046081 tightened sidecar_deps_installed() to require
node_modules/spectrum-ts, but test_runtime_record's _patch_spawn
fixture still created only bare node_modules/ — 2 tests red. Mirror
a real completed install.
tests/plugins/platforms/photon: 164/164 after; 158/164 before.
The sidecar auth token is generated at spawn (secrets.token_hex) and
existed only in the gateway process memory + sidecar child env, so
_standalone_send from cron subprocesses, hermes send, or the dashboard
structurally could not authenticate (#69960).
The adapter now writes <hermes-home>/runtime/photon-sidecar.json
({port, token, pid}, 0600, atomic tempfile+os.replace) once the sidecar
passes its /healthz readiness check, and deletes it in _stop_sidecar,
on every startup-failure path, and at disconnect so a stale record
never outlives a dead sidecar. _standalone_send falls back to the
record when PHOTON_SIDECAR_TOKEN is unset, validating the recorded pid
is alive first; a stale record yields a clear 'gateway appears to be
down' error. Docs note the gateway-must-be-running requirement and the
Photon-side shared-line initiation policy (#51897).
All Photon sidecar HTTP requests target 127.0.0.1 — they should
never be routed through a system HTTP proxy. When trust_env=True
(the default), httpx picks up macOS system proxy settings and
routes localhost requests through the proxy. If the proxy returns
a spurious response (e.g. 502), _reap_stale_sidecar() interprets
it as 'port in use by a non-sidecar process' and refuses to start,
yielding: 'pids: unknown, not a Photon sidecar'.
Set trust_env=False on all five httpx.AsyncClient call sites in
the Photon adapter so localhost sidecar communication bypasses
the system proxy entirely.
Addresses teknium1 review on #50983:
- cli.py's `hermes photon status` and adapter.py's _start_sidecar() still
used the old node_modules/-existence check while check_requirements()
had moved to a spectrum-ts content check. Extracted the check into a
shared sidecar_deps_installed(), used by all three, so an
empty/partial node_modules/ (aborted npm install) is rejected
consistently instead of only in check_requirements().
- _install_sidecar()'s success-path _NPM_ERROR_LOG.unlink() only caught
FileNotFoundError, so a PermissionError/OSError on a locked file would
propagate. Broadened to OSError.
- npm stderr was truncated only when read back in check_requirements();
an unbounded stderr was written to disk on every failed install. Now
truncated to _NPM_ERROR_LOG_MAX_CHARS before write_text().
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Problema
--------
Quando o npm install do sidecar Photon falhava, o Hermes descartava toda
evidencia e continuava normalmente - deixando o adapter de iMessage
silenciosamente ausente, sem nenhuma mensagem de erro acionavel.
Tres falhas independentes formavam o caminho de falha silenciosa:
1. check_requirements() sem logging
Cada branch de return False retornava sem emitir nenhum log. O core em
platform_registry.py so consome o bool de check_fn() e loga uma mensagem
generica com o install_hint - sem acesso ao motivo real da falha.
if not HTTPX_AVAILABLE: return False # sem log
if not shutil.which(node): return False # sem log
if not node_modules.exists: return False # sem log
2. node_modules/ parcialmente criado passava o guard (Risk 2)
npm cria node_modules/ antes de abortar em ENOSPC, timeout de rede ou
EACCES. O diretorio existia, check_requirements() retornava True (falso
positivo), o adapter era registrado, e o crash acontecia em runtime com
um erro de modulo ausente aparentemente nao relacionado ao setup.
3. stderr do npm descartado (Risk 3)
subprocess.run sem stderr=PIPE. O output de erro aparecia no terminal
durante o setup e sumia depois - diagnostico impossivel em CI/CD, Docker,
VPS headless, e qualquer reinstalacao posterior.
Correcoes
---------
adapter.py - check_requirements() agora loga por branch:
- httpx ausente -> logger.warning com nome do pacote
- node nao no PATH -> logger.warning com nome do binario e env var
- spectrum-ts ausente -> logger.debug com path do sidecar + ultimo erro npm
(DEBUG nao WARNING: estado normal pre-setup; check_fn() e chamado de
5 hot paths do core incluindo polling do /api/status)
adapter.py - content check em vez de existence check (Risk 2):
antes: if not (_SIDECAR_DIR / node_modules).exists()
depois: if not (_SIDECAR_DIR / node_modules / spectrum-ts).exists()
spectrum-ts e a unica dependencia do package.json. Checar sua presenca
garante que instalacao parcial/abortada e detectada no boot do gateway,
nao na primeira mensagem recebida via gRPC.
cli.py - stderr capturado e persistido (Risk 3):
subprocess.run passa agora stderr=subprocess.PIPE, text=True em ambas as
chamadas (npm ci e npm install fallback). O stderr capturado e:
- impresso em sys.stderr imediatamente (output visivel no terminal)
- persistido em _NPM_ERROR_LOG = sidecar/.photon-npm-error.log se
returncode != 0, limitado a 300 chars
- apagado de _NPM_ERROR_LOG se returncode == 0 (evita erro stale)
check_requirements() le _NPM_ERROR_LOG quando spectrum-ts esta ausente e
inclui o conteudo no DEBUG log - o erro do npm sobrevive ao terminal, ao
restart do gateway e a reinicializacao da maquina.
sidecar/.gitignore - adicionado node_modules/ e .photon-npm-error.log.
Isolamento - sem impacto no core:
- check_fn() continua retornando apenas bool; core nao e modificado
- Logging usa namespace plugins.platforms.photon.adapter, isolado de
gateway.* e hermes_cli.*
- Cada plugin tem seu proprio check_requirements() independente
- OSError no write/read de _NPM_ERROR_LOG e silenciado - nunca propaga
Testes - 24/24 passando:
test_check_requirements_risks.py (7 testes):
WARNING emitido quando httpx ausente
WARNING emitido quando node nao no PATH
DEBUG emitido (nao WARNING) quando spectrum-ts ausente, com path
node_modules/ vazio agora retorna False (Risk 2 resolvido)
_NPM_ERROR_LOG escrito no stderr do npm em falha
_NPM_ERROR_LOG apagado apos npm bem-sucedido
erro npm aparece no DEBUG log quando node_modules ausente
test_npm_error_log_regression.py (9 testes - vetores de falha da solucao):
return code contrato intacto (0 em sucesso, nao-zero em falha)
OSError no write do log silenciado, exit code ainda propagado
OSError no read do log silenciado, check_requirements() retorna False
stderr vazio nao cria arquivo de log
proc.stderr=None nao lanca AttributeError
log stale apagado apos reinstall bem-sucedido
DEBUG emitido mesmo sem log de erro (setup pela primeira vez)
Closes#50981
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
hermes photon setup unconditionally called regenerate_project_secret()
on every re-run, invalidating the credential held by a running sidecar
and causing all outbound sends to fail with AuthenticationError.
Now validates existing credentials via a lightweight list_users call
before deciding to regenerate. Only rotates when no credentials exist
or the existing ones are invalid, and warns the user to restart the
gateway when rotation occurs.
Fixes#50755
Maintainer follow-up to the #72763 salvage: check_photon_token_valid()
now delegates to the existing validate_photon_token() (session lookup +
/api/projects/) instead of a bespoke get-session-only probe, since the
device flow can mint tokens that pass the session check but fail the
project APIs that setup actually uses. Semantics preserved: definitive
auth rejection = stale, transient errors = probably-valid.
Two related bugs in hermes photon setup / gateway_setup:
Bug 1 -- stale token reused (401)
----------------------------------
_cmd_setup reused an existing dashboard token without validation.
The device token has a short TTL (~3-4 days observed); reusing a
stale token caused every management API call (find_project_by_name,
regenerate_project_secret, etc.) to fail with 401. The operator
saw confusing "spectrum provisioning failed: 401" errors.
Fix: check GET /api/auth/get-session before using the stored token.
On 401/403, clear the stale token with clear_photon_token() and
fall back to a fresh device-login flow automatically.
Bug 2 -- channel left disabled after successful setup
-----------------------------------------------------
After all five provisioning steps completed, config.yaml still had
photon.enabled: false, so the gateway never loaded the Photon
adapter. Every inbound iMessage hit Photon's offline auto-responder
without the operator being notified.
Fix: call write_platform_config_field('photon', 'enabled', True,
raw=True) as a final setup step so the gateway picks up the freshly
configured channel on its next start.
New public API in auth.py:
- clear_photon_token() -- discard stored token from auth.json
- check_photon_token_valid(token) -- lightweight session-check test
References: #72763
store_photon_token/store_project_credentials/store_user_numbers read-modify-
wrote auth.json without hermes_cli/auth.py's _auth_store_lock(), the
cross-process flock every other writer of that file (credential_pool
refresh, model_switch, fallback_cmd, nous_portal adapter, etc.) already
holds. A concurrent write from either side during the unlocked
load-mutate-save window silently drops the other side's update.
Review follow-up: if os.fdopen() raised before taking ownership of the
descriptor returned by os.open(), the cleanup handler unlinked the temp
file but leaked the fd. Close it explicitly on that path, mirroring the
credential-writer cleanup from #62837.
Strengthen the tests so the old writer could not pass them: an os.open
spy asserts O_CREAT | O_EXCL and an explicit 0o600 mode (the final-mode
check alone was also satisfied by the post-write chmod), and a forced
fdopen-failure test asserts the raw fd is closed and no temp file is
left behind.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
_save_auth() wrote the bearer token with tmp.open('w') — created at
process umask (typically 0o644) — and only chmod'ed to 0o600 after the
write, leaving a window where the token sat world-readable. The temp
name was also fixed and predictable (auth.json.tmp), so it could be
pre-planted (symlink attack).
Create the temp file with os.open(O_WRONLY|O_CREAT|O_EXCL, 0o600) and a
per-process random suffix, fsync before the atomic replace, and clean
the temp file up on failure. Mirrors hermes_cli/auth.py:_save_auth_store
(#19673, #21148), which hardened the same pattern in the core writer.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Photon (iMessage) has no real edit API for already-sent messages. When
streaming completes, the gateway attempts to edit the message to remove
the streaming cursor (▉). Without edit support, this cursor gets stuck
in the final message, corrupting Unicode characters.
This change sets SUPPORTS_MESSAGE_EDITING=False on PhotonAdapter, which
causes the gateway to suppress the streaming cursor entirely for this
platform (via _effective_cursor in gateway/run.py). This prevents the
stale tofu square (▉) from appearing in streamed iMessage responses.
Fixes#49253
Follow-up on the salvaged #71610 commits:
- acquire/release a scoped lock on relay_url+pubkey in connect/disconnect
(IRC pattern) so two profiles can't drive one Buzz identity — duplicate
replies and split de-dupe state; +2 tests
- negative-cache _resolve_user_name failures so a profile-less pubkey
doesn't re-hit 'users get' every poll sweep (flagged by @jethac on the PR)
- register user-guide/messaging/buzz in website/sidebars.ts (page was
unreachable — the #63359 trap)
On hosted relays `buzz dms list` reliably returns [] even when DM
conversations exist, so DMs leaked in via `channels list` (as entries
named "DM" with an empty description) and were seeded chat_type="group".
That put them behind the channel mention gate: "@Chip /whoami" worked
but an un-mentioned DM was silently dropped.
Classify from the Nostr tags of real traffic instead: a message another
user sends in a DM carries a structural ["p", <own pubkey>] tag even
when the text never mentions the agent, while in a real channel a
p-tag-to-self only ever accompanies a visible @mention (typed mention
or reply). A group conversation therefore latches to chat_type="dm" on
the first kind-9 event that is p-tagged to self WITHOUT a visible
mention in the content — guarded by channels-list metadata so a real
community channel (real name / non-empty description) is never
reclassified by a reply or mention that p-tags the agent.
- latch during history seeding too, so a leaked DM bypasses the
mention gate from the very first poll after connect
- keep `dms list` as a best-effort source, and scan `channels list`
as a fallback so DM conversations opened mid-run still get watched
- strip a leading @mention in DMs as well, so "@Chip /whoami" keeps
firing as a slash command after the conversation reclassifies
Refs #68871
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Channel mention-gating was hardcoded on. Add a configurable require_mention
(default True, preserving current behavior). When False, the agent responds to
every message in a watched channel, not only when @mentioned; DMs always
dispatch. Read from config.yaml gateway.platforms.buzz.extra.require_mention
with BUZZ_REQUIRE_MENTION env override, bridged via apply_yaml_config_fn like
the other settings. A leading mention is still stripped when present.
Refs #68871
Channel messages address the agent with a leading @mention (e.g. '@Chip
/whoami'). The adapter passed the raw content through, so the gateway's
is_command() check (text.lstrip().startswith('/')) never matched and slash
commands were routed as plain chat. Strip a leading mention (name, npub, or
hex form) before dispatch in channels, mirroring the Discord adapter. Also
cleans normal prompts ('@Chip what's up?' -> 'what's up?'). DMs are untouched.
Verified live: '@Chip /whoami' -> '/whoami' after connect populates identity.
Refs #68871
The Buzz adapter's check_requirements() reads config from env only, so a
config.yaml-only setup (relay URL in gateway.platforms.buzz.extra) failed the
check_fn gate and was silently skipped at startup. Add an apply_yaml_config_fn
hook that bridges buzz.extra -> BUZZ_* env vars, mirroring the Slack/Telegram
pattern; BUZZ_PRIVATE_KEY stays a .env secret.
Also fix send_reaction() to use buzz-cli's real flags (--event <id> --emoji),
replacing the non-existent --channel/--message-id flags that would have failed
on every message. Verified live against the hosted relay (accepted:true).
Refs #68871
Plugin-path adapter (zero core changes) connecting Hermes to a Buzz
community relay via the buzz CLI binary (JSON in/out, arg-list exec,
key passed via env only). Inbound uses a poll loop with per-channel
high-water marks seeded from newest (no history replay), event-id
de-dupe, self-echo suppression by pubkey, and mention gating in
channels (DMs always dispatch). Registers env_enablement, cron
home-channel delivery, and an out-of-process standalone sender,
mirroring the IRC plugin.
Verified against a live relay: connect -> send -> poll -> MessageEvent
round-trip, self-echo suppressed, clean disconnect.
Known limitation: polled inbound (default 4s); a websocket transport
(buzz-ws-client) is a future optimization.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Move U+FFFC placeholder detection before _record_last_inbound() so the
placeholder message id is not recorded as the reaction target
- Cancel pending U+FFFC tasks in disconnect() to prevent task leaks
- Check mimeType audio/x-caf in addition to filename for CAF→VOICE promotion,
fixing unnamed attachments that default to '(unnamed)'
- Add 7 Photon adapter tests: CAF named/unnamed promotion, U+FFFC no-dispatch,
U+FFFC not recorded as last inbound, U+FFFC+attachment cancel, timeout fire,
disconnect cleanup
- Add 6 transcription tests: ffmpeg conversion, afconvert fallback, all-fail,
CAF→WAV before Groq, conversion failure error, local provider skip
iMessage's gRPC cloud push fires before CloudKit syncs the attachment
metadata, emitting a U+FFFC placeholder as text. Instead of dispatching
a spurious message, start a non-blocking 15s wait. If the real attachment
arrives, cancel the timeout and process normally. If not, log a warning.
spectrum-ts classifies all inbound attachments as type 'attachment',
never 'voice'. Without this, .caf voice notes are routed as AUDIO or
DOCUMENT and bypass the STT pipeline.
The msg_type_str == "richText" branch reset msg_type to PHOTO/TEXT after
the rich-text item scan had already promoted it (e.g. a native voice item
→ VOICE), dropping voice notes from the auto-STT path. Only re-derive when
the scan left the type at TEXT.
Ports the root-cause analysis from PR #38276 (stale, targeted the deleted
gateway/platforms/dingtalk.py) onto the live plugin adapter, with
regression tests.
Refs #38211#38219#38276
- Promote EXT_MAP to a module-level constant for reuse
- Classify DingTalk image messages and image file attachments as PHOTO
- Extract DingTalk card/interactiveCard document link content with defensive parsing
- Handle None, empty, JSON, and plain-string card content fields
- Add coverage: 8 card + 4 interactiveCard + 5 _extract_media = 17 test cases
- Remove a shadowed duplicate TestExtractText class (pytest collected only the later one)
3 fixes for the dingtalk-platform plugin (plugins/platforms/dingtalk/adapter.py):
1. _extract_text: parse ASR recognition text from voice messages via
extensions['content']['recognition'], fall back to fileName for
file messages ("[文件] xxx")
2. _extract_media: handle audio/file/image msgtype_str with correct
MIME mapping; exclude audio from media_urls to preserve DingTalk's
built-in ASR result (avoids failed whisper re-transcription)
3. _resolve_media_codes: parse downloadCode from extensions['content']
for file/image message types
All tested with live DingTalk group chat: voice → recognition text,
PDF/Markdown file → filename + download.
Lark's native "audio" msg_type is an in-app voice recording — uploaded
audio files arrive as "file"/"media". But _resolve_normalized_message_type
resolved the "audio" preferred type to MessageType.AUDIO, which the gateway
treats as a non-transcribed file attachment (run.py: AUDIO -> audio_file_paths,
"never STT"; VOICE -> audio_paths, "always STT"). Result: a Feishu voice
note reached the agent as an untranscribable audio attachment and was
silently ignored — the user's spoken message never became text.
Every other platform that receives native voice notes (Telegram, Discord,
Slack, WhatsApp, Signal, Matrix, WeChat, WeCom, DingTalk, QQ, BlueBubbles,
Mattermost, Yuanbao) classifies them as MessageType.VOICE. Feishu was the
only one classifying them as AUDIO. This is the follow-up to #28993, which
added native voice-note transcription for Discord + DingTalk but did not
cover Feishu.
Return MessageType.VOICE for the "audio" branch. The branch is reached only
for Lark's top-level audio msg_type (set in the normalizer; file uploads map
to "document"), so VOICE is unconditionally correct here — no risk of
auto-transcribing an uploaded music/audio file.
- plugins/platforms/feishu/adapter.py: _resolve_normalized_message_type audio
branch returns VOICE instead of resolving to AUDIO via mime.
- tests/gateway/test_feishu.py: test_extract_audio_message_downloads_and_caches
asserted the old AUDIO behavior on a fixture literally named voice.ogg —
updated to expect VOICE (the corrected classification).
- tests/gateway/test_feishu_voice_message_type.py: new focused regression
tests (audio->VOICE with and without mime; photo/document/text unaffected).
Rebased onto current main: the Feishu platform moved from the single-file
gateway/platforms/feishu.py into the plugins/platforms/feishu/ package; the
fix applies to the same _resolve_normalized_message_type logic at its new home.
Verified the new voice tests fail when the branch resolves to AUDIO and pass
with VOICE, while the photo/document/text cases are unaffected either way.
Note: classification is mock-tested here; the downstream STT pipeline is the
shared, already-proven path (#28993). End-to-end verification on a live
Feishu account would be a welcome confirmation.
Refs #28993 (sibling-gap: Feishu was the platform left uncovered).
Keep one owner for PATH/local-prefix ffmpeg discovery: ffmpeg_utils now
delegates to tools.transcription_tools._find_ffmpeg_binary and only adds
the Discord-specific FFMPEG_PATH override and Windows winget fallback on
top (follow-up to PR #60627 by @LauraGPT, fixes#60624).
Discord's voice socket needs a brief warm-up before receiving clients
actually hear audio; the first ~100-200ms is lost, clipping the first
word/syllable of TTS playback. Prepend a configurable lead of silence to
speech on both playback paths:
- Mixer path: new _lead_silence_bytes() helper prepends PCM silence
(BYTES_PER_MS constant added to voice_mixer.py) before play_speech, on
both the reply and the pre-tool ack.
- Legacy FFmpegPCMAudio path: apply -af adelay=<ms>:all=1.
Tunable via discord.voice_fx.lead_silence_ms (default 200, 0 disables).
Fixes#66827
VoiceMixer duck-typed the discord.AudioSource interface (is_opus, read,
cleanup) but never inherited from it. discord.py's vc.play() does an
isinstance check and rejects non-AudioSource objects, causing the voice
fx mixer to silently fail with:
"Voice mixer failed to start: source must be an AudioSource not
VoiceMixer"
Added missing `import discord` and changed the class definition from
`class VoiceMixer:` to `class VoiceMixer(discord.AudioSource):`.
- Wire adapter._voice_input_callback at connect and reconnect so voice
transcription is forwarded without requiring /voice join (#60623).
- Add optional text_channel_id and source params to DiscordAdapter
.join_voice_channel() so automatic/programmatic voice joins can
establish the text-channel binding needed by _handle_voice_channel_input.
- Add TestVoiceInputCallbackWiring: asserts callback wiring on startup
and reconnect for Discord adapters with voice attributes.
Closes the #32029 gap: TelegramAdapter.send_voice passed captions raw with
no parse_mode, so auto-TTS captions (which carry the agent's markdown
reply) showed literal *asterisks*, backticks and [links](...). Captions
are now formatted to MarkdownV2 via the adapter's format_message when the
formatted text fits Telegram's 1024-char caption cap, with fallback to the
plain truncated caption when formatting overflows or the Bot API rejects
the entities (mirrors the text-send markdown fallback ladder).
Fixes#32029
Salvaged from PR #53157 (@LLQWQ). Three changes for native Feishu voice
bubble delivery:
1. Include audio duration in the file-upload body when uploading opus
files — Feishu renders 0:00 bubbles without it. Duration is extracted
by parsing the OGG container's last granule position (pure Python,
no ffprobe dependency).
2. Thread routing fallback: Feishu's create-message API rejects
msg_type='audio' with receive_id_type='thread_id' (error 99992402);
retry via the reply API against the thread's last message, then fall
back to chat_id routing.
3. (dropped) tools/tts_tool.py want_opus hunk — superseded by main's
OPUS_VOICE_PLATFORMS, which already includes feishu. The PR's stray
scripts/release.py hunk was also dropped (frozen AUTHOR_MAP policy;
mapping added under contributors/emails/ instead).
Fixes#45557
Refs #18831#16524
Salvaged from PR #18861 (@shellybotmoyer). Switch the central Opus
transcoders from CBR 64k (-vbr off) to VBR 48k with -application voip and
-compression_level 10. The old CBR settings produced lower-quality speech
audio and occasionally failed to trigger Telegram's native voice-bubble
rendering. Applied to _ffmpeg_transcode_to_opus (the single transcoder
behind _convert_to_opus and _repair_ogg_container), the Gemini WAV→OGG
path, and the Matrix adapter boundary transcoder added in this branch.
Fixes#18818
Salvaged from PR #68063 (@malaiwah). MatrixAdapter.send_voice now transcodes
any non-Ogg audio to Ogg/Opus at the adapter boundary (best-effort — the
original file is sent unchanged when ffmpeg is unavailable), so MSC3245
voice bubbles render even when a caller hands the adapter MP3/WAV audio.
_matrix_voice_metadata_for_file probing now runs via asyncio.to_thread so
ffprobe/ffmpeg subprocess timeouts can't stall the adapter event loop.
The PR's tools/tts_tool.py want_opus hunk was dropped: main's
OPUS_VOICE_PLATFORMS set (PR #73072) already includes matrix; the Matrix
opus-routing test is kept.
Refs #14841
Salvaged from PR #68063 (base commit, runner-path hunks superseded by the
platform-aware OPUS_VOICE_PLATFORMS fix in this branch). Element and other
Matrix clients render voice bubbles more reliably when m.audio events carry
duration and waveform metadata; probe both best-effort via ffprobe/ffmpeg.
- Change await self._send_read_receipt to asyncio.create_task to avoid
blocking message dispatch on slow bridge responses (matches BlueBubbles
pattern). Up to 5s per-message latency eliminated.
- Update tests: assert_called_once_with instead of assert_awaited_once_with
since the receipt is now scheduled, not directly awaited.
- Add send_read_receipts documentation to whatsapp.md following the
BlueBubbles docs pattern.
The per-IP httpx transports were built once in __init__ and never torn
down. A connect that reached ESTABLISHED and was then closed by the peer
left its socket in CLOSE_WAIT inside the pool, and the failure path only
logged and continued — so the poisoned pool was retained and leaked one
descriptor per retry.
With DNS for api.telegram.org failing, every poll fell through to the
seed IP and leaked another fd every ~2.5s. The bot gateway reached 177
CLOSE_WAIT sockets against launchd's 256 soft limit and wedged: accept()
on the gateway port, config reads and DNS resolution all failed with
EMFILE, which in turn made the primary path fail and fed the loop.
Build fallback transports lazily and discard them on a retryable connect
failure, and bound every pool at 8 connections (httpx defaults to 100,
so two seed IPs plus primary could alone exceed the fd ceiling).
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
MEDIA_DELIVERY_EXTS in gateway/platforms/base.py omitted .3gp, causing
MEDIA: tags with .3gp files to leak as plain text instead of being
extracted for native video delivery. _VIDEO_EXTS in
tools/send_message_tool.py and _MIGRATION_VIDEO_EXTS in the Feishu
adapter omitted .webm, causing .webm files to be classified as
documents instead of video on Telegram and other platforms.
Both extensions are already present in every gateway-side _VIDEO_EXTS
definition (run.py, kanban_watchers.py, weixin.py, base.py local).
Closes#71621, Closes#71603
Surface a user-visible delivery notice when non-streaming media dispatch
gets success=False after MEDIA tags were stripped, and validate forum
starter-message attachments the same way as direct channel sends (#66797).
Outbound MEDIA video/document tags on the Discord non-streaming path
were extracted (stripped from the visible text) but never delivered:
no attachment, no error, and no dispatch log line (#66797). The
open-handle discord.File plus singular file= form could race Discord's
multipart encoder after an earlier image batch on the same channel and
return a successful message carrying zero attachments.
Fix _send_file_attachment (used by send_video/send_document/
send_image_file) to:
- use a path-based discord.File via the plural files=[...] kwarg, the
same pattern as the working send_multiple_images batch path;
- pre-flight os.path.isfile and return a File not found result instead
of raising deep inside discord.File;
- fail loud when Discord accepts the message but attaches nothing, so
the dispatch loop surfaces a warning rather than a silent drop;
- add INFO dispatch logs (base non-image MEDIA fan-out, video send, and
file attachment) so a MEDIA:.mp4 cannot vanish without a trace.
Tests (tests/gateway/test_discord_send.py):
- path-based files=[...] kwarg used, singular file= unset;
- fail-loud when the returned message has no attachments;
- missing file fails fast without resolving the channel;
- forum-parent delivery routes through create_thread with files=[...];
- end-to-end image+video response routes the mp4 to send_video while
images still batch.
Fixes#66797
The identity-refresh TTL used 0.0 as the "never checked" sentinel and
compared it against time.monotonic(). That epoch is arbitrary and starts
near zero on a freshly-booted host, so on CI runners and containers
`monotonic() - 0.0` was itself below the TTL — "never checked" read as
"checked just now" and the first identity refresh was suppressed for the
first 5 minutes of uptime. The stale-handle recovery therefore did nothing
on exactly the machines most likely to be freshly booted.
Invisible on a long-lived dev box (uptime >> TTL); caught by CI.
- Sentinel is now None, meaning never checked and always stale.
- Both TTL comparison sites route through _bot_identity_is_fresh().
- Regression test pins the invariant under a faked 12s-uptime clock.
Verified by re-running the recheck tests against a simulated 3s-uptime
host: green with the fix, and restoring the 0.0 sentinel reproduces the
exact CI failure locally.
Renaming a Telegram bot's @username in BotFather silently stopped the
gateway from answering in groups.
PTB caches getMe() in Bot._bot_user and only rewrites it inside get_me(),
so after a rename the adapter kept comparing mentions against the OLD
handle. The exclusive-mention gate then saw the new @handle, failed to
match itself, and concluded the message was addressed to a different bot
— dropping it before the reply and wake-word fallbacks could run. Native
replies to the bot were discarded too. Polling mode recovered on the next
90s heartbeat; webhook mode never calls get_me() again, so it stayed dead
until restart.
Separately, the bot-handle pattern assumed every bot username ends in
"bot". Collectible (Fragment) usernames can be assigned to bots and drop
that suffix (@jarvis, @pic), so such a bot could not recognise its own
handle in the entity-less fallback and was suppressed by any message that
also named another bot.
- Route every mention comparison through _current_bot_username(), which
prefers the last observed handle over PTB's cache.
- Learn the live handle from inbound updates: Telegram stamps the current
username on our own messages and on reply_to_message. Guarded by user
id, so another account's handle is never adopted.
- Re-check identity out of band (TTL-bounded, one getMe per 5 min) when
the exclusive gate is about to drop a message — the exact stale-handle
symptom — so the mistake self-corrects instead of persisting.
- Refresh identity in webhook mode via a dedicated low-frequency loop,
cancelled on the same teardown fence as the heartbeat.
- Match our own handle by identity rather than shape. Foreign handles keep
the deliberate "...bot" narrowing so human @handles still never act as
routing hints (the intent behind ce4d857021).
Validation: 11 regression tests; sabotage runs confirm each behavioral
test fails with the fix reverted. 157 tests green across the Telegram
gating, reconnect, and topic-mode suites.
Pair conditional approval buttons into rows of two so the full Allow Once /
Session / Always / Deny set stays readable instead of one truncated 4x1 row.
Follow-up hardening for the salvaged #69240 readiness gate (#67498):
- _start_polling_once now returns its (generation, progress_event) pair
so the strict cold-start gate binds to exactly the generation it
started, instead of re-reading self._polling_progress_event which a
concurrent recovery task may have replaced with a newer generation's
event (the G1/G2 race flagged in the #69240 review).
- Strict cold start no longer schedules background polling recovery: a
polling error during the readiness wait is captured by a strict
callback and fails the connect attempt immediately with a loud
OSError, so GatewayRunner disposes the partial adapter and retries
with a fresh one — no more waiting out the full readiness deadline on
a generation that already errored, and no G2-on-partial-app healing.
- After readiness is proven the strict callback delegates every later
polling error to the real background-recovery callback, preserving
the existing degraded/reconnect semantics for the polling lifetime.
- The readiness-timeout error message now states the deadline and that
the gateway will retry with a fresh adapter (loud failure, not a
silent wait).
- Regression tests: current-generation progress connects; a polling
error during strict cold start fails fast without scheduling
background recovery (the #67498 idle-threads shape); stale-generation
progress is rejected.
Progresses #67498
Use wall deadlines for deleteWebhook and start_polling, then fail cold startup unless getUpdates proves progress. This lets the gateway discard partial PTB state and retry with a fresh adapter.\n\nRefs #67498
Follow-up to the salvaged #38985: guard the 4 bare read_text/write_text
sites its allowlist missed (google_chat thread-count store + oauth JSON)
and add whatsapp/google_chat to the AST guard test's file list.
AST-driven pass over every subprocess.run/Popen/check_output/check_call/call
with text=True (or universal_newlines=True) and no explicit encoding=:
append encoding='utf-8', errors='replace' at the kwarg site. 136 call
sites across 28 files (cli.py, hermes_cli/main.py, tools_config.py,
environments, computer_use, gateway, scripts, skills helpers, agent/*).
Together with the salvaged #55339/#60741 commits this closes out issue
#53428's bug class; the salvaged #60751 linter rule in
check-windows-footguns.py now enforces it repo-wide (verified: 807 files
scanned, zero findings).
On Windows with Chinese locale (GBK), subprocess.run(text=True) without
explicit encoding causes UnicodeDecodeError crashes. This fix adds
encoding='utf-8', errors='replace' to all subprocess.run() and
subprocess.Popen() calls that use text=True across 76 non-test Python files.
Fixes#53428 (master tracker for Windows GBK locale crash).
Note: credential_pool.py and electron changes excluded per reviewer request —
those will be submitted as separate focused PRs.
The Telegram adapter's connect retry loop could silently stall after
'Connecting to Telegram (attempt 1/8)...' with the event loop permanently
parked in select() — all threads idle, no attempt 2/8 ever scheduled.
Root cause analysis:
- The retry loop reused the same Application object across all
8 attempts. After a failed initialize() the app could be in a partially-
initialized state (closed httpx transports from ,
or flag set before the hang) causing subsequent calls
to silently skip real initialization.
- CancelledError (a BaseException, not an Exception) propagated silently
through all except handlers with no logging — the task driving the retry
loop could exit without any trace.
- No total watchdog bound existed for the entire retry loop; only per-attempt
timeouts via _await_with_thread_deadline. If the loop itself stalled
between attempts (between-attempt sleep, cleanup, or scheduling), there
was no timeout to catch it.
Fixes:
1. **Total watchdog deadline**: Compute a total deadline for the entire
connect loop (8 attempts × init_timeout + 120s margin). Before each
attempt, check the wall clock; if exceeded, raise OSError immediately
instead of attempting another initialize().
2. **Fresh Application per retry**: On each failed attempt, rebuild
via and re-register all handlers. The old
app is best-effort shutdown with . This ensures
each retry starts with a clean slate — no stale transports, no stale
flag, no leaked state from the previous attempt.
3. **BaseException logging + propagation**: Added
(placed LAST after all other handlers) to log CancelledError and other
non-Exception signals before propagating. Previously these exited the
retry loop silently with no log message.
4. ** block for app rebuild**: The clause runs after
every failed attempt that isn't the last, rebuilding the app and
discarding the old one regardless of which exception class caused the
failure.
Two related Slack delivery fixes for send_message text sends:
- Route Slack text delivery through _send_via_adapter so the live
in-process gateway adapter (multi-workspace aware, channel→client
mapping, adapter-side gates) is preferred, with the plugin's
_standalone_send as the out-of-process fallback — matching how the
media path already behaves.
- _standalone_send: SLACK_BOT_TOKEN can be a comma-separated list in
multi-workspace installs and slack_tokens.json carries OAuth
per-workspace tokens; the standalone Web-API path used to send the
literal comma-joined string, which Slack rejects as invalid_auth.
Try each token individually, retrying on token-scoped errors
(invalid_auth / not_in_channel / channel_not_found …) and stopping on
terminal ones. User-DM resolution (U…/W… targets) also tries each
token.
Adapted from #47547 by @replygirl — the original patched the legacy
tools/send_message_tool.py::_send_slack helper, which moved to the
Slack plugin's _standalone_send in #41112.
Salvaged from #47547
Previously, Slack sessions and the channel directory stored raw channel
IDs (e.g. C0ATFHY907L) as chat_name, making it impossible for operators
to identify channels in send_message listings or session data.
Changes:
- Add _channel_name_cache and _resolve_channel_name() to SlackAdapter
that calls conversations.info (channels) or users.info (DMs) with
in-memory caching to avoid repeated API calls
- Use resolved names in _handle_slack_message() build_source() calls
- Use cached names in _seed_assistant_thread_session()
- Enrich session-sourced entries in channel_directory._build_slack()
by cross-referencing API results and falling back to conversations.info
+ users.info for DMs and private channels not in the bot's scope
Fixes: channel directory and session origins now show readable names
like 'general' or 'John Doe' instead of 'C0xxx' / 'D0xxx'.
Follow-up to the #51899 pick, folding in the config-bridge half of the
competing PR #46925 (@bhanusharma, earliest submitter for the
ignored-channel gate):
- _apply_yaml_config: translate config.yaml slack.ignored_channels into
SLACK_IGNORED_CHANNELS (list or CSV), env-var-wins like every other
bridged Slack key.
- SlackAdapter._slack_ignored_channels / gateway.run's
_slack_ignored_channels_from_gateway_config: fall back to the
SLACK_IGNORED_CHANNELS env var when PlatformConfig.extra carries no
value, so top-level slack: blocks (which flow through the env bridge,
not extra) are honored at both the adapter and runner gates.
- conftest: force-clear SLACK_ALLOWED_CHANNELS / SLACK_IGNORED_CHANNELS /
SLACK_DISABLE_DMS between tests (config-loader side-effect leak class).
- Tests: env-bridge translation + precedence in test_config.py, env
fallback + extra-wins in test_slack_runner_ignored_channels.py.
Credit: #46925 by @bhanusharma proposed the same gate with the YAML→env
bridge; #51899 (picked as the base) carries the wider outbound/runner
coverage. Closes#46925 as consolidated here with first-submitter credit.
Hermes already identifies itself on outbound calls to model providers
and monitoring endpoints — see hermes_cli/models.py and
plugins/model-providers/gmi/__init__.py
("Attribution so GMI can identify traffic from Hermes Agent"). The
Slack adapter is the one outbound surface that doesn't carry the
same identifier, so HermesAgent-driven Slack traffic is
indistinguishable from any other Bolt-Python app at the Slack
platform layer.
This sets `user_agent_prefix=f"HermesAgent/{_HERMES_VERSION}"` on the
AsyncWebClient instances constructed in gateway/platforms/slack.py
and threads them through AsyncApp via its `client` kwarg. Both
kwargs are first-class in slack-sdk and slack-bolt; no new
dependencies. Resulting header looks like:
HermesAgent/<version> Python/3.x slackclient/3.x ...
No behavioral change for users — the Slack API ignores User-Agent
semantically; it lands in logs and analytics. Reversible.
Tests in tests/gateway/test_slack.py:
- TestSlackUserAgent pins the prefix shape and runs connect()
end-to-end (multi-token config) to assert every AsyncWebClient
carries the prefix and AsyncApp receives the pre-built client.
- TestSlackProxyBehavior fakes updated to tolerate the new kwargs
via **_kwargs so they don't break on future passthroughs.
.get("key", {}) only applies the default when the key is ABSENT.
When the key exists with value None (null in JSON), .get() returns
None and the subsequent .get() raises AttributeError.
Fix: replace .get("key", {}).get(...) with (.get("key") or {}).get(...)
which handles both missing keys AND None values.
8 instances across 6 files:
- gateway/run.py: tool_call function name check
- acp_adapter/server.py: tool name/description extraction
- gateway/platforms/qqbot/onboard.py: API response task_id
- gateway/platforms/yuanbao.py: message content parsing (x2)
- gateway/platforms/slack.py: block text extraction (x2)
- tui_gateway/server.py: error message extraction
Blank (or whitespace-only) answers to the home-channel prompt in the
interactive setup wizards previously left any previously saved
*_HOME_CHANNEL / *_HOME_ROOM env value in place, so operators could not
clear a stale home channel by re-running setup. Strip the prompt input
and call remove_env_value() on blank answers across the Discord, Slack,
Feishu, Matrix, Mattermost, WeCom and WhatsApp plugin setup wizards,
with per-adapter wizard tests covering set/clear/whitespace flows.
Squash of the three commits from PR #58421 (setup-wizard fix, matrix/
wecom extension, and 6-adapter test coverage) — one commit per
contributor on this salvage branch.
Fixes#12423
Salvaged from #58421
The C13 log-noise test (merged while this branch was in flight) builds a
bare SlackAdapter without __init__; the new _channel_teams ambiguity map
crashed it. getattr-guard, same pattern as the sibling defensive inits.
Salvaged from #59742 (downloads half only). Main already resolves the
event-level team id from the channel→workspace cache before downloads
(the file-EVENT half was covered by #30456), but when neither the event
nor the cache knows the workspace, both download helpers silently fell
back to the PRIMARY workspace token — Slack then returns an HTML login
page instead of file bytes for any non-primary workspace file.
Slack private file URLs embed the owning workspace id
(files-pri/<TEAM_ID>-<FILE_ID>/...), so _resolve_download_token now
prefers: explicit team_id -> URL-embedded team id -> primary token.
Both _download_slack_file and _download_slack_file_bytes route through
it.
Deferred from #59742 (out of this correctness cluster's scope): the
channel→team disk persistence, the pre-send conversations.info probe
loop, and the thread-file ingestion feature.
Reapplied from #59742 by @benjamin2026-dot onto the current adapter
(original targeted the pre-#30456 download sites).
Widen #48476's fence guarantees to the two splitters that still emitted
fence-broken chunks:
* GatewayStreamConsumer._split_text_chunks (fallback final send): close
the orphaned ``` at each chunk boundary and reopen it — with the
original language tag — on the next chunk, mirroring
BasePlatformAdapter.truncate_message's contract. Headroom is reserved
so balanced chunks stay within the platform limit.
* Slack block_kit._split_text (3000-char section chunking): same
close/reopen balancing for mrkdwn section text carrying fences.
With these, every chunk boundary — non-streaming send
(truncate_message), streaming overflow (_truncate_for_stream via
adapter.truncate_message per #45938), fallback final
(_split_text_chunks), final-send balance (ensure_closed_code_fences),
and Block Kit section splits — delivers fence-balanced chunks.
Regression tests probe each path with fenced fixtures, assert per-chunk
balance, limit compliance, language-tag reopening, and prose passthrough.
Slack's Block Kit `markdown` block accepts standard markdown (tables,
headers, task lists, fenced code with syntax highlighting, links) and
lets Slack translate it natively — eliminating the lossy markdown→mrkdwn
conversion for the rendered layout. Enable via
platforms.slack.extra.markdown_blocks.
Safety rails added on top of the original design:
* opt-in (default off) — Slack documents the block for 'apps that use
platform AI features' and does not guarantee availability across all
app types / surfaces, so unconditional adoption is not safe yet
* the mrkdwn-converted text field is ALWAYS kept as the
notification/search/accessibility fallback
* content over Slack's 12k cumulative markdown-block cap declines to the
rich_blocks renderer / plain text path
* the existing block-rejection retry (invalid_blocks / msg_too_long /
too_many_blocks) re-sends the plain mrkdwn payload, so an unsupported
surface degrades gracefully instead of dropping the message
* when both modes are enabled, markdown_blocks is preferred over the
local rich_blocks renderer; rich_blocks remains the fallback
Adapted from #8554 by @shivasymbl — the original patched the deleted
gateway/platforms/slack.py and switched unconditionally; reimplemented
against the plugin adapter's _maybe_blocks/sanitize_blocks pipeline.
Fixes#8552.
Slack mrkdwn has no table syntax — GFM pipe tables render as literal-pipe
noise with a raw |---|---| separator row. Wrap detected tables in ```
fences so they render as monospace preformatted text, and pad cells to
per-column max display width (East-Asian Wide / Full-width chars counted
as 2 columns) so columns stay aligned even with CJK content.
Tables already inside fenced code blocks are left untouched, and the
emitted fences carry no language tag so they compose with the
lang-tag-strip pass. This covers the plain-mrkdwn text path; the opt-in
rich_blocks path already renders native Block Kit table blocks.
Reapplied from #16648 by @kylezh — the original patched
gateway/platforms/slack.py, which was migrated to
plugins/platforms/slack/adapter.py in the plugin migration.
Fixes the non-rich_blocks table path described in #8552.
Slack's mrkdwn parser can fail to recognize the closing * of a bold span
when it is immediately preceded by a non-word character (), ], }, ., :,
em-dash, ...), mis-rendering the span and in reported cases truncating
the rest of the message. Insert a zero-width space (U+200B) between the
last character and the closing * whenever the last character is not
alphanumeric or underscore.
Reapplied from #35144 by @gonzalofrancoceballos — the original patched
gateway/platforms/slack.py, which was migrated to
plugins/platforms/slack/adapter.py in the plugin migration.
Slack's mrkdwn does not strip the optional language tag from fenced
code blocks like GitHub-flavored markdown does — it renders
```text\nfoo\n``` as a code block whose literal first line is "text".
The agent emitted ```text fences around raw command output, which
surfaced "text" as the first line of every such block.
Drop the tag from the opening fence in format_message() before stashing
the block behind a placeholder. Stripping only fires for a genuine
opening fence — a ``` at the start of a line, tagged with a single
token (no spaces or backticks) — and the original line ending is
preserved. The fence-protection regex deliberately matches loosely, so
a mid-line ``` (e.g. an inline ```span``` wrapping across a newline)
can be grouped as an "opening fence" whose first line is real content;
differential fuzzing against the pre-change formatter (40k generated
messages) confirms the only behavioral delta is the tag strip itself.
The Block Kit renderer is unaffected: render_blocks() intercepts fences
itself before mrkdwn_fn is applied, so this only changes the mrkdwn
surfaces that still go through format_message() — the plain-text
fallback field (notifications, search indexing, accessibility),
slash-command ephemeral replies, and standalone cron delivery.
Originally written against gateway/platforms/slack.py; ported to
plugins/platforms/slack/adapter.py after the adapter migration in
5600105478.
Manually verified against a live Slack workspace (pre-migration
adapter; the ```text case strips identically) — code blocks no longer
carry a literal "text" first line.
format_message unescapes already-escaped input before re-escaping, so that
pre-escaped text doesn't get double-escaped. That unescape was three
sequential str.replace calls, which re-scan each other's output:
"&lt;" --(& -> &)--> "<" --(< -> <)--> "<"
The & produced by the first replace pairs with the following "lt;" and
decodes a second time. "&lt;" is the wire form of the literal text
"<", so the text is silently destroyed: Slack receives "<" and renders
"<". Anyone writing about HTML or markup ("&lt;b&gt;" -> "<b>")
loses their literal text, with no error.
re.sub scans left-to-right and never re-scans its own replacements, so a
single pass fixes it. The escape pass on the next line is left untouched --
it is correctly ordered (& first, so the &s it inserts aren't re-escaped).
Only the double-decode cases change; every other input is byte-identical
before and after. This is the same round-trip invariant the neighbouring
test_pre_escaped_{ampersand,lt,gt}_not_double_escaped tests already assert,
extended to the case they miss. Affects the plain mrkdwn path (send,
edit_message) and Block Kit sections, which route section text through
format_message.
Build the full reaction pipeline on top of the #29916 base:
- Opt-in gate: slack.reaction_triggers (default OFF — reaction events
stay acked-and-dropped so busy channels don't wake the agent on every
emoji). 'true' routes reactions on the bot's OWN messages; an explicit
emoji-name list routes those emojis from any message (handoff flows).
- reaction_removed events now route too, distinguished by the
cross-platform text convention reaction:added:<emoji> /
reaction:removed:<emoji> (matches the Feishu and Photon adapters, so
agents and skills see one shape everywhere).
- Authorization: the reactor becomes the synthesized message's user, so
the early _is_user_authorized gate and allowed_channels whitelist
apply exactly as for typed messages. _hermes_force_process only skips
the mention requirement (a reaction on the bot's own message is
definitionally addressed to the bot), mirroring Feishu/Photon.
- Gateway hooks (#33111 by @johnkattenhorn): every human reaction on a
message item fires reaction:added / reaction:removed through the new
BasePlatformAdapter.set_reaction_handler → GatewayRunner
._handle_reaction_event → HookRegistry.emit, independent of the
routing opt-in. Documented in hooks.md.
- Channel handoff (#45265 by @Kev-fs): slack.reaction_trigger_target
routes the reaction turn to a configured channel (top-level via
_hermes_no_thread_response + reply-anchor suppression in
gateway/platforms/base.py) or C123:<ts> thread.
- Manifest: reaction_removed event subscription added alongside
reaction_added/reactions:read.
- Docs: slack.md Reaction Triggers section; hooks.md event table rows.
Also credits #44508 by @harrisonmedmedmetrics (inbound reaction_added
handling — same plumbing class, superseded by this consolidated shape).
Co-authored-by: johnkattenhorn <john.kattenhorn.personal@gmail.com>
Co-authored-by: Kev-fs <kevin@fleetsmarts.net>
Co-authored-by: harrisonmedmedmetrics <harrison@medmetricsrx.com>
Slack reaction_added events were explicitly acked and dropped, so a user
reacting to a bot message (👍 to approve, ✅ to acknowledge) produced
nothing. Forward them through the normal message pipeline as synthesized
MessageEvents whose text is the reaction emoji (translated to unicode
for common names), keeping the downstream auth gate, thread-context
fetch, dedup, and skill routing unchanged.
- Self-reactions and non-message items are dropped; reactions on
messages not sent by this bot are dropped (Feishu-adapter parity).
- The reacted-to message's thread parent becomes the synthesized
thread_ts so the reaction lands in the same session as a reply would.
- Manifest gains reactions:read scope + reaction_added bot event.
Salvaged from PR #29916 by @bpross.
Related: #33111, #44508, #45265 (same cluster).
Salvaged from PR #45702 (heartbeat half). A multi-minute turn showed a
static 'is thinking...' assistant status that reads as stuck and provokes
mid-turn 'you there?' pings. Derive the fallback status label from the
turn's elapsed time (>=30s → 'still working… (NmSSs)'), riding the
existing _keep_typing refresh — zero extra API calls.
Ported onto the current plugin adapter: the start time rides the tracked
_active_status_threads entry (workspace-scoped key), so it shares the
existing bounds/eviction and resets when stop_typing clears the status.
Explicit live-status phrases (set_status_text) and configured
typing_status_text always win; only the built-in default label changes.
The PR's other half (top-level channel follow-up coalescing) is NOT
included — dispatch semantics changed on main (busy-input active-turn
redirect, #30170 demotion) and need a fresh design pass.
Refs #45702. Co-authored-by: MrAbsaroka <mrabsaroka@gmail.com>
Follow-up hardening on top of the C14 cherry-picks (#57860/#44026/#66742/#60009):
- Slack file downloads (_download_slack_file/_download_slack_file_bytes)
now require an https URL on a Slack CDN host (files.slack.com,
*.slack.com Enterprise Grid, *.slack-files.com legacy shares) before
attaching the bot token. url_private/url_private_download only ever
point at the Slack CDN, so a forged file object from a malicious
workspace app or compromised event stream pointing the Bearer-token
download at an arbitrary PUBLIC host (token exfiltration) is now
refused — a hole #44026's generic private-IP SSRF check alone could
not close.
- The same two download paths now use create_ssrf_safe_async_client
(from #57860) so the preflight-validated hostname is resolved once,
validated, and dialed by IP — closing the DNS-rebinding TOCTOU window
for the token-bearing inbound fetches as well.
- #60009's slack_tokens.json permission warning is generalized into
utils.warn_if_credential_file_broadly_readable() (POSIX-only,
fail-quiet) and wired into the other read path with the same gap:
google_chat's load_user_credentials(). google_chat already writes
0o600 via _write_private_json; the read-time warning covers
hand-provisioned/legacy files. Nothing in-repo writes
slack_tokens.json (user/OAuth-provisioned), so there is no write
path to chmod for Slack.
Security tests both directions: non-CDN/lookalike/http URLs and
connect-time DNS rebinds are blocked before any TCP connect; real
files.slack.com, Enterprise Grid, and slack-files.com URLs still reach
the network layer; 0o600 files stay silent while 0o644/0o640 warn with
a chmod hint. A/B: all 10 new download-guard tests fail with the
hardening reverted and pass with it applied.
The OAuth multi-workspace token file contains plaintext bot tokens for
all saved Slack workspaces. Unlike the Google Chat adapter which sets
0o600 when writing credentials, the Slack token file has no permission
enforcement — a default umask 022 makes it world-readable.
Fix: check file permissions on read and emit a warning log with remediation
instructions if the file is group- or world-readable.
`SlackAdapter._fetch_thread_context` formats each prior thread message as
`{name}: {msg_text}` and joins them with newlines into the block the call
site prepends *raw* into the model turn (`text = thread_context + text`).
Both fields are attacker-influenceable — any thread participant sets their
own Slack display name and message text — and neither was neutralized, so
an embedded newline let a thread message break out of its line and pose as
a fresh markdown section (a fake "## SYSTEM" / "## Override" heading) inside
the context the model reads when first mentioned mid-thread.
This is the same indirect-prompt-injection vector already closed for the
sibling untrusted sinks: the sender-name prefix
(`neutralize_untrusted_inline_text`), the reply quote, and the relay
channel-context renderer. The Slack thread-context backfill — the default
whenever the bot is mentioned in a thread with no active session — was the
missed sink. (The existing `[unverified]` tagging marks *who* a message is
from; it does nothing about newline structure, so an authorized sender can
inject just as easily.)
Flatten both fields with `neutralize_untrusted_inline_text` before
interpolation. The body uses `max_chars=0` so message text is not truncated
(thread context caps the message *count*, never per-message length); the
display name keeps the default bound. `parent_text` keeps the raw message
(its own reply-context sink neutralizes separately). A well-behaved message
is preserved byte-for-byte.
Adds a regression test covering a hostile display name, a hostile message
body, benign passthrough, and the no-truncation guarantee.
_download_slack_file and _download_slack_file_bytes fetched Slack-supplied URLs with the bot token attached and follow_redirects=True, but without the is_safe_url pre-flight check or per-redirect guard that the outbound send_image path already uses. A URL that resolves to (or 3xx-redirects into) a private/internal address could reach internal services and leak the bot token (CWE-918). Add the same pre-flight + _ssrf_redirect_guard hook to both inbound sibling paths.
Several platform fetch paths called is_safe_url before constructing ordinary httpx clients, leaving a second DNS lookup at connection time. This preserved the rebinding window for Slack batch images, Feishu documents, Telegram URL-photo fallback, and WeCom remote media.
Route each path through create_ssrf_safe_async_client and the shared redirect guard so direct connections validate and dial vetted IPs while configured proxies remain an explicit trusted egress boundary. Add per-path regressions that change DNS from public at preflight to metadata at connect time.
The Skills Hub provenance fixture intentionally serves content over loopback. Opt that test-scoped server into private-address access so it keeps exercising the real HTTP transport without weakening production blocking.
Related #8033
Co-authored-by: teknium1 <127238744+teknium1@users.noreply.github.com>
Install connect-time DNS validation for Hermes-owned direct httpx clients so SSRF-sensitive fetch paths dial a vetted IP instead of re-resolving after preflight. This preserves Host/SNI semantics for direct HTTP(S) connections and keeps proxy routing as an explicit trusted egress boundary.
Wire the guarded clients into media cache downloads, vision downloads, Skills Hub direct/raw fetches, and platform attachment fetch paths that already perform SSRF preflight and redirect validation.
Fixes#8033
Co-authored-by: Tom Qiao <zqiao@microsoft.com>
Widening for #24117 ('is thinking...' stuck after the response was
sent): the stuck-thinking class is a missing-cleanup-on-error-path bug.
send()'s status clear was gated on thread_ts already being resolved and
on reaching the normal post-message path, which left the Slack
Assistant status visible when a turn ended through any sibling exit:
- exception BEFORE _resolve_thread_ts (slash-context handling,
formatting, DM resolution) — the 'if thread_ts: stop_typing' clear
never ran
- empty/whitespace-only final response (no_text guard early-return)
- ephemeral slash replies (Slack only auto-clears assistant status on
real thread replies; ephemerals never count)
Add _clear_thread_status_quietly() — a best-effort stop_typing wrapper
that never masks the caller's SendResult — and wire it at every send()
exit plus the finalize paths of edit_message. stop_typing already
handles the untracked-thread fallback (clearing an unset status is a
no-op on Slack's side), so this is pure coverage widening.
Also add the #18859 unit tests for _resolve_progress_thread_id's new
reply_in_thread gate (synthetic-thread drop, real-thread keep, event-id
fallback suppression, default unchanged).
A/B: 3 of the 4 new status-clear tests fail with the adapter widening
reverted and pass with it applied; the fourth pins the existing
cleanup-must-not-mask-result contract.
When reply_in_thread=false, top-level channel events carry their own
message ts as metadata.thread_id for session keying. Calling
assistant.threads.setStatus on that ts activated a Slack assistant
thread ('is thinking...') before the actual response was sent, and the
flat reply then never cleared it.
send_typing now routes through the same _resolve_thread_ts synthetic-
thread guard as message sending, and the gateway threads message_id
through progress/status metadata so the adapter can distinguish real
threads from synthetic top-level session keys.
Reapplied from #18859-sibling PR #17184 by @dorukardahan (both commits:
fix + progress-metadata test) onto current main via 3-way apply — the
original patched gateway/platforms/slack.py, moved to
plugins/platforms/slack/adapter.py in the plugin migration.
Progress/status callbacks (context-pressure, compression retries,
model fallback) route through _send_or_update_status_coro, which
edits the previous bubble for the same status_key when the adapter
implements send_or_update_status — but only Telegram did. On Slack
every status event posted a fresh thread message, so a compression
retry loop spammed a dozen out-of-order bubbles into the thread
('Context too large 1/3... 2/3... 3/3', fallback switches, etc.).
Implement send_or_update_status on the Slack adapter following the
Telegram pattern (#30045): first call posts and caches the message ts
per (channel, thread, status_key); subsequent calls edit that message
via chat.update. Edit failure drops the cached ts and falls back to a
fresh send. Cache is FIFO-bounded.
When the bot is mentioned mid-thread for the first time, the thread root
is very often the artifact the mention is about ("@bot what's in this
chart?" posted as a reply under an image) — but the root's image never
reached the agent, so it answered blind.
On the cold-start hydrate path (and only there), _collect_thread_root_images
reads the root message from the thread-context cache the immediately
preceding _fetch_thread_context call just populated (zero extra Slack API
calls in the normal case), downloads its image/* attachments through the
existing authenticated _download_slack_file helper, and delivers them as
media_urls/media_types on the same MessageEvent — upgrading the message
type to PHOTO so vision routing engages.
Scope and safety:
- One-time delivery by construction: the cold-start path is guarded by
_has_active_session_for_thread, so later turns in the same session can
never re-download or re-deliver. No new gateway/session plumbing needed.
- Bounded by _THREAD_ROOT_IMAGE_MAX (4); non-image root attachments stay
text-only markers.
- Slack Connect stubs (file_access=check_file_info) resolve via files.info.
- Best-effort: a failed download degrades to the [image: ...] marker from
the thread context — never an error turn.
- Also hardens the video mimetype fallback (mimetype can be empty) so
media_types entries are always non-None strings.
Adapted from #69185 by @KCAYAAI — the original plumbed MessageEvent media
through gateway/base.py, run.py and session.py with durable one-time
delivery markers (2,441 lines); this lands the user-visible behavior
adapter-locally by reusing the session guard already on the hydrate path.
Images and files posted in a Slack thread before the bot joins were
invisible to the agent: _fetch_thread_context renders text only, and a
caption-less image post was dropped from context entirely (empty text →
skip). "@bot what do you think of the chart above?" read as a question
about nothing.
_render_message_text now appends a compact, sanitized marker per file
attachment — [image: chart.png], [video: demo.mp4], [audio: note.m4a],
[file: report.pdf (application/pdf)] — so the agent can SEE that prior
thread messages carried attachments and ask for a re-share when it needs
the bytes. Filenames are stripped of newlines/brackets so a hostile name
can't fake context structure. Because both thread-context formatting and
parent-text rendering go through _render_message_text, markers appear on
the cold-start hydrate, the explicit-mention delta refresh, restart
rehydration, and reply_to_text.
Reapplied from #32315 onto the current adapter (original patched the
pre-plugin gateway/platforms/slack.py, moved in the plugin migration;
annotation labels reworked to per-file typed markers, download side
handled separately).
Follow-up hardening for the C13 log-noise cluster:
- plugins/platforms/slack/adapter.py: clarify button resolution logged
the full chosen option text at INFO (choice=%r). Choice text is user
content — log the choice INDEX at INFO and the (truncated, %.100r)
text at DEBUG only. Widens #58478's principle: no message content
above DEBUG level anywhere in the adapter.
- tests/gateway/test_slack_log_noise.py (new): behavioral suite pinning
the cluster's invariants:
* catch-all ack registered AFTER every named handler (registration
order is bolt's dispatch priority — no shadowing);
* catch-all fires for an unsubscribed event type (reaction_added),
logs only a DEBUG line naming the type, and never logs content;
* named handlers still dispatch (message → _handle_slack_message);
* end-to-end inbound message run leaves NO message text or block
content in any adapter log record (caplog at DEBUG);
* #30185's event-arrival diagnostic is metadata-only;
* clarify resolution: INFO carries index/user only, text is
DEBUG-only (fails with the adapter fix reverted — A/B verified).
Content-leak audit of all 139 logger call sites in the adapter found
two above-DEBUG leaks: the clarify choice line (fixed here) and none
else carrying message text; remaining sites log error strings, URLs
via safe_url_for_log, ids, and counts. The block-extraction DEBUG
preview was already removed by #58478 (chars= length only).
Without a catch-all handler, slack-bolt returns HTTP 404 for every
unhandled bot event (user_change, user_huddle_changed, reaction_added,
etc.) and never sends the Socket Mode ack. On active Slack workspaces
where the app is subscribed to high-volume events, this produces a
near-100% un-acked failure rate that crosses Slack's >95%/60-min
threshold and triggers automatic disabling of the app's Event
Subscriptions — silently killing all inbound event delivery.
Place a catch-all re.compile(r".*") handler AFTER the specific event
handlers so bolt's router matches those first. Truly unhandled events
are silently acked (200) and logged at DEBUG. The failure rate stays
near 0% regardless of which events the Slack app manifest subscribes to.
Fixes#6572
On vulnerable SQLite (e.g. 3.50.4), do not enable WAL for fresh/non-WAL
shared databases — prefer DELETE instead. Leave existing on-disk WAL
alone (no live downgrade under concurrent gateway/cron openers). Surface
Python/SQLite version details as a doctor warning (#69784).
Post-rebase consolidation over the merged C7/C16/C10 work:
- _ensure_dm_conversation now records workspace ownership via
_remember_channel_team and bounds _dm_conversation_cache (cap 5000)
- module-level _slack_dm_cache (C7 standalone path) bounded oldest-first
- _user_is_bot_cache (C10) bounded with _trim_oldest_dict_entries
- caption-mode contract tests updated for the C7+C8 merged media path
Widen the #19237 send_message fix to the live adapter: send, _upload_file,
send_multiple_images, send_image, send_video, send_document,
send_exec_approval, send_slash_confirm, and send_clarify now route bare
Slack user IDs (U.../W...) through a shared _ensure_dm_conversation helper
before calling chat.postMessage / files_upload_v2, which reject user IDs.
This closes the gap in #17261 where an attachment worked when replying in
a thread but failed when directed at a user DM, and extends the DM-open
fallback to clarify/approval Block Kit prompts so gated actions can reach
a user directly.
Resolution uses the workspace-scoped client (multi-workspace installs open
the DM with the right bot token), caches per (team, user), and records the
opened D... channel in the channel→team map. On failure the original
target passes through so the downstream API call surfaces the real Slack
error.
Fixes#17261
Refs #19236
Add early auth check in _handle_slack_message() that runs BEFORE any
API calls (thread context fetch, user name resolution, file downloads)
or file processing. Unauthorized users could previously trigger Slack
API calls and file downloads before the runner's _is_user_authorized
gate rejected them.
Same pattern as Telegram fix#54164: build a SessionSource and check
the runner's _is_user_authorized at the adapter level before event
construction consumes resources.
Fixes the gap where Slack has no adapter-level auth gate while Discord
and Telegram have adapter-level allowlists.
Slack could already deliver files in-channel via the gateway, but
send_message omitted MEDIA for Slack and told the model it was
unsupported — causing agents to inconsistently refuse PDF sends.
Wire Slack through files_upload_v2 in the standalone sender.
Port of the Slack half of #13855 (by @kshitijk4poor), reimplemented against
the plugin adapter (the original PR targets the deleted
gateway/platforms/slack.py and six other legacy adapters).
Channels listed in slack.require_mention_channels (config.yaml) or
SLACK_REQUIRE_MENTION_CHANNELS ALWAYS require an explicit @mention, even
when require_mention is false globally or the channel is in
free_response_channels — the opposite direction of free_response_channels.
Instead of duplicating the PR's inline reply-to-bot-thread/mentioned-thread/
session checks, the forced channel falls through to the SAME decision chain
as normal mention gating, so all five wake checks in
_should_wake_on_unmentioned_message keep applying (single decision path).
Credit: adapted from #13855 by @kshitijk4poor (Slack half only; the other
platform halves target deleted legacy adapters and are out of scope for
this cluster).
The ignore_other_user_mentions gate relied solely on is_mentioned, which
only recognises exact <@UID> markup, so a message mentioning the bot in
pipe form (<@UID|name>) alongside a leading other-user mention was
wrongly suppressed. The gate now also scans for the bot's own mention in
either markup before ignoring a message.
Claude-Session: https://claude.ai/code/session_01TKsNdptNdo9CqT2u7JMdkH
rob-maron
Jasmine Naderi
Teknium
Gille
Shannon Sands
峯岸 亮
vaibhavjnf
Nick K
huntsyea
Hermes Agent
SeoYeonKim
Frowtek
ygd58
Spark (DGX)
yoma
ScaleLeanChris
DI404N
joaomarcos
liuhao1024
webtecnica
pierrenode
solyanviktor-star
MattMaximo
kelsia14
Rob Zehner
Vivaan Dhawan
Florian Valade
Kronexoi
raywu
wuli666
LauraGPT
Jeffrey Cox
Ariel Tov Ben
luyifan
PRATHAMESH75
Carlos Diosdado
isheng-eqi
Hu
shellybotmoyer
Michel Belleau
kshitijk4poor
Cyrus
alexneyman
andy
Piyush Jagadish Bag
HexLab98
mannnrachman
Rod Boev
Stoltemberg
Hermes Agent
replygirl
dirtyren
Michael Brooks
annguyenNous
David Metcalfe
vexclawx31
byshubham
benjamin2026-dot
Jordan Hubbard
shivasymbl
Kyle Zhang
mzkarami
Gonzalo Franco Ceballos
z23
briandevans
Benjamin Ross
MrAbsaroka
AlexFucuson9
zapabob
Eugeniusz Gilewski
Doruk Ardahan
Jerome Iveson
2001Y
Minseo-Choi
Yemi
haran2001
dsad
Rob Zolkos
YLChen-007
saitama
Ben Phillips