Commit Graph

2110 Commits

Author SHA1 Message Date
ethernet 5aa121ecfd refactor(deps): read the lazy-install specs from the pyproject extras
tools/lazy_deps.py held a table of about 40 features, each with its own
literal pip specs. pyproject.toml declares the same packages as extras,
so every pin existed twice and the two copies drifted.

Each feature now names an extra, and the specs come from pyproject at
run time. The table is 218 lines shorter. A test asserts that each
feature names an extra that exists and resolves to at least one spec, so
a typo cannot ship.

A wheel install, such as Nix, has no pyproject.toml beside the code.
There the same table comes from the dist metadata: each spec of an
extra is one Requires-Dist line, and its marker names the extra.
Without this fallback, each entry point raised on a Nix install, and
ensure() raised even for a feature whose packages the build baked in
through extraDependencyGroups. That call must be a no-op.
is_available() and feature_install_command() catch the failure as well
now. Their callers sit in status paths with no try/except, and their
contracts are bool and Optional[str].

The security overrides already come from pyproject (the previous
commit). This commit moves the reader onto the shared _pyproject()
cache and the shared temp-file writer.

The tier-0 installer, `uv sync --extra <name>`, names the project with
--project. uv reads the project from its working directory, and the
agent runs from the user's working directory, not from the install
tree. Without the flag the sync failed outside a checkout, and the pip
ladder always ran instead.

install_specs gets the same managed-install guard as ensure(). A Nix
venv is in the read-only store, so the pip ladder could only fail with
EROFS after a 15s ensurepip attempt. It reports the Nix remedy instead.
A durable install target overrides the guard, as it does in ensure(),
because the NixOS container module sets HERMES_MANAGED=true with a
writable target.

Spec parsing goes to packaging.requirements.Requirement, which is
already a core dependency. The hand-written version kept the
environment marker attached to the version. SpecifierSet raised on it,
so _is_satisfied answered True for every installed version of a marked
package. Such a package can never upgrade.

Reading the specs from an extra exposed a second fault, in the record of
which features are active. active_features read specs[0] as the anchor
package, and extra composition put sounddevice there for [voice] and for
each wake extra. One local STT install then marked every audio feature
active, and `hermes update` installed the wake engines that the user
never enabled.

ensure() records each feature it serves in
$HERMES_HOME/lazy-features.json, and active_features reads that record.
A recorded feature still needs its anchor package installed, so an
uninstalled backend does not come back. The anchor is the first pin
written directly in the extra, not the first spec after expansion. A
test asserts that no two extras share an anchor.

There is no seeding for an install that predates the record. Its first
`hermes update` refreshes nothing. ensure() then repairs a stale pin at
each backend's start and records the feature, and the next update covers
it.

[stt-whisper] splits out of [voice]. faster-whisper transcribes audio
files and needs no microphone and no PortAudio, so the Docker image can
bake it. [voice] composes [stt-whisper] and [audio-io] and stays the
microphone stack. stt.faster_whisper maps to the new extra.

Removed with the table:

- The literal pin list in plugins/platforms/google_chat/oauth.py. Its
  pip path targeted /nix/store on a Nix install, which is read-only.
- The bare honcho-ai fallback in the honcho setup. An unpinned install
  accepts whatever PyPI serves, which is the hole this branch closes.
  Both call sites report the remedy for the deployment instead, through
  the now-public managed_install_reason.
- install_deps() in the google-workspace skill. The SDKs ship in the
  [google] extra, so a stripped environment is a broken install. The
  repair is `hermes update`. A pip run from the script writes to
  whichever interpreter it runs under, which is not always the one
  Hermes uses.
- tests/test_runtime_pins_are_locked.py, which scanned first-party
  source for pin literals. There are none left to find.
- The spec shape check in install_specs. The same plugin.yaml hands
  external_dependencies[].install to bash with shell=True, and the
  plugin's __init__.py is imported. Anyone who can write that file
  already runs code as the user.
2026-08-09 17:33:14 -04:00
ethernet c195b0988c fix(deps): hold the [tool.uv] overrides on the lazy-install path
`uv pip install` and `pip install` do not read [tool.uv]
override-dependencies from pyproject.toml. A backend whose transitive
deps cap a security-pinned package below its patched floor therefore
downgrades the core venv the first time that backend is enabled.

The measured case: the core venv ships cryptography 50.0.0. The first
DingTalk install pulls alibabacloud-tea-openapi 0.4.5, which caps
cryptography<49, and the resolver moves cryptography back to 48.0.1 —
with its three advisories. Pinning the floor next to the specs is not
a fix: the resolver satisfies it by walking tea-openapi back to
0.3.16, a two-year-old sdist build, and pinning both is unsatisfiable.

tools/lazy_deps.py now reads override-dependencies from pyproject.toml
and hands the list to both installer tiers: uv gets it as --overrides,
pip gets it as --constraint. pyproject.toml is the one source of
truth, so there is no second list to keep in sync. Lazy installs only
run from a source checkout — the one wheel-shaped install, Nix, seals
its venv and cannot lazy-install — so the file is always on disk.

This also covers the pynacl override: a lazy discord.py install caps
pynacl below the patched 1.6 floor, and would move the core venv back
to 1.5.0.

New tests hold the contract: the reader returns the pyproject list
verbatim, and both installer tiers receive it.
2026-08-09 17:33:14 -04:00
kshitij 326bdfb7a2 refactor: clean up gateway scope identity predicate and tests
- Remove dead use_systemd_scope = False assignment (leftover from
  the old try/except pattern, immediately overwritten).
- Update stale log label supervisor= -> in_supervised_gateway=
  to match the renamed variable.
- Convert autouse _mark_gateway_process fixture to opt-in
  _gateway_identity so negative tests start from a clean slate
  instead of undoing the fixture's env/PID mocks.
- Parametrize 4 near-duplicate negative tests (2 scenarios x
  pipe/PTY) into 2 parametrized tests, reducing ~130 lines to ~80.

76 tests pass, ruff clean, net -32 LOC.
2026-08-09 21:43:32 +05:30
bgrablin aa32e81141 fix(process-registry): bind gateway scope identity to pid 2026-08-09 21:43:32 +05:30
bgrablin ff5dfdecef fix(process-registry): keep CLI workers off controlling tty 2026-08-09 21:43:32 +05:30
kshitij f9f4fb4327 test: update systemd scope assertions for start_new_session=True
The #70716 regression fix changes popen_start_new_session from False to
True in the systemd-scope branch.  Update the assertion in
test_wraps_in_systemd_scope_when_supervisor_and_available and the
docstring in test_systemd_post_spawn_failure_never_kills_gateway_process_group.
2026-08-09 14:13:02 +05:30
Teknium 471baea520 feat(plugins): map portable Agent Plugins streamable-http entries into the native MCP runtime
Agent Plugins v1 packages with 'streamable-http' mcp.json entries now load
through Hermes' existing URL-based MCP client instead of being reported and
skipped. The stdio-only limitation was the agreed follow-up slice from
PR #81196.

Boundary rules from the v1 spec (§7.2.1) are enforced:
- URL must be absolute http(s), no user information, no fragment; plain
  HTTP only for localhost/loopback hosts.
- Configured package headers are never forwarded across a cross-origin
  redirect: translation marks entries strict_redirect_headers, and the
  redirect hook in the native runtime strips those headers (plus
  Authorization) whenever a redirect leaves the original origin. On mcp <
  1.24.0, where the client cannot hook redirects, such servers fail closed
  with an actionable upgrade message.
- Legacy 'sse' entries remain reported and skipped.

The redirect hook is extracted into a testable module-level factory
(_make_redirect_header_stripper); default behavior for native config
servers is unchanged (Authorization-only stripping).
2026-08-08 23:56:46 -07:00
rob-maron 7065407411
Add more FAL models to nous portal (#82019)
* add more FAL models to nous portal

* fix test

* minor fixes

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-08 19:59:12 -04:00
Teknium 66ea4e686d feat(media): default-on upscaling for sub-2MP image models (FAL + Krea)
Per review: upscaling should be the default behavior (like the original
flux-2-pro chain), not agent opt-in. Policy: every image model whose
native output is below ~2MP now sets upscale=True in its catalog —
users never silently get low-res images. Native hi-res models
(Seedream 5 Pro/Lite, Krea 2 Large) stay off to avoid paying to
upscale already-large output.

- FAL catalog: 16 models flipped to upscale=True (klein, z-image,
  nano-banana pro/2/2-lite, gpt-image 1.5/2, ideogram v3/v4, recraft
  v4/v4.1, qwen image/3, krea-2 medium on FAL, MAI 2.5 pro).
- Krea plugin: per-model upscale defaults (medium + medium-turbo ON at
  1.5K native; large OFF at 2K native), precedence explicit kwarg >
  image_gen.krea.upscale config > catalog default.
- The 'upscale' tool param remains as a per-call override in both
  directions (false = fast draft, true = force on hi-res/edits).
- Video unchanged: opt-in only (default-on would double every video's
  cost and latency).
- Sibling tests updated: routing/payload tests pass upscale=False where
  the assertion targets the generation submit; catalog test now pins
  the native-resolution policy instead of the flux-2-pro snapshot.
2026-08-08 14:49:28 -07:00
Teknium 137960c9aa feat(media): opt-in upscale pass for image_generate and video_generate across FAL and Krea
The generated-media surface previously had almost no upscaler coverage:
only fal-ai/flux-2-pro chained Clarity Upscaler (hardcoded catalog
default), every other image model returned ~1MP output with no high-res
path, and video had no upscaler at all. Krea's API treats the enhancer
as a standard second pass; this brings the same shape to Hermes.

- image_generate: new optional 'upscale' boolean in the tool schema.
  Explicit true chains the backend upscaler on ANY model (including
  edits); explicit false disables flux-2-pro's automatic default;
  omitted keeps per-model catalog behavior. Response now reports
  'upscaled' so the agent knows which resolution it got.
- FAL image path: explicit flag overrides the catalog 'upscale' default
  (Clarity Upscaler, 2x). Failure falls back to the native image.
- Krea plugin: upscale=true chains Krea Enhance
  (/generate/enhance/krea/enhance, 2x, prompt-guided) through the same
  BYO/managed base URL + auth as generation, with a best-effort poll
  loop that never fails a successful generation.
- video_generate: new optional 'upscale' boolean; FAL video plugin
  chains ByteDance SeedVR2 (fal-ai/seedvr/upscale/video, 2x factor
  mode). Providers without upscalers ignore the kwarg per the ABC
  contract (documented in both ABCs).

Validation: targeted suites green (123 tests across 6 files, including
new coverage for override-wins/default-kept/failure-fallback on all
three paths); live E2E on direct FAL verified both chains end-to-end
(klein 9b + Clarity upscaled image; pixverse-v6 1s 360p + SeedVR2
upscaled video).
2026-08-08 14:49:28 -07:00
Sora-bluesky 26eeb8568e fix(tools): decode git output as UTF-8 in working_diff on Windows
_run() used text=True without an encoding, so Windows decoded git's
UTF-8 output with the locale code page (cp932) and raised
UnicodeDecodeError on non-ASCII filenames or diff content, breaking
the "Never raises on git failure" contract in its docstring. Match
the utf-8 + errors="replace" policy checkpoint_manager's _run_git
already uses. Legacy cp932-encoded blob content degrades to
replacement characters instead of crashing; a test pins that
trade-off so it stays a documented choice.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-08 12:34:46 -07:00
Teknium 93be7f0117 test(file-ops): end-to-end regression suite for the UTF-8-flagged-as-binary class
Real-backend coverage for the dupe-swarm cluster: truncated-CJK and
Cyrillic sample cuts, utf-8-sig BOM, genuine binaries (PNG/ELF magic,
NUL-in-text), empty files, UTF-16 both endians (read-only pin), plus the
sibling sites — read_file_raw (patch/V4A, #80221), patch_replace, and
content search (#80308).

Closes #76886 #77047 #77842 #80221 #80251 #80308 #80922
2026-08-08 12:34:06 -07:00
Ayush Nangia e40315d53a fix(file-ops): classify binary files at the byte layer, not on transport-lossy text
Fixes the read_file half of #80308 and the class behind #80261, #80250,

The binary sniff sampled files via 'head -c 1000' through the terminal
transport, which decodes stdout with errors="replace". A multibyte
character cut at byte 1000 therefore arrived as U+FFFD, and
_is_likely_binary treated any U+FFFD as binary — flagging valid CJK and
emoji text as unreadable. At the text layer a stored replacement char
and a transport-manufactured one are indistinguishable, which is why
per-callsite adjustments kept leaving siblings open.

Sample as 'head -c 1000 | base64' so raw bytes survive the transport
(fail-open to the legacy heuristic when the transport cannot produce
clean base64), then classify bytes: NUL => binary; valid UTF-8 allowing
one incomplete multibyte sequence at the sample end => text; mid-stream
invalid UTF-8 (latin-1, true binaries) => read-only, preserving the
anti-mojibake guarantee the old check existed for. Files legitimately
containing U+FFFD become readable.
2026-08-08 12:34:06 -07:00
Adolanium 5945929d4b fix(tests): read and write test files as UTF-8 so the suite runs on Windows
`tests/hermes_cli/test_plugins_cmd.py::TestNoAutoActivation::test_compressor_default_ignores_plugin`
fails on every Windows machine:

    UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in
    position 47744: character maps to <undefined>

The test reads `run_agent.py` back as text to assert a removed comment is
gone, but called `open()` with no `encoding=`. Python then falls back to
the locale preferred encoding, which is cp1252 on a default Windows
install rather than UTF-8. `run_agent.py` contains nine bytes cp1252
leaves undefined, so the read raises before the assertion is reached. On
Linux and macOS the preferred encoding is UTF-8 and the same line is
fine, which is why CI never caught it.

That one line is the only active failure. The rest of this change closes
the same gap in the files it touches, which `scripts/check-windows-footguns.py`
flags and which the #71014 read_text campaign has been working through
elsewhere in the tree:

- `tests/hermes_cli/test_plugins_cmd.py`: nine bare `write_text`/`read_text`
  calls writing YAML manifests, config and plugin sources
- `tests/tools/test_web_tools_truncate.py`: reads stored extracted web text,
  which is arbitrary content from the internet
- `tests/stress/test_atypical_scenarios.py`: writes and reads worker task
  ids and a barrier file

All three files are now clean under `check-windows-footguns.py`.

Reads go through `Path.read_text(encoding="utf-8")` rather than
`open(...).read()`, which also closes the handle instead of leaving it to
the garbage collector. On Windows a live handle blocks tmpdir cleanup, so
that part is not cosmetic either.

No new test. The repaired test is the regression coverage: it fails
before this change and passes after, on Windows.
2026-08-08 12:33:19 -07:00
Teknium 9e6cfcda5a fix: finish the missing-encoding sweep — BOM-tolerant reads for user-edited stores
Complements the cherry-picked contributor fixes and closes out the
remaining sites of the 'missing explicit encoding' bug class, which is
now permanently gated by ruff PLW1514 (enabled repo-wide in
pyproject.toml and enforced by the blocking `ruff check .` step in
.github/workflows/lint.yml):

- tools/memory_tool.py: read MEMORY.md/USER.md via utf-8-sig so a
  Notepad BOM never glues U+FEFF onto the first entry (issue #10878,
  PR #10888 by @easyvibecoding — strict-decode contract of
  _read_raw_checked preserved rather than errors="replace", so
  undecodable files still refuse read-modify-write instead of being
  lossily rewritten). Regression tests included.
- tools/skills_tool.py: SKILL.md and skill file reads pinned to
  utf-8-sig + errors="replace" — deterministic across platforms instead
  of the locale fallback proposed in PR #51701 (superseded: falling back
  to cp1252/GBK makes the same skill render differently per host); .env
  reader aligned with the canonical utf-8-sig dialect in hermes_cli/config.py.
- agent/shell_hooks.py, hermes_cli/main.py, gateway/slash_commands.py:
  explicit utf-8 on the remaining fdopen/open text-mode sites flagged by
  the AlexFucuson9 sweep series (#56033 #56940 #65565 #66782 #66791).

Co-authored-by: easyvibecoding <easyvibecoding@users.noreply.github.com>
Co-authored-by: AlexFucuson9 <AlexFucuson9@users.noreply.github.com>
Co-authored-by: flyingdoubleg <wangzhe00zju@gmail.com>
Co-authored-by: LeonSGP43 <cine.dreamer.one@gmail.com>
2026-08-08 12:32:23 -07:00
nankingjing 3fee5c291b test(tools): cover UTF-8 BOM input in json_parse sandbox helper
Review asked for a BOM-prefixed JSON case alongside the existing
control-character coverage. The sandbox script now also feeds
json_parse a \ufeff-prefixed document and asserts the parsed value
round-trips (fails against the pre-fix helper, passes with the
BOM strip).
2026-08-08 12:32:23 -07:00
Theophilus Chinomona 45aa902c18 fix(process_registry): surrogateescape-safe PTY stdin writes (#79178) 2026-08-08 12:31:19 -07:00
Theophilus Chinomona 73cbc5e731 test(file_operations): pin early surrogate rejection over the backstop (#79178) 2026-08-08 12:31:19 -07:00
Theophilus Chinomona d6eda8d9c5 fix(file_operations): reject unencodable surrogates early, hash with surrogateescape (#79178) 2026-08-08 12:31:19 -07:00
Theophilus Chinomona b0594118ab fix(environments): surface stdin write failures as stdin_error (#79178) 2026-08-08 12:31:19 -07:00
Theophilus Chinomona c5a1a5d7b0 fix(environments): surrogateescape-safe stdin piping, always close stdin (#79178) 2026-08-08 12:31:19 -07:00
Teknium fce314eabd feat(skills): advisory SKILL.md convention linter on create
Adds tools/skill_linter.py — a soft companion to the hard frontmatter
validator. It encodes the CONTRIBUTING 'Skill authoring standards
(HARDLINE)' conventions that today only a human reviewer catches:

- shell-utility references in prose (`grep`/`sed`/`cat`...) that should
  name the native tool (search_files/patch/read_file)
- missing version/author/license/metadata.hermes block
- name != directory, invalid name format
- description over the 60-char prompt budget, marketing words
- dangling references/ links, forbidden scaffolding files
- POSIX-only script primitives without a platforms: gate

Findings are ADVISORY. skill_manage(create) attaches them as
lint_warnings + lint_hint in the success result; nothing is blocked
(the hard rejects already run in _validate_frontmatter). A CLI
(python -m tools.skill_linter <dir>) exits 1 only on ERROR-severity
findings so CI can gate on structural breakage without failing on nits.

Calibrated against the bundled skills/ tree: 76 advisory findings, exit 0,
no false positives after excluding repo-root scripts/ refs.

Inspired by MiniMax Code's skill-creator lint step; adapted to our
existing validator + skill_utils rather than a parallel system.
2026-08-08 11:12:27 -07:00
kshitij 73997c41bb fix(tts): split long speech by provider and platform limits
Salvage of PR #17973 by @TKCen (Sebastian Hänisch), re-implemented on
current main to preserve speed/instructions/provider params,
prepare_spoken_text normalization, OPUS_VOICE_PLATFORMS, is_write_denied
path security, microsecond timestamps, and the streaming-TTS gate.

- Split long TTS text into provider-safe chunks instead of truncating
- Pack generated audio against platform upload limits (Discord 10MB,
  Telegram 50MB, configurable via tts.delivery_profiles)
- Combine chunks with ffmpeg (OGG/Opus re-encoded, MP3 stream-copied)
- Multi-file delivery when combination fails or would exceed limits
- Remove hard [:4000] truncation from all callers (cli.py, voice.py,
  gateway/run.py, gateway/platforms/base.py)
- Gemini TTS raises ValueError instead of silently truncating when
  composed prompt exceeds the provider limit

Simplify-code fixes: removed dead all_touched_paths set, added
try/finally for scratch file cleanup on exception, clean error response
on chunk failure instead of leaking stale file_path.
2026-08-08 22:54:20 +05:30
Brooklyn Nicholson 406501fd97 feat(agent): read_window_below tool — which OS window is underneath the desktop app
Desktop-gated (desktop_ui toolset) metadata-only window awareness: the agent
can ask which application window sits directly behind the Hermes window
(app, title, bounds — never pixels). Rides the same blocking bridge as
read_terminal: the gateway emits window.read.request and the renderer
answers window.read.respond.
2026-08-08 12:17:50 -05:00
Teknium 7c2bc87f81 feat(read_extract): label each unreadable PDF gap with its preceding section text
The coverage warning listed bare page ranges, which tells the agent
WHERE the gaps are but not WHAT they contain — its only options were
guessing or OCRing everything. Each gap is now labeled with the last
text extracted before it (usually a section divider page), so the agent
can decide which gaps it actually needs and render/OCR only those.
Gap list capped at 20 entries with a summary line for pathological
alternating documents.
2026-08-08 05:51:33 -07:00
Teknium fe54ab4f98 fix(docker): close the cold-container and multi-backend gaps in attachment delivery
Follow-ups on the salvaged commit:

1. get_cache_directory_mounts() now CREATES missing staging dirs instead of
   skipping them. Docker snapshots the mount list at container creation, so
   a dir born later (first attachment, first clipboard image) dangled for
   the life of a persistent container. Empty bind mount costs nothing.

2. to_agent_visible_cache_path() translates per-backend instead of
   docker-only: docker/modal -> /root/.hermes, ssh/daytona/vercel_sandbox ->
   ~/.hermes (shell-expanded remotely; bytes arrive via file sync), local/
   singularity keep the host path (apptainer auto-binds the host home).
   Mirrors the proven _agent_cache_base_for_env heuristics.

Updated the two mount-list tests pinning the old skip behavior; added
per-backend translation coverage.
2026-08-08 05:44:18 -07:00
Teknium cbb8cee47d fix(read_file): surface document extraction failures instead of the generic binary-file error
When extraction of a binary document format (.pdf, .docx, .xlsx, Office,
EPUB…) fails for a specific reason — the anydoc size cap, an encrypted or
malformed file — read_file previously swallowed the ExtractionError at
debug level and fell through to the generic 'Cannot read binary file'
guard, so the agent never saw the actionable reason (e.g. 'Document too
large to convert (N bytes, limit is 52,428,800)').

read_file now returns the specific extraction failure for binary document
formats. Fallthrough behavior is preserved where a raw read is still
useful: .ipynb (plain JSON) and converter-unavailable PDFs keep their
historical raw-read path, and the 'Unsupported document type' shape (no
extra information) keeps the generic guard.

Follow-up to #80004, where the size-cap message was being generated but
never reached the agent.
2026-08-08 05:40:45 -07:00
Teknium cd9fbf9f19 test: convert NB2 catalog snapshot test to invariants; live-verified t2i+edit
Follow-up on the cherry-picked contribution from @michaelsam94 (#51794):
replace display-string/exact-value snapshot assertions with invariant
checks per the no-change-detector-tests policy. Live-tested
fal-ai/nano-banana-2 and fal-ai/nano-banana-2/edit through the real
payload builders: both pass.
2026-08-08 05:31:38 -07:00
michaelsam94 2b16a6b03c feat(image_gen): add FAL Nano Banana 2 model 2026-08-08 05:31:38 -07:00
Teknium 5dc0fa3889 fix: post-merge audit follow-ups for #81138/#81139/#81141/#81148
Four fix-forwards from the adversarial post-merge audit of the Aug 7
unreviewed merge batch:

- estop (#81148): is_engaged() now fails SAFE (engaged) on stat errors;
  the gateway estop gate lets recognized slash commands and replies owned
  by in-flight work (update prompts, clarify, slash-confirm, tool
  approvals, running sessions) through instead of consuming them; new
  gateway /pause [reason|off] command gives messaging-only operators an
  in-band engage/resume path (busy_policy=dispatch so it works mid-run).
- cron monitor mode (#81138): execution-mode invariants (monitor x
  no_agent, monitor_script x monitor_url, no_agent-requires-script) now
  have ONE owner (_validate_job_mode_invariants) called from BOTH
  create_job and update_job, so the create-time invariant can no longer
  be silently violated through the update door.
- cron notepad (#81139): remove_job now clears the job's notepad rows
  (clear_notepad was dead code -> orphaned KV state forever); clear is
  best-effort and no-ops without creating notepad.db.
- delegation batch gate (#81141): template-marker regex narrowed to
  multi-word placeholder shapes only (<feature name>, {file_path}) so
  generics (Vec<T>), HTML tags, JSON snippets, glob braces and f-string
  style no longer reject legitimate batches; duplicate-goal rejection
  removed (best-of-N fan-outs are legitimate).
2026-08-08 05:21:09 -07:00
Teknium 765940df79 fix(read_extract): keep scanned-PDF coverage warning on the backend bytes path
The salvaged bytes path (_extract_anydoc_bytes) bypassed the coverage
check added in #81680. Materialize transferred PDF bytes in a host temp
file for the pdftotext scan, and name the backend-visible path in the
recovery command rather than the temp file.
2026-08-08 05:13:46 -07:00
fangliquanflq 8de3ddb9ef fix(tools): preserve document extraction boundaries 2026-08-08 05:13:46 -07:00
Teknium 70c6cf8e7e feat: add new FAL video families and image models
Video (plugins/video_gen/fal): Seedance 2.5, MiniMax H3, Seedance 2.0
Mini, FLUX 3, Grok Imagine 1.5, Gemini Omni Flash (i2v-only). New
family capability flags:
- duration_int: endpoints that take duration as a JSON integer
- resolution_aliases: maps 720p/1080p-style values onto non-standard
  enums (H3's 768P/2K/4K)
- image_drop_keys: strips keys the family's i2v endpoint rejects
  (aspect_ratio on Seedance 2.5 / H3 / Grok 1.5)

Image (tools/image_generation_tool): Seedream 5.0 Pro (+edit) and
Lite, Ideogram V4 instant + fast, Qwen Image 3 (+edit), MAI Image 2.5
Pro, Nano Banana 2 Lite (+edit), Recraft V4.1.

Every new endpoint live-tested against fal.run through the real
payload builders + submit path: 18/18 pass (t2v, i2v, t2i, and edit
probes). Note: several new endpoints return HTTP 409 from the Nous
Portal FAL proxy allowlist until it is updated portal-side; BYOK
FAL_KEY works today and the existing 4xx guidance message covers it.
2026-08-08 04:31:55 -07:00
Coy Geek 530d37820c fix(terminal-tool): redact terminal error result fields
Force-redact every terminal exception and traceback field before JSON serialization, including environment creation, background startup, exhausted foreground retries, and the outer catch-all. Preserve the current command-aware, opt-out-respecting redact_terminal_output(output, command) behavior for successful output.
2026-08-08 04:28:19 -07:00
Teknium 89c14aeb9e fix(read_file): warn when PDF pages yield no text (scanned-image coverage gap)
anydoc converts the PDF text layer only and emits no image placeholders
or page markers, so a mostly-scanned PDF extracts 'successfully' into
section headers with empty bodies — silent data loss the model cannot
detect. Count per-page text via poppler pdftotext and prepend an
EXTRACTION COVERAGE WARNING naming the empty pages and the recovery
path (pdftoppm + vision_analyze, or the ocr-and-documents skill).

Found on a 311-page HOA resale package where 198 scanned pages
(CC&Rs, Bylaws, Articles, insurance certs) vanished without a trace.
2026-08-08 04:25:27 -07:00
Teknium 72eda946be fix(security): redact terminal exception results and ACP stderr logs (#77484)
Closes the last two emission gaps from #77484:

- tools/terminal_tool.py: both exception paths (generic except and
  TERMINAL_DEGRADED_MODE=fail) returned raw str(e) + traceback.format_exc()
  to the model — only the logger copy was redacted. Exception text can
  embed the failing command line and any secrets inline in it; both fields
  now pass through redact_sensitive_text.
- acp_adapter/entry.py: _setup_logging cleared root handlers and installed
  a plain logging.Formatter, bypassing redaction entirely on ACP stderr.
  Now uses RedactingFormatter like every other logging surface.

The other three gaps from #77484 (process(list), *_KEY regex variants,
control-char splits) were fixed in #80964/#80965.
2026-08-08 04:19:49 -07:00
Teknium 2a743e5f43 fix(image_gen): confine generation source images to the terminal backend
image_generate and video_generate forwarded model-supplied local paths to
provider plugins, which read them off the HOST filesystem regardless of
terminal backend — inconsistent with the confinement boundary vision/video
analysis enforce (GHSA-gpxw-6wxv-w3qq), and broken for sandbox-only files.

New dispatch-layer chokepoint (_confine_source_images): under a non-local
backend, path-like image_url / reference_image_urls resolve through
tools.image_source (media-cache host reads, bounded in-sandbox exec-read,
lazy env bring-up, credential guard, 50MB cap) and reach every provider as
data: URLs — which all backends already accept. URLs/data: pass through;
local backend is a no-op. xai_video_edit/extend already require public
HTTPS URLs, so no change needed there.
2026-08-08 04:19:38 -07:00
Ahmett101 f46636bfe2 fix(vision): retry container exec-read for Docker cold-start, surface stderr (#76566)
Under the Docker terminal backend, vision_analyze's first exec-read
sometimes returned empty / non-zero against a freshly started container,
producing 'could not read <path> inside the sandbox' on a file the agent
could cat seconds later. Cold pipe setup on the first exec against a
new container, not a permissions or mount problem.

Retry once after a short delay (150 ms covers Docker exec warm-up
without making a real failure feel sluggish). When every attempt still
fails, fold the container's first stderr line into the raised error so
the user can tell 'no such file' from 'permission denied' instead of
staring at one opaque message.

Tests cover the retry-then-succeed path, the diagnostic-on-exhausted
path, and confirm the existing single-attempt raise is preserved.
2026-08-08 03:59:42 -07:00
Teknium 9eb3ac50fe fix(video): route terminal-backend reads through the shared media resolver
Follow-up on the salvaged commit: replace the hand-rolled file_ops python3
exec-read with tools.image_source.resolve_image_source(permitted=('video',)),
so video_analyze gets the same pipeline as vision_analyze — media-cache host
reads, bounded head -c sandbox exec (no python3 dependency in the sandbox
image, no unbounded base64 stream), lazy env bring-up (#62825), the
credential-read guard, and the 50MB ingest cap.
2026-08-08 03:59:39 -07:00
dsad f2e936dad5 fix(video): read analyze inputs through terminal backend 2026-08-08 03:59:39 -07:00
kshitij c8e558c72c fix(tools): keep non-bash -c invocations covered by the shell guard
The _bash_exec_payload delegation rejected short-option bundles with
letters outside bash's alphabet, so 'zsh -yc', 'dash -Vc' and 'ksh -Gc'
scripts stopped being scanned — a fail-open regression for shells the
guard's _SHELL_EXECUTABLES explicitly covers. Try the bash grammar
first (catches operand-hidden -c), then fall back to the permissive
positional scan; a block-guard fails closed.
2026-08-08 14:56:38 +05:30
kshitij daa139c9e3 fix(tools): classify git bisect as a worktree mutation
bisect sat in _KNOWN_GIT_BUILTINS and was allowed in the running source
root, yet it repeatedly checks out commits — the exact module-version
skew this guard exists to prevent. Move it to _WORKTREE_MUTATIONS.
2026-08-08 14:56:38 +05:30
kshitij bb311b3951 fix(tools): parse bash option grammar before extracting the -c script
The guard's _shell_script_arg treated any leading option containing 'c'
as -c and looked no further, so 'bash -o pipefail -c "git checkout
main"' returned None and the script was never scanned (fail-open).
approval.py's _bash_exec_payload already parses bash's real option
grammar (-O/-o consume operands, short-option bundles, --init-file);
delegate to it instead of keeping a second, weaker parser.
2026-08-08 14:56:38 +05:30
Erosika 886092bc54 fix(tools): block worktree removal and moves of the running source root
`worktree` sat in _KNOWN_GIT_BUILTINS, so the guard returned safe for the
whole family. That allowed `git worktree remove [--force] <root>` and
`git worktree move <root> <dest>` against the very checkout this process
runs from, which the guard already treats as a source root when its .git
is a linked-worktree file.

Both name their target as an argument rather than acting on the cwd, so
they also slipped past the "is the cwd inside root" gate when run from
outside. Resolve the target against the command's cwd and block it when
it lands on the running root, from any directory. `worktree add`, list,
prune, lock, unlock, and operations on other worktrees stay allowed.
2026-08-08 14:56:38 +05:30
Erosika f0a3ef8bde fix(tools): harden live source checkout guard 2026-08-08 14:56:38 +05:30
Erosika ecbe6ef0dd feat(tools): hard-block self-repo git mutations in terminal_tool
Wire the self-repo guard in next to the gateway lifecycle hard-block,
before the force check — force=True cannot make the command safe, only
delay the crash. Local backend only: sandboxed backends cannot reach the
host checkout. The block message explains the version-skew mechanism and
redirects to git worktree add / a temp clone, or running the command
outside hermes with a restart after.
2026-08-08 14:56:38 +05:30
Erosika 206531a1e1 feat(tools): detect git mutations targeting the running source checkout
When hermes runs from a source/editable install, a git checkout/reset/pull
in its own repo swaps code on disk under the live interpreter. Modules
imported before the switch stay old while later lazy imports load new code,
producing delayed signature TypeErrors and tracebacks that don't match the
source, typically losing the in-flight turn.

New tools/self_repo_guard.py detects working-tree/ref mutations (checkout,
switch, reset, rebase, merge, pull, restore, stash, clean, cherry-pick,
revert) whose target repo is the source root the process runs from, via
cwd, git -C, cd chains, and subshell segments. Read-only git, commits,
fetch, and git worktree add stay allowed; packaged installs (no .git) are
inert.
2026-08-08 14:56:38 +05:30
Erosika ad59d55338 fix(tools): bound the exception text dispatch writes into its own log line
dispatch() called logger.exception with the exception interpolated into the
message. exc_info renders the same exception again in the traceback, so a
failing tool wrote its error body to the log twice. Every tool exception
passes through this one handler, so a large HTTP error body from any tool
landed here at full size.

Bound the message copy. The traceback still renders the exception once,
which is what an operator needs to place the failure.

Same double-write @arimu1 fixed in the vision, image, and TTS handlers in
#75938.
2026-08-08 14:56:38 +05:30
Erosika 84bc430073 fix(tools): bound the truncation log so it stops re-dumping the full body
The debug line that fires when an error body is truncated interpolated the
whole untruncated body, so capping the model-facing copy still wrote the
original to the log. A large HTTP error body — a Cloudflare challenge page
or a proxy 502 — reached the log at full size on every failed call.

Log a bounded prefix instead. It stays longer than the model-facing cap so
an operator still has something to diagnose with, but it no longer grows
with the size of the response body.

Reported for the logging handlers in #75938 by @arimu1; the same pattern
was present here.
2026-08-08 14:56:38 +05:30
Erosika 2181d2e7c2 fix(tools): bound tool error bodies at the dispatch boundary
tool_error() caps its message at _MAX_TOOL_ERROR_CHARS (2048), logging
the full body at DEBUG before trimming the context-bound copy.

Handlers that serialize exceptions directly -- json.dumps({"error":
str(exc), ...}) -- bypass that helper, so _normalize_handler_result
also runs every string result through _bound_json_error_result: if it
parses as a JSON object with an oversized string error field, only
that field is trimmed and the payload re-serialized. Non-error
results, non-JSON strings, and multimodal envelopes pass untouched.
2026-08-08 14:56:38 +05:30