* fix: Windows path handling in search_files rg calls and patch escape drift
Two related Windows failures from a live session (Windows 10, git-bash
terminal backend, winget-installed native ripgrep):
1. search_files was unusable on drive-letter paths. _escape_shell_arg
rewrites C:\... to the MSYS form /c/... so bash builtins resolve it,
but rg is a native Windows binary and Hermes disables MSYS argument
conversion for its bash subprocesses (MSYS_NO_PATHCONV=1 /
MSYS2_ARG_CONV_EXCL=*, see _apply_windows_msys_bash_env_defaults) —
so nothing ever translated /c/... back and every search failed with
'The system cannot find the path specified. (os error 3)'.
Fix: new _escape_native_tool_arg emits the forward-slash NATIVE form
(C:/Users/...), which native binaries accept, bash passes through
untouched, and MSYS builds also handle. Applied to the six rg call
sites (content search, --files search x2, zero-match probe x3); the
grep fallback keeps the MSYS form since MSYS grep wants it.
2. The patch tool silently doubled backslash runs when tool-call args
arrived JSON-escaped one extra time (file had \ where old_string
had \\). Similarity strategies (context_aware) matched the region
anyway and wrote new_string verbatim, corrupting every backslash run
(reproduced: 6 backslashes on the line became 12). _detect_escape_drift
now also blocks when every backslash run in old_string is exactly twice
its counterpart in the matched region and new_string repeats the
doubling — with guardrails so exact matches, intentional backslash
edits, model-corrected new_strings, and single weak-signal runs all
still apply. Blocking returns the standard escape-drift guidance so
the model re-reads and retries with correct counts.
Tests: TestEscapeNativeToolArg (5 cases, including an end-to-end
_search_with_rg command capture) and TestBackslashDoublingDrift (6
cases). The 8 pre-existing failures in tests/tools/test_file_operations.py
on a Windows host (umask/symlink POSIX assumptions) are identical on
unmodified main and unrelated.
* fix: shell linters get native Windows paths too (node C:\c\... double-prefix)
Same class as the rg fix: LINTERS commands (python -m py_compile,
node --check, npx tsc, go vet, rustfmt) invoke native Windows binaries,
but _check_lint interpolated the MSYS /c/... form. node resolves that
as C:\c\Users\... (double-prefixed), so on Windows hosts every .js
write reported a phantom ENOENT lint failure that could mask real
syntax errors (issue #84303). Route the {file} arg through
_escape_native_tool_arg like the rg call sites.
Regression test asserts node --check receives 'C:/...' and never
'/c/...'.
'Binary file - use appropriate tools' names a recovery the model may
not have — in a file-only toolset it thrashed for 41 turns / 178 tool
calls / 1.5M tokens on a PNG-behind-.txt (readtool eval, qwen3.8-max)
hunting for tools that did not exist. Name the type instead: 25 magic
signatures (images, archives, executables, media, SQLite), ftyp check
for ISO media, size in human units. 'Binary file (PNG image data,
4.1 KB) - cannot display as text.' answers what-is-this in one read.
Both ShellFileOperations refusal sites (read_file + read_file_raw) use
the shared describe_binary_file(); the extension-based guard keeps its
extension message (an extension is a claim; only sniffed content earns
a type name).
ShellFileOperations.read_file previously ran sed -n '{off},{end}p' bare, so
a file with one pathological line (e.g. a 50MB+ minified bundle on a single
line) shipped the entire line across the exec transport before Python's
per-line clamp (_add_line_numbers, MAX_LINE_LENGTH=2000) could trim it.
read_file now pipes through 'cut -b1-{4*max_line_length+1}' so the shell
bounds every line to 8001 bytes before the bytes ever reach Python.
UTF-8 finding: GNU 'cut -c' is byte-based despite its name (verified:
cutting a line of 2-byte 'é' at -c8004 splits a codepoint, leaving a bare
0xC3 lead byte). The transport decodes with errors='replace', so a split
codepoint becomes U+FFFD rather than raising — but a clamp of
max_line_length+1 BYTES would deliver under max_line_length CHARS for
multibyte text, so the Python clamp would never fire and truncation would
be silent. Using 4*max_line_length+1 bytes (UTF-8 max 4 bytes/codepoint)
guarantees any line longer than max_line_length chars still decodes to
more than max_line_length chars, so len(line) > max_line_length always
triggers the existing '... [truncated]' suffix, and any boundary U+FFFD
lands past char max_line_length where the clamp removes it — verified
empirically with fixtures ('é'*4001 splits at the byte boundary yet the
result contains no U+FFFD and ends with the truncated suffix). 'cut -b'
is used explicitly to document the byte semantics.
cut (unlike sed -n p) always newline-terminates its output, which would
grow a phantom empty final line on files without a trailing newline; the
final-page path now probes the last byte (tail -c 1 | wc -l) and strips
the artifact.
read_file_raw is untouched: it is documented as no-per-line-truncation.
Benchmark (50MB single-line fixture, /usr/bin/time -v, median of 3):
before: 191.1 MB peak RSS, 1260 ms wall
after: 97.8 MB peak RSS, 490 ms wall
Correctness identical in both arms: monster line returns the clamped
2000-char form + '... [truncated]', offset=2 returns the trailing normal
lines intact.
Tests: 153 passed, 0 failed, 4 skipped across the file-ops suites plus a
new tests/tools/test_read_shell_line_clamp.py pinning the monster-line
clamp, offset-past-monster reads, no-trailing-newline preservation, both
UTF-8 boundary cases, and read_file_raw's exemption. Two existing mocks
asserting the exact sed command string were updated for the pipeline.
The size probe every read path starts with — `wc -c < path` — opens the
path. On a FIFO with no writer, a socket, or a character device that never
reaches EOF, that read never returns, and read_file/read_file_raw/
read_file_bytes all pass no timeout to _exec. The turn wedges until the
process is killed.
The device blocklist in tools/file_tools.py cannot close this: it matches
literal /dev/* names, so it can only ever cover paths someone thought to
enumerate. A FIFO is a file type and can sit at any path.
Gate the probe behind `[ -f ]`, which stats instead of opening, and report
a path that exists but is not a regular file as such. A missing path keeps
its existing not-found handling.
A read past EOF returned content '900|' (a phantom line-number prefix
that looks like a real line) and an empty file returned '1|' — both
ambiguous silence: indistinguishable, from inside the model, from a
broken tool, so it re-reads and widens windows. Name the dead end and
its recovery instead: 'offset 900 is beyond the end of the file (412
lines total). Retry with offset <= 412.' / 'File is empty (0 bytes).'
Notes, not errors — a fact about the file is not a failure.
Boundary pinned by test: offset == total_lines still reads (an
off-by-one in a resume hint is a silently corrupted read).
Measured (file-only arm, 3 reps, control vs feature): qwen3.8-max
-18% tokens, -26% tool calls, -17% turns across the two affected
tasks; opus-4.8 flat (within rep noise); accuracy held 1.00.
NFC/NFD, narrow no-break space (U+202F), and curly quotes render
identically in a terminal — a model retyping a visually-correct path
gets 'file not found' and can never discover the byte mismatch on its
own. On not-found, canonicalize the requested name and compare against
directory entries; exactly ONE equivalent spelling reads transparently
with an explanatory note. Zero or several matches (homoglyph twins)
fall through — never guess between collisions.
Also: difflib.SequenceMatcher >=0.8 fallback in _suggest_similar_files
catches near-miss typos (AGENT.md -> AGENTS.md) that substring scoring
misses entirely.
Measured (file-only arm, 3 reps, control=guard-only vs feature):
unicode task qwen3.8-max 31k->16k tok (-48%), turns 6.7->3.7;
opus-4.8 57k->33k tok (-42%), turns 8.3->5.0; accuracy held 1.00.
near-miss: opus mildly better, qwen flat, no regressions.
Fixes the read_file half of #80308 and the class behind #80261, #80250,
The binary sniff sampled files via 'head -c 1000' through the terminal
transport, which decodes stdout with errors="replace". A multibyte
character cut at byte 1000 therefore arrived as U+FFFD, and
_is_likely_binary treated any U+FFFD as binary — flagging valid CJK and
emoji text as unreadable. At the text layer a stored replacement char
and a transport-manufactured one are indistinguishable, which is why
per-callsite adjustments kept leaving siblings open.
Sample as 'head -c 1000 | base64' so raw bytes survive the transport
(fail-open to the legacy heuristic when the transport cannot produce
clean base64), then classify bytes: NUL => binary; valid UTF-8 allowing
one incomplete multibyte sequence at the sample end => text; mid-stream
invalid UTF-8 (latin-1, true binaries) => read-only, preserving the
anti-mojibake guarantee the old check existed for. Files legitimately
containing U+FFFD become readable.
requires-python is >=3.11 so tomllib is always in stdlib; the
tomli fallback branch in _lint_toml_inproc was unreachable. Removes
the dependency from pyproject.toml + uv.lock and deletes the dead
try/except ImportError fallback in the code.
- write_file: encode content once, share bytes between bytes_written and
the sha256 verification (drops a second full-content encode per write)
- patch_parser: replace the except-TypeError retry around
write_file(pre_content=...) with signature-based feature detection so a
TypeError raised inside a capable implementation propagates instead of
triggering a duplicate write; tests for both duck-typing contracts
- tests: real-ops V4A BOM round-trip + _file_has_bom disk-probe guard
(the teknium1-review regression previously only covered by a fake)
- comment: document dirs_created's long-standing "parent ensured" meaning
Content that flowed through a surrogateescape decode (backend output via
patch_replace) can carry lone surrogates; a strict encode raises
UnicodeEncodeError where the old wc -c path could not. Mirrors the
existing sha256 verification encode.
Bug 1 (UTF-8 BOM loss on V4A UPDATE):
_file_has_bom() trusted pre_content for BOM detection, but the most
common pre_content provider — read_file_raw() — deliberately strips
BOMs so the agent never sees U+FEFF glyphs. Passing BOM-stripped
content through pre_content caused a false-negative: the method
returned False and write_file() silently removed the marker on rewrite.
Fix: _file_has_bom() now always probes the first 3 bytes on disk
(head -c 3), ignoring pre_content for BOM purposes. pre_content is
still used by two other consumers — line-ending detection and lint/LSP
delta computation — neither of which is affected by BOM stripping.
Bug 2 (backward compatibility):
_apply_update() called write_file(path, content, pre_content=...) as a
keyword argument. Duck-typed file_ops implementations that only
implement the two-argument write_file(path, content) contract would
raise TypeError.
Fix: wrap the call in try/except TypeError, falling back to the
two-argument form when the keyword is not accepted.
Also declare tomli in pyproject.toml (pre-existing conditional import
for pre-3.11 Python, caught by the pre-commit dep scan after staging
file_operations.py).
Tests:
Add TestV4ABomRoundTrip with two cases:
- UPDATE on BOM-bearing file preserves the marker
- UPDATE on plain file does not inject a BOM
Addresses teknium1 review on PR #55661.
write_file currently spawns up to 6 subprocesses per call:
1. mkdir -p (separate call before atomic write)
2. cat (to read pre-content for lint/BOM/line-ending detection)
3. _atomic_write (mktemp + write + mv — the essential one)
4. wc -c (to measure bytes written)
5. _check_lint_delta (post-write lint — also essential)
6. LSP snapshot (also essential)
This PR removes three of them without changing any observable behavior:
1. Fold mkdir -p into _atomic_write shell script (−1 subprocess/write)
The atomic write script already runs a single shell; adding mkdir -p
to it costs zero extra processes.
2. Add optional pre_content parameter to write_file (−1 subprocess/patch)
patch_replace and V4A _apply_update already read the file for fuzzy
matching. Passing that content as pre_content skips the redundant cat
inside write_file. Fully backward-compatible: callers that don't pass
pre_content still read from disk as before.
3. Replace wc -c with len(content.encode('utf-8')) (−1 subprocess/write)
We already have the content in memory; encoding it to get the byte count
is equivalent to wc -c for UTF-8 text.
4. Remove redundant _check_lint loop in apply_v4a_operations (−N subprocesses/V4A)
write_file already runs _check_lint_delta internally. The old code ran a
bare _check_lint(f) loop over all modified files — a re-read + re-lint
without post_content context. Now lint results propagate from write_file
via a four-tuple return, zeroing out the extra subprocesses.
Net effect:
- write_file: 6 → 3 subprocesses per call (new files)
- patch_replace: 6 → 5 subprocesses per call (pre_content skips cat)
- V4A multi-file patches: saves 1 subprocess per modified file
- A typical 4-file V4A patch drops from ~28 to ~16 subprocess calls
Integration regression between two merged PRs: #77102's rg-path early
return (added to skip the grep-era line-oriented warning) also skipped
the #77011 zero-match steering probes (case-insensitive, hidden-file,
literal-vs-regex), silencing them on the primary engine. Caught by
#77001's rebased CI run (its branch carried both features together for
the first time).
_search_content now runs the zero-match probe block for BOTH engines
and only exempts the rg path from the legacy line-oriented \n warning
(rg auto-enables --multiline; the grep fallback keeps the explanation).
A regex \n (or a raw newline) in a search_files content pattern cannot
match in rg's default line-oriented mode. It previously either
hard-errored ('the literal "\n" is not allowed in a regex' — 17
occurrences in the production window) or, after the newline-warning
patch, returned 0 matches with an explanation — either way the model's
cross-line search intent required a manual workaround.
The rg engine now detects the pattern shape (_pattern_has_regex_newline,
already used by the warning path: odd-backslash \n escape or raw
newline; escaped \\n literals excluded) and enables -U/--multiline up
front, noting the mode switch in the result warning. Plain patterns are
untouched; the old line-oriented explanation is retained for the grep
fallback engine, which has no multiline mode.
write_file confirmed only SIZE (wc -c) after writing — never content.
Models compensated by re-reading files immediately after writing them
(154 verify-reads in a 400k-msg production window), and a corrupted
write (truncated pipe, backend FS oddity) could silently pass.
The write path now compares the on-disk sha256 against the intended
content (one shell call). Three outcomes:
- match -> result carries verified: true; the schema tells the model
an explicit contract: do NOT re-read to check the write landed.
- mismatch -> hard error ('The write did not persist correctly'),
mirroring patch_replace's existing post-write verification.
- backend can't hash (no sha256sum) -> flag omitted, write unaffected.
Hashes the shim-adjusted content (after CRLF/BOM preservation) so
Windows-line-ending and BOM round-trips verify correctly; surrogatepass
encoding matches the rest of the codebase's hashing of model text.
Third zero-match steering tier: when a content search finds nothing in
visible files, probe once with rg --hidden --no-ignore --count-matches.
If the pattern exists only in dotdirs or gitignored files, the result
says so and tells the model to search the hidden path explicitly.
Found live by the benchmark battery: a task with a match inside
.hidden/ returned a bare 0 and the model missed the file entirely in
2/3 baseline runs.
Two dead-turn classes from production mining (state.db, recent window):
1. 13.9% of 19.6k content searches return 0 matches with no steering.
Now a 0-match content search runs one cheap rg -i --count-matches
probe (plus an rg -F probe when the pattern has regex metachars) and
attaches what it found: '0 exact matches, but N case-insensitive
matches — casing may be wrong' / 'N literal matches — metacharacters
need escaping'. True zero-match results stay clean (no noise).
2. 122 'Path not found' failures came from models passing several paths
in ONE path string ('dir1 dir2 dir3', comma lists). Instead of
failing wholesale, split the string, search every path that exists,
merge results, and report skipped parts in a warning. Single-path
misses keep the existing Similar-paths hint; all-missing multi-path
strings still error.
Both probes are bounded (count-only rg, 30s timeout, max 2 invocations)
and wrapped so a probe failure can never break the search result.
The #1 patch failure class in production (state.db mining, 250k-window)
is a re-send of an edit that already landed: 'old_string and new_string
are identical' (299 occurrences) plus a share of hunk-not-found errors
where the new text is already in the file. These errored, sending
models into re-read/re-patch loops.
New tools/fuzzy_match.is_already_applied(content, old, new) — a
conservative check requiring (1) non-trivial new_string (>=8 chars),
(2) EXACT presence of new_string, (3) old_string gone (unless
identical). Wired into three sites:
- patch_replace (replace mode): returns success + no_change: true +
an explicit note instead of the identical-strings / no-match error.
- V4A validation phase: an already-applied hunk validates as a no-op
so multi-hunk patches no longer fail wholesale when one hunk landed
in a prior call.
- V4A apply phase: mirrors the same skip so the two phases agree.
Genuine no-matches (new text absent) and half-applied renames (old
text still present) keep their error behavior — covered by tests.
Production mining (state.db, 28.5k read_file calls in the recent
window) shows 74.3% of reads truncated — nearly all by the 500-line
default, not the char budget (22,443 line-limit vs 13 char-budget
truncations). That churn produced 12,229 redundant re-reads, and after
a truncated result the most common next move was fleeing to terminal
cat/sed (4,608 times) — the pagination contract was not trusted.
Median truncated file is 2,422 total lines, so a 2000-line default
makes 44% of today's truncated first-reads complete in one call while
the unchanged ~100K-char budget still caps worst-case result size
(same ceiling as before: 500 lines x 2000-char line cap = the same
100K). Schema max was already 2000.
Touchpoints: DEFAULT_READ_LIMIT + both read_file signatures
(file_operations.py), read_file_tool + schema text (file_tools.py),
execute_code sandbox stub docs (code_execution_tool.py), 3 tests
pinning the old default.
Two DATA-LOSS bugs in ShellFileOperations found in a core-tools audit,
each reproduced live against current main:
1. Non-UTF-8 file content silently corrupted on read->write. The terminal
env decodes stdout with errors='replace', so a latin-1/8859 file's bytes
arrive as U+FFFD before _is_likely_binary inspects them. U+FFFD is
'printable', so the >30%-non-printable check never flagged it, and the
agent would read the mojibake and write it back, permanently replacing the
original bytes. Fix: treat a sample containing U+FFFD as binary (read-only).
2. Writing through a symlink destroyed the link and orphaned the target. The
atomic temp-file + 'mv -f' swap replaced the symlink itself with a plain
file; the real target was never updated. Fix: resolve the link with
readlink -f/realpath first and recompute the temp dir from the resolved
target so the mv stays same-filesystem atomic. Broken links fall back to
the original path (no regression).
Both verified with sabotage-checked regression tests (fail without the fix).
Proper UTF-8 text (incl. non-ASCII) and plain-file writes are unaffected.
Follow-ups on top of #70888's cherry-picked fix:
- Replace the $((0666 & ~0$u)) shell arithmetic with POSIX who-less
'chmod "=rw"'. zsh (reachable via _find_bash's $SHELL fallback on
bash-less hosts) parses leading-zero constants as decimal and silently
chmods a garbage mode (e.g. 0210); the symbolic form is spec-identical
across bash/dash/busybox-ash/zsh and degrades to mktemp's 0600
(pre-fix behavior) rather than corrupting perms if chmod rejects it.
- Move the new-file chmod after the content stream so the temp file
stays owner-writable while cat runs.
- Run the chmod on a '[ ! -e "$t" ]' check after cat instead of the
stat/else branch, keeping the overwrite path untouched.
- Update the stale perms comment #70856 called out (new files did NOT
land with default umask perms pre-fix).
- Tests: select the atomic-write script by content instead of call
order (the previous last-call capture only worked because the bare
MagicMock's falsy-exit early return suppressed later execs), assert
behavior at explicit umasks 0022/0002/0077 via parametrize, add an
overwrite mode-preservation regression guard, and dedupe the
real-subprocess env fake into make_real_subprocess_env() shared with
TestSearchFilesFallbackHiddenPaths.
(webtecnica's email mapping already exists in contributors/emails/ on
current main; the PR's check-attribution red was stale-base only.)
# Conflicts:
# tests/tools/test_file_operations.py
Return actionable errors when HERMES_WRITE_SAFE_ROOT blocks a path instead of
labeling every denial as a protected credential file. Wire the helper through
write_file, patch, delete/move, and the Copilot ACP shim; sync docs examples.
#63113 rewrote native drive paths in ShellFileOperations, but init_session
/_wrap_command still embedded C:/... hermes-snap paths from get_temp_dir.
MSYS arg-converts those during bash -l and surfaces Directory \drivers\etc
— including for relative write_file targets, since the wrapper is the fault.
Add _bash_safe_path, override BaseEnvironment._quote_shell_path on
LocalEnvironment (no base→local import), and normalize mixed /c/Users\...
paths in file ops.
Co-authored-by: xxxigm <tuancanhnguyen706@gmail.com>
ShellFileOperations builds bash commands (wc/head/sed/cat/tee ...) with the
target path as an argument. On a Windows/Git-Bash host a native `C:\...` path
has its backslashes eaten by bash (and mangled by the msys runtime even when
single-quoted) — the "Directory \drivers\etc does not exist; exiting — update
your msys package" class of failures. Rewrite a native drive path to forward
slashes in `_escape_shell_arg`, reusing the env layer's `_windows_to_msys_path`.
Both `C:/...` and `/c/...` fix the backslash bug (the MSYS coreutils resolve
either via the POSIX API). We emit `/c/...` purely for consistency: it's the
same form `_windows_to_msys_path` already produces for the terminal `cd`
(LocalEnvironment._quote_cwd_for_cd), so shell file ops and `cd` share one
helper and one path form.
Scoped from #55481, which also patched BaseEnvironment._quote_cwd_for_cd — but
LocalEnvironment already overrides that through `_windows_to_msys_path`, so on a
real Windows host the base branch never ran (the cwd is already `/c/...`).
Co-authored-by: konsisumer <der@konsi.org>
safe_load() raises ComposerError on multi-document streams (k8s manifests)
and ConstructorError on application-defined tags (CloudFormation !Sub,
Ansible !vault) — both valid YAML syntax. Now that the linter's verdict is
a fail-closed write gate, those false positives would refuse legitimate
writes outright. Switch to yaml.parse() (scanner+parser only), which still
catches real syntax failures.
write_file() previously called _atomic_write() first and only ran the
JSON/YAML/TOML/Python syntax check afterward as an informational lint
delta -- a parse failure never set the top-level `error` key, so a
corrupt structured-data write still landed on disk (and file_tools.py's
files_modified gating, which keys off `error`, silently reported it as
a successful modification).
Move the in-process syntax check for JSON/YAML/TOML ahead of
_atomic_write() and refuse the write outright on a parse failure: no
temp file, no rename, nothing touches disk, and the result carries a
top-level `error` so callers correctly see it as unmodified.
Deliberately scoped to _FAIL_CLOSED_INPROC_EXTS (JSON/YAML/TOML), not
all of LINTERS_INPROC -- .py is excluded because this codebase's own
test fixtures (TestPatchReplacePostWriteVerification et al.) write
arbitrary non-Python text through *.py paths purely to exercise
write-mechanics; a hard block there broke 3 previously-passing tests
during development. Python keeps its pre-existing non-blocking
lint-delta report.
Adds tests/tools/test_write_file_syntax_gate.py: invalid JSON/YAML/YML/
TOML refused with nothing written (new file) and nothing modified
(existing file); valid JSON/YAML still written byte-for-byte; a
non-linted extension with garbage content is unaffected; invalid Python
is confirmed NOT hard-refused (still just reported).
* feat(search_files): path-grouped lossless densification of content matches
Content-mode search_files results repeat the {path,line,content} JSON keys
and the full path string for every match. Group consecutive same-path matches
under one path header with indented '<line>: <content>' rows — lossless (every
path/line/content byte preserved), self-describing (matches_format key), and
readable by the model with no decode step.
57.8% mean token reduction on real search_files content outputs (422-output
corpus), fires on 97% of them. Gated at >=5 matches; below that the verbose
array is left untouched. Default to_dict(densify=False) is unchanged, so no
other caller is affected.
ripgrep emits matches path-ordered, so consecutive grouping never reorders
results.
* test: accept densify kwarg in _FakeSearchResult.to_dict
The search loop-detection tests stub SearchResult with a fake whose
to_dict() must mirror the real signature now that it takes densify=.
* test(search_files): edge-case losslessness battery for densification
Adversarial single-line content (colons, indentation, unicode/emoji, empty,
trailing whitespace, quotes+commas), paths with spaces, and an explicit
one-line-per-match invariant documenting the ripgrep contract the format
relies on (0/6775 real match contents contained a newline).
The error guard in _search_with_rg/_search_with_grep was unreachable and,
if it had fired, would have discarded valid results.
Two root causes:
1. Unreachable. Both methods pipe the search through `| head` with no
pipefail, so the pipeline reported head's exit code (0), masking rg/grep's
error code (2). The guard never fired. Worse, because _exec merges stderr
into stdout (stderr=subprocess.STDOUT), the error text was then parsed as
bogus match lines instead of being surfaced — the user got garbage matches
with no indication the search failed.
2. Latent results-dropping. The original `not result.stdout.strip()` check
was always False on error (error text lives in stdout), and the
`hasattr(result, 'stderr')` branch was dead code (ExecuteResult has no
stderr field). A naive broadening to `exit_code == 2` would have nuked
real matches whenever rg/grep also hit a non-fatal error (e.g. one
unreadable file in a tree that otherwise matched), which both tools signal
with exit 2.
Fix:
- Prefix the piped command with `set -o pipefail` so rg/grep's real exit
status propagates. rg exits 0 on a truncating head; grep exits 141
(SIGPIPE), so the strict `== 2` guard ignores truncated-success.
- Add _split_tool_diagnostics() to separate tool diagnostics from match
output by tool prefix and output shape. Diagnostics never become matches;
on a hard error they are the message to surface.
- Only surface an error when exit==2 AND no usable match payload remains, so
partial errors keep their real matches.
Tests: tests/tools/test_search_error_guard.py drives both methods through the
real local backend (hard error surfaced, partial error keeps matches,
truncation no false error, files_only/count exclude diagnostics) plus unit
coverage for the splitter.
Supersedes #39710.
* fix(desktop): triage 24 GUI quality-of-life fixes across sidebar, composer, tool cards, messaging, and platform plumbing
A grab-bag of high-leverage UX fixes plus a few backend touches that the
GUI needs to behave correctly on Windows.
Sidebar / sessions
- Decrement $sessionsTotal on delete + archive so "Load N more" stops
claiming removed rows are still on the server.
- Hide the "Group by workspace" toggle when no unpinned sessions exist.
- Accept Cmd/Ctrl+N as a "new session" accelerator (in addition to bare
Shift+N), and render the kbd hint per-platform.
- Switch the statusbar to overflow-x-clip so untitled sessions don't
paint a horizontal scrollbar at the bottom of the window.
Messaging + Cron
- Add [-webkit-app-region: no-drag] to the page-search input so clicks
reach the field instead of routing to the OS window-drag handler.
- Replace single-letter PlatformAvatar with brand glyphs from
@icons-pack/react-simple-icons (telegram, discord, matrix, signal,
whatsapp, mattermost, wechat, qq, ...). Letter monogram fallback for
Slack / Dingtalk / Feishu / WeCom (removed from Simple Icons at brand
owner request).
- Drop the duplicate "Create first cron" button in the empty state.
Composer
- Dedupe pasted images by (name, size, lastModified, type) instead of
Blob identity; Chromium hands us the same screenshot via both
clipboard.items and clipboard.files with fresh File instances.
- Enable spellcheck on the contentEditable, configure Chromium's
spellchecker with the system locale on whenReady, and add
replaceMisspelling + "Add to dictionary" entries to the context menu.
- Render user messages through a minimal markdown pipeline (inline
backtick code + fenced ``` blocks) while keeping @file:/@image:
directive chips intact.
- max-h-[60vh] overflow-y-auto + collisionPadding on the prompt-snippet
submenu.
- Bake cursor-pointer into the <Button> primitive (with
disabled:cursor-default) and into titlebarButtonClass.
Dialogs + tabs + version
- Default DialogContent now has max-h-[85vh] overflow-y-auto so long
bodies scroll instead of falling off-screen.
- Right-rail preview tabs close on middle-click (button === 1), with an
onMouseDown swallow to suppress Chromium autoscroll.
- New refreshDesktopVersion() helper called from About mount, after
every update check, and on throttled window focus so About reflects
the just-installed binary.
Keys + Artifacts + Terminal
- Drop the global "Show advanced" toggle in KeysSettings. Provider
groups now default-expand when they have any key set.
- Extend openExternalUrl to handle file:// via shell.openPath, with
showItemInFolder fallback when the OS can't open the file.
- New lib/ansi.ts SGR parser + <AnsiText> component, applied to
terminal/execute_code tool output.
- ToolView gained stdout / stderr / rendersAnsi; tool-fallback renders
the two streams as separate labeled blocks with stderr in a neutral
tone (not destructive — many CLIs log info on stderr).
- Drop 'stderr' from ERROR_MSG_KEYS in tool-result-summary.
Paths + platform
- resolveHermesCwd skips process.cwd() when packaged and prefers a
user-configurable default project directory.
- New hermes:setting:defaultProjectDir:{get,set,pick} IPC handlers +
preload bridge + global.d.ts typing + a "Default project directory"
row in Sessions settings.
- FileOperations.delete_path(path, recursive=True) on the abstract
base; ShellFileOperations.delete_file rewritten to run a cross-
platform python3 -c snippet so deletes work on Windows shells (which
have no rm/rm -rf). Fallback to `python` when `python3` isn't on PATH.
- README troubleshooting block split into macOS/Linux + Windows
PowerShell recipes.
- Tightened renderer favicon links in index.html + added color-scheme
and theme-color meta.
Backend lifecycle (renderer-side mitigation)
- New noteSessionActivity() heartbeat + session.ts watchdog: an
8-minute silence on the stream auto-clears stuck $workingSessionIds
entries so "Session Busy" never gets permanently wedged. Wired into
useSessionStateCache so every state update refreshes the timer.
i18n spike
- docs/desktop-i18n-rfc.md scoping a future language-switcher PR
(recommends react-intl, audits IME/RTL/CJK in the composer +
chat bubbles, 4-PR rollout plan, ~3-4 eng-weeks for the first
non-English locale).
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(desktop): replace native OS scrollbar in portaled dropdown menus
Radix's DropdownMenuPrimitive.Portal renders content under document.body,
outside the `.scrollbar-dt` scope on #root. Whenever a menu's max-height
clipped its content (even by a pixel — common for the composer "+" menu
that opens upward near the bottom of the window), the user saw the OS's
chunky native scrollbar painted across the whole menu.
Bake a thin, slot-styled scrollbar onto DropdownMenuContent and
DropdownMenuSubContent via [scrollbar-width:thin] + WebKit pseudo-element
arbitrary variants. The submenu also gets a max-h tied to
--radix-dropdown-menu-content-available-height so long snippet lists scroll
cleanly instead of running off the bottom of the viewport. Drop the now-
redundant max-h-[60vh] override on the prompt-snippet submenu.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(desktop): unbork dropdown menu — submenu opens, parent isn't a circle
Two regressions from the previous dropdown-scrollbar fix:
- The parent menu rendered as a rounded oval. Long Tailwind v4 arbitrary-
variant strings like [&::-webkit-scrollbar-thumb]:rounded-full inside a
cn() call were being mis-resolved so the `rounded-full` leaked onto the
menu container itself. Replaced the whole tower of arbitrary variants
with a real `.dt-portal-scrollbar` class in styles.css that mirrors what
`.scrollbar-dt` already does for #root descendants. Plain CSS, no Tailwind
parser ambiguity.
- The Prompt snippets submenu didn't open. Radix publishes
--radix-dropdown-menu-content-available-height on Content but NOT on
SubContent, so the `max-h` bound to that variable computed to 0 and the
submenu collapsed to zero height. Switched SubContent to a fixed
max-h-80 (≈20rem) which is plenty for a snippet list and never collapses.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(desktop): promote prompt snippets from Radix submenu to a real Dialog
The submenu refused to open when the parent dropdown was anchored at the
bottom of the window (composer "+" button) — Radix's collision detection +
SubContent positioning was fighting us. Rather than keep tuning side /
sideOffset / collisionPadding / max-h until something stuck, replace the
DropdownMenuSub with a clicked DropdownMenuItem that opens a proper
Dialog.
Side benefits over the submenu:
- Each snippet gets a description line, so a glance is enough to pick one.
- Focus management is handled by Dialog automatically.
- Easy to grow (search, custom user snippets, categories) without
another round of Radix positioning bugs.
Also extract types/interfaces to the bottom of the file per workspace
convention.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(desktop): move cron 'New cron' button off the top bar into the body
Reverses the previous direction on cron empty-state dedup. The body
button is more discoverable for first-time users (it's anchored next to
the "No scheduled jobs yet" copy that explains the feature) and frees
the top bar from a global CTA that wasn't pulling its weight.
- Empty (zero jobs): EmptyState renders the "Create first cron" button
again, like the original design.
- Empty (search filtered out all jobs): no button, just "Try a broader
search query" copy.
- Has jobs: small inline header above the list shows `N/M active` plus
a single "New cron" button (right-aligned). The rows themselves
already cover edit/pause/trigger/delete, so this is the only "create"
affordance.
Also drop the dead `<div className="hidden">…</div>` enabledCount line
the previous patch left behind; the count is now visible in the new
header instead of hidden.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(desktop): address Copilot review on PR 37536
- sessions-settings: guard the WHOLE bridge call rather than chaining
`?.settings.foo().then(...)` — the latter throws when
`window.hermesDesktop` is undefined (non-Electron / Vitest contexts)
because the chain short-circuits to `undefined.then(...)`.
- file_operations: drop `Path.unlink(missing_ok=True)` (Py>=3.8) so the
generated delete snippet still works on remote backends running
Python 3.7. The existing FileNotFoundError handler covers the same
case and works back to 3.4.
- ansi.test.ts: add focused Vitest coverage for the SGR parser
(basic/bright colors, bold toggles, default-fg reset, coalescing,
256-color / truecolor arg consumption, non-SGR CSI drop, empty SGR
full-reset) so future refactors can't silently regress terminal
rendering.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(desktop/updates): swallow refreshDesktopVersion bridge errors
`refreshDesktopVersion()` is called best-effort with `void` from
`checkUpdates()`, `startUpdatePoller()`, and the window focus handler.
If the IPC bridge rejects (main process shutting down during reload,
bridge not yet ready on first paint), the rejection surfaces as an
unhandled promise rejection in the renderer. Wrap the call in try/catch
and return null on failure so callers can keep the existing
fire-and-forget pattern safely.
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore(desktop): drop work duplicated by other in-flight PRs
- composer/text-utils.ts: revert paste-image dedupe — PR #37596
ships the same fix with a cleaner content-key approach and a
Vitest file (text-utils.test.ts). Letting that PR own the change.
- docs/desktop-i18n-rfc.md: delete the i18n scoping RFC — PR #37568
has already shipped a working i18n surface (homegrown nanostores
`t()` helper over en/zh dictionaries), so the RFC's framework
recommendation (`react-intl`) is now obsolete and would just
contradict the implementation that's actually landing.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
The compact "<n>|content" gutter from #35368 is now the sole behavior.
Removes the HERMES_READ_GUTTER=padded escape hatch and its env lookup —
no legacy fixed-width path to maintain. Padding was pure token overhead
(~48% more tokens than bare content, ~16% more than compact) with no
measured accuracy gain in the original A/B.
- file_operations.py: drop env lookup + os import; gutter always f"{i}|{line}"
- tests: drop the padded env-override test; compact assertions retained
read_file's gutter used a fixed-width zero/space-padded prefix
(" 1|content"). The padding is pure token overhead: measured with
cl100k on real Hermes source, the padded gutter costs ~48% more tokens
than bare content and ~16% more than a compact "<n>|content" gutter,
because the leading spaces tokenize into extra tokens on every line.
Switched the default to the compact "<n>|content" form. An A/B
(Sonnet 4.6 via OpenRouter, 2 passes, 4-task battery, every claim
verified against ground truth) showed:
- padded : 4/4 PASS both passes
- compact : 4/4 PASS both passes ← keeps line-referencing + patch
- none : 3/4 PASS both passes ← dropping numbers entirely made
the model hand-count lines and answer off-by-one (33 vs 34)
So we keep the line numbers (the model genuinely uses them to reference
lines) but drop the wasteful padding — capturing ~14% of the read-token
cost with zero measured accuracy change. Dropping numbers entirely
(the larger 33% saving) is rejected: it regresses line-referencing.
patch/fuzzy_match never consumed the gutter (they match old_string text
and compute char offsets internally), so editing is unaffected. No
downstream parser keys on the fixed-width columns. HERMES_READ_GUTTER=
padded restores the legacy format for anyone relying on alignment.
Tests: updated the 3 format assertions to the compact gutter; added an
env-override test for the legacy padded format. 209 file-tool tests green.
Some Windows editors prepend an invisible UTF-8 BOM (U+FEFF) to text
files. We had no awareness of it, so: read_file surfaced a phantom
U+FEFF as the first character; patch matches against the true first
line could miss; and a write/patch round-trip silently stripped the
marker, changing the file's byte signature.
Now:
- read_file / read_file_raw strip a single leading BOM so the model
never sees it (only on the first chunk — the marker lives at byte 0).
- patch_replace strips the BOM before fuzzy-matching (so an exact
first-line match works) and its post-write verification compares
BOM-stripped content.
- write_file restores the BOM when the original file had one and the
new content doesn't, mirroring the existing line-ending preservation
(detect on disk via a cheap `head -c 3` probe or reuse pre_content,
re-prepend across the edit). Guards against double-BOM.
Mid-content U+FEFF is left alone (it's data there, not a file marker).
Tests: TestBomHandling (real LocalEnvironment) — read-strips, raw-read
strips, write preserves, no-BOM-when-original-had-none, no-double-BOM,
patch round-trip preserves, patch matches first line through a BOM,
plus helper unit tests. 208 file-tool tests green.
* Inspired by Claude Code: /compress here [N] — boundary-aware 'summarize up to here'
Adds a user-chosen compression boundary to the existing /compress command.
/compress here [N] summarizes everything except the most recent N exchanges
(default 2), which are preserved verbatim — letting the user pick the
compression boundary instead of relying on the automatic token-budget heuristic.
Inspired by Claude Code's Rewind 'Summarize up to here' action (v2.1.139,
Week 20, May 2026): https://code.claude.com/docs/en/whats-new/2026-w20
- hermes_cli/partial_compress.py: pure split/parse helpers + seam-alternation
guard (shared by CLI and gateway).
- cli.py / gateway/run.py: route 'here [N]' / '--keep N' to partial compression;
compress only the head, re-append the verbatim tail through the seam guard.
- Preserves message-flow role alternation (seam guard merges any illegal
user->user / assistant->assistant adjacency).
- Reuses the existing _compress_context session-rotation/lock machinery — no
changes to the compression core.
- Bare /compress (full) and /compress <focus> behavior unchanged.
Tests: 12 helper unit tests + 5 CLI integration tests + E2E (interleaved
tool-call transcript, degenerate/multimodal seams, real handler path).
* fix(file-tools): make write_file/patch atomic (temp-file + rename)
write_file streamed content straight into the target via `cat > path`, so
a crash, SIGKILL, or truncated pipe mid-write left the file half-written
and corrupt. patch_replace routes through write_file, so it shared the flaw.
Now writes stream into a temp file in the SAME directory and `mv` it over
the target — a real same-filesystem rename, which is atomic on POSIX and on
every terminal backend (local/docker/ssh/modal). A failed write leaves the
original byte-intact and leaks no temp file. The existing file's mode is
preserved across the swap (stat + chmod, GNU/BSD), and content still rides
stdin so there's no ARG_MAX limit. A trap cleans the temp on any error path.
Tests: added TestAtomicWrite (real LocalEnvironment, no mocks) covering
inode-change-on-overwrite, mode preservation, failed-write-leaves-original,
no-temp-leak, special chars, and patch routing. Updated two mocks in
test_file_operations.py that keyed on the literal `cat >` write command to
key on the stdin_data behavioral signal instead. 200 file-tool tests green.
Vulture + per-symbol verification (whole-repo grep incl. tests, string
literals, getattr, decorator/registry/argparse dispatch) confirmed each of
these has zero callers anywhere — not reachable via any dynamic-dispatch path,
not referenced by tests, not re-exported.
Removed:
- acp_adapter/tools.py: _build_patch_mode_content
- agent/anthropic_adapter.py: read_claude_managed_key (diagnostics-only, never called)
- agent/bedrock_adapter.py: get_bedrock_model_ids
- agent/browser_registry.py: get_active_browser_provider
- agent/chat_completion_helpers.py: _take_request_client (x2 nested closures, never invoked)
- gateway/platforms/weixin.py: _rewrite_headers_for_weixin, _rewrite_table_block_for_weixin
- hermes_cli/banner.py: _skin_branding
- hermes_cli/debug.py: _delete_hint
- hermes_cli/gateway.py: _setup_email, _setup_sms, _setup_yuanbao
(platform keys absent from the _builtin_setup_fn dispatch dict; handled by
the _setup_standard_platform fallback)
- hermes_cli/kanban_db.py: set_max_runtime, active_run
- hermes_cli/kanban_diagnostics.py: severity_of_highest, _latest_clean_event_ts
- hermes_cli/main.py: _build_provider_choices, cmd_portal
(portal subcommand is wired via portal_cli.add_parser, not this wrapper)
- hermes_cli/model_switch.py: CustomAutoResult (orphaned by the switch_model() extraction)
- hermes_cli/models.py: format_model_pricing_table, fetch_nous_account_tier
- hermes_cli/portal_cli.py: _nous_portal_base_url
- hermes_cli/proxy/server.py: handle_models_fallback (defined but never registered on the router)
- tools/computer_use/cua_backend.py: _parse_element, _is_arm_mac
- tools/file_operations.py: _get_safe_write_root (prod uses the imported
agent.file_safety.get_safe_write_root directly)
- tools/skills_tool.py: _load_category_description
Also dropped two imports left unused by the removals:
- tools/file_operations.py: get_safe_write_root alias
- tools/computer_use/cua_backend.py: import platform
Pure deletion: -551 LOC. No behavior change. Test files covering the edited
modules pass (640/640); the broader suite's pre-existing/env-dependent
failures reproduce unchanged on origin/main.
* remove Vercel AI Gateway provider and Vercel Sandbox terminal backend
Both Vercel-hosted integrations are removed end-to-end. Users on the AI
Gateway should switch to OpenRouter or one of the other aggregators
(Nous Portal, Kilo Code). Users on the Vercel Sandbox backend should
switch to Docker, Modal, Daytona, or SSH.
What's removed:
- `plugins/model-providers/ai-gateway/` provider plugin
- `hermes_cli/vercel_auth.py` Vercel-Sandbox auth helper
- `tools/environments/vercel_sandbox.py` terminal backend
- `ai-gateway` provider wiring across auth, doctor, setup, models,
config, status, providers, main, web_server, model_normalize, dump
- `vercel_sandbox` backend wiring across terminal_tool, file_tools,
code_execution_tool, file_operations, approval, skills_tool,
environments/local, credential_files, lazy_deps, prompt_builder,
cli, gateway/run
- `AI_GATEWAY_BASE_URL` constant, `_AI_GATEWAY_HEADERS` auxiliary-client
header set, run_agent base-URL header/reasoning special-cases
- `[vercel]` pyproject extra and `vercel`/`vercel-workers` from uv.lock
- env vars: `AI_GATEWAY_API_KEY`, `AI_GATEWAY_BASE_URL`, `VERCEL_TOKEN`,
`VERCEL_PROJECT_ID`, `VERCEL_TEAM_ID`, `VERCEL_OIDC_TOKEN`,
`TERMINAL_VERCEL_RUNTIME`
- Tests: deletes test_ai_gateway_models.py and
test_vercel_sandbox_environment.py; scrubs references across 23
surviving test files (no entire tests deleted unless they were
dedicated to AI Gateway / Sandbox)
- Docs: provider tables, env-var reference, setup guides, security
notes, tool config, terminal-backend tables — English plus zh-Hans
i18n parity
- `hermes-agent` skill: provider table entry and remote-backend list
What stays (intentional):
- `popular-web-designs/templates/vercel.md` — CSS design reference,
unrelated to Vercel-the-AI-product
- `x-vercel-id` in `stream_diag.py` headers — generic Vercel CDN
response header, useful diag signal on any Vercel-hosted endpoint
- `vercel-labs/agent-browser` URL in browser config — lightpanda
browser project, different OSS effort
- `userStories.json` historical contributor entry mentioning Vercel
Sandbox — archive, not active docs
Validation:
- 1153 tests in the 22 targeted files pass (`scripts/run_tests.sh`)
- Full repo `py_compile` clean
- Live import of every touched module + invariant check (no
`ai-gateway` in `PROVIDER_REGISTRY`, no `_AI_GATEWAY_HEADERS`, no
`vercel_sandbox` in `_REMOTE_TERMINAL_BACKENDS`)
* test: convert profile-count check from change-detector to invariant
The hardcoded "== 34" assertion broke when ai-gateway was removed.
Per AGENTS.md change-detector-test guidance, assert the relationship
(registry count >= number of plugin dirs) instead of a literal count.
Counts shift when providers are added/removed; that's expected.
Three granular patch-tool refinements from the Roo Code deep-dive (#507).
## Indentation preservation (fuzzy_match.py)
When fuzzy_find_and_replace matches via a non-exact strategy, the file's
indentation may differ from what the LLM sent in old_string/new_string
(common case: model sends zero-indent old/new for a method body that
lives inside an 8-space-indented class). Before this commit the
replacement was spliced in verbatim, producing a file with a broken
indent level that may still parse but is logically wrong.
The fix computes the indent delta between old_string's first meaningful
line and the matched region's first meaningful line, then re-indents
every line of new_string by that delta. Exact-strategy matches are
untouched (passthrough). Same approach as Roo Code's
multi-search-replace.ts:466-500.
## CRLF preservation (file_operations.py)
Models nearly always send tool args with bare LF endings (JSON-encoded),
but the file on disk may have CRLF (Windows-line-ending configs, .bat,
.cmd, .ini files). Before this commit:
- write_file silently normalized CRLF to LF on every overwrite
- patch produced mixed-ending files: the substituted region had LF,
the surrounding context kept CRLF
The fix detects the file's existing line endings (via pre_content if
already read for lint/LSP, otherwise a tiny head -c 4096 probe), and
normalizes the entire write to that ending. New files are written
verbatim (no detection possible).
## Per-file failure escalation (file_tools.py)
When the agent fails to patch the same file 3+ times in a row, the
existing 'old_string not found' hint isn't strong enough — the model
keeps retrying with variations against a stale view of the file.
The fix tracks consecutive failures per (task_id, resolved_path) and
injects an escalating hint after 3 failures: 'This is failure #N
patching X. Stop retrying. Either re-read fresh, use longer context,
or fall back to write_file.' Counter resets on a successful patch to
the same path.
## Validation
- 22 new tests across tests/tools/test_fuzzy_match.py (5),
test_line_ending_preservation.py (12), test_patch_failure_tracking.py (5)
- All existing tests pass (165/165 in the touched files)
- E2E verified with real _handle_patch / _handle_write_file calls
against real CRLF files and real failure loops
Closes part of #507. The remaining open items in #507 (2b start_line
hint, behavioral rules) were declined after audit:
- 2b adds schema bloat for a problem the existing 'multiple matches'
contract already handles
- Behavioral rules conflict with the personality system
Items 1, 2d, 2e, 3, 4 of #507 were already landed in earlier work.
* fix(lint): skip per-file shell linter when LSP will handle the file
`_check_lint` ran `npx tsc --noEmit FILE.ts` after every `.ts`/`.tsx`
edit. `tsc` ignores `tsconfig.json` when given an explicit file argument
(documented quirk) and defaults to no-lib / ES5, so every ES2015+ stdlib
reference reports as missing:
- `Cannot find global value 'Promise'`
- `Cannot find name 'Map' / 'Set' / 'ReadonlySet' / 'Iterable'`
- `Property 'isFinite' does not exist on type 'NumberConstructor'`
- `Module 'phaser' can only be default-imported using esModuleInterop`
- `import.meta is only allowed when --module is es2020+`
On real TypeScript projects this floods the `lint` field on
WriteResult / PatchResult with up to 25K tokens of false positives
per edit. The delta filter in `_check_lint_delta` is supposed to mask
them, but a tiny edit shifts line numbers and every phantom resurfaces
as "introduced by this edit". The result is a 1MB+ phantom-error dump
on every patch that eats the agent's context budget. Same shape for
`.go` (`go vet` outside a module) and `.rs` (`rustfmt --check` outside
a Cargo project).
PR #24168 added an LSP tier on top of this — real `tsserver` / `gopls`
/ `rust-analyzer` diagnostics surface in the separate `lsp_diagnostics`
field. But the broken shell linter kept running underneath, so the
phantom-error dump kept happening even when LSP was giving us a clean
authoritative signal.
This change short-circuits the shell linter for the structurally-broken
extensions (`.ts`, `.tsx`, `.go`, `.rs`) when an LSP server is active
and claims the file via `LSPService.enabled_for(path)`. The LSP tier
runs as before and carries the real diagnostics in `lsp_diagnostics`.
Other shell linters (`py_compile`, `node --check`) keep running
unconditionally — they're fast, file-local, and correct.
Default behavior (LSP disabled, LSP misconfigured, remote backend, file
outside a workspace) is unchanged — the existing fallback paths trigger
when `_lsp_will_handle` returns False, so users who haven't opted into
LSP get the same shell-linter behavior they had before.
Drive-by: `.tsx` was missing from the `LINTERS` table entirely, so TS
React files got no post-edit syntax check at all. Added it for
symmetry; in practice it now hits the LSP-skip path.
Tests:
- `tests/agent/lsp/test_shell_linter_lsp_skip.py` — 14 tests covering:
* skip happens for each redundant extension when LSP claims the file
(asserted by patching `_exec` to raise on any shell-linter call)
* shell linter still runs when LSP is inactive (regression guard)
* `.py` / `.js` continue to run unconditionally even with LSP active
* `_lsp_will_handle` is exception-safe: returns False on None
service, remote backend, or `enabled_for` raising
* `.tsx` is in both `LINTERS` and `_SHELL_LINTER_LSP_REDUNDANT`
- All pre-existing tests in `tests/agent/lsp/` and
`tests/tools/test_file_operations*.py` still pass (233/233).
* fix(lint): address Copilot review on #29054
Two fixes from copilot-pull-request-reviewer on PR #29054:
1. `.tsx` regression with LSP disabled
(https://github.com/NousResearch/hermes-agent/pull/29054#discussion_r3271017282)
The first revision added `.tsx` to the `LINTERS` table so that
TypeScript React files would hit the LSP skip path. Side effect:
when LSP is *disabled* (the default), `.tsx` edits would suddenly
run `npx tsc --noEmit FILE.tsx` and inherit the same phantom-error
dump this PR is supposed to fix. Pre-PR behavior was implicit
`skipped` (no `LINTERS` entry); restore that.
- Remove `.tsx` from `LINTERS`.
- Remove `.tsx` from `_SHELL_LINTER_LSP_REDUNDANT` (the skip path
is unreachable without a `LINTERS` entry — falls through to
`ext not in LINTERS` first).
- When LSP IS enabled, `.tsx` is still covered by the LSP tier
via `_maybe_lsp_diagnostics` (typescript-language-server's
`extensions` tuple includes `.tsx`), so the diagnostics still
surface — just on the `lsp_diagnostics` channel, not `lint`.
- Update test_shell_linter_lsp_skip.py to reflect this contract
(drop `.tsx` from the parametrize lists; add
`test_tsx_stays_out_of_linters_table_for_default_compatibility`
and `test_tsx_default_check_lint_returns_skipped`).
2. V4A patches dropped `WriteResult.lsp_diagnostics`
(https://github.com/NousResearch/hermes-agent/pull/29054#discussion_r3271017295)
`tools/patch_parser.py::apply_v4a_operations` calls
`file_ops.write_file()` per operation, then calls `_check_lint()`
directly afterwards — but never propagates `WriteResult.lsp_diagnostics`
to the `PatchResult`. The shell-linter skip introduced in this PR
makes the gap visible: a `.ts` / `.go` / `.rs` V4A patch with LSP
active would return `lint = {f: {skipped: True}}` and zero
diagnostics from any channel.
- `_apply_add` and `_apply_update` now return
`Tuple[bool, str, Optional[str]]` where the third element is
`WriteResult.lsp_diagnostics` (or `None` on failure / no diags).
- `_apply_delete` and `_apply_move` stay 2-tuples — they don't
produce diagnostics, no write goes through `write_file`.
- `apply_v4a_operations` accumulates per-file diagnostics blocks
and surfaces a combined block on `PatchResult.lsp_diagnostics`.
Each block already carries its `<diagnostics file="...">` header
from `LSPService.report_for_file`, so concatenation preserves
per-file attribution.
Tests added (`test_patch_parser.py::TestV4ALspDiagnosticsPropagation`):
- ADD op: `WriteResult.lsp_diagnostics` flows to `PatchResult`
- UPDATE op: same
- No diagnostics → `PatchResult.lsp_diagnostics is None` (not "")
- Multi-file patch: combined block contains every per-file block
Verification:
- Targeted test scope: 257/257 pass
(tests/agent/lsp/, tests/tools/test_file_operations*.py,
tests/tools/test_patch_parser.py)
- Wider sweep: 5400 pass; 11 failures all pre-existing on origin/main
(file_staleness / file_read_guards / file_state_registry — unrelated
macOS /var/folders tmp-path sensitivity issues, confirmed by
re-running on a clean origin/main checkout)
* docs(test): align shell-linter LSP skip docstring with .tsx behavior
Copilot review feedback (review #4324947616, comment #3271049036):
the test module docstring still listed .tsx alongside .ts/.go/.rs in
the skip contract, but .tsx is now intentionally NOT in LINTERS or
_SHELL_LINTER_LSP_REDUNDANT. Updated the bullet list to drop .tsx from
the skip contract and added a paragraph documenting why .tsx is left
out (preserves pre-PR implicit-skip behavior for LSP-disabled users;
LSP coverage still happens via _maybe_lsp_diagnostics).
* test(lsp): drop unused tmp_path from _make_fops helper
Copilot review #3271069484: the helper accepted tmp_path but never
used it. Callers still need tmp_path themselves for the file they're
asserting against, so we just drop the helper's parameter.
Pre-existing diagnostics below an edit point used to surface as 'LSP
diagnostics introduced by this edit' whenever the edit deleted or
inserted lines. The delta-filter key included the diagnostic's
range, so the same logical error reported at a different line in
the post-edit snapshot looked like a brand new diagnostic.
Concrete case: deleting 14 lines in cli.py caused Pyright errors at
lines 9873, 10590, 12413, 13004 (unrelated to the edit) to be
reported as introduced by it.
Fix: build a piecewise-linear line-shift map (via difflib's
SequenceMatcher) from pre and post content, and remap baseline
diagnostics into post-edit coordinates before the set-difference.
Diagnostics in deleted regions drop out cleanly; diagnostics below
the edit shift by the right amount; diagnostics above are untouched.
The strict (range-aware) equality key stays — so a genuinely new
instance of an identical error class at a different line still
surfaces as new.
Pieces:
- agent/lsp/range_shift.py — build_line_shift, shift_diagnostic_range,
shift_baseline. Pure functions, no LSP state.
- agent/lsp/manager.py — LSPService.get_diagnostics_sync gains an
optional line_shift kwarg; baseline is shift_baseline'd before
computing the seen-set. _diag_key keeps the strict range key.
- tools/file_operations.py — write_file captures pre_content for any
LSP-handled extension (not just LINTERS_INPROC) and passes pre/post
to _maybe_lsp_diagnostics, which builds the shift map.
- New _lsp_handles_extension helper guards the pre_content read.
Trade-offs preserved:
- Genuinely new same-class errors at different lines still surface
(content-only key would have swallowed them).
- Pre-existing errors at unshifted positions still get filtered
(covered by the strict-key path with no shift).
- Best-effort: when pre_content can't be captured (file didn't
exist, permissions), the unshifted comparison still catches
most pre-existing errors; the edge case it misses is a new file
with a non-empty baseline, which is structurally impossible.
Three follow-ups to PR #24168 found during live E2E testing on TS/bash files:
1. typescript-language-server now installs the typescript SDK (tsserver)
alongside it. Without that sibling install, initialize() failed with
"Could not find a valid TypeScript installation" and the server was
marked broken — no diagnostics ever reached the agent. New extra_pkgs
field on INSTALL_RECIPES makes that explicit and reusable for future
peer-dep cases.
2. _check_lint now treats "linter command exists on PATH but cannot
actually run" as skipped instead of error. The motivating case is
npx tsc when typescript is not in node_modules — npx prints its
"This is not the tsc command you are looking for" banner and exits
non-zero, which previously blocked the LSP semantic tier (gated on
success or skipped). Pattern-matched per base command (npx,
rustfmt, go) so genuine lint errors still flow through normally.
3. hermes lsp status now surfaces a Backend warnings section when
bash-language-server is installed but shellcheck is missing. The
server itself spawns fine but bash-language-server delegates
diagnostics to shellcheck — without it on PATH the integration
looks alive but never reports any problems. Same warning is
logged once at server spawn time.
Validation:
- 12 new tests in tests/agent/lsp/test_install_and_lint_fixes.py:
* recipe carries typescript SDK
* _install_npm passes both pkg + extras to npm CLI
* backwards compat: recipes without extras still work
* _backend_warnings quiet when bash absent / both present
* _backend_warnings fires when bash installed without shellcheck
* status output includes the Backend warnings section
* _looks_like_linter_unusable catches the npx tsc banner
* real TS type errors not misclassified as unusable
* unfamiliar linters fall through normally
* _check_lint returns skipped on npx tsc unusable
* _check_lint returns error on real tsc type errors
- Full lsp + file_operations test suite: 245/245 pass
- Live E2E:
* try_install("typescript-language-server") installs both packages
into node_modules
* write_file(bad.ts, ...) returns lint=skipped + lsp_diagnostics
with two real TS errors (was lint=error, no lsp_diagnostics)
* hermes lsp status renders the shellcheck warning when bash is
installed but shellcheck is not on PATH
* feat(lsp): semantic diagnostics from real language servers in write_file/patch
Wire ~26 language servers (pyright, gopls, rust-analyzer, typescript-language-server,
clangd, bash-language-server, ...) into the post-write lint check used by write_file
and patch. The model now sees type errors, undefined names, missing imports, and
project-wide semantic issues introduced by its edits, not just syntax errors.
LSP is gated on git workspace detection: when the agent's cwd or the file being
edited is inside a git worktree, LSP runs against that workspace; otherwise the
existing in-process syntax checks are the only tier. This keeps users on
user-home cwds (Telegram/Discord gateway chats) from spawning daemons.
The post-write check is layered: in-process syntax check first (microseconds),
then LSP semantic diagnostics second when syntax is clean. Diagnostics are
delta-filtered against a baseline captured at write start, so the agent only
sees errors its edit introduced. A flaky/missing language server can never
break a write -- every LSP failure path falls back silently to the syntax-only
result.
New module agent/lsp/ split into:
- protocol.py: Content-Length JSON-RPC framer + envelope helpers
- client.py: async LSPClient (spawn, initialize, didOpen/didChange,
ContentModified retry, push/pull diagnostic stores)
- workspace.py: git worktree walk-up + per-server NearestRoot resolver
- servers.py: registry of 26 language servers (extension match,
root resolver, spawn builder per language)
- install.py: auto-install dispatch (npm install --prefix, go install
with GOBIN, pip install --target) into HERMES_HOME/lsp/bin/
- manager.py: LSPService (per-(server_id, root) client registry, lazy
spawn, broken-set, in-flight dedupe, sync facade for tools layer)
- reporter.py: <diagnostics> block formatter (severity-1-only, 20-per-file)
- cli.py: hermes lsp {status,list,install,install-all,restart,which}
Wired into tools/file_operations.py:
- write_file/patch_replace now call _snapshot_lsp_baseline before write
- _check_lint_delta gains a third tier: LSP semantic diagnostics when
syntax is clean
- All LSP code paths swallow exceptions; write_file's contract unchanged
Config: 'lsp' section in DEFAULT_CONFIG with enabled (default true),
wait_mode, wait_timeout, install_strategy (default 'auto'), and per-server
overrides (disabled, command, env, initialization_options).
Tests: tests/agent/lsp/ -- 49 tests covering protocol framing (encode and
read_message round-trip, EOF/truncation/missing Content-Length), workspace
gate (git walk-up, exclude markers, fallback to file location), reporter
(severity filter, max-per-file cap, truncation), service-level delta filter,
and an in-process mock LSP server that exercises the full client lifecycle
including didChange version bumps, dedup, crash recovery, and idempotent
teardown.
Live E2E verified end-to-end through ShellFileOperations: pyright
auto-installed via npm into HERMES_HOME, baseline captured, type error
introduced, single delta diagnostic surfaced with correct line/column/code/
source, then patch fix removes the diagnostic from the output.
Docs: new website/docs/user-guide/features/lsp.md page covering supported
languages, configuration knobs, performance characteristics, and
troubleshooting; cli-commands.md updated with the 'hermes lsp' reference;
sidebar updated.
* feat(lsp): structured logging, backend gate, defensive walk caps
Cherry-picks the substantive ideas from #24155 (different scope, same
problem space) onto our PR.
agent/lsp/eventlog.py (new): dedicated structured logger
``hermes.lint.lsp`` with steady-state silence. Module-level dedup sets
keep a 1000-write session at exactly ONE INFO line ("active for
<root>") at the default INFO threshold; clean writes log at DEBUG so
they never reach agent.log under normal config. State transitions
(server starts, no project root for a file, server unavailable) fire
at INFO/WARNING once per (server_id, key); novel events (timeouts,
unexpected errors) fire WARNING per call. Grep recipe: ``rg 'lsp\\['``.
agent/lsp/manager.py: wire the eventlog into _get_or_spawn and
get_diagnostics_sync so users can answer "did LSP fire on this edit?"
with a single grep, plus surface "binary not on PATH" warnings once
instead of silently retrying every write.
tools/file_operations.py: backend-type gate. ``_lsp_local_only()``
returns False for non-local backends (Docker / Modal / SSH /
Daytona); ``_snapshot_lsp_baseline`` and ``_maybe_lsp_diagnostics``
now skip entirely on remote envs. The host-side language server
can't see files inside a sandbox, so this prevents pretending to
lint a file the host process can't open.
agent/lsp/protocol.py: 8 KiB cap on the header block in
``read_message``. A pathological server that streams headers
without ever emitting CRLF-CRLF would have looped forever consuming
bytes; now raises ``LSPProtocolError`` instead.
agent/lsp/workspace.py: 64-step cap on ``find_git_worktree`` and
``nearest_root`` upward walks, plus try/except containment around
``Path(...).resolve()`` and child ``.exists()`` calls. Defensive
against pathological inputs (symlink loops, encoding errors,
permission failures mid-walk) — the lint hook is hot-path code and
must never raise.
Tests:
- tests/agent/lsp/test_eventlog.py: 18 tests covering steady-state
silence (clean writes stay DEBUG), state-transition INFO-once
semantics (active for, no project root), action-required
WARNING-once (server unavailable), per-call WARNING (timeouts,
spawn failures), and the "1000 clean writes => 1 INFO" contract.
- tests/agent/lsp/test_backend_gate.py: 5 tests verifying
_lsp_local_only / snapshot_baseline / maybe_lsp_diagnostics skip
the LSP layer for non-local backends and route correctly for
LocalEnvironment.
- tests/agent/lsp/test_protocol.py: new test_read_message_rejects_runaway_header
exercising the 8 KiB cap.
Validation:
- 73/73 LSP tests pass (49 original + 18 eventlog + 5 backend-gate + 1 framer cap)
- 198/198 pass when run alongside existing file_operations tests
- Live E2E re-run with pyright still surfaces "ERROR [2:12] Type
... reportReturnType (Pyright)" through the full path, then patch
fix removes it on the next call.
* feat(lsp): atexit cleanup + separate lsp_diagnostics JSON field
Two improvements salvaged from #24414's plugin-form alternative,
keeping our core-integrated design:
1. atexit cleanup of spawned language servers
----------------------------------------------------------------
``agent/lsp/__init__.get_service`` now registers an ``atexit``
handler on first creation that tears down the LSPService on
Python exit. Without this, every ``hermes chat`` exit was
leaking pyright/gopls/etc. processes for a few seconds while
their stdout buffers drained -- they got reaped by the kernel
eventually but a watchful ``ps aux`` would catch them.
The handler runs once per process (gated by
``_atexit_registered``); idempotent ``shutdown_service``
ensures double-fire is a no-op. Errors during shutdown are
swallowed at debug level since by the time atexit fires the
user has already seen the agent's final response.
2. Separate ``lsp_diagnostics`` field on WriteResult / PatchResult
----------------------------------------------------------------
Previously the LSP layer folded its diagnostic block into the
``lint.output`` string, conflating the syntax-check tier with
the semantic tier. The agent (and any downstream parsers) now
read syntax errors and semantic errors as independent signals:
{
"bytes_written": 42,
"lint": {"status": "ok", "output": ""},
"lsp_diagnostics": "<diagnostics file=...>\nERROR [2:12] ..."
}
``_check_lint_delta`` returns to its original two-tier shape
(syntax check + delta filter); ``write_file`` and
``patch_replace`` independently fetch LSP diagnostics via
``_maybe_lsp_diagnostics`` and pass them into the new field.
``patch_replace`` propagates the inner write_file's
``lsp_diagnostics`` so the outer PatchResult carries the patch's
delta correctly.
Tests: 19 new
- tests/agent/lsp/test_lifecycle.py (8 tests): atexit registration
fires once and only once across N get_service calls; the
registered callable is our internal shutdown wrapper;
shutdown_service is idempotent and safe when never started;
exceptions during shutdown are swallowed; inactive service is
cached so we don't rebuild on every check.
- tests/agent/lsp/test_diagnostics_field.py (11 tests): WriteResult
/ PatchResult dataclass shape, to_dict include/omit semantics,
channel separation (lint and lsp_diagnostics carry independent
signals), write_file populates the field via
_maybe_lsp_diagnostics only when the syntax tier is clean,
patch_replace propagates the field forward from its internal
write_file.
Validation:
- 92/92 LSP tests pass (73 prior + 8 lifecycle + 11 diagnostics field)
- 217/217 pass with file_operations + LSP combined
- Live E2E reverified: clean writes -> both fields empty/none; type
error introduced -> lint clean (parses), lsp_diagnostics carries
the pyright reportReturnType block; patch fix -> both fields
clean again.
* fix(lsp): broken-set short-circuit so a wedged server isn't paid every write
Discovered while auditing failure paths: a language server binary that
hangs (sleep forever, no LSP traffic on stdin/stdout) caused EVERY
subsequent write to re-pay the 8s snapshot_baseline timeout. Five
writes = ~64s of dead time.
The bug: ``_get_or_spawn`` adds the (server_id, root) pair to
``_broken`` inside its inner exception handler, but when the OUTER
``_loop.run`` timeout fires, it cancels the inner task before that
handler runs. The pair never makes it to broken-set, so the next
write re-enters the spawn path and re-pays the timeout.
Fix:
- New ``_mark_broken_for_file`` helper at the service layer marks
the (server_id, workspace_root) pair broken from the OUTSIDE when
the outer timeout fires. Called from the except branches in
``snapshot_baseline``, ``get_diagnostics_sync`` (asyncio.TimeoutError
+ generic Exception). Also kills any orphan client process that
survived the cancelled future, fire-and-forget with a 1s ceiling.
- ``enabled_for`` now consults the broken-set BEFORE returning True.
Files in already-broken (server_id, root) pairs short-circuit to
False, so the file_operations layer skips the LSP path entirely
with no spawn cost. Until the service is restarted (``hermes lsp
restart``) or the process exits.
- A single eventlog WARNING is emitted on first mark-broken so the
user knows which server gave up. Subsequent edits in the same
project stay silent.
Tests: 7 new in tests/agent/lsp/test_broken_set.py — covers the
key shape (server_id, per_server_root), enabled_for short-circuit,
sibling-file skip in same project, project isolation (broken in
A doesn't affect B), graceful no-op for missing-server / no-workspace,
and an end-to-end test that snapshots after a failure and verifies
the next ``enabled_for`` returns False.
Validation:
- Live retest of the wedged-binary scenario: 5 sequential writes,
first 8.88s (the one snapshot timeout), subsequent four ~0.84s
(no LSP cost). Down from 5x12.85s = 64s before this fix.
- 99/99 LSP tests pass (92 prior + 7 broken-set)
- 224/224 pass with file_operations + LSP combined
- Happy path E2E reverified — clean write, type error introduced,
patch fix all behave correctly with the new broken-set logic.
Note: the FIRST write to a wedged binary still pays 8s (the
snapshot_baseline timeout). We could shorten that, but pyright/
tsserver normally take 2-3s and slow CI rust-analyzer can need
5+ seconds, so 8s is the conservative ceiling. Subsequent writes
are instant.
wanquanyang
Teknium
Drexuxux
Ayush Nangia
Theophilus Chinomona
fangliquanflq
Kevin Yin
kshitij
kshitijk4poor
阿泥豆
webtecnica
HexLab98
Brooklyn Nicholson
Neo Guyver
Andres Sommerhoff
Austin Pickett