The coverage warning listed bare page ranges, which tells the agent
WHERE the gaps are but not WHAT they contain — its only options were
guessing or OCRing everything. Each gap is now labeled with the last
text extracted before it (usually a section divider page), so the agent
can decide which gaps it actually needs and render/OCR only those.
Gap list capped at 20 entries with a summary line for pathological
alternating documents.
When extraction of a binary document format (.pdf, .docx, .xlsx, Office,
EPUB…) fails for a specific reason — the anydoc size cap, an encrypted or
malformed file — read_file previously swallowed the ExtractionError at
debug level and fell through to the generic 'Cannot read binary file'
guard, so the agent never saw the actionable reason (e.g. 'Document too
large to convert (N bytes, limit is 52,428,800)').
read_file now returns the specific extraction failure for binary document
formats. Fallthrough behavior is preserved where a raw read is still
useful: .ipynb (plain JSON) and converter-unavailable PDFs keep their
historical raw-read path, and the 'Unsupported document type' shape (no
extra information) keeps the generic guard.
Follow-up to #80004, where the size-cap message was being generated but
never reached the agent.
The salvaged bytes path (_extract_anydoc_bytes) bypassed the coverage
check added in #81680. Materialize transferred PDF bytes in a host temp
file for the pdftotext scan, and name the backend-visible path in the
recovery command rather than the temp file.
anydoc converts the PDF text layer only and emits no image placeholders
or page markers, so a mostly-scanned PDF extracts 'successfully' into
section headers with empty bodies — silent data loss the model cannot
detect. Count per-page text via poppler pdftotext and prepend an
EXTRACTION COVERAGE WARNING naming the empty pages and the recovery
path (pdftoppm + vision_analyze, or the ocr-and-documents skill).
Found on a 311-page HOA resale package where 198 scanned pages
(CC&Rs, Bylaws, Articles, insurance certs) vanished without a trace.
The anydoc path from #79781 passed every covered file straight to
to_markdown with no pre-check. anydoc loads the whole document through
its Rust core and the read_file char budget only applies after
conversion, so one large PDF or deck could pin a tool turn and spike
RAM.
_extract_anydoc now rejects inputs over MAX_ANYDOC_BYTES (50 MB) with
ExtractionError before calling the converter, which routes them to the
existing read_file fallthrough instead of converting. No timeout is
added: the conversion is a synchronous Rust call that cannot be
cancelled from Python, so a thread-based deadline would bound the wait
but leave the RAM burn running in the background.
The first _anydoc() load cached None on any failure (network blip,
missing wheel, pip race), so one bad first try disabled document
extraction for the rest of the process. Long-lived gateway and desktop
workers never recovered.
Failed loads now cool down for ANYDOC_RETRY_SECONDS and retry instead
of sticking, and a lock serializes first use so parallel readers cannot
double-install or race a failure into the cache. Successful loads are
still cached for the process lifetime.
read_file's auto-extraction covered only the stdlib trio (.ipynb/.docx/
.xlsx). firecrawl-anydoc (MIT, Rust core, imports as `anydoc`) converts
Word, PowerPoint, Excel — including legacy .doc/.ppt/.xls — OpenDocument,
RTF, EPUB, and PDF to clean Markdown through one shared document model.
Wiring follows the footprint ladder: no new tool, no hard dependency.
- tools/read_extract.py gains an ANYDOC_EXTENSIONS set that is active
only when the converter imports; the stdlib extractors remain
authoritative for their three formats so behavior is identical with
or without the package.
- tools/lazy_deps.py adds tool.doc_extract (firecrawl-anydoc==0.1.6),
installed on first read of such a file with prompt=False so read_file
can never block. Lazy-only for now: the package's first release was
2026-08-04, inside uv's 14-day exclude-newer quarantine, so the
mirrored pyproject extra lands after it clears.
- Any anydoc ConvertError maps to ExtractionError, falling back to the
existing path/binary handling instead of erroring the tool.
Tests: real-binding suite skips cleanly when the wheel is absent
(verified: 15 passed/3 skipped without it, 18 passed with it), plus an
absent-dep contract class that pins the fallback regardless of local
install state.
Second, deeper pass over tools/gateway/hermes_cli plus first pass over
the trees wave 1 missed (acp, acp_adapter, skills, computer_use, docker,
dashboard, conformance, monitoring, secret_sources, hermes_state,
providers). Same rubric as wave 1 (AGENTS.md test policy); security,
alternation/caching invariants, issue-number regressions, and E2E kept.
Real test-quality fixes found and rooted out along the way:
- tests/tools/test_command_guards.py made real auxiliary-LLM HTTPS calls
(DEFAULT_CONFIG smart-approval leaked in) — pinned approval
mode=manual via autouse fixture: 17.4s → 0.4s.
- test_model_switch_custom_providers.py / test_user_providers_model_switch.py
silently probed live provider catalogs (~2s/test) — stubbed
cached_provider_model_ids/provider_model_ids/fetch_api_models.
- test_telegram_noise_filter.py: 15-platform copy-paste matrix over
shared gateway.run logic → 3 representative platforms (55s → 3.9s).
- test_gateway_shutdown.py: stop()'s 5s interrupt-deadline loop spun on
MagicMock agents — interrupt.side_effect now clears _running_agents
(22s → 1.0s).
- test_gateway_inactivity_timeout.py poll-harness timings shrunk 3-5x
(24s → 1.1s); test_mcp_stability.py backoff/SIGTERM-grace sleeps
patched (15.4s → 2.5s); test_async_delegation.py negative-drain wait
5s → 0.5s.
- test_telegram_init_deadline.py: loop-block margin restored to 1.0s
with rationale comment — the watchdog-dump assertion needs the loop
blocked well past deadline+grace under parallel load (flaked once in
the 40-worker verification run at a 0.2s margin).
Verification: full hermetic suite via scripts/run_tests.sh —
2,438 files, 21,718 tests passed, 0 failed, 293.9s wall.
Suite totals vs original baseline: 46,820 → 19,757 test functions
(−57.8%), wall 583.5s → 293.9s (−50%), subprocess CPU 13,564s → 11,623s.
Teknium
fangliquanflq
Adolanium