Efficiency review (measured with timeit probes) found two unbounded
costs on adversarial inputs:
- The masked-range rebuild copied the whole string once per range
(O(n*k)): 50k tiny heredocs took 1.7s. Replaced with a single-pass
segment join over the (sorted, non-overlapping) ranges: 152ms, and
newlines are now counted on the original command instead of
re-slicing.
- After the last '<<' occurrence no opener can start, but the scanner
still walked the remaining text per-char: one heredoc followed by a
1MB tail cost ~150ms. An rfind bound breaks out of the unit loop
once the scan passes it: 0.3ms.
Typical commands are unaffected (the '<<' fast path already returns
first). 30/30 guard tests pass; mutation check re-run on the final
stack (no-op mutation -> 11 tests fail, restore -> green).
The previous commit's regex-based stripper removed EVERY heredoc body,
which review flagged as bypassable: a fake '<<EOF' marker inside a
comment or quoted string enters the unterminated path and swallows a
later REAL background operator, and unquoted ('cat <<EOF' — expansion
runs) or shell-consumed ('bash <<'EOF'' — body IS shell) bodies are
executable content that must stay visible to the guard.
Replace it with tools/shell_heredoc.strip_inert_heredoc_bodies(), a
conservative shell-state scanner: a body is masked ONLY when every
delimiter on the opener is quoted (no expansion), every heredoc is
terminated by an exact delimiter line, the opener composes a single
command (no list/pipeline operators, no nested $()/backtick/process-
substitution scope), and the consumer is an allowlisted non-shell
interpreter (python/osascript/cat). Anything ambiguous is returned
unchanged — a false positive on exotic syntax is acceptable; hiding a
real background operator is not. Masked bodies become newlines so line
structure is preserved for MULTILINE regexes.
The helper is a standalone stdlib-only module (precedent:
tools/ansi_strip.py) because the same heredoc-as-data false-positive
class exists in the blocked-command regex checks (#83104) and the
gateway lifecycle guard (#81721/#79835, cron/lifecycle_guard.py) —
which must not import the terminal-tool module graph.
Adapted from Wolfram Ravenwolf's security-hardened rework of #63788
(69c7663c6de6b6cb05bf99203fa39673efe01ccf); test scenarios for the
bypass cases derive from his suite.
Co-authored-by: Wolfram Ravenwolf <github.com@wolfram.ravenwolf.de>
kshitij