hermes-agent/scripts
kshitij 2b6897f982
fix(computer-use): target Linux app windows reliably (#63725)
* fix(computer-use): target Linux app windows reliably

Resolve app filters through the canonical cua-driver MCP app metadata and join running app PIDs back to windows. Preserve an exact selected window across capture_after, support direct capture by pid/window_id, and send the active window ID for coordinate pointer actions on Linux.

Co-authored-by: annguyenNous <annguyenNous@users.noreply.github.com>
Co-authored-by: grimmjoww578 <willies578@gmail.com>
Co-authored-by: ai-ag2026 <261867348+ai-ag2026@users.noreply.github.com>

* fix(computer-use): address review on PR #63725

Address three review comments from @f-trycua:

1. type_text, press_key, and hotkey now carry _active_window_id and
   fail closed when it is missing. Previously they sent only the PID,
   so CUA Driver fell back to the first window for that PID — input
   could reach the wrong window in multi-window apps.

2. Coordinate scroll x/y are now capability-gated behind
   input.scroll.coordinates. CUA Driver 0.7.1 Linux schema rejects
   x/y on scroll; omitting them when the driver doesn't advertise
   support avoids the schema rejection while still routing via
   window_id.

3. Windows are sorted by z_index descending (higher = front, per CUA
   Driver semantics) instead of ascending. Null z_index (Wayland) is
   coerced to 0 in _ingest_windows so it doesn't crash the sort and
   sorts to the back instead of being selected as the capture target.

---------

Co-authored-by: LeonSGP43 <cine.dreamer.one@gmail.com>
Co-authored-by: annguyenNous <annguyenNous@users.noreply.github.com>
Co-authored-by: grimmjoww578 <willies578@gmail.com>
Co-authored-by: ai-ag2026 <261867348+ai-ag2026@users.noreply.github.com>
2026-07-16 00:34:20 +05:30
..
ci
…
lib
…
tests
…
whatsapp-bridge
…
LIVETEST_README.md
…
analyze_livetest.py
…
benchmark_browser_eval.py
…
build_model_catalog.py
…
build_skills_index.py
…
check-windows-footguns.py
…
check_subprocess_stdin.py
…
contributor_audit.py
…
dev-sandbox.sh
…
discord-voice-doctor.py
…
docker_config_migrate.py
…
docker_rebootstrap_nous_session.py
…
hermes-gateway
…
install.cmd
…
install.ps1
…
install.sh
…
install_psutil_android.py
…
keystroke_diagnostic.py
…
kill_modal.sh
…
lint_diff.py
…
profile-tui.py
…
release.py
…
run_tests.sh
…
run_tests_parallel.py
…
sample_and_compress.py
…
tool_search_livetest.py
…