hermes-agent/scripts
Teknium ced8e30217 feat(scripts): reproducible core-toolset A/B eval harness (toolperf_abeval)
Ships the hard A/B evaluation used for the August 2026 core-toolset
performance batch (#77056) as a reusable harness: 9 error-inducing trap
tasks derived from measured production waste classes, two-arm
PYTHONPATH-only comparison, ATOF-trace-based scoring, resume-safe
batteries.

Hardened from the original one-off: paths de-hardcoded (ABEVAL_ROOT /
ABEVAL_HOME), encoding= on all file IO, startup crashes retry on resume
instead of polluting cells, post-hoc grading fix for err_inline_script
baked in. Live-smoked end to end (baseline arm, qwen3-coder-30b,
err_multi_dir: exit 0, correct on-disk verification, resume record
written).
2026-08-05 13:43:30 -07:00
..
ci fix(ci): follow artifact download redirect without auth 2026-08-05 11:16:18 -04:00
lib fix(install): restore installs — engines floor no shipping toolchain can meet 2026-08-01 22:56:52 -05:00
observability fix(monitoring): drain terminal lifecycle events 2026-07-24 18:54:45 +00:00
sandbox feat(dev-sandbox): support fake installer / fake main / git clones 2026-08-04 17:36:26 -04:00
tests test(install): exercise 8.3 normalization by running install.ps1, not by parsing it 2026-08-04 15:34:32 -06:00
toolperf_abeval feat(scripts): reproducible core-toolset A/B eval harness (toolperf_abeval) 2026-08-05 13:43:30 -07:00
whatsapp-bridge fix(whatsapp): guard bridge reconnect against hangs and unhandled rejections 2026-08-03 10:03:57 +05:30
LIVETEST_README.md test(tool-search): add live A/B harness, drop checked-in transcripts 2026-05-29 02:04:12 -07:00
add_contributor.py fix(scripts): accept legacy consecutive-hyphen GitHub logins in add_contributor 2026-07-24 22:40:15 -07:00
analyze_livetest.py fix: remove dead f-string prefixes via ruff F541 (216 sites) (#52336) 2026-07-05 13:42:46 -07:00
audit_pr_attribution.py feat(ci): auto-fixable contributor attribution — audit_pr_attribution.py + gate points at it 2026-08-02 15:26:59 -07:00
benchmark_browser_eval.py
build_model_catalog.py feat(models): catalog-labeled silent default — GLM-5.2 marked "default": true in the model catalog 2026-07-15 00:10:31 -07:00
build_skills_index.py fix(skills-hub): include owner in ClawHub source URLs and add retry on 429 (#51236) 2026-07-31 22:33:11 -07:00
capture-cage-terminal.sh feat(devshell): add terminal UI capture tools (#69629) 2026-07-22 17:28:21 -04:00
check-windows-footguns.py fix(windows): sweep remaining bare read_text/write_text sites + linter rule 2026-07-24 17:10:39 -07:00
check_subprocess_stdin.py fix(windows): sweep remaining bare read_text/write_text sites + linter rule 2026-07-24 17:10:39 -07:00
contributor_audit.py chore: release v0.20.0 (2026.8.3) 2026-08-03 09:57:23 -07:00
dev-sandbox.sh feat(dev-sandbox): support fake installer / fake main / git clones 2026-08-04 17:36:26 -04:00
discord-voice-doctor.py
docker_config_migrate.py refactor: config auto-migration support floor at v12 + deprecated shim retirement 2026-07-29 16:44:31 -07:00
docker_rebootstrap_nous_session.py fix(auth): apply newer hosted bootstrap session (#64612) 2026-07-16 08:52:47 +10:00
generate_conformance_vectors.py feat(conformance): vector generator — native renderers as executable spec (#71666) 2026-07-26 16:52:25 +10:00
hermes-gateway
install.cmd fix(docs): update all install instructions everywhere 2026-06-04 21:07:45 -04:00
install.ps1 fix(install): resolve 8.3 profile aliases so a built desktop app stops reporting failure 2026-08-04 15:34:32 -06:00
install.sh fix(install): sync the lockfile engines mirrors with the manifests 2026-08-01 23:28:28 -05:00
install_psutil_android.py revert(windows): roll back terminal-popup PRs #53791 #53810 #53829 (#53853) 2026-06-27 15:59:00 -07:00
iso-certify.py fix(windows): sweep remaining unguarded text-mode subprocess sites codebase-wide 2026-07-24 11:45:57 -07:00
keystroke_diagnostic.py
kill_modal.sh
lint_diff.py fix(windows): sweep remaining bare read_text/write_text sites + linter rule 2026-07-24 17:10:39 -07:00
micro_compaction_report.py feat(agent): report context occupancy, not just tokens saved 2026-07-31 17:44:19 +05:30
profile-tui.py fix(windows): sweep remaining bare read_text/write_text sites + linter rule 2026-07-24 17:10:39 -07:00
release.py fix(credential-pool): clear exhaustion state on key rotation (#22622) 2026-08-04 11:35:06 -06:00
run_tests.sh fix(test-runner): native Windows venv probe + glyph-safe stdio 2026-07-29 21:30:53 -07:00
run_tests_parallel.py fix(test-runner): native Windows venv probe + glyph-safe stdio 2026-07-29 21:30:53 -07:00
sample_and_compress.py fix: remove dead f-string prefixes via ruff F541 (216 sites) (#52336) 2026-07-05 13:42:46 -07:00
smoke_nemo_relay_shared_metrics.py Merge updated client resource metrics into active-install metrics 2026-08-04 15:10:41 -07:00
tool_search_livetest.py bench: Unreal-scale live benchmark — Epic's real 830 UE 5.8 schemas replayed (Opus 4.8) 2026-07-26 08:26:09 -07:00
tool_search_livetest2.py fix(lint): encode remaining write_text calls in the tool_search livetest harness 2026-07-26 11:54:03 -07:00
tool_search_livetest_ue.py fix(lint): explicit encoding on probe-file open in UE harness (PLW1514 + windows-footguns) 2026-07-26 08:26:09 -07:00
tool_search_livetest_ue_disc.py bench: discovery-bound suite — paraphrase/absence/survey tasks isolate the listing's structural advantage 2026-07-26 08:26:09 -07:00
tool_search_livetest_ue_hard.py bench: adversarial 830-tool gauntlet — confusion clusters, type-aware error mocks, strict scoring 2026-07-26 08:26:09 -07:00