Commit Graph

74 Commits

Author SHA1 Message Date
Teknium 411a07481b feat(skills): add decorrelated review lenses to sdlc-review
Teach the kanban reviewer to vary its inspection lens per review round
instead of repeating the same framing: round 1 reads the artifact cold
before the implementer narrative, round 2 checks out and empirically
executes the work, round 3+ audits strictly against the original
acceptance criteria and every prior request_changes item. The round is
derived from the changes_requested entries already visible in the
reviewer's worker context (live-verified against build_worker_context
across two real request_review/request_changes rounds on an isolated
board). Also adds a lens-variation note for parallel delegate_task
review fan-outs. Contract test updated with section order and lens
assertions.
2026-08-10 13:04:13 -07:00
Jakub Wolniewicz b6a14d8297 docs(skills): modernize SDLC review guidance 2026-08-10 12:43:46 -07:00
Teknium 34b6bbb96a feat(skills): add bundled merge-reconciler skill for neutral multi-agent conflict resolution
Adds skills/autonomous-ai-agents/merge-reconciler — a bundled skill teaching
a neutral third-party agent to resolve git merge conflicts between two
agents' branches: gather both diffs + intents, classify each hunk
(disjoint-intent / same-question-different-answer / superseded), resolve
under an impartiality contract, verify, and hand back a per-hunk summary.
Procedure was live-tested end-to-end against a real conflict fixture.

Includes contract tests (tests/skills/test_merge_reconciler_skill.py) and a
kanban docs cross-reference (en + zh-Hans): assign a third neutral profile a
reconciliation card with both conflicted cards as parents.
2026-08-10 11:02:57 -07:00
teknium1 55982159dd feat(tests): CI-enforce skill authoring standards; clear all remaining debt
New tests/skills/test_authoring_standards.py parametrizes every bundled +
optional SKILL.md (1148 checks) against the mechanically-verifiable subset
of the hardline standards:
- required frontmatter fields (name/description/version/author/license/
  platforms) + tags
- frontmatter name == directory name
- description <= 60 chars, ends with period, no marketing words
- related_skills resolve in-repo
- no machine-local paths
- <= 100k chars
Grandfather dict for legacy debt ships EMPTY — all pre-existing violations
fixed in this PR:

- 13 frontmatter names canonicalized to their directory names (the install
  identifier); all related_skills references updated (comfyui -> stable-
  diffusion). Fixes the class behind PR #42788's report; also fixes
  here.now's invalid dot-name.
- optional-skills/devops/cli -> inference-sh-cli (dir was the generic
  'cli'; fm name was right) incl. docs pages (en + zh-Hans), catalog row,
  sidebar entry.
- pytorch-fsdp: 157k generated 'Quick Reference' dump moved to
  references/common-patterns.md; SKILL.md 159k -> 2.5k with a pointer.
- research-paper-writing: 31.7k Phase 5 drafting section moved to
  references/phase5-paper-drafting.md; SKILL.md 103k -> 71k.

Docs regenerated with scope discipline.
2026-08-08 15:43:00 -07:00
teknium1 65710ca186 chore(skills/competitor-news-monitor): cron-recipe shape + competitor-watch blueprint
Skill polish (hardline standards):
- description 247 -> 55 chars; author credits Ben Barclay (benbarclay) first
- restructured into Setup (foreground, once) / Tick (each scheduled run)
  phases with explicit cronjob(action='create') wiring and a state file
  at ~/.hermes/competitor-watches/
- dropped dangling 'change-monitor-and-notify' related_skills entry
- Hermes-tool framing (web_search, web_extract, blogwatcher for feeds)
- coverage honesty: source failure = unknown coverage, cutoff advances
  only on success

Blueprint half:
- new 'competitor-watch' Automation Blueprint (companies/categories/time/
  recurrence/deliver slots) loading the skill, [SILENT] no-news path,
  catalog now 16 blueprints; blueprints index regenerated

Tests: 12 skill tests incl. setup/tick split, coverage-honesty guards,
blueprint registration, and the catalog-wide skills-resolve invariant.
2026-08-08 14:09:22 -07:00
teknium1 91a545ab1e chore(skills/social-media-content-calendar): tighten to hardline standards, ship optional
- description 210 -> 57 chars; author credits Ben Barclay (benbarclay) first
- optional-skills/creative/ (marketing vertical, narrowest audience of
  the batch)
- dropped phantom 'image-generation-workflow' ref; visuals via the
  image_generate tool
- honest handoff language: platforms without connectors end at approved
  drafts marked handed-off, never claimed as published
- tests (10) incl. phantom-ref and honest-handoff guards
- docs regen scoped: per-skill page + one catalog row + one sidebar line
2026-08-08 12:05:19 -07:00
teknium1 99fa93035d chore(skills/weekly-review-planning): hardline polish + wire task blueprints to their skills
Skill polish:
- description 208 -> 57 chars; author credits Ben Barclay (benbarclay) first
- connector framing (google-workspace, obsidian, notion, email-inbox-triage)
- modern section order; boilerplate folded into step-local rules

Blueprint wiring (completes the batch's recipe integration):
- weekly-review blueprint loads weekly-review-planning; prompt follows the
  skill's seven-section shape, drafts-only
- morning-brief blueprint loads google-workspace; prompt points at
  references/daily-brief.md when connected
- important-mail blueprint loads email-inbox-triage
- blueprints index regenerated

Tests: 13 skill tests + two catalog invariants (every blueprint skills=
entry resolves to a real bundled skill; the four task blueprints are wired
to their procedure skills). 32 green across both files.
2026-08-08 11:53:51 -07:00
Teknium 36f73df139 fix(skills): widen BOM-tolerant reads to all comfyui workflow-JSON call paths
The salvaged fix covered run_workflow.py and hardware_check.py. The same
locale-default read of user-authored workflow JSON exists in five sibling
scripts (auto_fix_deps, check_deps, extract_schema, health_check,
run_batch) — same bug class, same utf-8-sig fix. Invariant test extended
to pin all nine read sites.

The pdf half of the original PR is superseded: those scripts were
replaced wholesale by the clean-room rewrite (#81890), which ships
UTF-8-explicit I/O enforced by its own invariant test.
2026-08-08 11:20:51 -07:00
William Chastain 50f742f8ed fix(skills): pin text-mode file I/O to UTF-8 in comfyui and pdf skill scripts
The bundled comfyui and pdf skills read and write text files with the
locale-default codec. Both declare platforms: [linux, macos, windows], so
these paths run on hosts where that codec is not UTF-8 (cp1252 on US
Windows, cp936 on Chinese Windows, ASCII under LC_ALL=C).

Readers (the live bugs):

- run_workflow.py load_schema() and the main() workflow read parse
  user-authored JSON. A non-ASCII label crashes json.load with
  UnicodeDecodeError under a non-UTF-8 locale, and a file saved from a
  Windows GUI editor carries a UTF-8 BOM that json.load rejects with
  JSONDecodeError. Both are read as utf-8-sig, which is BOM-tolerant and
  identical to utf-8 on BOM-less input. This differs from adecb0d1a,
  which used plain utf-8 for the pdf form JSON; those payloads are
  agent-authored and BOM-free by construction, these are not.
- hardware_check.py reads /proc/version and /proc/meminfo. Both are
  Linux-gated so Windows never reaches them, but the C locale defaults to
  ASCII, so they pin plain utf-8. No BOM is possible on /proc.

Writers (not currently broken):

- extract_form_structure.py and extract_form_field_info.py write their
  JSON with json.dump, whose default ensure_ascii=True keeps the bytes
  pure ASCII. Pinned anyway because the codec is the writer's contract,
  not a property of what the caller happens to dump.

wf_path.open() is a Path.open() site that check-windows-footguns.py
deliberately does not flag (per the rule comment: "Path.open() is ALSO
affected ... and can be audited separately"). It is fixed here because it
is the same bug 156 lines from a site the checker does flag, and line 623
of the same file already uses read_text(encoding="utf-8").

Adds tests/skills/test_comfyui_skill.py with contract assertions plus two
live regressions that run load_schema in a child interpreter under
LC_ALL=C with PYTHONUTF8=0, and extends the office skill tests with writer
contract assertions. All 8 new tests fail without this change.

Note that pyproject.toml exempts skills/** from ruff PLW1514
(unspecified-encoding) because skill scripts are partly user-authored.
This change does not touch that exemption; the sites are fixed by hand,
the same way adecb0d1a did.
2026-08-08 11:20:51 -07:00
teknium1 20fece3b42 chore(skills/product-price-monitor): cron-recipe shape + price-watch blueprint
Skill polish (hardline standards):
- description 199 -> 58 chars; author credits Ben Barclay (benbarclay) first
- moved research/ -> productivity/ (consumer task, not research)
- restructured into Setup (foreground, once) / Tick (each scheduled run)
  phases with explicit cronjob(action='create') wiring and a state file
  at ~/.hermes/price-watches/
- dropped phantom 'flight-research' related_skills/prose refs
- Hermes-tool framing (web_extract, browser_navigate)

Blueprint half:
- new 'price-watch' Automation Blueprint (item/condition/interval_h/
  deliver slots) loading the skill via skills=(...), [SILENT] no-alert
  path, catalog now 15 blueprints; blueprints index regenerated

Tests: 12 skill tests incl. setup/tick split, state discipline, blueprint
registration + schedule resolution; existing blueprint catalog suite green
(33 total across both files).
2026-08-08 11:19:31 -07:00
Teknium 51570f4da7 feat: replace Anthropic office document skills with clean-room MIT implementations
The bundled docx, xlsx, powerpoint, and pdf skills were adapted from
Anthropic's document skills and carried their proprietary LICENSE.txt
(no derivatives, no redistribution). Flagged as critical license
findings by the SkillEvaluator Tier 1 scan of our skill tree.

This replaces all four with clean-room rewrites:

- Authored from scratch against library knowledge only (python-docx,
  openpyxl, python-pptx, pypdf/reportlab/pdfplumber — all MIT/BSD) by
  isolated subagents given functional specs, with an explicit
  prohibition on reading the prior skill content or anthropics/skills;
  session transcripts retained as provenance evidence.
- MIT licensed (LICENSE file per skill), author: Nous Research.
- Each skill: SKILL.md to house standards + argparse helper scripts
  with UTF-8-explicit I/O + its own e2e pytest suite (fixtures built
  on the fly, non-ASCII round-trips run under LC_ALL=C).
- All four pass SkillEvaluator Tier 1 pii+unicode+lint 3/3.

tests/skills/test_office_document_skills.py rewritten against the new
contracts: MIT/no-Anthropic-text invariants, scripts documented in
SKILL.md, argparse CLI shape, and a no-locale-default-open() check
(which caught and fixed a real gap: pdfplumber text reads are fine,
but the invariant scan now guards every future script).

Docs pages regenerated for the four skills (scoped; unrelated
generator drift excluded).

Honest capability deltas vs the old versions are documented per
SKILL.md (e.g. tracked-changes accept/reject and OOXML XSD validation
are not reimplemented; form flattening limits stated).
2026-08-08 10:46:20 -07:00
teknium1 a6ede70c2a chore(skills/meeting-action-items): tighten to hardline standards
- description 178 -> 59 chars
- author credits Ben Barclay (benbarclay) first
- dropped phantom 'Linear' connector from prose (points at notion/
  github-issues/user's tracker instead)
- Hermes-tool framing (read_file for transcripts)
- template boilerplate folded into step-local rules and skill-specific
  verification
- tests at tests/skills/test_meeting_action_items_skill.py (10 passing,
  incl. phantom-connector guard and reconcile-before-create discipline)
- docs regen scoped: per-skill page + one catalog row + one sidebar line
2026-08-08 10:06:39 -07:00
teknium1 ac662c3f71 chore(skills/google-workspace): fold daily-brief into references/, not a sibling skill
The brief is single-connector (every command comes from google-workspace),
so it ships as references/daily-brief.md with a pointer + load trigger in
SKILL.md — progressive disclosure instead of a new skill-index entry.
Contributor's procedure preserved (half-open day windows, mail-to-meeting
linking with fuzzy-match discipline, 7-section brief, bounded actions);
credit noted in the reference header. Tests (8) guard the wiring and
disciplines. Version 1.1.0 -> 1.2.0.
2026-08-08 05:59:37 -07:00
teknium1 ef9d5f8c06 chore(skills/github-issue-to-pr): de-router, fold in maintainer issue-to-PR discipline
Rewrote from a sibling-skill routing table into a skill that carries its
own procedure, and folded in generalized rules from maintainer practice:

- full-thread reads (gh issue view --comments; newest comment = live state)
- duplicate-PR sweep (issue number + keyword variants) before any code
- design-intent check via git log -p -S alongside premise reproduction
- fix the class: sweep sibling call sites into the same PR
- sabotage run: prove the regression test fails without the fix
- open the PR immediately (PR dispatches CI; CI latency is the long pole)
- close the loop: comment the issue with the PR link

Also: description 205 -> 59 chars, author credits Ben Barclay first,
modern section order, boilerplate trimmed, tests (10) incl. a
router-pattern guard, scoped docs regen.
2026-08-08 05:26:27 -07:00
teknium1 90badaa284 chore(skills/email-inbox-triage): tighten to hardline standards
- description 219 -> 58 chars
- author credits Ben Barclay (benbarclay) first
- modern section order; trimmed template safety boilerplate into
  step-local rules and a skill-specific verification checklist
- tests at tests/skills/test_email_inbox_triage_skill.py (9 passing)
- docs regen scoped: per-skill page + one catalog row + one sidebar line
2026-08-08 04:19:33 -07:00
teknium1 78bc9acdf1 chore(skills/document-to-action-items): promote to bundled tier
Fleet audit showed these task skills are commonly needed across users;
shipping bundled per Teknium's direction. Docs and tests follow the
bundled paths.
2026-08-07 10:35:42 -07:00
teknium1 7b8d0d800c chore(skills/document-to-action-items): tighten to hardline standards, move to optional
- description 214 -> 59 chars
- author credits Ben Barclay (benbarclay) first
- moved skills/productivity -> optional-skills/productivity (not a daily driver)
- dropped dangling 'linear' related_skills entry; prose points at approved destinations
- framed steps through Hermes tools (read_file, web_extract, xlsx, notion)
- trimmed template safety/verification boilerplate to doc-specific rules
- modern section order (When to Use / Procedure / Pitfalls / Verification)
- tests at tests/skills/test_document_to_action_items_skill.py (8 passing)
- docs regen scoped: per-skill page + one catalog row + one sidebar line
2026-08-07 10:35:42 -07:00
Teknium 5aa798fecc feat(skills): actual-setup optional skill + provider docs
- optional-skills/devops/actual-setup: field-tested setup skill contributed
  by shl0ms, updated for the first-class 'actual' provider (the original
  targeted a custom-provider config that now collides with the built-in name)
- docs: providers.md section + tables, environment-variables.md, quickstart.md
- tests/skills: frontmatter + first-class-provider conformance checks
2026-08-05 14:08:32 -07:00
Teknium a6defd4f15 fix(skills): match evidence quotes through markdown markup
Live-run findings from a real fact-checking task (ankylosing spondylitis
genetics, 7 authoritative sources) against the new mode:

- Verbatim check rejected a legitimate quote because web_extract returns
  markdown: the MedlinePlus sentence is "including _[ERAP1](https://...)_,
  _[IL1A](...)_" on the wire but plain prose to a reader. The agent was
  forced onto a weaker evidence fragment — the opposite of the point.
  Matching now canonicalizes inline links to their label and drops
  emphasis/code markers and backslash escapes on both sides, so quoting
  the sentence a reader sees works. Paraphrases are still rejected.
- Escaped asterisks (HLA-B\*27) no longer have to be reproduced in the
  quote, so extractor artifacts stop leaking into rendered evidence.
- New `render --replace-in <draft>`: rewrites a draft's Sources block in
  place, idempotently. Previously the only path was hand-slicing the
  file, which also tripped over the emitted heading being `## Sources`
  while the prose said "Sources:".
- verify stats: report the provenance total that the percentage is
  actually computed from (cited + [unverified], counted once), and print
  the line as `info:` instead of `warn:` when nothing is wrong. The old
  line printed 17 cited / 2 unverified next to 72%, which does not
  reconcile — a sentence can be both.
- SKILL.md documents the emitted heading, --replace-in, and exactly what
  counts as a prose sentence for --min-coverage.

7 new tests (47 total) using the real MedlinePlus/Frontiers markup;
6 sabotage runs, all red.
2026-08-02 16:18:28 -07:00
Teknium 4660673a3c feat(skills): add fact-checking mode to grounded-citations
Extends the citation ledger with evidence-backed fact-checking:

- New `quote` subcommand attaches verbatim supporting quotes to a
  source; the quote is rejected unless it appears verbatim
  (whitespace/case-insensitive) in the fetched page text, so a
  paraphrase or misremembered figure cannot masquerade as evidence.
- `verify --evidence` fails a draft whose cited sources carry no
  attached quote.
- `render --style evidence` prints each source's quotes beneath its
  URL, showing the claim -> source -> exact-text chain.
- `[unverified]` marker declares model-knowledge claims; counts toward
  --min-coverage so provenance is declared for every sentence without
  forcing fake citations.
- SKILL.md: new Fact-Checking Mode section + pitfalls; version 1.1.0.
- 10 new tests (40 total), all proven live by sabotage runs.

Covers the fact-checking/evidence-transparency half of #28289.
2026-08-02 16:18:28 -07:00
teknium1 43c79cd84a feat(skills): add grounded-citations skill for verifiable sourcing
Answers and written deliverables that rest on retrieved information now get
inline numbered citations plus a mechanically-rendered Sources list, with a
persistent ledger that makes a hallucinated citation detectable.

- skills/research/grounded-citations/scripts/sources.py: stdlib citation
  ledger (add/ingest/list/render/verify) at
  $HERMES_HOME/cache/citations/ledger.json, profile-aware, O_EXCL-locked so
  parallel subagents sharing a ledger can't collide on ids
- SKILL.md: cite-while-drafting procedure, register-at-retrieval rule,
  pitfalls, verification gate
- references/citation-formats.md: per-target placement (markdown, LaTeX/PDF,
  docx, pptx, xlsx, wiki, BibTeX handoff to research-paper-writing)
- references/grounding-rationale.md: why numbered ids (ALCE 2305.14627,
  WebGPT 2112.09332, Perplexity marker conventions), and how this relates to
  the in-process registry in PR #44833
- tests/skills/test_grounded_citations_skill.py: 30 tests
2026-08-02 14:27:47 -07:00
briandevans e75336d597 fix(cli): preserve symlinked config.yaml in the migration script's atomic write
The inlined temp-file + os.replace in openclaw_to_hermes.dump_yaml_file
replaced a symlinked config.yaml with a regular file, silently detaching
managed deployments that symlink ~/.hermes/config.yaml into a dotfiles repo or
profile package. The bare path.write_text it replaced followed the link, and
utils.atomic_replace -- which the hermes_cli twin reaches through
atomic_write_text -- resolves the link for exactly this reason (#16743).

Mirror that here: resolve the symlink before creating the temp file so the
rename lands on the real file, and fall back to copyfile on EXDEV/EBUSY now
that the target can live on another device. Covered by a regression test that
fails when the resolution is removed.

Also guard the permission-denied test for Windows: os.geteuid does not exist
there and chmod-based denial is unreliable, so skip on non-POSIX.
2026-08-02 14:55:32 +05:30
briandevans 981a598646 fix(cli): stop hermes import-agent from destroying an unreadable config.yaml
agent_import.py carries a private load_yaml_file/dump_yaml_file pair that
returned {} for an absent file AND for a present file it could not read or
parse. Three importers -- import_permission_allowlist, import_permission_denylist
and import_mcp_servers -- read config.yaml through it, merge one section into
the result, and write the whole mapping straight back. So a YAML syntax error,
a permission problem or a broken mount meant the importer replaced every
setting the user had with only the one to three keys it merged, and still
reported the item as "imported". The write was a bare path.write_text(), so an
interrupted import truncated the file instead.

Distinguish the two cases at the read. Absent, or present but empty, still
yields {} so first-time creation works. Present but unreadable, unparseable, or
not a mapping raises ConfigReadError; the three sites funnel through a new
load_target_config() that records the refusal as a per-item error and leaves the
file byte-identical. Dry-run refuses too, rather than previewing an "imported"
that would destroy the config. dump_yaml_file now writes through
utils.atomic_write_text, which the module already imports and uses for the
memory store.

This is the invariant hermes_cli/config.py already enforces for its own writers
via require_readable_config_before_write / atomic_config_write, whose docstring
names this exact root cause and calls itself "the single chokepoint every
config-update path should use". agent_import.py has its own helper pair and so
was never covered; it was the last config.yaml writer without the guard.

The identical helper pair lives in openclaw_to_hermes.py, the script this module
was ported from, where twelve config.yaml read-modify-write sites share the same
defect; fixed there too. Its refusal is recorded at the run_if_selected dispatch
point, which flips the existing _config_apply_blocked flag so the remaining
config-mutating options short-circuit instead of each rediscovering the same
unreadable file. The atomic write is inlined with tempfile + os.replace because
that script runs standalone with only the stdlib on its path.
2026-08-02 14:55:32 +05:30
Eugeniusz Gilewski 64dd865912 fix(deps): repair Google transitive security floors (#72108)
Google API and authentication packages permit vulnerable httplib2 and pyasn1
transitives, while the Workspace and Google Chat runtime installers previously
treated any importable version as sufficient. Existing environments could
therefore remain vulnerable after the project dependency pins were repaired.

Carry the fixed versions through the Google and Vertex extras, lazy feature
requirements, lockfile, and both runtime installers. Route the documented
Google Chat installation path through its maintained secure requirements
instead of an unconstrained direct pip command.

Detect stale distributions, install only unsatisfied requirements, and verify
the result before continuing. Behavioral tests cover those repair invariants
without freezing manifests, lockfiles, or complete package sets.

Related #72108
Extracted from #72840
Co-authored-by: Teknium <127238744+teknium1@users.noreply.github.com>
2026-07-31 23:18:38 -07:00
Teknium 859573d283 test: import LAZY_DEPS directly and make httplib2 pin a floor invariant
Replace the AST source-parse of tools/lazy_deps.py with a real import
(the module is importable; only setup.py legitimately needs AST since
it is a side-effectful standalone script), and convert the exact
httplib2==0.32.0 snapshot assertion into a >=0.32.0 floor so routine
future bumps don't break the test.
2026-07-31 22:28:21 -07:00
Yorkstone Supplies (sycamoregroupltd) 37e42808e7 fix(security): pin httplib2==0.32.0 in setup.py REQUIRED_PACKAGES (GHSA-j5g9-f88f-gfj3)
The previous fix (904ade32b) pinned httplib2==0.32.0 in pyproject.toml's
google extra and tools/lazy_deps.py's skill.google_workspace, but missed
a third install path: skills/productivity/google-workspace/scripts/setup.py
REQUIRED_PACKAGES. A user following the --install-deps path could still
resolve httplib2 via unpinned ranges.

This commit:
1. Exact-pins all four Google packages in REQUIRED_PACKAGES to match
   pyproject.toml and lazy_deps.py contracts exactly.
2. Adds a focused regression test that parses setup.py's REQUIRED_PACKAGES
   via AST and asserts every pin matches the other two install paths.

Changelog: fix(security), test(security)
2026-07-31 22:28:21 -07:00
Teknium 39975613b1
test: prune wave 2 + speed fixes — 28,106 → 19,757 test functions, suite wall 315s → 294s
Second, deeper pass over tools/gateway/hermes_cli plus first pass over
the trees wave 1 missed (acp, acp_adapter, skills, computer_use, docker,
dashboard, conformance, monitoring, secret_sources, hermes_state,
providers). Same rubric as wave 1 (AGENTS.md test policy); security,
alternation/caching invariants, issue-number regressions, and E2E kept.

Real test-quality fixes found and rooted out along the way:
- tests/tools/test_command_guards.py made real auxiliary-LLM HTTPS calls
  (DEFAULT_CONFIG smart-approval leaked in) — pinned approval
  mode=manual via autouse fixture: 17.4s → 0.4s.
- test_model_switch_custom_providers.py / test_user_providers_model_switch.py
  silently probed live provider catalogs (~2s/test) — stubbed
  cached_provider_model_ids/provider_model_ids/fetch_api_models.
- test_telegram_noise_filter.py: 15-platform copy-paste matrix over
  shared gateway.run logic → 3 representative platforms (55s → 3.9s).
- test_gateway_shutdown.py: stop()'s 5s interrupt-deadline loop spun on
  MagicMock agents — interrupt.side_effect now clears _running_agents
  (22s → 1.0s).
- test_gateway_inactivity_timeout.py poll-harness timings shrunk 3-5x
  (24s → 1.1s); test_mcp_stability.py backoff/SIGTERM-grace sleeps
  patched (15.4s → 2.5s); test_async_delegation.py negative-drain wait
  5s → 0.5s.
- test_telegram_init_deadline.py: loop-block margin restored to 1.0s
  with rationale comment — the watchdog-dump assertion needs the loop
  blocked well past deadline+grace under parallel load (flaked once in
  the 40-worker verification run at a 0.2s margin).

Verification: full hermetic suite via scripts/run_tests.sh —
2,438 files, 21,718 tests passed, 0 failed, 293.9s wall.
Suite totals vs original baseline: 46,820 → 19,757 test functions
(−57.8%), wall 583.5s → 293.9s (−50%), subprocess CPU 13,564s → 11,623s.
2026-07-29 13:39:40 -07:00
briandevans 8a9ab8b56b fix(cli): stop shredding an existing MEMORY.md on hermes import-agent
memories/MEMORY.md is the "§"-delimited store written by MemoryStore, not a
markdown document. parse_existing_memory_entries() fell back to
extract_markdown_entries() -- the *source* parser for CLAUDE.md / AGENTS.md --
whenever the destination held no delimiter, which is exactly the case for a
single-entry store or one that was hand-edited or shell-appended. That
extractor skips fenced code blocks, skips table rows, splits a block into one
entry per bullet and reflows paragraphs. The shredded result was then written
straight back over the user's store and reported as "Imported", with no backup
to recover from.

Parse the destination the way MemoryStore._parse_entries does: split on
ENTRY_DELIMITER only, so a store with no delimiter is one intact entry.
extract_markdown_entries() is unchanged and still used on the sources, where
it is correct.

Also restore the safety net the port dropped. The openclaw migration script
this module was ported from calls maybe_backup(destination) before rewriting a
memory store; the port did not. Snapshot the store to <name>.bak.<unix_ts>
(same naming as MemoryStore._backup_drifted_file), refuse to rewrite when the
snapshot fails, and write via temp file + atomic rename so an interrupted
import cannot leave a truncated store and a symlinked MEMORY.md stays a
symlink.

The identical fallback lives in openclaw_to_hermes.py, where it is reached
from migrate_memory() (memories/MEMORY.md and memories/USER.md) and
migrate_daily_memory(); fixed there too.
2026-07-29 16:49:07 +05:30
Teknium 9b97dea1e6 fix(skills): parse stored GitHub credentials without scanner false positives
Co-authored-by: Syed Annas <28944679+AnnasMazhar@users.noreply.github.com>
Co-authored-by: Bryan Neva <13835061+bryanneva@users.noreply.github.com>
2026-07-26 20:59:26 -07:00
solyanviktor-star adecb0d1a9 fix(skills): read OOXML parts as bytes and form JSON as UTF-8 in office skill scripts
The bundled office skills (#68595) read user documents and agent-authored
payloads with the locale-default codec:

- docx/powerpoint validators/base.py opened OOXML part XML in text mode
  before handing it to lxml. On Windows (cp1251/GBK) the bytes decode to
  mojibake that lxml then parses, so validation runs against silently
  corrupted document text; on locales where the UTF-8 bytes don't decode
  the validator crashes with UnicodeDecodeError instead of validating.
  Opening as bytes lets lxml honor the encoding declared in the XML prolog.

- The pdf form scripts (fill_fillable_fields, fill_pdf_form_with_annotations,
  create_validation_image, check_bounding_boxes) read the fields JSON the
  agent authors — UTF-8 by construction — with the locale codec, so
  non-ASCII form values (any Cyrillic/CJK/accented input) either crash or
  get written into the user's PDF as mojibake. The json.dump writers use
  ensure_ascii=True and were already safe; only the readers needed pinning.

Adds a contract test asserting every document/payload reader is
locale-independent, plus a live regression test that runs
check_bounding_boxes.py on a non-ASCII fields.json under a forced
non-UTF-8 locale — it fails without the fix on both POSIX (C locale)
and Windows (cp1251 chokes on the 0x98 byte of U+2018).
2026-07-24 17:10:39 -07:00
chancelu 049af61d64 fix: handle non-UTF-8 files in OpenClaw migration script 2026-07-24 17:10:39 -07:00
Teknium 6441b05888 fix(skills): keep xurl SKILL self-contained; move x_search routing to gated surfaces
Follow-up on the salvaged commit: the xurl skill loads even when x_search
isn't registered (check_fn-gated on xAI credentials), so per the
cross-toolset reference rule the skill must not name it. Replaced the
skill's x_search routing block and workflow step with skill-native
wording (raw engageable posts, authenticated context, write-evidence
rule). The cross-surface comparison stays on surfaces where both are
known to exist: x-search feature docs, toolset description, tools-config
setup note, and the x_search tool schema (kept generic, no tool names).
Rewrote the routing tests to pin the placement contract, including that
the skill never names credential-gated surfaces.
2026-07-23 21:06:47 -07:00
Julien Talbot b9b100da11 docs(xai): clarify x_search vs xurl routing without schema cross-refs
Make the x_search / xurl boundary explicit in the skill, feature docs,
toolset metadata, setup note, and reference pages, while keeping the
model-facing x_search schema generic (no static xurl name).

Regression tests assert behavioral routing invariants rather than frozen
prose snapshots. Drop the stale CI-only plugin/hangup hunks already on
main so this rebases cleanly.
2026-07-23 21:06:47 -07:00
iammuhammadfurqan 2978a9e9c2 fix(skills): rename pinecone-research, shorten descriptions, add scripts + tests
- Rename research/pinecone to pinecone-research (distinct from mlops/pinecone)
- Shorten mlops/pinecone description to <=60 chars
- Add rag_pipeline.py and memory_manager.py scripts
- Add test_pinecone_research_skill.py (10 tests: frontmatter, scripts, naming)
- Remove unconditional description: prefix strip from extract_skill_description()
2026-07-23 21:06:33 -07:00
teknium1 e321b83392 feat(skills): add tldraw-offline agent scripting skill
Optional skill for driving the tldraw offline desktop app via its local
HTTP control API (the same curl-based path the app's own agent skills use
for Codex/Claude Code/Cursor/Gemini) — read the canvas, make live edits,
and write embedded document scripts.

Grounded in the app's bundled script-context.d.ts and agent playbook, and
in the real tldraw SDK v5 shape schema:
- document-script contract: export default function ({ editor, helpers, signal })
- HTTP API: /api/search, /api/doc/:id/exec, /api/doc/:id/script-workspace,
  /api/doc/:id/script-status (bearer token from server.json, re-read per call)
- shape schema table validated against @tldraw/tlschema (scripts/validate_shapes.mjs, 3/3)
- interactive-UI example (scripts/counter.js) + diagram-generation (scripts/main.js)
- honest verification boundary: click->state logic verified via /exec dispatch
  (0->1->2->1->0), with documented host caveats (inotify watcher, Electron
  background-click rejection)

Tests: tests/skills/test_tldraw_offline_skill.py (15 passing).
2026-07-23 08:07:00 -07:00
Teknium afb7bf6a5a
feat(skills): bundle docx, xlsx, and pdf office skills; refresh powerpoint (#68595)
Non-technical users asking for Word docs, spreadsheets, or PDF work had
no bundled skill coverage — docx/xlsx creation required discovering and
installing hub skills, and PDF manipulation had no skill at all beyond
OCR extraction and nano-pdf edits.

- skills/productivity/docx: create (docx-js), edit (unzip -> XML -> zip),
  tracked changes, comments, validation. Adapted from anthropics/skills.
- skills/productivity/xlsx: openpyxl creation/editing, mandatory
  LibreOffice recalc gate, formula-compatibility rules, financial-model
  conventions. Points at optional excel-author for finance-grade work.
- skills/productivity/pdf: merge/split/rotate/watermark/encrypt, form
  filling (AcroForm + flat overlay scripts), text/table extraction,
  reportlab creation, forms.md + reference.md companions.
- skills/productivity/powerpoint: synced to current upstream pptx skill —
  richer pptxgenjs corruption footguns, template workflow, validate.py +
  validators + thumbnail.py, font-substitution QA guidance; drops the
  stale pack.py/editing.md/pptxgenjs.md workflow files.
- Cross-linked ocr-and-documents, nano-pdf, excel-author via
  related_skills so each office skill routes to its siblings.
- deliverable-mode docs mention the new skills; regenerated per-skill
  docs pages, catalogs, and sidebar.
- tests/skills/test_office_document_skills.py: frontmatter contracts,
  referenced-script existence, schema-map integrity, cross-link
  resolution, script compilation.

E2E validated: docx create->render->edit->validate, xlsx recalc
(SUM + _xlfn.TEXTJOIN evaluate correctly), pdf create->merge->extract,
pptx generate->validate->thumbnail.
2026-07-21 05:39:23 -07:00
teknium1 5f171e36ab chore(skills/mcp-oauth-remote-gateway): move to optional-skills + modernize
- Move skills/mcp/ -> optional-skills/mcp/ (niche remote-deployment
  workflow; bundled tier is for daily-driver skills)
- Frontmatter: description 421 -> 57 chars, add platforms gating
  [linux, macos] (bash pipelines + gateway hosts), credit Ben Barclay
  first in author
- Document the built-in flow's own escape hatches (paste-back prompt,
  ssh -N -L port-forward) as cheap first fallbacks before manual
  token surgery; scope the skill to no-TTY messaging-gateway contexts
- Frame execution through the terminal / execute_code tools
- Tests: tests/skills/test_mcp_oauth_remote_gateway_skill.py covers
  all four diagnostic branches, atomic --write persistence, 0600
  perms, httpx UA on the wire, no-secrets-in-stdout, and frontmatter
  invariants (9 passing)
- Regen auto-gen docs page + one-line catalog row + sidebar entry
  (scoped; unrelated generator drift reverted)
2026-07-16 01:28:32 -07:00
SHL0MS 5218c8a1d3 blocked-tail pass: blind opt-out default, email fallback, peopleconnect delete-wipes-suppression
from a live blocked-sites pass (no PII):
- posture shift: blind opt-out is the DEFAULT, not a fallback -- submit on every site with an
  accessible removal channel even without first confirming a listing (own identifiers to the broker's
  own official channel = still least-disclosure). guided flows double as the authoritative search.
- blocked-form rule: when a form is automation-hostile (hard captcha / cloudflare / datadome /
  slide-to-verify), default to the broker's CITED rights-email rather than recording blocked.
- captcha policy clarified: never defeat behavioral/token/slider challenges; ok to read a static
  distorted-text or plain-arithmetic captcha on the subject's own opt-out; stop if the whole
  submission is rejected after a correct answer (fingerprinting the automation, not grading it).
- intelius/peopleconnect: delete-wipes-suppression is field-confirmed -- a deletion-complete email
  means the suppression is gone and the subject re-lists cluster-wide; re-run suppression and verify
  the Control step reads "suppressed". guided-mode session persists; DOB is an <input type=date>.
- new records: addresses.json (intelius front-end, cluster-covered) and socialcatfish.json
  (cited rights-email lane + automation-hostile form).
- new references/site-playbooks.md: per-site game-plan matrix (8 blocked-tail sites), the meta-search
  no-op skip-list (idcrawl/lullar/yasni/webmii/namesdir/itools/skipease), and the infopay /
  peopleconnect backend clusters. OSINT-list triage taxonomy added to methods.md.
- state-machine.md: fixed doc drift + documented submitted->not_found illegal (resolves as
  awaiting_processing), blocked->submitted via action_selected, operator_manual_check, --evidence & pitfall.

tests: standalone 99, PR 97 (+1 cluster-coverage regression); ruff + windows-footguns clean.
2026-07-03 15:21:04 -04:00
SHL0MS a35ac25437 add `cdp`: launch/detect operator chrome over CDP for phase-2 browser + webmail
phase-2 work (sending webmail, clearing session-bound gates like peopleconnect guided-mode) needs
the operator's own logged-in browser, not a cloud browser. new `pdd.py cdp`:
- finds chrome/chromium/brave/edge (macos/linux/windows), launches it detached on a dedicated debug
  profile ($HERMES_HOME/chrome-debug) with --remote-debugging-port, waits for the port, prints the
  CDP endpoint (webSocketDebuggerUrl)
- `--check`: report whether a debug browser is already live (never double-launches)
- `--print`: emit the exact command for the operator to run themselves
- doctor, SKILL.md, and methods.md all point at it
- windows-safe detach (start_new_session on posix, DETACHED_PROCESS on windows); stdlib only

tests: standalone 98, PR 96 (+6 cdp); ruff + windows-footguns clean.
2026-07-03 13:16:06 -04:00
SHL0MS f8e36f0f31 field-report fixes: dob pre-warn, .env creds, show cmd, false-positive guards
from a live run (NY subject, 43 brokers):
- fanout default 8->5 (8+ batches time out)
- setup/doctor read $HERMES_HOME/.env so creds hermes already loads are detected
- new `show <subject> <broker>`: reads back case state+evidence for cheap parent re-verify
- intelius: requires.dob + 5-step guided-mode gate; planner pre-warns when dob is missing
- rehold.json: property-record != PII (an address-only match is not_found, not removable)
- tps/fps: match_signal_notes tell the scanner to ignore SEO-templated titles
- methods.md: browser backends (scan vs execute + operator chrome over CDP), property/SEO callouts
- doctor: warn when browser email-mode pairs with a cloud scan backend (needs operator chrome/CDP)
- ledger: found->not_found retract (false-positive), blocked->human_task_queued
- autopilot: indirect-exposure web-form fallback; drop a stray f-string

tests: standalone 92 pass; ruff clean.
2026-07-03 13:05:28 -04:00
SHL0MS aaea22f89c fix(unbroker): suppress-first for PeopleConnect (deletion undoes suppression)
PeopleConnect is the exception to deletion-beats-suppression: "DELETE MY
USER DATA" also deletes suppressions on file, and deletion does not stop the
people-search sites from showing you (public records re-list). Suppression
is the effective lever and must be maintained.

- intelius.json: deletion.prefer=false; playbook/quirks/notes rewritten with
  the verbatim privacy-center language; delete is the data-purge-only path.
- autopilot: honor deletion.prefer -> prefer_suppression when false.
- methods.md / SKILL.md / README: exception called out.
- tests updated + prefer-flag routing test (86 tests).
2026-07-02 22:29:10 -04:00
SHL0MS c2828f2b9b feat(skills): add security/unbroker (autonomous data-broker removal)
unbroker finds where a consenting person's info is exposed across data
brokers and people-search sites and files the removals, running as far as
each site allows and handing only genuinely human-only steps (hard CAPTCHA,
gov-ID, phone, fax) back as an end-of-run digest.

- Deterministic stdlib CLI (scripts/pdd.py) owns config, dossiers+consent,
  the broker DB, tier planning, the ledger, email, and the autonomous
  action queue; the agent scans/submits with native tools (web_extract,
  browser_*, delegate_task, cronjob, terminal).
- Verify-before-disclose, least-disclosure (never volunteers SSN), consent
  gate, opaque ids, optional age-at-rest encryption, file-locked ledger.
- Jurisdiction-aware (CCPA/CPRA, GDPR, generic); CA DROP one-shot covers
  the state registry (~545) in a single request; BADBOOL + curated
  people-search coverage; scheduled re-scan for re-listing.
- No CAPTCHA-solving services or anti-bot bypass; browser email mode needs
  no stored password.
- 85 hermetic tests (tests/skills/test_unbroker_skill.py; SMTP/IMAP via
  injected fakes, registry via CSV fixtures). Ships placeholder data only.

Broker dataset adapted from BADBOOL (Yael Grauer, CC BY-NC-SA 4.0).
2026-07-02 21:16:10 -04:00
Teknium ff08e60c63
feat(skills): add cloudflare-temporary-deploy optional skill (#50849)
* chore: re-trigger CI (workflows did not dispatch on prior head)

* feat(skills): add cloudflare-temporary-deploy optional skill

Optional web-development skill teaching the agent to deploy a Worker to a
live workers.dev URL with no Cloudflare account via 'wrangler deploy
--temporary' (Wrangler 4.102.0+). Cloudflare provisions a throwaway,
claimable account valid for 60 minutes — ideal for an autonomous
write->deploy->verify loop with no OAuth/signup hard stop.

- SKILL.md: when/when-not, prereqs (unauth requirement, version floor),
  step-by-step deploy + verify flow, product limits table, pitfalls
  (hidden flag, stale global wrangler, auth-present error, rate limits,
  workers.dev edge cache), verification.
- scripts/parse_deploy_output.py: stdlib-only parser extracting live URL,
  claim URL, account name/state, expiry, deploy status from wrangler output.
- tests/skills/test_cloudflare_temporary_deploy_skill.py: 16 tests incl.
  a real-output regression case.

Verified live end-to-end: temporary account created with no creds,
deployed to a live URL, curl confirmed body, redeploy reused the account.
2026-06-22 12:14:30 -07:00
Teknium 7130d60861
feat(providers): remove google-gemini-cli + google-antigravity OAuth providers (#50492)
* feat(providers): remove google-gemini-cli + google-antigravity OAuth providers

Google now actively bans accounts for third-party tools that piggyback on
Gemini CLI / Antigravity / Code Assist OAuth, and because abuse prevention
sits at a backend layer the ban can extend to the entire Google account
(Gmail/Drive), with a second violation being permanent.
Ref: https://github.com/google-gemini/gemini-cli/discussions/20632

Removes both OAuth inference providers entirely (modules, provider profiles,
auth/runtime/config/models wiring, the /gquota Code Assist quota command,
the antigravity-cli optional skill, desktop + docs surface in en + zh-Hans).
The API-key 'gemini' provider (GOOGLE_API_KEY/GEMINI_API_KEY against
generativelanguage.googleapis.com) is unaffected and stays fully supported.

* fix(skills): keep the antigravity-cli skill — only the OAuth provider is removed

The antigravity-cli optional skill orchestrates the external `agy` binary as
a coding-agent tool via the terminal tool — it does NOT wrap Hermes inference
through the banned google-antigravity OAuth provider, so it carries none of
the account-ban risk that motivated removing that provider. Restore the skill,
its docs page, the sidebar entry, and the optional-skills catalog row. The
google-antigravity / google-gemini-cli inference providers stay fully removed.
2026-06-21 19:53:27 -07:00
Ben Barclay 7c00ffd92c
fix(google-workspace): fall back to uv when venv has no pip (#39516)
The Hermes Docker image's venv is built with `uv sync`, which does not
bootstrap pip into the venv. When the google-workspace setup script needs
to install its deps and the running interpreter has no pip,
`sys.executable -m pip install` dead-ends with "No module named pip"
(reported via Discord support).

install_deps() now falls back to `uv pip install --python <interpreter>`
when the pip path fails and uv is on PATH. uv installs into the exact
interpreter the script is running under without needing pip present, so
the pip-less venv self-heals (e.g. a dep evicted on image update, or a
build without the [google]/[all] extra). On environments with neither
pip nor uv, the [google] extra hint is printed as before.

Verified E2E against nousresearch/hermes-agent:latest: under the venv
python with a missing dep, --install-deps now prints "Dependencies
installed." and exits 0 instead of failing.

Adds TestInstallDeps regression coverage: pip path, uv fallback,
uv-not-consulted-when-pip-works control, and both no-installer-available
and uv-also-fails failure cases.
2026-06-05 13:30:02 +10:00
helix4u 51a2c07016 fix(skills): document xurl X Article ingestion 2026-06-03 15:11:57 -07:00
Donovan Yohan 8bd00607dc fix(google-workspace): handle Gmail header casing case-insensitively
Normalize Gmail API message header names to lowercase before lookup so
gmail get/search/reply populate to/subject/from regardless of the casing
the message was stored with. Emit conventional MIME header casing
(To/Subject/Cc/From) on send and reply.

Fixes #34806

Co-authored-by: Donovan Yohan <donovan-yohan@users.noreply.github.com>
2026-05-30 02:38:18 -07:00
kshitijk4poor 66827f8947 chore: prune unused imports and duplicate import redefinitions
Remove unused imports (F401) and duplicate/shadowed import
redefinitions (F811) across the codebase using ruff's safe
autofixes. No behavioral changes -- imports only.

- ~1400 safe autofixes applied across 644 files (net -1072 lines)
- __init__.py re-exports preserved (excluded from F401 removal so
  public re-export surfaces stay intact)
- Re-exports that are imported or monkeypatched by tests but look
  unused in their defining module are kept with explicit # noqa:
  F401 (gateway/run.py load_dotenv; run_agent re-exports from
  agent.message_sanitization, agent.context_compressor,
  agent.retry_utils, agent.prompt_builder, agent.process_bootstrap,
  agent.codex_responses_adapter)
- Unsafe F841 (unused-variable) fixes deliberately skipped -- those
  can change behavior when the RHS has side effects
- ruff lints remain disabled in pyproject.toml (only PLW1514 is
  selected); this is a one-time cleanup, not a config change

Verification:
- python -m compileall: clean
- pytest --collect-only: all 27161 tests collect (zero import errors)
- core entry points import clean (run_agent, model_tools, cli,
  toolsets, hermes_state, batch_runner, gateway)
- static scan: every name any test imports directly from an edited
  module still resolves
2026-05-28 22:26:25 -07:00
Zyrixtrex 87c6edc1d0 fix(skills): add timeout to Google OAuth urlopen calls 2026-05-19 00:11:44 -07:00
kshitij 5fba236644
chore: ruff auto-fix PLR6201 resweep — tuple → set in membership tests (#27355)
Six days after #23937 (608 fixes) the codebase had accumulated 241 new
PLR6201 violations. Same mechanical `x in (...)` → `x in {...}` fix,
same zero-risk profile: set lookup is O(1) vs O(n) for tuple and the
two are semantically equivalent for hashable scalar membership tests.

All 241 instances fixed via `ruff check --select PLR6201 --fix
--unsafe-fixes`, zero remaining. Every changed value is a hashable
scalar (str/int/None/enum/signal); no risk of unhashable runtime
errors. No behavior change.

Test plan:
- 119 files changed, +244/-244 (net zero) — exactly one-line edits
- `ruff check` clean afterward
- Compile checks pass on the largest touched files (cli.py, run_agent.py,
  gateway/run.py, gateway/platforms/discord.py, model_tools.py)
- Subset broad test run on tests/gateway/ tests/hermes_cli/ tests/agent/
  tests/tools/: 18187 passed, 59 pre-existing failures (verified against
  origin/main with the same shape — identical failure count, identical
  category — all xdist test-order flakes unrelated to this change)

Follows the same template as PR #23937 ([tracker: #23972](https://github.com/NousResearch/hermes-agent/issues/23972)).
2026-05-17 02:29:41 -07:00