anthropic-skills/skills/skill-creator/scripts
Varun Sahni c3d2ee7b0e fix: fix two bugs in run_eval.py causing 0% trigger rates
Bug 1: When the child process exits, the remaining stdout buffer was read
but never parsed. The loop broke immediately after reading the leftover data,
discarding any trigger evidence in the final bytes. Fixed by parsing the
remaining buffer after the read loop exits.

Bug 2: Parallel workers shared a single .claude/commands/ directory, so
each worker saw every other worker's temporary skill files. With N workers,
each had roughly a 1/N chance of being credited for its own trigger. Fixed
by giving each worker a unique subdirectory (eval-worker-N) so workers are
isolated.

Closes #1552

## Changes
- skills/skill-creator/scripts/run_eval.py:
  - Extracted line-parsing logic into _parse_buffer_lines() helper
  - Helper returns None when more data is needed, True/False for definitive verdict
  - Added post-loop buffer parsing to fix Bug 1
  - Added worker_id parameter to run_single_query to fix Bug 2
  - Each worker writes to .claude/commands/eval-worker-N/ subdirectory
  - run_eval() passes worker_idx % num_workers as worker_id
  - Added from __future__ import annotations for Python 3.9 compatibility

## Testing
- Unit tests for _parse_buffer_lines in test_fixes.py: 9 tests covering
  stream events, fallback assistant messages, state continuation, and
  edge cases. All pass.
- Verified AST parsing succeeds on Python 3.9.6
2026-08-14 02:14:39 +05:30
..
__init__.py chore: export latest skills (#465) 2026-02-24 20:28:38 -08:00
aggregate_benchmark.py chore: export latest skills (#465) 2026-02-24 20:28:38 -08:00
generate_report.py chore: export latest skills (#465) 2026-02-24 20:28:38 -08:00
improve_description.py skill-creator: drop ANTHROPIC_API_KEY requirement from description optimizer (#547) 2026-03-06 12:06:23 -08:00
package_skill.py chore: export latest skills (#465) 2026-02-24 20:28:38 -08:00
quick_validate.py Update skill-creator and make scripts executable (#350) 2026-02-06 13:19:32 -08:00
run_eval.py fix: fix two bugs in run_eval.py causing 0% trigger rates 2026-08-14 02:14:39 +05:30
run_loop.py skill-creator: drop ANTHROPIC_API_KEY requirement from description optimizer (#547) 2026-03-06 12:06:23 -08:00
utils.py chore: export latest skills (#465) 2026-02-24 20:28:38 -08:00