claw-code/tests
YeonGyu-Kim 5a8d7d9d58 docs+test: cycle #29 — document + lock text-mode vs JSON-mode exit divergence
Cycle #29 dogfood found a real pinpoint: cross-mode exit code divergence.

## The Pinpoint

Dogfooding the CLI revealed that unknown subcommand errors return different
exit codes depending on output mode:

  $ python3 -m src.main nonexistent-cmd                        # exit 2
  $ python3 -m src.main nonexistent-cmd --output-format json   # exit 1

ERROR_HANDLING.md documented the exit-code contract (1=parse, 2=timeout)
but did NOT explicitly state the contract applies only to JSON mode. Text
mode follows argparse defaults (exit 2 for any parse error), which
violates the documented contract when interpreted generally.

A claw using text mode with 'claw nonexistent' would see exit 2 and
misclassify as timeout per the docs. Real protocol contract gap, not
implementation bug.

## Classification

This is a DOCUMENTATION gap, not a behavior bug:
- Text mode follows argparse convention (reasonable for humans)
- JSON mode normalizes to documented contract (reasonable for claws)
- The divergence is intentional; only the docs were silent about it

Fix = document the divergence explicitly + lock it with tests.

NOT fix = change text mode exit code to 1 (would break argparse
conventions and confuse human users).

## Documentation Changes

ERROR_HANDLING.md:
1. Added IMPORTANT callout in Quick Reference section:
   'The exit code contract applies ONLY when --output-format json is
    explicitly set. Text mode follows argparse conventions.'
2. New 'Text mode vs JSON mode exit codes' table showing exact divergence:
   - Unknown subcommand: text=2, json=1
   - Missing required arg: text=2, json=1
   - Session not found: text=1, json=1 (app-level, identical)
   - Success: text=0, json=0 (identical)
   - Timeout: text=2, json=2 (identical, #161)
3. Practical rule: 'always pass --output-format json'

## Tests Added (5)

TestTextVsJsonModeDivergence in test_cross_channel_consistency.py:

1. test_unknown_command_text_mode_exits_2 — text mode argparse default
2. test_unknown_command_json_mode_exits_1 — JSON mode contract normalized
3. test_missing_required_arg_text_mode_exits_2 — same for missing args
4. test_missing_required_arg_json_mode_exits_1 — same normalization
5. test_success_path_identical_in_both_modes — success exit identical

These tests LOCK the expected divergence so:
- Documentation stays aligned with implementation
- Future changes (either direction) are caught as intentional
- Claws trust the docs

## Test Status

- 217 → 222 tests passing (+5)
- Zero regressions

## Discipline

This cycle follows the cycle #28 template exactly:
- Dogfood probe revealed real friction (test said exit=2, docs said exit=1)
- Minimal fix shape (documentation clarification, not code change)
- Regression guard via tests
- Evidence-backed, not speculative

Relationship to #181:
- #181 fixed env.exit_code != process exit (WITHIN JSON mode)
- #29 clarifies exit code contract scope (ONLY JSON mode)
- Both establish: exit codes are deterministic, but only when --output-format json

---

Classification (per cycle #24 calibration):
- Red-state bug? ✗ (behavior was reasonable, docs were incomplete)
- Real friction? ✓ (docs/code divergence revealed by dogfood)
- Evidence-backed? ✓ (test suite probed both modes, found the gap)

Source: Jobdori cycle #29 proactive dogfood — in response to Clawhip nudge
for pinpoint hunting. Found that text-mode errors return exit 2 but
ERROR_HANDLING.md implied exit 1 was the parse-error contract universally.
2026-04-29 19:33:29 +09:00
..
test_cancel_observed_field.py feat: #164 Stage B CLOSURE — turn-loop JSON + cancel_observed coverage + CLAWABLE promotion 2026-04-29 19:33:28 +09:00
test_cli_parity_audit.py feat: #164 Stage B CLOSURE — turn-loop JSON + cancel_observed coverage + CLAWABLE promotion 2026-04-29 19:33:28 +09:00
test_command_graph_tool_pool_output_format.py fix: #169 — command-graph and tool-pool now accept --output-format; diagnostic inventory JSON parity 2026-04-29 19:33:28 +09:00
test_cross_channel_consistency.py docs+test: cycle #29 — document + lock text-mode vs JSON-mode exit divergence 2026-04-29 19:33:29 +09:00
test_exec_route_bootstrap_output_format.py feat: #180 implement --version flag for metadata protocol (#28 proactive demand) 2026-04-29 19:33:29 +09:00
test_flush_transcript_cli.py fix: #166 — flush-transcript now accepts --directory / --output-format / --session-id; session-creation command parity with #160/#165 lifecycle triplet 2026-04-29 19:33:28 +09:00
test_json_envelope_field_consistency.py feat: #173 — wrap_json_envelope() applied to all 13 clawable commands (LOOP CLOSED) 2026-04-29 19:33:28 +09:00
test_load_session_cli.py feat: #173 — wrap_json_envelope() applied to all 13 clawable commands (LOOP CLOSED) 2026-04-29 19:33:28 +09:00
test_parse_error_envelope.py fix: #179 — JSON mode now fully suppresses argparse stderr + preserves real error message 2026-04-29 19:33:28 +09:00
test_porting_workspace.py feat: #173 — wrap_json_envelope() applied to all 13 clawable commands (LOOP CLOSED) 2026-04-29 19:33:28 +09:00
test_run_turn_loop_cancellation.py fix: #164 Stage A — cooperative cancellation via cancel_event in submit_message 2026-04-29 19:33:28 +09:00
test_run_turn_loop_continuation.py fix: #163 — remove [turn N] suffix pollution from run_turn_loop; file #164 timeout-cancellation followup 2026-04-29 19:33:28 +09:00
test_run_turn_loop_permissions.py fix: #159 — run_turn_loop no longer hardcodes empty denied_tools; permission denials now parity-match bootstrap_session 2026-04-29 19:33:28 +09:00
test_run_turn_loop_timeout.py fix: #164 Stage A — cooperative cancellation via cancel_event in submit_message 2026-04-29 19:33:28 +09:00
test_session_store.py fix(#160): harden delete_session contract — idempotency, race-safety, typed partial-failure 2026-04-29 19:33:28 +09:00
test_show_command_tool_output_format.py fix: #167 — show-command and show-tool now accept --output-format flag; CLI parity with session-lifecycle family 2026-04-29 19:33:28 +09:00
test_submit_message_budget.py fix: #162 — budget-overflow no longer corrupts session state in submit_message 2026-04-29 19:33:28 +09:00
test_submit_message_cancellation.py fix: #164 Stage A — cooperative cancellation via cancel_event in submit_message 2026-04-29 19:33:28 +09:00