Commit Graph

21 Commits

Author SHA1 Message Date
Christian Moellmann 895a5fbaee fix(interviews): accept stringified ints in all 4 subagent validators
Real LLMs (observed with anthropic/claude-haiku-4-5 on a 23-agent run)
sometimes return Likert values as JSON strings ('3' not 3). The 4 subagent
validators rejected this with isinstance(v, int), losing ~30% of agents at
N=23. Added a shared coerce_int helper in base.py that accepts ints and
numeric strings, rejects bools/floats/garbage, and is now used by:

- Longitudinal: response values 1-5
- Diversity: Q-sort placements -3..+3 and 6 Likert axes 1-7
- Delphi: R2 and R3 importance/plausibility 1-5
- Scenario: 4 dimensions 1-7

Validators now coerce in place so downstream code sees ints regardless of
the wire format. Added 8 tests (4 unit on coerce_int + 4 per-subagent
contract tests showing stringified values are accepted).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 14:03:34 +02:00
Christian Moellmann 6a53c110b7 feat(interviews): capture raw LLM output on schema-validation failures
Adds SchemaValidationFailure exception carrying both retry attempts' raw
output, so audit.jsonl preserves what the model actually said when an
agent's response can't be coerced into the instrument schema. Lets us
diagnose persona-vs-format failures without re-running. Two new tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 13:40:43 +02:00
Christian Moellmann 6e1489fe08 fix(interviews): wire Zep updater/memory/hooks correctly for production runs (C1-C5)
Five tightly-coupled fixes that were causing the interview subsystem to silently
degrade in production:

- C1+C2: `_build_orchestrator` now resolves `graph_id` from
  `SimulationManager().get_simulation(sim_id).graph_id` (the real persisted
  state) instead of a `graph_id.txt` that nothing in the codebase writes.
  `ZepGraphMemoryUpdater(graph_id=...)` is now called with the correct
  positional argument; the bare `try/except Exception` that was swallowing the
  TypeError is replaced with a narrow fallback that logs explicitly.
- C3: `SimulationManager._on_ready_hooks` / `_on_completed_hooks` are now
  class-level (mirroring `SimulationRunner._on_completed_callbacks`).
  Hooks registered at app startup now survive across the per-request
  `SimulationManager()` instances created by the Flask API, so the T0
  longitudinal auto-survey actually fires.
- C4: `ZepGraphMemoryUpdater` gains an explicit `add_text_episode(graph_id, text)`
  method for synchronous text writes. `InterviewZepWriter._emit` no longer
  silently falls back to a dict-shaped `add_activity` call that the real
  implementation rejects (its `add_activity` requires an `AgentActivity`
  dataclass).
- C5: `FileSystemPersonaProvider.agent_to_entity()` builds an
  `{agent_id: zep_entity_uuid}` map from the persisted profile files; the map
  is now passed to `ZepMemoryProvider` so `get_entity_with_context` is called
  with real Zep UUIDs instead of `str(agent_id)`. To make this work,
  `OasisProfileGenerator._save_reddit_json` and `_save_twitter_csv` now persist
  `source_entity_uuid` (Reddit JSON: optional field; Twitter CSV: appended
  column).

Tests: 51 unit + 2 integration pass (was 40 + 2). New tests lock in each fix:
- `test_hooks_survive_across_instances` (C3)
- `test_build_orchestrator_reads_graph_id_from_state` (C1+C2+C5)
- `test_build_orchestrator_falls_back_when_state_missing` (C1+C2)
- `test_emit_uses_add_text_episode_with_graph_id`,
  `test_emit_raises_when_updater_lacks_add_text_episode`,
  `test_real_updater_exposes_add_text_episode` (C4)
- `test_agent_to_entity_from_reddit_json`,
  `test_agent_to_entity_empty_when_no_field`,
  `test_agent_to_entity_falls_back_to_twitter_csv`,
  `test_agent_to_entity_reddit_takes_precedence` (C5)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 13:27:47 +02:00
Christian Moellmann 6b04ea5c27 feat(interviews): auto-trigger lifecycle hooks + bridge SimulationRunner→Manager on COMPLETED
- Add backend/app/services/interviews/lifecycle.py with install_hooks() that
  registers on_ready (pre-survey) and on_completed (post-survey + synthesis)
  daemon-thread callbacks on a SimulationManager.
- Add SimulationRunner.register_on_completed() / _fire_on_completed() so
  external callbacks can be notified when _monitor_simulation transitions to
  COMPLETED (both exit-code-0 path and simulation_end event path).
- Wire both in app/__init__.py: create singleton SimulationManager, install
  lifecycle hooks, and register its _notify_on_completed with SimulationRunner.
- Add test_lifecycle.py: verifies install_hooks registers one callable for each
  of ready and completed.
- All 40 unit tests + 2 integration tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 12:51:13 +02:00
Christian Moellmann 61f13a806d test(interviews): end-to-end pipeline test + content-aware LLM stubs for all 4 subagents
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 12:40:53 +02:00
Christian Moellmann 52bae0a3da feat(interviews): Flask blueprint /api/interview with task-based async + CSV export
Add /api/interview blueprint with POST pre/post/rerun, GET status/results/synthesis/export.csv endpoints. Background tasks tracked by UUID in module-level dict. Add register_blueprints() helper to api/__init__.py and wire app factory through it. Add UPLOADS_DIR to Config with env-override default.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 12:37:04 +02:00
Christian Moellmann bc07170dbf feat(interviews): persona + Zep memory adapters bridging existing services to interview subsystem
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 12:34:03 +02:00
Christian Moellmann d79c81d2b7 feat(interviews): synthesiser emits cross-method report + tidy CSV + limitations section
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 12:32:14 +02:00
Christian Moellmann 3322bcb20c feat(interviews): on_ready / on_completed hook registry on SimulationManager
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 12:29:30 +02:00
Christian Moellmann b3e2039817 feat(interviews): orchestrator with two-phase lifecycle, parallel fan-out, isolated failures
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 12:27:47 +02:00
Christian Moellmann cca67365b9 feat(interviews): Zep writer adapts add_activity/add_text_episode for per-agent + aggregate episodes
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 12:24:33 +02:00
Christian Moellmann 998cf1ac27 feat(interviews): JSONL/JSON storage layout with run_id directories and latest pointer
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 12:23:06 +02:00
Christian Moellmann ae4941df8e feat(interviews): scenario subagent with 4 futures × 4 dimensions + polarity matrix
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 12:21:21 +02:00
Christian Moellmann 5d7111b54e feat(interviews): Delphi subagent (3 rounds: open, rate, revise) + convergence metrics
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 12:19:07 +02:00
Christian Moellmann 75762ccc18 feat(interviews): diversity subagent with Q-sort + 6 Likert axes + PCA/k-means typology
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 12:16:21 +02:00
Christian Moellmann 0fcb815cde feat(interviews): longitudinal subagent + 12-item Likert instrument
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 12:12:46 +02:00
Christian Moellmann 289a0cff56 feat(interviews): StakeholderInterviewer base with in-character prompting and schema retry
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 12:10:01 +02:00
Christian Moellmann eb3c3629c1 feat(interviews): LLM stub mode for deterministic CI tests
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 12:08:29 +02:00
Christian Moellmann 29be754ff4 feat(interviews): YAML instrument loader with pydantic validation and hash freezing
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 12:06:52 +02:00
Christian Moellmann f1898b4eac feat(interviews): add pydantic models for instruments and responses
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 12:04:45 +02:00
Christian Moellmann f63bc5542a chore(interviews): add deps and pytest scaffold for interview subsystem
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 12:00:09 +02:00