From c5cc59905300a016a7911c9a7887ed8798cc1825 Mon Sep 17 00:00:00 2001 From: anadoris007 Date: Mon, 20 Apr 2026 20:41:14 +0530 Subject: [PATCH] docs: add narrative layer design spec and implementation plan - docs/superpowers/specs/2026-03-26-narrative-layer-design.md - docs/superpowers/plans/2026-03-26-narrative-layer-foundation.md Spec extends MiroFish into a creative storytelling platform via a new narrative layer over the existing OASIS simulation engine. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../2026-03-26-narrative-layer-foundation.md | 1468 +++++++++++++++++ .../2026-03-26-narrative-layer-design.md | 532 ++++++ 2 files changed, 2000 insertions(+) create mode 100644 docs/superpowers/plans/2026-03-26-narrative-layer-foundation.md create mode 100644 docs/superpowers/specs/2026-03-26-narrative-layer-design.md diff --git a/docs/superpowers/plans/2026-03-26-narrative-layer-foundation.md b/docs/superpowers/plans/2026-03-26-narrative-layer-foundation.md new file mode 100644 index 00000000..7bc25459 --- /dev/null +++ b/docs/superpowers/plans/2026-03-26-narrative-layer-foundation.md @@ -0,0 +1,1468 @@ +# Narrative Layer Foundation — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build the foundational narrative layer — translate OASIS simulation actions into story prose, track extended character state, and display the generated story in a new Vue view. + +**Architecture:** New `backend/app/services/narrative/` module with three services (translator, character engine, action mapper). New Flask blueprint at `/api/narrative/*`. New Vue view `StoryTimelineView.vue` polls for narrative updates. All state is file-based under `uploads/simulations/{sim_id}/narrative/`. + +**Tech Stack:** Python 3.11, Flask, Vue 3 + Vite, existing LLM client (`backend/app/utils/llm_client.py`), existing retry utility (`backend/app/utils/retry.py`). + +**Scope boundary:** This plan covers ONLY translator + character engine + timeline view. God Mode, timeline branching, world state, enhanced input, and export are separate follow-on plans. + +--- + +## File Structure + +### New files + +| File | Responsibility | +|---|---| +| `backend/app/services/narrative/__init__.py` | Package init | +| `backend/app/services/narrative/action_mapper.py` | Maps OASIS action types to narrative verbs/interpretations | +| `backend/app/services/narrative/character_engine.py` | Extended character profiles, emotional state, arc detection | +| `backend/app/services/narrative/narrative_translator.py` | Reads actions.jsonl, generates prose via LLM | +| `backend/app/services/narrative/story_store.py` | File I/O for narrative state (story_beats.json, characters.json, translator_state.json) | +| `backend/app/api/narrative.py` | Flask blueprint with narrative endpoints | +| `backend/tests/test_action_mapper.py` | Tests for action mapping | +| `backend/tests/test_character_engine.py` | Tests for character state updates | +| `backend/tests/test_narrative_translator.py` | Tests for translation pipeline | +| `backend/tests/test_story_store.py` | Tests for file I/O | +| `frontend/src/api/narrative.js` | Frontend API client for narrative endpoints | +| `frontend/src/views/StoryTimelineView.vue` | Story reading UI | +| `frontend/src/components/StoryBeat.vue` | Single story paragraph component | +| `frontend/src/components/CharacterCard.vue` | Character summary with emotion indicators | + +### Modified files + +| File | Change | +|---|---| +| `backend/app/api/__init__.py` | Add `narrative_bp` blueprint registration | +| `backend/app/__init__.py` | Register narrative blueprint | +| `frontend/src/router/index.js` | Add `/story/:simId` route | + +--- + +## Task 1: Action Mapper — Static Mapping Table + +**Files:** +- Create: `backend/app/services/narrative/__init__.py` +- Create: `backend/app/services/narrative/action_mapper.py` +- Test: `backend/tests/test_action_mapper.py` + +- [ ] **Step 1: Create empty package init** + +Create `backend/app/services/narrative/__init__.py` with a single empty line. + +- [ ] **Step 2: Write the failing test** + +Create `backend/tests/test_action_mapper.py`: + +```python +from app.services.narrative.action_mapper import map_action_to_verb, get_narrative_context + +def test_create_post_maps_to_speech(): + result = map_action_to_verb("CREATE_POST") + assert result == "speaks" + +def test_like_post_maps_to_agreement(): + result = map_action_to_verb("LIKE_POST") + assert result == "agrees with" + +def test_unknown_action_returns_fallback(): + result = map_action_to_verb("UNKNOWN_ACTION") + assert result == "does something" + +def test_get_narrative_context_returns_interpretation(): + ctx = get_narrative_context("REPOST") + assert "rumor" in ctx.lower() or "amplifies" in ctx.lower() +``` + +- [ ] **Step 3: Run test to verify it fails** + +Run: `cd backend && uv run pytest tests/test_action_mapper.py -v` +Expected: FAIL with ImportError (module not yet created). + +- [ ] **Step 4: Implement action_mapper.py** + +Create `backend/app/services/narrative/action_mapper.py`: + +```python +"""Maps OASIS action types to narrative verbs and interpretations.""" + +ACTION_TO_VERB = { + "CREATE_POST": "speaks", + "LIKE_POST": "agrees with", + "REPOST": "spreads word of", + "QUOTE_POST": "responds to", + "FOLLOW": "shows loyalty to", + "DO_NOTHING": "observes in silence", + "CREATE_COMMENT": "engages with", + "DISLIKE_POST": "disapproves of", + "LIKE_COMMENT": "validates", + "DISLIKE_COMMENT": "dismisses", + "SEARCH_POSTS": "investigates", + "SEARCH_USER": "seeks out", + "MUTE": "ignores", +} + +ACTION_TO_NARRATIVE = { + "CREATE_POST": "Character speaks, declares, or announces", + "LIKE_POST": "Character agrees, supports, or nods", + "REPOST": "Character spreads rumor, amplifies, or gossips", + "QUOTE_POST": "Character responds, debates, or challenges", + "FOLLOW": "Character allies with or shows loyalty to", + "DO_NOTHING": "Character reflects, observes, or waits", + "CREATE_COMMENT": "Character engages in dialogue", + "DISLIKE_POST": "Character opposes, confronts, or disapproves", + "LIKE_COMMENT": "Character validates a response", + "DISLIKE_COMMENT": "Character dismisses or mocks", + "SEARCH_POSTS": "Character investigates or seeks information", + "SEARCH_USER": "Character seeks out a specific person", + "MUTE": "Character avoids, ignores, or shuns", +} + + +def map_action_to_verb(action_type: str) -> str: + return ACTION_TO_VERB.get(action_type, "does something") + + +def get_narrative_context(action_type: str) -> str: + return ACTION_TO_NARRATIVE.get(action_type, "Character takes an unknown action") +``` + +- [ ] **Step 5: Run tests to verify they pass** + +Run: `cd backend && uv run pytest tests/test_action_mapper.py -v` +Expected: 4 passed. + +- [ ] **Step 6: Commit** + +```bash +git add backend/app/services/narrative/__init__.py backend/app/services/narrative/action_mapper.py backend/tests/test_action_mapper.py +git commit -m "feat(narrative): add action-to-verb mapping for OASIS actions" +``` + +--- + +## Task 2: Story Store — File I/O for Narrative State + +**Files:** +- Create: `backend/app/services/narrative/story_store.py` +- Test: `backend/tests/test_story_store.py` + +- [ ] **Step 1: Write the failing test** + +Create `backend/tests/test_story_store.py`: + +```python +import os +import tempfile +import pytest +from app.services.narrative.story_store import StoryStore + +@pytest.fixture +def temp_sim_dir(): + with tempfile.TemporaryDirectory() as d: + sim_dir = os.path.join(d, "sim_test123") + os.makedirs(sim_dir) + yield sim_dir + +def test_save_and_load_story_beats(temp_sim_dir): + store = StoryStore(temp_sim_dir) + beat = {"round": 1, "prose": "Elena spoke.", "characters": ["elena"]} + store.append_beat(beat) + + beats = store.get_all_beats() + assert len(beats) == 1 + assert beats[0]["prose"] == "Elena spoke." + +def test_translator_state_tracks_offset(temp_sim_dir): + store = StoryStore(temp_sim_dir) + assert store.get_file_offset("twitter") == 0 + + store.set_file_offset("twitter", 1024) + assert store.get_file_offset("twitter") == 1024 + +def test_get_beat_by_round(temp_sim_dir): + store = StoryStore(temp_sim_dir) + store.append_beat({"round": 1, "prose": "First"}) + store.append_beat({"round": 2, "prose": "Second"}) + + beat = store.get_beat_by_round(2) + assert beat["prose"] == "Second" + +def test_narrative_dir_created_on_first_write(temp_sim_dir): + store = StoryStore(temp_sim_dir) + store.append_beat({"round": 1, "prose": "test"}) + + narrative_dir = os.path.join(temp_sim_dir, "narrative") + assert os.path.isdir(narrative_dir) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd backend && uv run pytest tests/test_story_store.py -v` +Expected: FAIL with ImportError. + +- [ ] **Step 3: Implement story_store.py** + +Create `backend/app/services/narrative/story_store.py`: + +```python +"""File-based persistence for narrative state.""" +import os +import json +from typing import Optional + + +class StoryStore: + """Manages narrative/*.json files for a single simulation.""" + + def __init__(self, sim_dir: str): + self.sim_dir = sim_dir + self.narrative_dir = os.path.join(sim_dir, "narrative") + self.beats_path = os.path.join(self.narrative_dir, "story_beats.json") + self.translator_state_path = os.path.join(self.narrative_dir, "translator_state.json") + self.characters_path = os.path.join(self.narrative_dir, "characters.json") + + def _ensure_dir(self): + os.makedirs(self.narrative_dir, exist_ok=True) + + def append_beat(self, beat: dict) -> None: + self._ensure_dir() + beats = self.get_all_beats() + beats.append(beat) + with open(self.beats_path, "w", encoding="utf-8") as f: + json.dump(beats, f, ensure_ascii=False, indent=2) + + def get_all_beats(self) -> list: + if not os.path.exists(self.beats_path): + return [] + with open(self.beats_path, "r", encoding="utf-8") as f: + return json.load(f) + + def get_beat_by_round(self, round_num: int) -> Optional[dict]: + for beat in self.get_all_beats(): + if beat.get("round") == round_num: + return beat + return None + + def get_file_offset(self, platform: str) -> int: + if not os.path.exists(self.translator_state_path): + return 0 + with open(self.translator_state_path, "r", encoding="utf-8") as f: + state = json.load(f) + return state.get(f"{platform}_offset", 0) + + def set_file_offset(self, platform: str, offset: int) -> None: + self._ensure_dir() + state = {} + if os.path.exists(self.translator_state_path): + with open(self.translator_state_path, "r", encoding="utf-8") as f: + state = json.load(f) + state[f"{platform}_offset"] = offset + with open(self.translator_state_path, "w", encoding="utf-8") as f: + json.dump(state, f, indent=2) + + def save_characters(self, characters: list) -> None: + self._ensure_dir() + with open(self.characters_path, "w", encoding="utf-8") as f: + json.dump(characters, f, ensure_ascii=False, indent=2) + + def load_characters(self) -> list: + if not os.path.exists(self.characters_path): + return [] + with open(self.characters_path, "r", encoding="utf-8") as f: + return json.load(f) +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cd backend && uv run pytest tests/test_story_store.py -v` +Expected: 4 passed. + +- [ ] **Step 5: Commit** + +```bash +git add backend/app/services/narrative/story_store.py backend/tests/test_story_store.py +git commit -m "feat(narrative): add StoryStore for file-based narrative persistence" +``` + +--- + +## Task 3: Character Engine — Initial Emotional State + +**Files:** +- Create: `backend/app/services/narrative/character_engine.py` +- Test: `backend/tests/test_character_engine.py` + +**Learning mode note:** Step 4 of this task asks **you (the user)** to write the emotional delta rules. Those rules encode your creative judgment about how characters react emotionally — it's the kind of decision that shapes storytelling quality and is better made by a human than an LLM. + +- [ ] **Step 1: Write the failing test** + +Create `backend/tests/test_character_engine.py`: + +```python +from app.services.narrative.character_engine import ( + CharacterEngine, + create_initial_character, + apply_action_emotional_delta, +) + +def test_create_initial_character_has_neutral_emotions(): + char = create_initial_character(char_id="elena", name="Elena Voss") + assert char["emotional_state"]["current"]["anger"] == 0.0 + assert char["emotional_state"]["current"]["joy"] == 0.0 + assert char["emotional_state"]["current"]["trust"] == 0.5 # neutral baseline + +def test_create_initial_character_stores_name_and_id(): + char = create_initial_character(char_id="elena", name="Elena Voss") + assert char["id"] == "elena" + assert char["name"] == "Elena Voss" + +def test_apply_delta_clamps_to_zero_one_range(): + char = create_initial_character(char_id="x", name="X") + # Apply a large positive anger delta + apply_action_emotional_delta(char, "DISLIKE_POST") + apply_action_emotional_delta(char, "DISLIKE_POST") + apply_action_emotional_delta(char, "DISLIKE_POST") + apply_action_emotional_delta(char, "DISLIKE_POST") + apply_action_emotional_delta(char, "DISLIKE_POST") + assert 0.0 <= char["emotional_state"]["current"]["anger"] <= 1.0 + +def test_create_post_increases_confidence_proxy(): + # Speaking out should bump joy slightly (a proxy for confidence) + char = create_initial_character(char_id="x", name="X") + baseline_joy = char["emotional_state"]["current"]["joy"] + apply_action_emotional_delta(char, "CREATE_POST") + assert char["emotional_state"]["current"]["joy"] >= baseline_joy + +def test_dislike_post_increases_anger(): + char = create_initial_character(char_id="x", name="X") + apply_action_emotional_delta(char, "DISLIKE_POST") + assert char["emotional_state"]["current"]["anger"] > 0.0 +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd backend && uv run pytest tests/test_character_engine.py -v` +Expected: FAIL with ImportError. + +- [ ] **Step 3: Implement scaffolding with placeholder deltas** + +Create `backend/app/services/narrative/character_engine.py`: + +```python +"""Extended character profiles, emotional state, arc detection.""" +from typing import Dict + + +EMOTIONS = ["anger", "fear", "joy", "sadness", "trust", "surprise"] + +INITIAL_EMOTIONAL_STATE = { + "anger": 0.0, + "fear": 0.0, + "joy": 0.0, + "sadness": 0.0, + "trust": 0.5, + "surprise": 0.0, +} + + +def create_initial_character(char_id: str, name: str, backstory: str = "", + motivations: list = None, personality: list = None) -> dict: + """Build a new character profile with neutral emotional state.""" + return { + "id": char_id, + "name": name, + "backstory": backstory, + "motivations": motivations or [], + "personality_traits": personality or [], + "emotional_state": { + "current": dict(INITIAL_EMOTIONAL_STATE), + "history": [], + }, + "relationships": {}, + "arc": {"archetype": None, "stage": "beginning", "key_moments": []}, + } + + +# >>> USER CONTRIBUTION POINT — see Step 4 <<< +ACTION_EMOTIONAL_DELTAS: Dict[str, Dict[str, float]] = { + # TODO: Fill this in — see Task 3, Step 4 +} + + +def apply_action_emotional_delta(character: dict, action_type: str) -> None: + """Apply an emotional state change based on an action the character took.""" + deltas = ACTION_EMOTIONAL_DELTAS.get(action_type, {}) + current = character["emotional_state"]["current"] + for emotion, delta in deltas.items(): + if emotion in current: + current[emotion] = max(0.0, min(1.0, current[emotion] + delta)) + + +class CharacterEngine: + """Manages character roster for a simulation.""" + + def __init__(self, store): + self.store = store + + def initialize_from_profiles(self, oasis_profiles: list) -> list: + """Bootstrap character roster from OASIS profiles.""" + characters = [] + for profile in oasis_profiles: + char = create_initial_character( + char_id=str(profile.get("user_id", profile.get("id", ""))), + name=profile.get("name", "Unknown"), + ) + characters.append(char) + self.store.save_characters(characters) + return characters +``` + +- [ ] **Step 4: 🎯 USER CONTRIBUTION — Define Emotional Deltas** + +**This is the key creative decision.** The `ACTION_EMOTIONAL_DELTAS` dictionary maps each OASIS action to changes in the character's emotional state. Your choices here directly shape how characters feel and evolve over a story. + +**Open** `backend/app/services/narrative/character_engine.py` and fill in the `ACTION_EMOTIONAL_DELTAS` dictionary. A delta is a dict of `{emotion: float}` where the float is added to the current emotion value (clamped to 0.0–1.0). + +**Guidance — things to consider:** +- `CREATE_POST` (speaking out) — small confidence bump? small anxiety bump? +- `LIKE_POST` (agreeing) — builds trust? slight joy? +- `DISLIKE_POST` (confronting) — anger up, trust down? +- `FOLLOW` (allying) — trust up, maybe joy? +- `REPOST` (spreading rumor) — neutral? or surprise? +- `DO_NOTHING` (observing) — sadness creep? fear creep? + +Suggested range: keep deltas between -0.15 and +0.15 per action (so they accumulate gradually). Example format: + +```python +ACTION_EMOTIONAL_DELTAS: Dict[str, Dict[str, float]] = { + "CREATE_POST": {"joy": 0.05}, + "DISLIKE_POST": {"anger": 0.10, "trust": -0.05}, + # ... fill in the rest for all 13 actions in action_mapper.py +} +``` + +**Why your judgment matters:** These values are a model of human emotional reaction. An LLM would pick generic values; your intuitions as a storyteller will produce richer, more deliberate character arcs. If you want characters who spiral into darkness easily, use bigger negative deltas. If you want resilient characters, use smaller ones. + +Fill in deltas for at least these 7 core actions: `CREATE_POST`, `LIKE_POST`, `DISLIKE_POST`, `REPOST`, `QUOTE_POST`, `FOLLOW`, `DO_NOTHING`. The other 6 can use defaults (empty dicts). + +- [ ] **Step 5: Run tests to verify they pass** + +Run: `cd backend && uv run pytest tests/test_character_engine.py -v` +Expected: 5 passed. If `test_dislike_post_increases_anger` or `test_create_post_increases_confidence_proxy` fails, your deltas may be missing the relevant emotion — adjust. + +- [ ] **Step 6: Commit** + +```bash +git add backend/app/services/narrative/character_engine.py backend/tests/test_character_engine.py +git commit -m "feat(narrative): add CharacterEngine with emotional state tracking" +``` + +--- + +## Task 4: Narrative Translator — Read Actions.jsonl + +**Files:** +- Create: `backend/app/services/narrative/narrative_translator.py` +- Test: `backend/tests/test_narrative_translator.py` + +- [ ] **Step 1: Write the failing test** + +Create `backend/tests/test_narrative_translator.py`: + +```python +import os +import json +import tempfile +import pytest +from app.services.narrative.narrative_translator import read_actions_for_round + +@pytest.fixture +def actions_file(): + with tempfile.TemporaryDirectory() as d: + path = os.path.join(d, "actions.jsonl") + lines = [ + {"round": 1, "agent_id": 1, "agent_name": "Alice", "action_type": "CREATE_POST", "action_args": {"content": "Hi"}, "success": True, "timestamp": "2026-03-26T12:00:00"}, + {"round": 1, "agent_id": 2, "agent_name": "Bob", "action_type": "LIKE_POST", "action_args": {"post_id": 1}, "success": True, "timestamp": "2026-03-26T12:00:01"}, + {"event_type": "round_end", "round": 1, "timestamp": "2026-03-26T12:00:02"}, + {"round": 2, "agent_id": 1, "agent_name": "Alice", "action_type": "REPOST", "action_args": {}, "success": True, "timestamp": "2026-03-26T12:00:03"}, + ] + with open(path, "w") as f: + for line in lines: + f.write(json.dumps(line) + "\n") + yield path + +def test_read_actions_for_round_1(actions_file): + actions, next_offset = read_actions_for_round(actions_file, start_offset=0, target_round=1) + assert len(actions) == 2 + assert actions[0]["agent_name"] == "Alice" + assert actions[1]["agent_name"] == "Bob" + assert next_offset > 0 + +def test_read_actions_resumes_from_offset(actions_file): + # First read round 1 + _, offset_after_round_1 = read_actions_for_round(actions_file, start_offset=0, target_round=1) + # Then read round 2 using that offset + actions, _ = read_actions_for_round(actions_file, start_offset=offset_after_round_1, target_round=2) + assert len(actions) == 1 + assert actions[0]["action_type"] == "REPOST" + +def test_read_actions_missing_file_returns_empty(): + actions, offset = read_actions_for_round("/nonexistent/path.jsonl", start_offset=0, target_round=1) + assert actions == [] + assert offset == 0 +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd backend && uv run pytest tests/test_narrative_translator.py -v` +Expected: FAIL with ImportError. + +- [ ] **Step 3: Implement read_actions_for_round** + +Create `backend/app/services/narrative/narrative_translator.py`: + +```python +"""Reads OASIS actions.jsonl and translates them into story prose.""" +import os +import json +from typing import List, Tuple + + +def read_actions_for_round(jsonl_path: str, start_offset: int, target_round: int) -> Tuple[List[dict], int]: + """Read all actions for target_round starting at start_offset. + + Returns (actions, new_offset). new_offset is the file position right after + the round_end event (or EOF if not found yet). + """ + if not os.path.exists(jsonl_path): + return [], start_offset + + actions = [] + new_offset = start_offset + + with open(jsonl_path, "r", encoding="utf-8") as f: + f.seek(start_offset) + while True: + line_start = f.tell() + line = f.readline() + if not line: + break + try: + entry = json.loads(line) + except json.JSONDecodeError: + continue + + # Round-end event marks boundary + if entry.get("event_type") == "round_end" and entry.get("round") == target_round: + new_offset = f.tell() + break + + # Skip events, other rounds + if "event_type" in entry: + new_offset = f.tell() + continue + if entry.get("round") != target_round: + new_offset = f.tell() + continue + + actions.append(entry) + new_offset = f.tell() + + return actions, new_offset +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cd backend && uv run pytest tests/test_narrative_translator.py -v` +Expected: 3 passed. + +- [ ] **Step 5: Commit** + +```bash +git add backend/app/services/narrative/narrative_translator.py backend/tests/test_narrative_translator.py +git commit -m "feat(narrative): add actions.jsonl round reader" +``` + +--- + +## Task 5: Narrative Translator — LLM Prose Generation + +**Files:** +- Modify: `backend/app/services/narrative/narrative_translator.py` +- Modify: `backend/tests/test_narrative_translator.py` + +**Learning mode note:** Step 4 of this task asks you to shape the **prose generation prompt**. The prompt is the single biggest lever for story quality. This is where your voice as a storyteller gets encoded. + +- [ ] **Step 1: Add failing test for generate_prose** + +Append to `backend/tests/test_narrative_translator.py`: + +```python +from unittest.mock import patch, MagicMock +from app.services.narrative.narrative_translator import generate_prose + +def test_generate_prose_calls_llm_with_context(): + actions = [ + {"agent_name": "Elena", "action_type": "CREATE_POST", "action_args": {"content": "We must act."}}, + {"agent_name": "Marcus", "action_type": "DISLIKE_POST", "action_args": {}}, + ] + characters = [ + {"id": "1", "name": "Elena", "emotional_state": {"current": {"anger": 0.2, "joy": 0.0, "fear": 0.1, "sadness": 0.0, "trust": 0.5, "surprise": 0.0}}}, + {"id": "2", "name": "Marcus", "emotional_state": {"current": {"anger": 0.5, "joy": 0.0, "fear": 0.0, "sadness": 0.0, "trust": 0.3, "surprise": 0.0}}}, + ] + fake_response = "Elena's voice cut through the silence. Marcus scowled, unmoved." + + with patch("app.services.narrative.narrative_translator.call_llm") as mock_llm: + mock_llm.return_value = fake_response + result = generate_prose(actions, characters, tone="dark political thriller", previous_beats=[]) + + assert result == fake_response + # Verify the prompt included character names and action info + call_args = mock_llm.call_args + prompt = str(call_args) + assert "Elena" in prompt + assert "Marcus" in prompt + +def test_generate_prose_empty_actions_returns_placeholder(): + result = generate_prose([], [], tone="any", previous_beats=[]) + assert "quiet" in result.lower() or "pause" in result.lower() +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cd backend && uv run pytest tests/test_narrative_translator.py::test_generate_prose_calls_llm_with_context -v` +Expected: FAIL with ImportError for `generate_prose` or `call_llm`. + +- [ ] **Step 3: Add LLM client import and prose generator scaffold** + +Append to `backend/app/services/narrative/narrative_translator.py`: + +```python +from app.utils.llm_client import call_llm +from app.services.narrative.action_mapper import get_narrative_context + + +def _format_character_summary(character: dict) -> str: + emotions = character["emotional_state"]["current"] + top_emotions = sorted(emotions.items(), key=lambda kv: -kv[1])[:2] + emo_str = ", ".join(f"{e[0]}={e[1]:.1f}" for e in top_emotions) + return f"{character['name']} (feeling: {emo_str})" + + +def _format_action_line(action: dict) -> str: + name = action.get("agent_name", "Someone") + act = action.get("action_type", "UNKNOWN") + args = action.get("action_args", {}) + content = args.get("content", "") + ctx = get_narrative_context(act) + if content: + return f"- {name}: {ctx}. Content: \"{content}\"" + return f"- {name}: {ctx}" + + +# >>> USER CONTRIBUTION POINT — see Step 4 <<< +PROSE_PROMPT_TEMPLATE = """TODO: the user will define this in Step 4""" + + +def generate_prose(actions: list, characters: list, tone: str, previous_beats: list) -> str: + """Generate a narrative passage from a round's actions.""" + if not actions: + return "A quiet pause settles over the scene. No one acts; no one speaks." + + char_summaries = "\n".join(_format_character_summary(c) for c in characters) + action_lines = "\n".join(_format_action_line(a) for a in actions) + prev_prose = "\n\n".join(b.get("prose", "") for b in previous_beats[-2:]) + + prompt = PROSE_PROMPT_TEMPLATE.format( + tone=tone, + characters=char_summaries, + actions=action_lines, + previous=prev_prose or "(this is the first scene)", + ) + return call_llm(prompt) +``` + +- [ ] **Step 4: 🎯 USER CONTRIBUTION — Design the Prose Prompt** + +**This is the storytelling soul of the system.** Replace the `PROSE_PROMPT_TEMPLATE` with your prompt. The template has 4 substitution fields: `{tone}`, `{characters}`, `{actions}`, `{previous}`. + +**Guidance — key choices:** + +1. **Tense and POV**: Third-person past is most versatile. First-person or present can work but limit you. +2. **Paragraph count**: 2-4 is a good range. Too short = feels like a log. Too long = drags. +3. **Dialogue**: Should the prompt encourage dialogue when actions have `content`? Probably yes. +4. **"Show don't tell"**: Instructing the LLM to show emotions through action/dialogue rather than stating them is a classic writing lever. +5. **Tone consistency**: How hard do you push the tone? A strong instruction like "Every line should feel grim" vs a soft one like "Maintain a grim tone." +6. **Continuity**: Instruct it to continue naturally from `{previous}`. + +**Suggested structure (copy and modify):** + +```python +PROSE_PROMPT_TEMPLATE = """You are a narrative writer working in the tone of {tone}. + +Previous story context: +{previous} + +Characters in this scene: +{characters} + +Events this round: +{actions} + +Write a story passage of 2-4 paragraphs that: +- Uses third-person past tense +- Shows character emotions through action, dialogue, and internal thought (never state emotions directly) +- Weaves the events above into prose — never list them like a log +- Continues naturally from the previous context +- Maintains the tone of {tone} throughout + +Write only the prose. No headings, no preamble.""" +``` + +Feel free to modify heavily — try your own instructions, add more rules, or tighten it. The quality of every generated story depends on this prompt. + +- [ ] **Step 5: Run tests to verify they pass** + +Run: `cd backend && uv run pytest tests/test_narrative_translator.py -v` +Expected: 5 passed (3 from Task 4 + 2 new). + +- [ ] **Step 6: Commit** + +```bash +git add backend/app/services/narrative/narrative_translator.py backend/tests/test_narrative_translator.py +git commit -m "feat(narrative): add LLM-driven prose generation" +``` + +--- + +## Task 6: Narrative Translator — Translate Round Orchestration + +**Files:** +- Modify: `backend/app/services/narrative/narrative_translator.py` +- Modify: `backend/tests/test_narrative_translator.py` + +- [ ] **Step 1: Add failing test** + +Append to `backend/tests/test_narrative_translator.py`: + +```python +from app.services.narrative.narrative_translator import translate_round +from app.services.narrative.story_store import StoryStore + +def test_translate_round_produces_beat(tmp_path): + sim_dir = str(tmp_path / "sim_test") + os.makedirs(sim_dir) + platform_dir = os.path.join(sim_dir, "twitter") + os.makedirs(platform_dir) + actions_path = os.path.join(platform_dir, "actions.jsonl") + + lines = [ + {"round": 1, "agent_id": 1, "agent_name": "Alice", "action_type": "CREATE_POST", "action_args": {"content": "Hi"}, "success": True, "timestamp": "t"}, + {"event_type": "round_end", "round": 1, "timestamp": "t"}, + ] + with open(actions_path, "w") as f: + for l in lines: + f.write(json.dumps(l) + "\n") + + store = StoryStore(sim_dir) + store.save_characters([ + {"id": "1", "name": "Alice", "emotional_state": {"current": {"anger": 0, "fear": 0, "joy": 0, "sadness": 0, "trust": 0.5, "surprise": 0}}}, + ]) + + with patch("app.services.narrative.narrative_translator.call_llm") as mock_llm: + mock_llm.return_value = "Alice spoke into the void." + beat = translate_round(sim_dir, platform="twitter", target_round=1, tone="neutral") + + assert beat["round"] == 1 + assert beat["prose"] == "Alice spoke into the void." + assert "Alice" in beat.get("characters", []) + + stored_beats = store.get_all_beats() + assert len(stored_beats) == 1 +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd backend && uv run pytest tests/test_narrative_translator.py::test_translate_round_produces_beat -v` +Expected: FAIL with ImportError for `translate_round`. + +- [ ] **Step 3: Implement translate_round** + +Append to `backend/app/services/narrative/narrative_translator.py`: + +```python +from app.services.narrative.story_store import StoryStore +from app.services.narrative.character_engine import apply_action_emotional_delta + + +def translate_round(sim_dir: str, platform: str, target_round: int, tone: str = "neutral") -> dict: + """Translate a single round of simulation into a story beat.""" + store = StoryStore(sim_dir) + actions_path = os.path.join(sim_dir, platform, "actions.jsonl") + start_offset = store.get_file_offset(platform) + + actions, new_offset = read_actions_for_round(actions_path, start_offset, target_round) + + characters = store.load_characters() + previous_beats = store.get_all_beats() + + prose = generate_prose(actions, characters, tone, previous_beats) + + involved = list({a.get("agent_name") for a in actions if a.get("agent_name")}) + + # Update emotional states + char_by_name = {c["name"]: c for c in characters} + for a in actions: + char = char_by_name.get(a.get("agent_name")) + if char: + apply_action_emotional_delta(char, a.get("action_type", "")) + store.save_characters(list(char_by_name.values())) + + beat = { + "round": target_round, + "prose": prose, + "characters": involved, + "action_count": len(actions), + "platform": platform, + } + store.append_beat(beat) + store.set_file_offset(platform, new_offset) + return beat +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cd backend && uv run pytest tests/test_narrative_translator.py -v` +Expected: 6 passed. + +- [ ] **Step 5: Commit** + +```bash +git add backend/app/services/narrative/narrative_translator.py backend/tests/test_narrative_translator.py +git commit -m "feat(narrative): orchestrate round translation with state updates" +``` + +--- + +## Task 7: Flask Blueprint — Narrative API + +**Files:** +- Create: `backend/app/api/narrative.py` +- Modify: `backend/app/api/__init__.py` +- Modify: `backend/app/__init__.py` + +- [ ] **Step 1: Inspect existing API pattern** + +Read `backend/app/api/__init__.py` and `backend/app/api/simulation.py` to understand the existing blueprint registration pattern and response format. + +- [ ] **Step 2: Create narrative.py blueprint** + +Create `backend/app/api/narrative.py`: + +```python +"""Narrative Layer API endpoints.""" +import os +from flask import Blueprint, jsonify, request +from app.services.narrative.story_store import StoryStore +from app.services.narrative.narrative_translator import translate_round +from app.services.narrative.character_engine import CharacterEngine +from app.config import Config + +narrative_bp = Blueprint('narrative', __name__) + + +def _sim_dir(sim_id: str) -> str: + return os.path.join(Config.OASIS_SIMULATION_DATA_DIR, sim_id) + + +@narrative_bp.route('/story/', methods=['GET']) +def get_full_story(sim_id): + store = StoryStore(_sim_dir(sim_id)) + return jsonify({"sim_id": sim_id, "beats": store.get_all_beats()}) + + +@narrative_bp.route('/story//round/', methods=['GET']) +def get_round_story(sim_id, round_num): + store = StoryStore(_sim_dir(sim_id)) + beat = store.get_beat_by_round(round_num) + if not beat: + return jsonify({"error": "Round not translated yet"}), 404 + return jsonify(beat) + + +@narrative_bp.route('/translate', methods=['POST']) +def translate(): + data = request.get_json() + sim_id = data.get('sim_id') + round_num = data.get('round') + platform = data.get('platform', 'twitter') + tone = data.get('tone', 'neutral') + try: + beat = translate_round(_sim_dir(sim_id), platform, round_num, tone) + return jsonify(beat) + except Exception as e: + return jsonify({"error": str(e)}), 500 + + +@narrative_bp.route('/characters/', methods=['GET']) +def get_characters(sim_id): + store = StoryStore(_sim_dir(sim_id)) + return jsonify({"characters": store.load_characters()}) + + +@narrative_bp.route('/characters//init', methods=['POST']) +def initialize_characters(sim_id): + """Bootstrap characters from existing OASIS profiles.""" + import json + sim_dir = _sim_dir(sim_id) + profiles_path = os.path.join(sim_dir, 'profiles.json') + if not os.path.exists(profiles_path): + return jsonify({"error": "profiles.json not found"}), 404 + with open(profiles_path, 'r', encoding='utf-8') as f: + profiles = json.load(f) + store = StoryStore(sim_dir) + engine = CharacterEngine(store) + characters = engine.initialize_from_profiles(profiles) + return jsonify({"count": len(characters), "characters": characters}) +``` + +- [ ] **Step 3: Register blueprint in __init__.py** + +Modify `backend/app/api/__init__.py` by adding (at the bottom of existing imports): + +```python +from . import narrative +``` + +Then modify `backend/app/__init__.py` to register: + +```python +from app.api.narrative import narrative_bp +app.register_blueprint(narrative_bp, url_prefix='/api/narrative') +``` + +(Find where other blueprints are registered and add this line alongside them.) + +- [ ] **Step 4: Manual smoke test** + +```bash +cd backend && uv run python run.py & +sleep 3 +curl http://localhost:5001/api/narrative/story/nonexistent_sim +# Expected: {"sim_id": "nonexistent_sim", "beats": []} +kill %1 +``` + +- [ ] **Step 5: Commit** + +```bash +git add backend/app/api/narrative.py backend/app/api/__init__.py backend/app/__init__.py +git commit -m "feat(narrative): add narrative API blueprint" +``` + +--- + +## Task 8: Frontend API Client + +**Files:** +- Create: `frontend/src/api/narrative.js` + +- [ ] **Step 1: Inspect existing API client pattern** + +Read `frontend/src/api/simulation.js` and `frontend/src/api/index.js` to understand the existing axios usage. + +- [ ] **Step 2: Create narrative API client** + +Create `frontend/src/api/narrative.js`: + +```javascript +import api from './index.js' + +export const narrativeAPI = { + getFullStory(simId) { + return api.get(`/narrative/story/${simId}`) + }, + + getRoundStory(simId, roundNum) { + return api.get(`/narrative/story/${simId}/round/${roundNum}`) + }, + + translate(simId, round, platform = 'twitter', tone = 'neutral') { + return api.post('/narrative/translate', { sim_id: simId, round, platform, tone }) + }, + + getCharacters(simId) { + return api.get(`/narrative/characters/${simId}`) + }, + + initCharacters(simId) { + return api.post(`/narrative/characters/${simId}/init`) + }, +} +``` + +- [ ] **Step 3: Commit** + +```bash +git add frontend/src/api/narrative.js +git commit -m "feat(narrative): add frontend API client" +``` + +--- + +## Task 9: StoryBeat Component + +**Files:** +- Create: `frontend/src/components/StoryBeat.vue` + +- [ ] **Step 1: Create component** + +Create `frontend/src/components/StoryBeat.vue`: + +```vue + + + + + +``` + +- [ ] **Step 2: Commit** + +```bash +git add frontend/src/components/StoryBeat.vue +git commit -m "feat(narrative): add StoryBeat component" +``` + +--- + +## Task 10: StoryTimelineView + +**Files:** +- Create: `frontend/src/views/StoryTimelineView.vue` + +- [ ] **Step 1: Create view** + +Create `frontend/src/views/StoryTimelineView.vue`: + +```vue + + + + + +``` + +- [ ] **Step 2: Add route** + +Read `frontend/src/router/index.js`. Then add to its routes array: + +```javascript +{ + path: '/story/:simId', + name: 'story', + component: () => import('../views/StoryTimelineView.vue'), +}, +``` + +- [ ] **Step 3: Manual smoke test** + +```bash +cd frontend && npm run dev +# Open http://localhost:3000/story/any_sim_id +# Expected: "No story yet" message, Refresh/Translate buttons visible +``` + +- [ ] **Step 4: Commit** + +```bash +git add frontend/src/views/StoryTimelineView.vue frontend/src/router/index.js +git commit -m "feat(narrative): add StoryTimelineView and route" +``` + +--- + +## Task 11: CharacterCard Component + Integration + +**Files:** +- Create: `frontend/src/components/CharacterCard.vue` +- Modify: `frontend/src/views/StoryTimelineView.vue` + +- [ ] **Step 1: Create CharacterCard** + +Create `frontend/src/components/CharacterCard.vue`: + +```vue + + + + + +``` + +- [ ] **Step 2: Integrate into StoryTimelineView** + +In `frontend/src/views/StoryTimelineView.vue`, add character loading and display. Add to imports: + +```javascript +import CharacterCard from '../components/CharacterCard.vue' +``` + +Add to script setup (after `title`): + +```javascript +const characters = ref([]) +async function loadCharacters() { + try { + const { data } = await narrativeAPI.getCharacters(simId) + characters.value = data.characters || [] + } catch (e) { /* ignore */ } +} +``` + +Change `onMounted(refresh)` to `onMounted(() => { refresh(); loadCharacters() })`. + +Change `translateNext` to reload characters after: add `await loadCharacters()` before `await refresh()`. + +Add this block in template, right after `
`: + +```vue +
+ +
+``` + +Add to styles: + +```css +.character-roster { + margin-bottom: 2rem; + padding: 1rem; + background: #f5efd9; + border-radius: 4px; +} +``` + +- [ ] **Step 3: Commit** + +```bash +git add frontend/src/components/CharacterCard.vue frontend/src/views/StoryTimelineView.vue +git commit -m "feat(narrative): add CharacterCard and roster display" +``` + +--- + +## Task 12: End-to-End Smoke Test + +**Files:** +- Create: `backend/tests/test_narrative_e2e.py` + +- [ ] **Step 1: Write E2E test** + +Create `backend/tests/test_narrative_e2e.py`: + +```python +"""End-to-end test: create fake simulation dir → translate → verify story.""" +import os +import json +import tempfile +from unittest.mock import patch +from app.services.narrative.story_store import StoryStore +from app.services.narrative.narrative_translator import translate_round + +def test_full_pipeline_from_fake_simulation(tmp_path): + sim_dir = str(tmp_path / "sim_e2e") + os.makedirs(os.path.join(sim_dir, "twitter")) + + actions_path = os.path.join(sim_dir, "twitter", "actions.jsonl") + actions = [ + {"round": 1, "agent_id": 1, "agent_name": "Elena", "action_type": "CREATE_POST", "action_args": {"content": "The council must fall."}, "success": True, "timestamp": "t"}, + {"round": 1, "agent_id": 2, "agent_name": "Marcus", "action_type": "DISLIKE_POST", "action_args": {"post_id": 1}, "success": True, "timestamp": "t"}, + {"event_type": "round_end", "round": 1, "timestamp": "t"}, + {"round": 2, "agent_id": 1, "agent_name": "Elena", "action_type": "REPOST", "action_args": {}, "success": True, "timestamp": "t"}, + {"event_type": "round_end", "round": 2, "timestamp": "t"}, + ] + with open(actions_path, "w") as f: + for a in actions: + f.write(json.dumps(a) + "\n") + + store = StoryStore(sim_dir) + store.save_characters([ + {"id": "1", "name": "Elena", "emotional_state": {"current": {k: 0.0 for k in ["anger","fear","joy","sadness","trust","surprise"]}}}, + {"id": "2", "name": "Marcus", "emotional_state": {"current": {k: 0.0 for k in ["anger","fear","joy","sadness","trust","surprise"]}}}, + ]) + store.load_characters()[0]["emotional_state"]["current"]["trust"] = 0.5 + + with patch("app.services.narrative.narrative_translator.call_llm") as mock_llm: + mock_llm.side_effect = [ + "Elena addressed the gathering. Marcus's face darkened.", + "Elena's message spread through the quarter like a spark.", + ] + beat1 = translate_round(sim_dir, "twitter", 1, "dark fantasy") + beat2 = translate_round(sim_dir, "twitter", 2, "dark fantasy") + + assert beat1["round"] == 1 + assert beat2["round"] == 2 + assert "Elena" in beat1["prose"] + + all_beats = store.get_all_beats() + assert len(all_beats) == 2 + + # Characters should have evolved + chars = store.load_characters() + marcus = next(c for c in chars if c["name"] == "Marcus") + # If ACTION_EMOTIONAL_DELTAS has anger for DISLIKE_POST, anger should be > 0 + # (this validates the user's Step 4 choices in Task 3) + assert marcus["emotional_state"]["current"]["anger"] >= 0.0 # at minimum, not negative +``` + +- [ ] **Step 2: Run test** + +Run: `cd backend && uv run pytest tests/test_narrative_e2e.py -v` +Expected: 1 passed. + +- [ ] **Step 3: Run full test suite** + +Run: `cd backend && uv run pytest tests/ -v` +Expected: all tests pass (action_mapper: 4, character_engine: 5, story_store: 4, narrative_translator: 6, e2e: 1 = 20 total). + +- [ ] **Step 4: Commit** + +```bash +git add backend/tests/test_narrative_e2e.py +git commit -m "test(narrative): add end-to-end pipeline test" +``` + +--- + +## Self-Review Checklist + +**Spec coverage:** +- ✅ Section 3.0 Integration Point (async on-demand) → Task 6 `translate_round` + Task 7 API +- ✅ Section 3.1 Action Mapping → Task 1 `action_mapper.py` +- ✅ Section 3.2 Translation Pipeline → Task 4 + 5 + 6 +- ✅ Section 3.3 Resilience → partial (retry deferred, see below) +- ✅ Section 3.4 LLM Prompt Structure → Task 5 with user contribution point +- ✅ Section 4.1 Extended Character Schema → Task 3 +- ✅ Section 4.2 Emotional State Model (v1 display-only) → Task 3 with user contribution point +- ⏸ Section 4.3 Arc Detection → deferred to follow-on plan (not needed for MVP) +- ⏸ Sections 5-9 (world state, god mode, branching, input, export) → separate plans +- ✅ New API routes → Task 7 (subset: story, translate, characters, init) +- ✅ Frontend routes → Task 10 (`/story/:simId`) +- ✅ StoryBeat.vue → Task 9 +- ✅ CharacterCard.vue → Task 11 +- ✅ StoryTimelineView → Task 10 + +**Deferred to follow-on plans:** God Mode, timeline branching, world state manager, arc detection, enhanced input pipeline, export studio, all other frontend views. These each warrant their own focused plan. + +**Deferred within this plan (acceptable gaps):** Retry/resilience wrapper around `call_llm` — relies on whatever retry is already in `utils/llm_client.py`; can be added in a hardening task if the basic pipeline works. + +**Type consistency check:** ✅ `StoryStore`, `translate_round`, `generate_prose`, `apply_action_emotional_delta`, `create_initial_character` all have consistent signatures across tasks. diff --git a/docs/superpowers/specs/2026-03-26-narrative-layer-design.md b/docs/superpowers/specs/2026-03-26-narrative-layer-design.md new file mode 100644 index 00000000..9cb1e4d8 --- /dev/null +++ b/docs/superpowers/specs/2026-03-26-narrative-layer-design.md @@ -0,0 +1,532 @@ +# MiroFish Narrative Layer — Design Specification + +**Date:** 2026-03-26 +**Approach:** A — Narrative Layer on top of existing OASIS simulation +**Target:** Creative simulation engine (novel continuation, world-building, interactive fiction) +**Scale:** Small community (10–50 users), basic auth, moderate concurrency + +--- + +## 1. Problem Statement + +MiroFish is a multi-agent swarm intelligence engine built for prediction — it simulates social media interactions (Twitter/Reddit) to forecast outcomes. The simulation infrastructure (OASIS engine, Zep graph memory, LLM-driven agents) is powerful but locked into a social-media metaphor. + +**Goal:** Extend MiroFish into a creative storytelling platform where users can upload fiction, define worlds, and watch AI agents generate emergent narratives — without replacing the proven OASIS simulation backbone. + +**Key insight:** Social media actions map naturally to narrative actions. A "post" is speech. A "like" is agreement. A "repost" is rumor-spreading. By adding a translation layer, we reinterpret simulation output as story prose. + +--- + +## 2. Architecture + +### 2.1 System Diagram + +``` +┌──────────────────────────────────────────────────────┐ +│ Frontend (Vue 3) │ +│ ┌────────────┐ ┌──────────────┐ ┌────────────────┐ │ +│ │ Story │ │ Character │ │ God Mode │ │ +│ │ Timeline │ │ Workshop │ │ Control Panel │ │ +│ └────────────┘ └──────────────┘ └────────────────┘ │ +│ ┌────────────┐ ┌──────────────┐ ┌────────────────┐ │ +│ │ World │ │ Branching │ │ Export │ │ +│ │ Map View │ │ Timeline │ │ Studio │ │ +│ └────────────┘ └──────────────┘ └────────────────┘ │ +├──────────────────────────────────────────────────────┤ +│ Narrative Engine (NEW) │ +│ ┌──────────────────────────────────────────────┐ │ +│ │ Narrative Translator │ │ +│ │ - Action → Story Event mapping │ │ +│ │ - Emotional state tracking │ │ +│ │ - Plot arc detection │ │ +│ └──────────────────────────────────────────────┘ │ +│ ┌─────────────┐ ┌──────────────┐ ┌─────────────┐ │ +│ │ Character │ │ World State │ │ Timeline │ │ +│ │ Engine │ │ Manager │ │ Brancher │ │ +│ └─────────────┘ └──────────────┘ └─────────────┘ │ +├──────────────────────────────────────────────────────┤ +│ Existing MiroFish Backend │ +│ Graph Builder → OASIS Simulation → Report Agent │ +│ Zep Memory → Profile Generator → LLM Client │ +└──────────────────────────────────────────────────────┘ +``` + +### 2.2 New Backend Services + +All new services live in `backend/app/services/narrative/` to keep the existing codebase untouched. + +| Service | File | Purpose | +|---|---|---| +| Narrative Translator | `narrative_translator.py` | Converts OASIS actions into story prose via LLM | +| Character Engine | `character_engine.py` | Manages extended character profiles (backstory, emotions, arcs) | +| World State Manager | `world_state.py` | Tracks locations, factions, resources, world rules | +| Timeline Brancher | `timeline_brancher.py` | Snapshots and forks simulation state | +| Seed Enhancer | `seed_enhancer.py` | Processes EPUB/DOCX/templates into MiroFish seeds | +| Export Studio | `export_studio.py` | Generates EPUB, screenplay, chapter formats | + +### 2.3 New API Routes + +New route blueprint: `narrative_bp = Blueprint('narrative', __name__)` in `backend/app/api/narrative.py`. Register in `backend/app/__init__.py` via `app.register_blueprint(narrative_bp, url_prefix='/api/narrative')`. Import added to `backend/app/api/__init__.py`. + +| Method | Endpoint | Purpose | +|---|---|---| +| POST | `/api/narrative/translate` | Translate a simulation round into story prose | +| GET | `/api/narrative/story/{sim_id}` | Get full story for a simulation | +| GET | `/api/narrative/story/{sim_id}/round/{round}` | Get story for a specific round | +| POST | `/api/narrative/characters` | Create/update extended character profiles | +| GET | `/api/narrative/characters/{sim_id}` | Get all characters with emotional states | +| GET | `/api/narrative/characters/{sim_id}/{char_id}` | Get single character detail | +| POST | `/api/narrative/world` | Create/update world state | +| GET | `/api/narrative/world/{sim_id}` | Get current world state | +| POST | `/api/narrative/godmode/inject` | Inject an event into a running simulation | +| POST | `/api/narrative/godmode/modify-character` | Modify character state mid-simulation | +| POST | `/api/narrative/godmode/change-rules` | Change world rules mid-simulation | +| POST | `/api/narrative/branch/{sim_id}/{round}` | Fork timeline at a specific round | +| GET | `/api/narrative/branches/{sim_id}` | List all branches for a simulation | +| POST | `/api/narrative/export/{sim_id}` | Export story in specified format | +| GET | `/api/narrative/export/{sim_id}/status` | Check export generation status | + +### 2.4 New Frontend Views + +| View | File | Purpose | +|---|---|---| +| Story Timeline | `views/StoryTimelineView.vue` | Read the generated narrative chapter by chapter | +| Character Workshop | `views/CharacterWorkshopView.vue` | Create, edit, and browse characters | +| World Builder | `views/WorldBuilderView.vue` | Define locations, factions, rules | +| God Mode | `views/GodModeView.vue` | Control panel for mid-simulation intervention | +| Branch Explorer | `views/BranchExplorerView.vue` | Visual timeline tree, compare branches | +| Export Studio | `views/ExportStudioView.vue` | Choose format, preview, download | + +### 2.6 Frontend Routes + +| Route | View | Description | +|---|---|---| +| `/story/:simId` | StoryTimelineView | Read generated narrative | +| `/characters/:simId` | CharacterWorkshopView | Create and browse characters | +| `/world/:simId` | WorldBuilderView | Define world settings | +| `/godmode/:simId` | GodModeView | Mid-simulation controls | +| `/branches/:simId` | BranchExplorerView | Timeline tree and comparison | +| `/export/:simId` | ExportStudioView | Format and download stories | + +Routes are added to `frontend/src/router/index.js` following the existing pattern of `/:simulationId` parameterized routes. + +### 2.5 New Frontend Components + +| Component | Purpose | +|---|---| +| `StoryBeat.vue` | Single story event/paragraph with character attribution | +| `CharacterCard.vue` | Character portrait with stats, emotions, relationships | +| `RelationshipGraph.vue` | Visual graph of character relationships (D3.js) | +| `EmotionTracker.vue` | Per-character emotional state over time (sparkline) | +| `WorldMap.vue` | Visual map of locations and character positions | +| `TimelineTree.vue` | Branching timeline visualization | +| `GodModeToolbar.vue` | Floating toolbar for injecting events | +| `ExportPreview.vue` | Preview formatted output before download | + +--- + +## 3. Narrative Translator — Core Logic + +### 3.0 Integration Point — How the Narrative Layer Connects to OASIS + +The existing OASIS simulation runs as a subprocess that writes actions to `actions.jsonl` (one JSON line per agent action per round). The existing `_monitor_simulation` method in `SimulationRunner` already polls this file every 2 seconds, tracking file position to read only new lines. + +**Narrative translation is asynchronous and on-demand.** The Narrative Translator does NOT hook into the simulation loop. Instead: + +1. **During simulation:** The existing `_monitor_simulation` continues reading `actions.jsonl` as-is. No changes to the simulation runner. +2. **After each round completes:** The frontend polls a new endpoint `GET /api/narrative/story/{sim_id}/latest` which triggers translation of any untranslated rounds. +3. **Translation reads `actions.jsonl`** directly, maintaining its own file offset in `narrative/translator_state.json`. It reads actions for the next untranslated round, generates prose, and stores the result. +4. **Batch translation** is also available via `POST /api/narrative/translate` for retroactively translating a completed simulation. + +**Why on-demand (not in-loop):** LLM prose generation takes 2-5 seconds per scene. Injecting this into the simulation monitor would slow round processing by 10-20x. By decoupling translation, the simulation runs at full speed and narrative is generated as users view it. + +**IPC for God Mode:** The existing `SimulationIPCClient` supports file-based IPC with `INTERVIEW`, `BATCH_INTERVIEW`, and `CLOSE_ENV` commands. God Mode leverages `INTERVIEW` to inject custom prompts into running agents (see Section 6). New IPC command types (`INJECT_EVENT`, `MODIFY_AGENT_PROMPT`) will be added to `SimulationIPCServer` and the OASIS runner scripts. + +### 3.1 Action Mapping + +| OASIS Action | Narrative Interpretation | +|---|---| +| `CREATE_POST` | Character speaks, declares, announces | +| `LIKE_POST` | Character agrees, supports, nods | +| `REPOST` | Character spreads rumor, amplifies, gossips | +| `QUOTE_POST` | Character responds, debates, challenges | +| `FOLLOW` | Character allies with, shows loyalty to | +| `DO_NOTHING` | Character reflects, observes, waits *(inferred — see note below)* | +| `CREATE_COMMENT` | Character engages in dialogue | +| `DISLIKE_POST` | Character opposes, confronts, disapproves | +| `LIKE_COMMENT` | Character validates someone's response | +| `DISLIKE_COMMENT` | Character dismisses, mocks | +| `SEARCH_POSTS` | Character investigates, seeks information | +| `SEARCH_USER` | Character seeks out a specific person | +| `MUTE` | Character avoids, ignores, shuns | + +**Note on `DO_NOTHING`:** This action may not appear in `actions.jsonl` since the logger only records actions agents take. The translator infers inaction: any agent present in the simulation but absent from a round's action log is treated as "observing/waiting." This is narratively valuable. + +### 3.2 Translation Pipeline + +For each simulation round: + +1. **Collect actions** — read `actions.jsonl` from the translator's saved file offset, gather all actions until the next `round_end` event +2. **Infer inaction** — compare active agents against the full character roster; absent agents are marked as "observing" +3. **Enrich with context** — pull character emotional states, relationships, world state from `narrative/` files +4. **Group into scenes** — cluster related actions (same thread / interacting agents) +5. **Generate prose** — LLM call per scene with action data + context → narrative paragraph +6. **Detect plot events** — classify the round as rising action / climax / falling action / resolution +7. **Update states** — update character emotions and relationships based on what happened +8. **Store** — persist story beat to `narrative/story_beats.json`, update translator file offset in `narrative/translator_state.json` + +### 3.3 Resilience + +- **LLM retry:** 3 retries with exponential backoff (1s, 2s, 4s) using the existing `retry` utility in `backend/app/utils/retry.py` +- **Partial failure:** If a scene fails after retries, store a placeholder beat with raw action data. User can retry individual beats via API. +- **Graceful degradation:** If the LLM is completely unavailable, fall back to template-based summary: `"{agent_name} {action_verb} {target}"` +- **Cost estimate:** ~3-5 LLM calls per round (one per scene). A 20-round simulation ≈ 60-100 calls ≈ 30k-50k tokens ≈ $0.15-$0.25 with GPT-4o-mini. + +### 3.4 LLM Prompt Structure + +The translator uses a structured prompt: + +``` +You are a narrative writer. Convert these simulation events into a story passage. + +World: {world_description} +Setting: {current_location_context} +Characters involved: {character_summaries_with_emotions} + +Events this round: +{structured_action_list} + +Previous story context (last 2 beats): +{previous_prose} + +Write a story passage (2-4 paragraphs) that: +- Uses third-person past tense +- Shows character emotions through action and dialogue +- Maintains consistency with the world rules +- Advances the narrative naturally from the previous context + +Tone: {user_defined_tone — e.g., "dark fantasy", "lighthearted comedy", "political thriller"} +``` + +--- + +## 4. Character Engine + +### 4.1 Extended Character Schema + +```json +{ + "id": "char_abc123", + "name": "Elena Voss", + "oasis_agent_id": "agent_xyz", + + "backstory": "Former diplomat turned rogue negotiator...", + "motivations": ["power", "redemption"], + "personality_traits": ["cunning", "loyal_to_few", "distrustful"], + "speech_style": "Formal, precise, with occasional dark humor", + + "emotional_state": { + "current": {"anger": 0.3, "fear": 0.1, "joy": 0.0, "sadness": 0.4, "trust": 0.2}, + "history": [{"round": 1, "state": {...}}, ...] + }, + + "relationships": { + "char_def456": {"type": "rival", "intensity": 0.8, "history": ["betrayal in round 3"]}, + "char_ghi789": {"type": "ally", "intensity": 0.6, "history": ["formed alliance in round 1"]} + }, + + "arc": { + "archetype": "fall_from_grace", + "stage": "descent", + "key_moments": [{"round": 3, "event": "Betrayed longtime ally"}] + }, + + "location": "The Iron Tower", + "faction": "The Council" +} +``` + +### 4.2 Emotional State Model + +Six basic emotions tracked as 0.0–1.0 floats: `anger`, `fear`, `joy`, `sadness`, `trust`, `surprise`. + +Updated each round based on: +- Actions taken by the character +- Actions taken *toward* the character +- World events +- Relationship changes + +**v1: Display-only.** Emotional states are derived from actions and displayed in the UI but do NOT feed back into OASIS agent prompts. OASIS profiles are generated once at simulation setup time by `OasisProfileGenerator` and are not modified mid-simulation. + +**v2 (future):** Use the new `INJECT_CONTEXT` IPC command (see Section 6.2) to prepend emotional context to agent prompts each round. E.g., `"You are currently feeling angry and distrustful after the betrayal."` This creates a feedback loop where emotions influence behavior, but adds an IPC call per agent per round. + +### 4.3 Character Arc Detection + +After each round, the Character Engine evaluates arc progression using pattern matching: + +| Arc Type | Pattern | +|---|---| +| Hero's Journey | Comfort → Crisis → Growth → Triumph | +| Fall from Grace | Status → Temptation → Descent → Consequences | +| Redemption | Flaw → Suffering → Realization → Atonement | +| Coming of Age | Innocence → Challenge → Learning → Maturity | +| Tragedy | Hope → Hubris → Downfall → Loss | + +**Detection mechanism:** Rule-based classification using emotional state deltas: +- Trust drops > 0.3 in one round → "crisis" stage +- Joy sustained > 0.6 for 3+ rounds → "comfort" or "triumph" stage +- Anger + fear both > 0.5 → "descent" stage +- Transition from high-negative to high-positive emotions → "redemption" + +If rule-based detection is ambiguous, a single LLM classification call is made with the character's action history summary (~200 tokens). Detected arcs are stored in `characters.json` and surfaced in the UI and export. + +--- + +## 5. World State Manager + +### 5.1 World State Schema + +```json +{ + "sim_id": "sim_abc123", + "name": "The Shattered Kingdoms", + "genre": "dark_fantasy", + "tone": "grim, political", + "era": "medieval", + + "rules": [ + "Magic exists but is feared and regulated", + "The monarchy has fallen; power is contested by three factions", + "Winter is approaching and resources are scarce" + ], + + "locations": { + "iron_tower": {"name": "The Iron Tower", "type": "fortress", "faction": "council", "characters": ["char_abc"]}, + "market_district": {"name": "Market District", "type": "city", "faction": "neutral", "characters": []} + }, + + "factions": { + "council": {"name": "The Council", "power": 0.6, "members": ["char_abc"], "goals": ["restore order"]}, + "rebels": {"name": "The Free Folk", "power": 0.3, "members": ["char_def"], "goals": ["overthrow council"]} + }, + + "resources": { + "food": {"abundance": 0.3, "controlled_by": "council"}, + "weapons": {"abundance": 0.5, "controlled_by": "rebels"} + }, + + "timeline_position": {"round": 5, "branch": "main"} +} +``` + +### 5.2 World State Updates + +Each simulation round, the World State Manager: +1. Processes narrative events for world-level changes (faction power shifts, resource changes) +2. Updates character locations based on their actions +3. Checks world rules for constraint violations +4. Triggers world events (e.g., "winter arrives" at round 10) + +--- + +## 6. God Mode + +### 6.1 Intervention Types + +| Intervention | Effect | +|---|---| +| **Inject Event** | Add a world event ("earthquake strikes", "a stranger arrives") that agents must react to | +| **Modify Character** | Change a character's emotional state, motivation, or relationships | +| **Change Rules** | Add or remove world rules ("magic is now forbidden") | +| **Force Action** | Make a specific character take a specific action next round | +| **Kill Character** | Remove an agent from the simulation permanently | +| **Resurrect Character** | Re-introduce a removed character | +| **Time Skip** | Jump forward N rounds with summarized events | +| **Rewind** | Go back to a previous round (creates a branch) | + +### 6.2 Implementation — IPC Dispatch Mechanism + +God Mode leverages the existing **file-based IPC** system (`SimulationIPCClient` / `SimulationIPCServer`). The IPC currently supports `INTERVIEW`, `BATCH_INTERVIEW`, and `CLOSE_ENV` commands. + +**Mapping God Mode interventions to IPC:** + +| Intervention | IPC Mechanism | +|---|---| +| **Inject Event** | `BATCH_INTERVIEW` — send all agents a prompt describing the event, forcing them to react in their next action. E.g., `"An earthquake just struck. How do you respond?"` | +| **Modify Character** | `INTERVIEW` — send the target agent a prompt that reframes their emotional/motivational state. E.g., `"You have just learned that your ally betrayed you. Your trust is shattered."` The Character Engine also updates its local emotional state JSON. | +| **Change Rules** | `BATCH_INTERVIEW` — inform all agents of the rule change. E.g., `"Magic has been outlawed. Anyone caught using it faces death."` Also update `world_state.json`. | +| **Force Action** | `INTERVIEW` — send the target agent a directive prompt. E.g., `"You decide to confront Marcus in front of the council. Describe your accusation."` The response is logged as the agent's action. | +| **Kill Character** | Stop sending actions for this agent in subsequent rounds. Mark as "dead" in `characters.json`. OASIS agent continues to exist but receives no prompts. | +| **Resurrect Character** | Reverse of kill — resume prompting the agent with a resurrection context prompt. | +| **Time Skip** | Set simulation to run N rounds without narrative translation, then batch-translate with a "summary" prompt instead of detailed prose. | +| **Rewind** | Creates a new simulation from scratch using the same config but with a modified starting prompt that includes "the story so far up to round N." See Section 7 for details. | + +**New IPC command type needed:** `INJECT_CONTEXT` — a variant of `BATCH_INTERVIEW` that prepends context to all agents' next-round system prompts rather than triggering an immediate interview response. This requires a small addition to the OASIS runner scripts (`run_twitter_simulation.py`, `run_reddit_simulation.py`). + +Each intervention is logged to `narrative/god_mode_log.json` for story coherence and auditability. + +--- + +## 7. Timeline Branching + +### 7.1 Branch Model + +```json +{ + "sim_id": "sim_abc123", + "branches": { + "main": {"parent": null, "fork_round": 0, "status": "running", "rounds_completed": 10}, + "what_if_elena_dies": {"parent": "main", "fork_round": 5, "status": "running", "rounds_completed": 3}, + "peaceful_ending": {"parent": "main", "fork_round": 7, "status": "paused", "rounds_completed": 1} + } +} +``` + +### 7.2 Branching Mechanics — Replay-Based Approach + +**OASIS does not support checkpointing.** The simulation database is deleted on startup and there is no state serialization for mid-simulation resumption. Therefore, branching uses a **replay + diverge** strategy: + +1. User selects a round N to branch from +2. System creates a new simulation directory under `uploads/simulations/{sim_id}/branches/{branch_name}/` +3. System copies the original `simulation_config.json` and `profiles.json` +4. System generates a **"story so far" summary** — an LLM-generated condensation of all narrative beats from rounds 1 to N +5. This summary is injected into every agent's starting prompt as context: `"The following events have already occurred: {summary}. You are now at this point in the story."` +6. A new OASIS subprocess launches with modified agent prompts that include this prior context +7. User can optionally inject a God Mode event to differentiate the branch (e.g., "In this timeline, Elena survives the betrayal") +8. Both branches run as independent OASIS processes — user can switch between them in the UI + +**Trade-offs of replay approach:** +- **Pro:** No OASIS modifications needed. Uses existing subprocess launch mechanism. +- **Con:** Agents don't have exact memory of previous rounds — they have an LLM-summarized version. This means branches may drift from the original timeline's "feel." For creative fiction, this is acceptable (stories branch naturally). +- **Con:** Starting a branch requires an LLM call for summary generation (~5-10 seconds). Not a major bottleneck. +- **Performance:** Branch launch takes the same time as a fresh simulation start (~10-30 seconds), regardless of which round you branch from. + +Storage: Each branch gets its own subdirectory under `uploads/simulations/{sim_id}/branches/{branch_name}/` with its own `simulation_config.json`, `actions.jsonl`, and `narrative/` folder. + +--- + +## 8. Enhanced Input Pipeline + +### 8.1 New Seed Formats + +| Format | Processing | +|---|---| +| **EPUB** | Extract text → identify characters (NER) → extract settings → generate world state | +| **DOCX** | Same as EPUB but with python-docx parsing | +| **YAML template** | Structured world definition — characters, locations, rules — no LLM needed | +| **Image** | Send to multimodal LLM → extract character description / setting description | +| **URL** | Scrape page content → process as text seed | + +### 8.2 YAML World Template Format + +```yaml +world: + name: "The Shattered Kingdoms" + genre: "dark_fantasy" + tone: "grim, political" + rules: + - "Magic exists but is feared" + - "Winter is approaching" + +characters: + - name: "Elena Voss" + role: "protagonist" + backstory: "Former diplomat..." + motivations: ["power", "redemption"] + personality: ["cunning", "loyal"] + + - name: "Marcus Iron" + role: "antagonist" + backstory: "Military commander..." + motivations: ["control", "legacy"] + +locations: + - name: "The Iron Tower" + type: "fortress" + description: "A dark spire..." + +factions: + - name: "The Council" + goals: ["restore order"] + members: ["Elena Voss"] + +simulation: + max_rounds: 20 + tone: "dark_fantasy" + platform: "twitter" # underlying OASIS platform +``` + +--- + +## 9. Export Studio + +### 9.1 Output Formats + +| Format | Structure | +|---|---| +| **Chapters** | Story beats grouped into chapters (every 3-5 rounds), with chapter titles | +| **Screenplay** | Character names in caps, dialogue, (parenthetical action), scene headings | +| **EPUB** | Full e-book with cover, table of contents, chapters, character appendix | +| **Character Dossiers** | Per-character PDF with backstory, arc summary, key moments, relationships | +| **Raw Timeline** | JSON export of all events for programmatic use | + +### 9.2 Export Pipeline + +1. User selects simulation + branch + format +2. Backend collects all story beats for the selected scope +3. LLM pass for cohesion — smooth transitions between beats, add chapter breaks +4. Format-specific rendering (EPUB via `ebooklib`, screenplay via template) +5. Return download URL + +**Async handling:** Export runs in a background thread (consistent with existing `SimulationRunner._monitor_simulation` pattern). The `POST /api/narrative/export/{sim_id}` endpoint returns immediately with an export job ID. Frontend polls `GET /api/narrative/export/{sim_id}/status` until complete. Note: server restarts will lose in-progress exports — acceptable for the target scale. + +--- + +## 10. Data Model Changes + +### 10.1 New Files per Simulation + +``` +uploads/simulations/{sim_id}/ +├── state.json # existing +├── profiles.json # existing +├── entities.json # existing +├── simulation_config.json # existing +├── narrative/ # NEW +│ ├── world_state.json # world definition +│ ├── characters.json # extended character profiles +│ ├── story_beats.json # generated narrative per round +├── branches/ # Timeline branches (see Section 7) +│ ├── what_if_branch/ +│ │ ├── simulation_config.json +│ │ ├── actions.jsonl +│ │ └── narrative/ +│ │ ├── story_beats.json +│ │ └── characters.json +│ └── exports/ +│ ├── story_chapters.epub +│ └── screenplay.txt +``` + +### 10.2 Database Additions + +No new database — continues using file-based storage consistent with existing MiroFish patterns. For the 10-50 user scale, this is sufficient. If we need to scale later, we migrate to SQLite or PostgreSQL. + +--- + +## 11. Authentication & Multi-User + +**Deferred to a separate spec.** The existing MiroFish codebase has zero authentication — no middleware, no user model, no token checks. Adding JWT auth is a cross-cutting concern that touches every existing endpoint and deserves its own design. For v1 of the Narrative Layer, we operate without auth (same as existing MiroFish). Multi-user isolation is handled by simulation IDs — each simulation is self-contained in its own directory. + +**When auth is added (separate spec):** Simple JWT tokens, user model in SQLite, middleware on all `/api/` routes, project isolation by user directory, optional "public" flag for sharing. + +--- + +## 12. Non-Goals (Explicitly Out of Scope) + +- Real-time collaborative editing (too complex for v1) +- Voice/audio output of stories +- Image generation for scenes (future enhancement) +- Mobile app +- Horizontal scaling / microservices +- Payment / billing