RomanNum3ral
/
MicroFish
mirror of https://github.com/666ghj/MiroFish.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/main' into fix/23-externalize-chinese-frontend-strings 

# Conflicts:
#	frontend/src/components/Step2EnvSetup.vue
#	frontend/src/components/Step4Report.vue
#	frontend/src/views/Process.vue
This commit is contained in:
Dominik Seemann 2026-05-11 11:16:40 +02:00
parent 960e3ba81d 056f3664be
commit 9df4e155ff
139 changed files with 23458 additions and 5312 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

26
.github/workflows/i18n-cjk-guard.yml vendored Normal file
View File

@ -0,0 +1,26 @@
name: i18n CJK Guard
on:
pull_request:
branches: [main]
permissions:
contents: read
jobs:
guard:
runs-on: ubuntu-latest
timeout-minutes: 1
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Run i18n CJK guard
run: python scripts/ci/i18n_cjk_guard.py

5
.kiro/specs/i18n-ci-guard/baseline.txt Normal file
View File

@ -0,0 +1,5 @@
# Per-path CJK baseline for the i18n CI guard.
# Format: <path>\t<count>. Sorted lexicographically.
# Refresh via: python scripts/ci/i18n_cjk_guard.py --update-baseline
backend/app 307
frontend/src 124

544
.kiro/specs/i18n-ci-guard/design.md Normal file
View File

@ -0,0 +1,544 @@
# Design — i18n-ci-guard
## Overview
This feature installs a permanent, PR-time CI guard that blocks
regressions of the project's English-by-default state. It performs two
checks: `locales/en.json` must contain zero CJK characters, and the
total CJK match count under `backend/app/` and `frontend/src/` must not
exceed a committed per-path baseline. The guard is a single Python
script invoked by a single GitHub Actions workflow.
**Purpose**: This feature delivers an automatic regression gate to the
i18n initiative so reviewers do not have to spot CJK reintroductions
by eye.
**Users**: Project maintainers and PR authors. Maintainers gain a
hard regression gate; PR authors gain a script they can run locally to
catch regressions before pushing.
**Impact**: Adds the project's first `pull_request`-triggered CI
workflow. No production source under `backend/app/`, `frontend/src/`,
or `locales/` is modified by this spec — only new files are added.
### Goals
- Fail any PR that introduces a CJK character into `locales/en.json`.
- Fail any PR whose CJK match count under `backend/app/` or
`frontend/src/` exceeds the committed baseline.
- Print a single actionable failure message that includes the exact
command a contributor must run if the regression is intentional.
- Run end-to-end under sixty seconds on `ubuntu-latest`.
- Be reproducible verbatim on a developer machine with Python ≥3.11
and `git`.
### Non-Goals
- Re-implementing the full classification pipeline from
`.kiro/specs/i18n-e2e-english-verification/` (that work belongs to
PR #27).
- Auto-updating the baseline on `main`.
- Translating any production source to satisfy a higher baseline. The
initial baseline is recorded against `main` and only ratchets down
over time.
- Gating commits at pre-commit time. The guard is CI-only; a future
spec may wrap it in a hook.
## Boundary Commitments
### This Spec Owns
- The guard script `scripts/ci/i18n_cjk_guard.py` and its CLI
contract.
- The workflow `.github/workflows/i18n-cjk-guard.yml` and its
trigger configuration.
- The baseline file `.kiro/specs/i18n-ci-guard/baseline.txt` and its
format.
- The pass/fail semantics of both checks.
### Out of Boundary
- Any change to files under `backend/app/`, `frontend/src/`, or
`locales/` — except `locales/en.json` if it is found to contain CJK
during initial baseline calibration (a remediation translation would
be a separate spec/PR).
- The classification heuristics in PR #27's `classify.py`.
- Pre-commit hooks; IDE integrations; alternative scoped paths beyond
`backend/app/` and `frontend/src/`.
### Allowed Dependencies
- Python ≥3.11 standard library.
- `git` (for `git grep -nIP` invocation).
- `actions/checkout@v4` and `actions/setup-python@v5` from the
GitHub Actions Marketplace.
### Revalidation Triggers
- Adding a third scoped path → baseline file format changes; consumers
(none today) re-check.
- Changing the regex range → audit pipeline alignment must be
re-confirmed.
- Switching from `pull_request` to `merge_group` or other event →
required-status-check rules in branch protection must be re-checked.
## Architecture
### Existing Architecture Analysis
- **Repo layout**: monorepo split by runtime (`backend/`, `frontend/`)
with shared `locales/` at root. The guard scopes its scan to
`backend/app/`, `frontend/src/`, and `locales/en.json`, matching the
audit pipeline's canonical scope.
- **Existing scripts pattern**: `scripts/<purpose>.py` for developer
tools. The new `scripts/ci/` subdirectory introduces a clear,
CI-only home without disturbing the existing developer scripts.
- **Existing CI**: `.github/workflows/docker-image.yml` is tag-only.
No `pull_request` workflow exists. The new workflow is additive and
does not affect the docker-image workflow.
### Architecture Pattern & Boundary Map
```mermaid
flowchart LR
PR[Pull Request to main] -->|trigger| WF[.github/workflows/i18n-cjk-guard.yml]
WF -->|setup-python + checkout| RUN[python scripts/ci/i18n_cjk_guard.py]
RUN -->|read| EN[locales/en.json]
RUN -->|git grep -nIP| BAPP[backend/app/]
RUN -->|git grep -nIP| FSRC[frontend/src/]
RUN -->|read| BL[.kiro/specs/i18n-ci-guard/baseline.txt]
RUN -->|exit 0 or 1| WF
WF -->|status| PR
DEV[Developer terminal] -->|python scripts/ci/i18n_cjk_guard.py| RUN
DEV -->|--update-baseline| RUN
RUN -.->|writes| BL
```
**Architecture Integration**:
- **Selected pattern**: single-purpose script + thin workflow.
Matches the project's existing `scripts/<purpose>.py` convention.
- **Domain boundaries**: the guard is a pure verification tool with no
side effects on production code. Its only writeable surface is the
baseline file, and only when explicitly invoked with
`--update-baseline`.
- **Existing patterns preserved**: stdlib-only Python tooling
(precedent: `scripts/check_i18n_logs.py`); single-file workflows in
`.github/workflows/`.
- **New components rationale**: a new file rather than an extension of
an existing script — the existing script is scoped to a fixed
module list and is not a regression gate.
- **Steering compliance**: respects layer-based structure (script
lives at repo root in `scripts/ci/`, not under `backend/` or
`frontend/`), no new heavy dependencies, no `os.getenv` calls
outside `backend/app/config.py`.
### Technology Stack
| Layer | Choice / Version | Role in Feature | Notes |
|-------|------------------|-----------------|-------|
| Frontend / CLI | Python 3.11 stdlib (`argparse`, `json`, `re`, `subprocess`, `pathlib`, `sys`) | Guard CLI | Stdlib only — Req 5.5 |
| Backend / Services | n/a | — | Guard does not touch backend services |
| Data / Storage | Plain-text baseline file under `.kiro/specs/` | Per-path count store | One line per path, `<path>\t<count>` |
| Messaging / Events | n/a | — | — |
| Infrastructure / Runtime | GitHub Actions `ubuntu-latest`, `actions/checkout@v4`, `actions/setup-python@v5` | PR-time runner | `fetch-depth: 1` is sufficient |
## File Structure Plan
### Directory Structure
```
scripts/
└── ci/
└── i18n_cjk_guard.py # Guard CLI (new)
.github/
└── workflows/
└── i18n-cjk-guard.yml # PR-time workflow (new)
.kiro/specs/i18n-ci-guard/
├── spec.json # (existing, updated)
├── requirements.md # (existing)
├── gap-analysis.md # (existing)
├── research.md # (existing)
├── design.md # (this file)
├── tasks.md # (created in next phase)
└── baseline.txt # Per-path CJK match counts (new)
```
### Modified Files
- `.kiro/specs/i18n-ci-guard/spec.json` — phase / approval fields
updated by Kiro flow only.
- No production source files are modified by this spec.
## System Flows
### Guard execution (default mode)
```mermaid
sequenceDiagram
participant CI as GitHub Actions
participant Script as i18n_cjk_guard.py
participant Repo as Working tree
participant BL as baseline.txt
CI->>Script: python scripts/ci/i18n_cjk_guard.py
Script->>Repo: read locales/en.json
Script->>Script: scan for CJK chars
alt en.json has CJK
Script-->>CI: exit 1 + per-key findings
else en.json clean
Script->>Repo: git grep -nIP backend/app/
Script->>Repo: git grep -nIP frontend/src/
Script->>BL: read baseline counts
alt any current count > baseline
Script-->>CI: exit 1 + per-path delta + refresh hint
else within baseline
Script-->>CI: exit 0 + summary
end
end
```
### Baseline refresh
```mermaid
sequenceDiagram
participant Dev as Developer
participant Script as i18n_cjk_guard.py
participant Repo as Working tree
participant BL as baseline.txt
Dev->>Script: python scripts/ci/i18n_cjk_guard.py --update-baseline
Script->>Repo: git grep -nIP backend/app/
Script->>Repo: git grep -nIP frontend/src/
Script->>BL: write per-path counts (sorted)
Script-->>Dev: exit 0 + new counts
```
The two checks run in fixed order: en.json first (cheap, decisive),
then per-path counts. Both run under all conditions; the script does
not short-circuit after the first failure so the contributor sees the
complete diagnostic in one CI log.
## Requirements Traceability
| Requirement | Summary | Components | Interfaces | Flows |
|-------------|---------|------------|------------|-------|
| 1.1 | Scan en.json for CJK | `i18n_cjk_guard.py` | CLI default mode | Guard execution |
| 1.2 | Fail with key:line per offender | `i18n_cjk_guard.py` | CLI stderr output | Guard execution |
| 1.3 | Report clean state | `i18n_cjk_guard.py` | CLI stdout summary | Guard execution |
| 1.4 | Hard error if file missing | `i18n_cjk_guard.py` | CLI stderr + exit 1 | Guard execution |
| 2.1 | Count CJK matches per scoped path | `i18n_cjk_guard.py` | `git grep -nIP` invocation | Guard execution |
| 2.2 | Read baseline counts | `i18n_cjk_guard.py`, `baseline.txt` | File read | Guard execution |
| 2.3 | Fail on regression | `i18n_cjk_guard.py` | Exit 1 | Guard execution |
| 2.4 | Pass when within baseline | `i18n_cjk_guard.py` | Exit 0 | Guard execution |
| 2.5 | Skip binary files | `git grep -I` | — | Guard execution |
| 2.6 | Tracked-only scope | `git grep` default | — | Guard execution |
| 3.1 | Per-key locale failure detail | `i18n_cjk_guard.py` | CLI stderr lines | Guard execution |
| 3.2 | Per-path regression detail | `i18n_cjk_guard.py` | CLI stderr lines | Guard execution |
| 3.3 | Print refresh command | `i18n_cjk_guard.py` | CLI stderr footer | Guard execution |
| 3.4 | Success summary lines | `i18n_cjk_guard.py` | CLI stdout | Guard execution |
| 4.1 | Baseline under spec dir | `baseline.txt` | File path | — |
| 4.2 | Diff-friendly text format | `baseline.txt` | File format | — |
| 4.3 | Refresh via flag | `i18n_cjk_guard.py` | `--update-baseline` | Baseline refresh |
| 4.4 | No implicit baseline writes | `i18n_cjk_guard.py` | CLI default mode | Guard execution |
| 4.5 | Hard error if baseline missing | `i18n_cjk_guard.py` | Exit 1 + message | Guard execution |
| 5.1 | PR-only trigger to main | `i18n-cjk-guard.yml` | `on.pull_request.branches` | — |
| 5.2 | Checkout PR head | `i18n-cjk-guard.yml` | `actions/checkout@v4` | — |
| 5.3 | Surface output on failure | `i18n-cjk-guard.yml` | Default GH log | — |
| 5.4 | Pass on exit 0 | `i18n-cjk-guard.yml` | Default | — |
| 5.5 | Stdlib-only, no third-party | `i18n_cjk_guard.py`, `i18n-cjk-guard.yml` | — | — |
| 5.6 | ≤60s runtime | `i18n-cjk-guard.yml` | `timeout-minutes: 1` | — |
| 6.1 | Same result locally | `i18n_cjk_guard.py` | CLI | — |
| 6.2 | Single stable entry point | `scripts/ci/i18n_cjk_guard.py` | Path | — |
| 6.3 | No env vars / secrets | `i18n_cjk_guard.py` | CLI | — |
## Components and Interfaces
| Component | Domain/Layer | Intent | Req Coverage | Key Dependencies | Contracts |
|-----------|--------------|--------|--------------|------------------|-----------|
| `i18n_cjk_guard.py` | CI script | Two-check guard CLI | 1.1–6.3 | `git`, Python stdlib | Service (CLI) |
| `i18n-cjk-guard.yml` | CI workflow | Run guard on every PR to main | 5.1–5.6 | `actions/checkout@v4`, `actions/setup-python@v5` | Batch / Job |
| `baseline.txt` | Data | Per-path baseline counts | 4.1, 4.2, 2.2 | — | State (file) |
### CI Script
#### `i18n_cjk_guard.py`
| Field | Detail |
|-------|--------|
| Intent | Run two CJK-regression checks; optionally refresh the baseline |
| Requirements | 1.1, 1.2, 1.3, 1.4, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 3.1, 3.2, 3.3, 3.4, 4.1, 4.3, 4.4, 4.5, 5.5, 6.1, 6.2, 6.3 |
| Owner / Reviewers | i18n maintainers |
**Responsibilities & Constraints**
- Owns the canonical guard semantics: which paths are scoped, which
regex is canonical, what counts as a regression.
- Runs in pure Python 3.11 stdlib + a single `git` subprocess per
scoped path.
- Never modifies any file other than the baseline file, and only when
invoked with `--update-baseline`.
- Always runs both checks (does not short-circuit), so a single CI log
shows every failure mode at once.
**Dependencies**
- Inbound: `i18n-cjk-guard.yml` workflow; developers running locally.
- Outbound: `git` subprocess (`git grep`, `git rev-parse`).
- External: none.
**Contracts**: Service [x] / API [ ] / Event [ ] / Batch [ ] / State [x]
##### Service Interface (CLI)
```text
i18n_cjk_guard.py [--update-baseline] [--baseline PATH] [--repo-root PATH]
```
Type-annotated module signature (Python type hints, public functions
only):
```python
def main(argv: list[str]) -> int: ...
def run_check(repo_root: pathlib.Path, baseline_path: pathlib.Path) -> int:
"""Run both checks; return 0 on success, 1 on any failure."""
def update_baseline(repo_root: pathlib.Path, baseline_path: pathlib.Path) -> int:
"""Refresh the baseline file with current per-path counts; return 0."""
def scan_locale_cjk(en_json_path: pathlib.Path) -> list[LocaleFinding]:
"""Return a list of (key, line_number, snippet) tuples for every
CJK occurrence in locales/en.json. Empty list when clean."""
def count_path_cjk(repo_root: pathlib.Path, scoped_path: str) -> int:
"""Return the number of CJK match lines under scoped_path,
using `git grep -nIP '[\\x{4e00}-\\x{9fff}]' -- <scoped_path>`."""
def read_baseline(baseline_path: pathlib.Path) -> dict[str, int]:
"""Parse the baseline file. Each non-empty, non-comment line is
'<path>\\t<count>'. Raise BaselineError on any malformed input
or missing file."""
def write_baseline(baseline_path: pathlib.Path, counts: dict[str, int]) -> None:
"""Atomically overwrite the baseline file with sorted entries
and a single trailing newline."""
```
Where:
```python
LocaleFinding = tuple[str, int, str] # (dotted_key, line_number, snippet)
SCOPED_PATHS: tuple[str, ...] = ("backend/app", "frontend/src")
EN_JSON_REL_PATH: str = "locales/en.json"
CJK_PATTERN: str = "[\\x{4e00}-\\x{9fff}]" # passed to git grep -P
CJK_RE: re.Pattern[str] = re.compile(r"[一-鿿]")
SNIPPET_MAX_LEN: int = 80
```
- **Preconditions**: invoked with CWD at the repo root or
`--repo-root` set; `git` is on `$PATH`; the working tree is the
intended scan target.
- **Postconditions** (default mode): exit 0 iff both checks pass;
exit 1 otherwise. Stdout receives the success summary; stderr
receives findings on failure. The baseline file is unchanged.
- **Postconditions** (`--update-baseline`): the baseline file is
rewritten to current per-path counts and exit 0 is returned.
- **Invariants**: regex range, scoped paths, and baseline file path
are constants — no env-var override.
##### State Management
- **State model**: a dict `{<scoped_path>: <count>}` parsed from
the baseline file.
- **Persistence**: plain-text file at
`.kiro/specs/i18n-ci-guard/baseline.txt`. Atomic write via
`tmp + os.replace`.
- **Concurrency**: single-writer (developer running
`--update-baseline`); CI workers only read.
**Implementation Notes**
- Output format mirrors `scripts/check_i18n_logs.py`:
`<file>:<line>: <reason>: <snippet>` on stderr, summary on stdout,
trailing `OK` or `N issues`.
- The exact refresh command printed on regression failure is:
`python scripts/ci/i18n_cjk_guard.py --update-baseline`.
- `count_path_cjk` invokes `git grep` via `subprocess.run` with
`check=False`; `git grep` exits 1 when there are zero matches, so
the function treats exit codes 0 and 1 as success and any other
code as a hard error.
- Localised key extraction for `en.json` walks the parsed JSON dict;
line numbers are obtained by re-reading the file as text and
matching the value's first textual occurrence.
- Risks: see `research.md` § Risks & Mitigations.
### CI Workflow
#### `i18n-cjk-guard.yml`
| Field | Detail |
|-------|--------|
| Intent | Run the guard on every PR to `main` |
| Requirements | 5.1, 5.2, 5.3, 5.4, 5.5, 5.6 |
| Owner / Reviewers | i18n maintainers |
**Contracts**: Batch / Job [x]
##### Batch / Job Contract
- **Trigger**: `on: pull_request: branches: [main]`.
- **Input / validation**: PR head ref checkout via
`actions/checkout@v4` with `fetch-depth: 1`. Python set up via
`actions/setup-python@v5` with `python-version: '3.11'`.
- **Output / destination**: pass/fail status surfaced as a GitHub
Actions check on the PR. Script stdout/stderr appears in the
workflow log.
- **Idempotency & recovery**: re-running the workflow re-evaluates the
same working tree; no persistent side effects on the runner.
##### Workflow shape (sketch)
```yaml
name: i18n CJK Guard
on:
pull_request:
branches: [main]
jobs:
guard:
runs-on: ubuntu-latest
timeout-minutes: 1
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 1
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- run: python scripts/ci/i18n_cjk_guard.py
```
### Baseline Data File
#### `baseline.txt`
| Field | Detail |
|-------|--------|
| Intent | Persist the per-path CJK match-count baseline |
| Requirements | 2.2, 4.1, 4.2 |
**Contracts**: State [x]
##### Format
```text
# Per-path CJK baseline for the i18n CI guard.
# Format: <path>\t<count>. Sorted lexicographically.
# Refresh via: python scripts/ci/i18n_cjk_guard.py --update-baseline
backend/app <int>
frontend/src <int>
```
- One header block of `#`-prefixed comments (parser ignores).
- Blank lines ignored.
- Lines must match `^(?P<path>[^\t\n]+)\t(?P<count>\d+)$`.
- Trailing newline mandatory.
## Data Models
### Domain Model
- `LocaleFinding` — value object
`(dotted_key: str, line_number: int, snippet: str)`.
- `PathCount` — pair `(scoped_path: str, count: int)`. The full
baseline is a `dict[str, int]` keyed by scoped path.
Invariants:
- `count` is a non-negative integer.
- `scoped_path` is one of `SCOPED_PATHS`.
- `LocaleFinding.snippet` is at most `SNIPPET_MAX_LEN` characters,
truncated with an ellipsis when needed.
## Error Handling
### Error Strategy
- All non-zero exits are accompanied by a stderr message identifying
the failing check, the offending file or path, and (for regressions)
the refresh command. The script never raises uncaught exceptions
past `main()` in normal flow; unexpected I/O errors propagate as
`OSError` with a clear traceback so CI logs surface them clearly.
### Error Categories and Responses
- **Locale failure** (Req 1.2): one stderr line per offending key
(`locales/en.json:<line>: cjk-in-en: <key> = <snippet>`), then a
trailing `N issues` summary.
- **Regression failure** (Req 3.2): one stderr line per regressed
path (`<path>: cjk-regression: baseline=<b> current=<c> delta=+<d>`)
followed by a one-line refresh hint:
`# refresh via: python scripts/ci/i18n_cjk_guard.py --update-baseline`.
- **Missing en.json** (Req 1.4): stderr `locales/en.json: missing
catalogue file`, exit 1.
- **Missing or malformed baseline** (Req 4.5): stderr
`<baseline-path>: missing or malformed; refresh via …`, exit 1.
- **`git grep` unavailable / non-PCRE**: stderr
`git grep failed: <stderr>`, exit 1.
### Monitoring
- The guard is a single short-lived script. All observability is
delegated to GitHub Actions logs (stdout/stderr, run duration).
No external telemetry.
## Testing Strategy
### Unit Tests (Python)
Place tests under `scripts/ci/tests/test_i18n_cjk_guard.py` (or invoke
the script directly via subprocess in a tmp git repo). The project's
test runner is `pytest` (already used by `backend/`), but the new
tests must be runnable with `python -m pytest` from the repo root
without backend dependencies. Tests are scoped to:
1. `scan_locale_cjk` — clean catalogue returns empty list; planted CJK
value returns a single `LocaleFinding` with the correct key and
line number.
2. `count_path_cjk` — given a tmp git repo with N planted CJK lines,
returns N; binary file matches are excluded; untracked file
matches are excluded.
3. `read_baseline` / `write_baseline` round-trip — write counts,
re-read, equal.
4. `read_baseline` malformed input — non-tab line → `BaselineError`.
5. `run_check` end-to-end — passing baseline → exit 0; regressed
baseline → exit 1 and stderr contains the refresh command.
### Integration Tests
1. Workflow shape — `actionlint` (optional, if installed locally) on
`i18n-cjk-guard.yml`. At minimum, `python -c "import yaml;
yaml.safe_load(open('.github/workflows/i18n-cjk-guard.yml'))"` for
YAML validity.
2. Local end-to-end — run
`python scripts/ci/i18n_cjk_guard.py` from the repo root with the
committed baseline; expect exit 0 on a clean checkout of `main`.
3. Refresh end-to-end — run with `--update-baseline`; verify
baseline file is rewritten and a second default run is exit 0.
### Performance / Load
- Single-pass `git grep` over the scoped paths runs in <2 s on the
current repo. The workflow's `timeout-minutes: 1` is a hard ceiling
per Req 5.6.
## Optional Sections
### Security Considerations
- The guard reads only tracked text files; no secrets are accessed.
- The workflow uses `GITHUB_TOKEN` only implicitly via
`actions/checkout`; no additional permissions are requested
(`permissions:` block omitted relies on the repo default of
`contents: read`, which is sufficient).

169
.kiro/specs/i18n-ci-guard/gap-analysis.md Normal file
View File

@ -0,0 +1,169 @@
# Gap Analysis — i18n-ci-guard
Comparison of the approved requirements against the current MiroFish
codebase, focused on what already exists, what is missing, and what
options the design phase should choose between.
## 1. Current State Investigation
### Domain assets already in the repo
- **`scripts/check_i18n_logs.py`** — Python-stdlib-only, exit-code-based
i18n verification script. Uses the same canonical CJK regex
`[一-鿿]` (`U+4E00..U+9FFF`) the new guard needs, prints findings as
`<file>:<line>: <reason>: <snippet>`, and was written for ticket #6.
Strong precedent for the new guard's CLI surface and output format.
- **`scripts/_apply_translations.py`, `scripts/_codemod_i18n.py`,
`scripts/_merge_locale_keys.py`** — i18n tooling sibling scripts.
Convention is to keep auxiliary i18n scripts under `scripts/` at the
repo root.
- **`.github/workflows/docker-image.yml`** — only existing GH Actions
workflow; triggers on tag pushes and `workflow_dispatch`. No PR-time
workflow exists yet, so the new guard introduces the project's first
PR-blocking CI check.
- **PR #27 / branch `chore/i18n-10-e2e-english-verification`** — defines
the audit methodology referenced by the ticket. Its `audit_cjk.sh`
uses `git grep -nIP '[\x{4e00}-\x{9fff}]' -- backend/app frontend/src
locales/en.json` — the canonical scoped scan command. PR #27 is open;
the new guard must work with or without it merged.
- **`.kiro/specs/<feature>/`** — established home for spec artefacts.
`i18n-externalize-backend-logs/` is the closest precedent for an
i18n-flavoured spec.
- **`locales/en.json`, `locales/zh.json`, `locales/languages.json`** —
shared i18n source consumed by both runtimes.
### Conventions extracted
- Auxiliary scripts: `scripts/<purpose>.py`, Python ≥3.11 stdlib only,
shebang `#!/usr/bin/env python3`, double-quoted strings, snake_case,
Google-style docstrings on the module and public functions.
- Output format: `<file>:<line>: <reason>: <snippet>`, summary line
`OK` or `N issues`, exit `0`/`1`.
- Reuse the canonical regex `[一-鿿]` rather than re-deriving range
literals.
- 4-space indent, ≤120 cols, no trailing whitespace, single trailing
newline (`.claude/rules/dev-guidelines.md`).
### Integration surfaces
- **CI**: GitHub Actions, `.github/workflows/`. `ubuntu-latest` runner,
Python 3.11+ via `actions/setup-python@v5` (use the same version
pin already present in the docker-image workflow ecosystem if any).
- **Repo layout boundaries** scoped by the audit: `backend/app/`,
`frontend/src/`, `locales/en.json` — all live at repo root or two
levels deep.
- **Git working tree**: the guard relies on `git grep -I` for tracked,
text-only matches; this binds the guard to a runner that has `git`
available (true on `ubuntu-latest` and on developer machines).
## 2. Requirement-to-Asset Map
| Req | Need | Existing asset | Gap |
| --- | --------------------------------- | ----------------------------------------------------------------------------------------------- | ----------- |
| 1 | CJK scan of `locales/en.json` | `scripts/check_i18n_logs.py` already loads `locales/*.json` and runs the canonical regex. | Missing — new guard must scan en.json specifically and emit `key:line` per offender. |
| 2 | CJK count under `backend/app/` and `frontend/src/` against baseline | Audit `audit_cjk.sh` (PR #27) demonstrates `git grep -nIP` is the canonical scan; no baseline file exists yet on main. | Missing — no per-path counter, no baseline file. |
| 3 | Actionable failure messaging | `check_i18n_logs.py` output format reusable. | Missing — need refresh-baseline command in failure text. |
| 4 | Baseline file lifecycle | None. | Missing — file format and refresh subcommand to design. |
| 5 | GH Actions PR integration | `.github/workflows/` directory exists; one tag-only workflow. | Missing — new `pull_request` workflow. |
| 6 | Local reproducibility | Existing scripts run locally with stdlib; same pattern reusable. | None — covered by following the existing pattern. |
## 3. Implementation Approach Options
### Option A — Extend `scripts/check_i18n_logs.py`
Add a new `--cjk-guard` mode (catalogue scan + per-path baseline diff)
to the existing script, then call it from the new workflow.
- ✅ One file to maintain; reuses the regex constant and CLI.
- ❌ The existing script is tightly scoped to the in-scope backend
modules and the parity check. Mixing a PR-gating regression check into
it dilutes its intent and grows it past the SRP line that the
surrounding scripts respect.
- ❌ The existing script targets a fixed list of backend modules; the
new guard scans whole subtrees. The two scopes don't fit one CLI.
### Option B — New, focused script `scripts/ci/i18n_cjk_guard.py` + new workflow (recommended)
A new directory `scripts/ci/` holds CI-only scripts; the guard is a
single file that performs both checks and supports a `--refresh-baseline`
flag. New workflow `.github/workflows/i18n-cjk-guard.yml` runs it on
every PR to `main`.
- ✅ Clean separation: production-i18n script (`check_i18n_logs.py`)
and CI-gating script (`i18n_cjk_guard.py`) live side by side without
overlapping responsibilities.
- ✅ Mirrors the established convention of one script per
responsibility under `scripts/`.
- ✅ The baseline file lives under the spec dir
(`.kiro/specs/i18n-ci-guard/baseline.txt`), matching the ticket's
"baseline must be committed and reviewable" requirement.
- ❌ One more file in the repo, but the file is small (~150 LoC).
### Option C — Hybrid: shared `cjk_scan.py` helper + thin guard script
Factor the regex + git-grep logic into a tiny shared helper consumed by
both `check_i18n_logs.py` and the new guard.
- ✅ DRY for the regex constant.
- ❌ Premature abstraction: today the only shared element is one
one-line regex. The two scripts have different scopes, output
formats, and consumers. Pulling a helper out now satisfies
consistency without paying for itself; defer until a third caller
appears.
### Recommendation
**Option B**. It matches the project's established "one focused script
per responsibility" convention, isolates the new CI surface from
existing i18n scripts, and keeps the baseline file collocated with
spec metadata where reviewers expect to find it.
## 4. Research Items for Design Phase
- **Baseline file format**: prefer a stable, line-oriented text format
over JSON to minimize diff churn (e.g., `path<TAB>count` per line,
trailing newline). Confirm in design.
- **`git grep` invocation portability**: `git grep -nIP` works on all
modern git builds (≥2.4 ships PCRE2). `ubuntu-latest` ships ≥2.40.
No portability concern; record the assumption explicitly.
- **`fetch-depth`** for the `actions/checkout@v4` step: `git grep`
scans the working tree, not history, so a shallow clone (`fetch-depth:
1`) is sufficient.
- **Workflow timeout budget**: capture the empirical runtime of the
full scan locally (already measured: a single `git grep` over the
scoped paths runs in <2 seconds with ~3.6k matches). The 60-second
ceiling in Req 5 is comfortable.
- **Failure-message refresh command** wording: the design should pin
the exact command shown to contributors so it stays one stable
string developers can copy.
- **Initial baseline values**: with `git grep -nIP '[\x{4e00}-\x{9fff}]'`
on the current branch — `backend/app` = 2707, `frontend/src` = 902,
`locales/en.json` = 0. The committed baseline must be regenerated
against `main` at implementation time so it reflects the merge target.
## 5. Effort & Risk
- **Effort**: **S** (1–3 days). Small, self-contained additions
(one Python script, one workflow file, one baseline file, plus the
spec). All patterns already exist in the repo.
- **Risk**: **Low**. No production-source changes, no new dependencies,
no architectural shifts. The only failure mode is a noisy guard
blocking unrelated PRs — mitigated by the per-path baseline ratchet.
## 6. Recommendations for Design Phase
- Adopt **Option B** (new focused script + new workflow + baseline file
under spec dir).
- Lock in the canonical regex `[一-鿿]` and the canonical scan command
`git grep -nIP '[\x{4e00}-\x{9fff}]' -- <path>` to keep this guard
bytewise-aligned with the audit pipeline.
- Use a line-oriented baseline format keyed by scoped path; explicit
`--refresh-baseline` (or equivalent) subcommand updates it; no
implicit overwrite.
- Output: machine-friendly findings on stderr, summary on stdout,
exit `0`/`1`.
- The workflow should run only on `pull_request` to `main` (Req 5.1)
with `fetch-depth: 1` and `actions/setup-python@v5`. No third-party
packages.
- Baseline counts must be recomputed against `main` before the PR
ships; do not commit baselines from a feature branch's working tree.

189
.kiro/specs/i18n-ci-guard/requirements.md Normal file
View File

@ -0,0 +1,189 @@
# Requirements Document
## Project Description (Input)
Add a permanent CI guard that runs an i18n CJK audit on every pull request.
Linked GitHub issue: #26 (.ticket/26.md).
The guard must fail a PR build when:
1. locales/en.json contains any CJK character (range U+4E00..U+9FFF), or
2. The total count of CJK matches across backend/app/ and frontend/src/ regresses (i.e. exceeds) a committed baseline value.
## Introduction
The i18n initiative has driven the project toward English-by-default UI, logs,
prompts, and documentation. Manual audits (see PR #27, the
`i18n-e2e-english-verification` spec) have repeatedly surfaced regressions
where Chinese strings re-enter the codebase. This spec installs a permanent,
self-contained CI guard that runs on every pull request and fails the build
when (a) `locales/en.json` is no longer CJK-clean, or (b) the total CJK match
count under `backend/app/` and `frontend/src/` regresses against a committed
baseline.
The guard is intentionally minimal: it captures the two highest-signal checks
from the larger audit pipeline so it can run on every PR with a sub-minute
budget and without depending on the (currently unmerged) verification spec.
The committed baseline lets the project ratchet down gaps over time without
blocking unrelated PRs on pre-existing CJK content.
## Boundary Context
- **In scope**:
- A locally runnable Python script that performs both guard checks on the
current working tree.
- A baseline file committed under the spec directory recording the
accepted CJK match counts per scoped path.
- A GitHub Actions workflow that runs the script on every pull request
targeting `main` and fails the build when either check fails.
- A clear, actionable failure message (which path regressed, baseline
value, current value, command to update the baseline).
- **Out of scope**:
- The full classification pipeline (`classify.py`, `render_report.py`,
`post_comment.sh`) from the unmerged `i18n-e2e-english-verification`
spec — those scripts perform deeper audit work and are not required
for the PR-time guard.
- Auto-updating the baseline on `main` (the baseline is a normal
reviewable file).
- Translation work itself; this spec only enforces a regression gate.
- Any change to production source under `backend/app/`, `frontend/src/`,
or `locales/` apart from translations needed to satisfy the guard
against its own initial baseline.
- **Adjacent expectations**:
- PR #27 (`chore/i18n-10-e2e-english-verification`) provides the
methodology referenced here. This spec must remain functional whether
PR #27 has been merged or not.
- The guard reuses the canonical CJK regex range
`[一-鿿]` already established by that audit.
## Requirements
### Requirement 1: Locale-catalogue CJK cleanliness check
**Objective:** As a maintainer of the English locale catalogue, I want every
PR to fail when `locales/en.json` reintroduces any CJK character, so that the
English catalogue stays CJK-free.
#### Acceptance Criteria
1. When the guard script is run from the repository root, the i18n CI Guard
shall scan the contents of `locales/en.json` for any character in the
range `U+4E00..U+9FFF`.
2. If `locales/en.json` contains at least one such character, the i18n CI
Guard shall exit with a non-zero status and report each offending
`key:line` pair on standard output.
3. While `locales/en.json` contains zero such characters, the i18n CI Guard
shall report the catalogue as CJK-clean.
4. If `locales/en.json` is missing or unreadable, the i18n CI Guard shall
exit with a non-zero status and emit an explicit error message naming
the missing file.
### Requirement 2: Backend/frontend CJK regression check against committed baseline
**Objective:** As a maintainer of English support across the codebase, I
want every PR to fail when the total CJK match count under `backend/app/`
or `frontend/src/` exceeds a committed baseline, so that the codebase
ratchets monotonically toward English-only without blocking PRs on
pre-existing CJK content.
#### Acceptance Criteria
1. When the guard script is run, the i18n CI Guard shall count the total
number of CJK matches (range `U+4E00..U+9FFF`, line-level, text files
only) under each of the scoped paths `backend/app/` and `frontend/src/`.
2. The i18n CI Guard shall read the baseline counts from a single
committed baseline file under the spec directory.
3. If the current count for any scoped path exceeds the baseline count for
that path, the i18n CI Guard shall exit with a non-zero status.
4. While the current count for every scoped path is less than or equal to
the baseline, the i18n CI Guard shall exit with status zero for this
check.
5. The i18n CI Guard shall ignore matches inside binary files
(image, font, archive, lockfile, or other non-text formats) by relying
on `git grep -I` semantics.
6. The i18n CI Guard shall scope its scan to tracked files only (matches
in untracked or ignored files shall not contribute to the count).
### Requirement 3: Actionable failure messaging
**Objective:** As a contributor whose PR was rejected by the guard, I want
the failure message to tell me exactly what regressed and how to fix it,
so that I can either translate the offending content or — when intentional —
update the baseline through normal review.
#### Acceptance Criteria
1. If the locale-catalogue check fails, the i18n CI Guard shall print, for
each offending entry: the dotted catalogue key, the line number in
`locales/en.json`, and a truncated snippet of the value.
2. If the regression check fails, the i18n CI Guard shall print, for each
regressed scoped path: the path name, the baseline count, the current
count, and the delta.
3. If the regression check fails, the i18n CI Guard shall print the exact
shell command a contributor must run locally to refresh the baseline
file so the PR can be re-reviewed against the new value.
4. The i18n CI Guard shall print, on success, a one-line summary per check
confirming the catalogue is CJK-clean and the per-path counts are at or
below baseline.
### Requirement 4: Baseline file lifecycle
**Objective:** As a reviewer enforcing English support, I want the baseline
to live in the repository as a small, human-readable file that only changes
through code review, so that downward ratcheting is intentional and
auditable.
#### Acceptance Criteria
1. The i18n CI Guard shall store the baseline as a single committed file
under `.kiro/specs/i18n-ci-guard/`.
2. The baseline file shall record one count per scoped path, in a stable,
diff-friendly text format (no JSON line shuffling, no trailing
whitespace).
3. When the guard script is invoked with an explicit "refresh baseline"
subcommand or flag, the i18n CI Guard shall overwrite the baseline file
with the current per-path counts and exit with status zero.
4. While no refresh flag is supplied, the i18n CI Guard shall never modify
the baseline file.
5. If the baseline file is missing at check time, the i18n CI Guard shall
exit with a non-zero status and instruct the contributor to refresh it.
### Requirement 5: GitHub Actions PR integration
**Objective:** As a project maintainer, I want every pull request targeting
`main` to be gated by the guard, so that no merge silently regresses the
English-only state of the catalogue or codebase.
#### Acceptance Criteria
1. The i18n CI Guard workflow shall trigger on every `pull_request` event
whose base ref is `main`.
2. While the workflow runs, the i18n CI Guard shall check out the PR head
commit with full history sufficient for `git grep` to scan tracked
files.
3. When the guard script exits with non-zero status, the workflow shall
fail and surface the script's standard output and standard error in the
GitHub Actions log.
4. When the guard script exits with status zero, the workflow shall pass.
5. The workflow shall use only Python from the standard
`actions/setup-python` distribution and tools already available on the
GitHub-hosted `ubuntu-latest` runner (`bash`, `git`); it shall not
install third-party Python packages.
6. The workflow shall complete within sixty seconds of wall-clock time on
a clean `ubuntu-latest` runner.
### Requirement 6: Local reproducibility
**Objective:** As a developer preparing a PR, I want to run the same guard
locally before pushing, so that I can catch regressions before CI does.
#### Acceptance Criteria
1. When the guard script is invoked from a developer machine that has
Python 3.11 or newer and `git` available, the i18n CI Guard shall
produce the same pass/fail result and the same per-path counts that
it would produce in CI for the same working tree.
2. The i18n CI Guard shall expose a single, stable invocation entry point
(a script under `scripts/ci/`) documented in the spec's design and
README touchpoints.
3. The i18n CI Guard shall require zero environment variables or secrets
to run locally.

175
.kiro/specs/i18n-ci-guard/research.md Normal file
View File

@ -0,0 +1,175 @@
# Research & Design Decisions — i18n-ci-guard
## Summary
- **Feature**: `i18n-ci-guard`
- **Discovery Scope**: Simple Addition (one Python script + one GH Actions
workflow + one baseline file). Extension-flavoured because it builds on
established `scripts/` conventions and the canonical CJK regex used by
the larger audit pipeline.
- **Key Findings**:
- The canonical CJK match command `git grep -nIP '[\x{4e00}-\x{9fff}]'
-- <path>` is already used by the unmerged audit pipeline (PR #27)
and is portable on every git ≥2.4 (`ubuntu-latest` ships ≥2.40).
- `scripts/check_i18n_logs.py` is a strong CLI/style precedent:
Python-stdlib-only, exit `0`/`1`, output as `<file>:<line>:
<reason>: <snippet>`, canonical regex `[一-鿿]`.
- The repository has no existing `pull_request`-triggered GH Actions
workflow; this guard introduces the first one. The only existing
workflow (`.github/workflows/docker-image.yml`) runs on tag pushes
only.
- Current per-path counts on this branch:
`backend/app=2707, frontend/src=902, locales/en.json=0`. These are
sample counts; the committed baseline must be regenerated against
`main` at implementation time.
## Research Log
### Canonical scan command
- **Context**: Requirement 2 needs a stable per-path CJK count and
Requirement 5.5 forbids third-party packages.
- **Sources Consulted**:
- `audit_cjk.sh` from PR #27 commit `3481408`.
- `git grep` man page.
- **Findings**:
- `git grep -nIP '[\x{4e00}-\x{9fff}]' -- <path>` returns one match
per matching line in tracked, text-only files. `-I` excludes binary
files; `-P` enables PCRE2 so the `\x{...}` Unicode range works.
- This matches the input format consumed by the existing audit
classifier, so the guard's match counts are directly comparable
across pipelines.
- **Implications**:
- The guard re-uses this exact command; no new dependencies.
- Because `-I` skips binary files and tracked-only is the default,
Requirements 2.5 and 2.6 are satisfied by the command itself
rather than by additional script logic.
### Baseline file format
- **Context**: Requirement 4 needs a diff-friendly committed baseline.
- **Sources Consulted**:
- Diff churn behaviour of JSON vs. line-oriented text in this repo's
history (e.g. `locales/*.json` PR diffs frequently re-key, while
plain-text `parity.txt` from PR #27 reads cleanly).
- **Findings**:
- Line-oriented `<path>\t<count>` files produce minimal diffs and
require no JSON parser.
- A two-line file (one per scoped path) is large enough to be
self-explanatory and small enough to never line-shuffle.
- **Implications**:
- Use plain text, sorted by path, single trailing newline. Reject
the file as malformed if the script cannot parse it (Req 4.5).
### Locale-catalogue scan path
- **Context**: Requirement 1 wants `key:line` per CJK offender in
`locales/en.json`.
- **Sources Consulted**:
- `scripts/check_i18n_logs.py` (`flatten_keys` reuse pattern).
- `check_parity.py` from PR #27 (`flatten`, `[cjk-in-en]` block).
- **Findings**:
- Both precedents flatten the locale dict and run the canonical
regex against each leaf string value. Line numbers are derivable
by re-reading the file as text and matching the value's first
occurrence (good enough for an actionable error message).
- Empty-string values and non-string leaf values (booleans, null)
are skipped.
- **Implications**:
- Implement a tiny flatten-then-scan helper inside the guard
script; do not add a new shared utility module.
### GH Actions trigger and budget
- **Context**: Requirements 5.1, 5.5, 5.6.
- **Sources Consulted**:
- GitHub-hosted runners reference (`ubuntu-latest`).
- `actions/setup-python@v5` README.
- **Findings**:
- `ubuntu-latest` has Python 3.10+ pre-installed; `actions/setup-python@v5`
pins to 3.11 in <5 s.
- A single `git grep` over the scoped paths runs in <2 s on this
repo (~3.6k matches). End-to-end the workflow comfortably fits
inside the 60 s ceiling.
- **Implications**:
- Use `actions/checkout@v4` with `fetch-depth: 1`,
`actions/setup-python@v5` with `python-version: '3.11'`, and run
the script directly. No caching layer needed.
## Architecture Pattern Evaluation
| Option | Description | Strengths | Risks / Limitations | Notes |
|--------|-------------|-----------|---------------------|-------|
| A. Extend `check_i18n_logs.py` | Add `--cjk-guard` mode to existing script | Reuses one file | Conflates two scopes; existing script is module-scoped, guard is subtree-scoped | Rejected |
| B. New `scripts/ci/i18n_cjk_guard.py` + new workflow | Single-purpose script + workflow + baseline file | Clean SRP; matches "one script per responsibility" precedent | One additional file | **Selected** |
| C. Shared `cjk_scan.py` helper + thin guard | Factor regex/git-grep into helper | DRY for regex constant | Premature abstraction; only one shared symbol today | Rejected |
## Design Decisions
### Decision: Single-purpose CI script + GH Actions workflow (Option B)
- **Context**: Requirements 1–6 demand a small, self-contained guard.
- **Alternatives Considered**: A (extend), C (shared helper).
- **Selected Approach**: New script `scripts/ci/i18n_cjk_guard.py`,
new workflow `.github/workflows/i18n-cjk-guard.yml`, baseline file
`.kiro/specs/i18n-ci-guard/baseline.txt`.
- **Rationale**: Matches the project's "one focused script per
responsibility" convention; isolates a CI-blocking surface from the
existing i18n developer scripts; keeps the baseline collocated with
the spec for review traceability.
- **Trade-offs**: One more file in `scripts/` vs. tighter cohesion.
- **Follow-up**: When a third caller wants the canonical regex, factor
it out then.
### Decision: Plain-text baseline format
- **Context**: Requirement 4.2 demands stable, diff-friendly format.
- **Alternatives Considered**: JSON, YAML.
- **Selected Approach**: One line per scoped path: `<path>\t<count>`,
sorted lexicographically by path, single trailing newline.
- **Rationale**: Zero parser dependency; predictable diffs; trivial
to refresh atomically.
- **Trade-offs**: Less expressive than JSON (no nested structure), but
the data model is two integers — nesting is unnecessary.
### Decision: Refresh via `--update-baseline` subcommand-style flag
- **Context**: Requirement 4.3 needs an explicit refresh path.
- **Alternatives Considered**: Separate `update_baseline.py` script;
Makefile target.
- **Selected Approach**: Single script with two modes: default (check
+ exit 0/1) and `--update-baseline` (overwrite baseline + exit 0).
- **Rationale**: One CLI surface to remember; the failure message
prints the exact command to run.
- **Trade-offs**: Slightly more conditional logic in one script;
acceptable given the small total LoC.
### Decision: Workflow runs only on `pull_request` to `main`
- **Context**: Requirement 5.1.
- **Alternatives Considered**: Run on `push` to all branches as well;
run on `pull_request` to any base branch.
- **Selected Approach**: `on.pull_request.branches: [main]` only.
- **Rationale**: Aligns with how the existing project uses `main` as
the protected branch (see `gh pr list` history; every feature PR
targets `main`). Avoids redundant runs on intra-branch chains.
- **Trade-offs**: A direct push to `main` would not be guarded — but
branch protection already discourages that path (per
`dev-guidelines.md`).
## Risks & Mitigations
- **Risk**: Baseline drifts upward unintentionally during
`--update-baseline` runs, hiding real regressions.
- *Mitigation*: Failure message instructs contributors to refresh
*only when intentional*; the baseline file is reviewed in the same
PR diff. Acceptance Criteria 3.3 makes this explicit.
- **Risk**: `git grep -P` not built with PCRE on a developer's local
git build (rare on Linux/macOS, possible on minimal Windows builds).
- *Mitigation*: The guard prints a clear error if `git grep` exits
non-zero with PCRE mode; documents Python ≥3.11 + git ≥2.20 as
prerequisites.
- **Risk**: Baseline counts captured on a feature branch include
changes not yet on `main`, mis-anchoring the ratchet.
- *Mitigation*: The implementation task explicitly recomputes
baseline against `origin/main` before committing; documented in
`tasks.md`.
## References
- PR #27 audit pipeline (`audit_cjk.sh`, `check_parity.py`,
`classify.py`) — methodology source of truth.
- `scripts/check_i18n_logs.py` — CLI/style precedent.
- `git grep` man page — `-n`, `-I`, `-P` flag semantics.
- GitHub Actions `actions/setup-python@v5` and `actions/checkout@v4`
README pages.

24
.kiro/specs/i18n-ci-guard/spec.json Normal file
View File

@ -0,0 +1,24 @@
{
"feature_name": "i18n-ci-guard",
"created_at": "2026-05-08T00:25:37Z",
"updated_at": "2026-05-08T00:40:00Z",
"language": "en",
"phase": "tasks-generated",
"approvals": {
"requirements": {
"generated": true,
"approved": true
},
"design": {
"generated": true,
"approved": true
},
"tasks": {
"generated": true,
"approved": true
}
},
"ready_for_implementation": true,
"ticket": "26",
"ticket_url": "https://github.com/salestech-group/MiroFish/issues/26"
}

157
.kiro/specs/i18n-ci-guard/tasks.md Normal file
View File

@ -0,0 +1,157 @@
# Implementation Tasks — i18n-ci-guard
> Approved spec: see `requirements.md`, `design.md`, `research.md`,
> `gap-analysis.md` in this directory.
## Tasks
- [x] 1. Foundation: scaffold the CI guard script with stable CLI surface and stdlib-only dependencies
- [x] 1.1 Create the empty guard script and CLI skeleton
- Place the new script at the path designated by the design (`scripts/ci/`).
- Establish the module docstring, the canonical CJK regex constant, the
scoped-paths constant tuple, and the `argparse` parser exposing default
check mode plus an explicit `--update-baseline` flag and a
`--baseline` path override.
- Confirm the script exits 0 on a smoke `--help` invocation and rejects
unknown flags with non-zero exit.
- Observable: running `python scripts/ci/i18n_cjk_guard.py --help` from
the repo root prints usage text containing every documented flag and
exits 0; running with an unknown flag exits non-zero.
- _Requirements: 5.5, 6.2, 6.3_
- _Boundary: i18n_cjk_guard.py_
- [x] 2. Core: implement the two CJK checks
- [x] 2.1 Implement the locale-catalogue scan
- Recursively walk the parsed `locales/en.json` dict, applying the
canonical regex to every string leaf to gather offending entries.
- Compute the source line number by re-reading the file as text and
matching the value's first textual occurrence; truncate snippets to
the documented snippet length.
- On a missing or unreadable catalogue file, emit a clear stderr
message and exit non-zero.
- Observable: against a synthetic clean catalogue, the function returns
an empty list; against a synthetic catalogue with one CJK value, it
returns exactly one finding tuple with the correct dotted key and
line number.
- _Requirements: 1.1, 1.2, 1.3, 1.4, 3.1_
- _Boundary: i18n_cjk_guard.py_
- [x] 2.2 (P) Implement the per-path CJK count via `git grep`
- Invoke `git grep -nIP '[\x{4e00}-\x{9fff}]' -- <scoped_path>` for each
scoped path; treat exit codes 0 (matches found) and 1 (no matches) as
success, any other exit code as a hard error reported on stderr.
- Count lines of stdout; the result for a zero-match path must be the
integer `0`, never an exception.
- Reject working-tree states where `git` is not available or PCRE is
not enabled, with a clear stderr message.
- Observable: against a tmp git repository with N planted CJK lines
under a scoped path, the function returns N; with zero CJK content,
it returns 0; binary files and untracked files do not contribute.
- _Requirements: 2.1, 2.4, 2.5, 2.6_
- _Boundary: i18n_cjk_guard.py_
- [x] 2.3 Implement baseline file read/write with strict format
- Parse the baseline file as `<path>\t<count>` lines, ignoring `#`
comments and blank lines, raising a typed error on malformed input
or missing file.
- Write atomically (`tmp + os.replace`) with sorted entries, a single
header comment block, and a single trailing newline.
- Observable: a round-trip write/read of a deterministic counts dict
yields the same dict; a baseline file containing a non-tab line is
rejected with a clear error; the baseline file ends with exactly one
`\n`.
- _Requirements: 4.2, 4.3_
- _Boundary: i18n_cjk_guard.py_
- [x] 3. Integration: wire the two checks into the default and refresh modes
- [x] 3.1 Compose the default check mode
- Run both checks under all conditions (do not short-circuit), so a
single CI log shows every failure in one pass.
- Print a one-line success summary per check on stdout when both pass.
- On locale failure, print `<file>:<line>: <reason>: <snippet>` lines
on stderr and a trailing `N issues` summary; on regression failure,
print `<path>: cjk-regression: baseline=<b> current=<c> delta=+<d>`
lines plus the exact verbatim refresh command.
- Surface a non-zero exit when either check fails and exit 0 only when
both pass.
- Observable: against a working tree with the committed baseline at or
above the current count and a CJK-clean en.json, exit code is 0 and
stdout contains the success summary; planting one CJK char in
en.json or planting enough new CJK lines to break the baseline
yields exit 1 and the documented stderr text.
- _Requirements: 1.2, 1.3, 1.4, 2.2, 2.3, 2.4, 3.1, 3.2, 3.3, 3.4, 4.4, 4.5_
- _Boundary: i18n_cjk_guard.py_
- [x] 3.2 Compose the `--update-baseline` mode
- When the flag is provided, recompute current per-path counts and
overwrite the baseline file via the atomic writer; print the new
counts on stdout; exit 0.
- When the flag is absent, never write the baseline file under any
code path.
- Observable: invoking with `--update-baseline` rewrites the baseline
file's contents to match current counts and exits 0; running the
default mode immediately afterward exits 0.
- _Requirements: 4.3, 4.4_
- _Boundary: i18n_cjk_guard.py_
- [x] 4. Establish the committed baseline anchored to `main`
- [x] 4.1 Capture initial baseline counts against `main`
- Operate from a tree that reflects `origin/main`'s state for the
scoped paths (e.g., a fresh checkout, a worktree at `origin/main`,
or `git checkout origin/main -- backend/app frontend/src` followed
by a clean revert) so the committed baseline does not over- or
under-count relative to the merge target.
- Run `--update-baseline` to materialize the counts; confirm the
resulting file is exactly two non-comment data lines (one per
scoped path) sorted lexicographically.
- Observable: the baseline file is committed to
`.kiro/specs/i18n-ci-guard/baseline.txt` and `python scripts/ci/i18n_cjk_guard.py`
against the same `main`-aligned tree exits 0.
- _Requirements: 4.1, 4.2_
- _Boundary: baseline.txt_
- [x] 5. Wire the guard into GitHub Actions on every PR to `main`
- [x] 5.1 Add the PR-time workflow
- Create the workflow file at the path designated by the design,
triggered on `pull_request` whose base ref is `main`.
- Set explicit minimal permissions (`contents: read`), a one-minute
job timeout, `actions/checkout@v4` with `fetch-depth: 1`, and
`actions/setup-python@v5` pinned to Python 3.11.
- The single executable step invokes the guard script with no
arguments; the workflow surfaces the script's stdout and stderr in
the GitHub Actions log without filtering.
- Observable: the workflow YAML parses cleanly; on a PR with no CJK
regression, the job passes; on a PR that introduces a CJK regression
or CJK in en.json, the job fails and the log shows the documented
failure messages.
- _Requirements: 5.1, 5.2, 5.3, 5.4, 5.5, 5.6_
- _Boundary: i18n-cjk-guard.yml_
- [x] 6. Validation: tests and end-to-end checks
- [x] 6.1 Add unit and integration tests for the guard script
- Cover the locale scan against a synthetic clean catalogue and a
synthetic CJK-tainted catalogue, asserting findings tuples match.
- Cover the per-path counter against a tmp git repo with both N>0
and N=0 planted CJK lines, asserting the zero-match path exits
cleanly with a count of 0.
- Cover the baseline read/write round-trip and the malformed-input
rejection path.
- Cover the default mode end-to-end (pass and fail paths) with the
expected exit codes and stderr fragments, including the verbatim
refresh command on regression failure.
- Observable: `python -m pytest scripts/ci/tests/test_i18n_cjk_guard.py`
from the repo root passes locally with stdlib-only Python.
- _Requirements: 1.1, 1.2, 1.3, 1.4, 2.1, 2.4, 2.5, 2.6, 3.3, 4.3, 4.5, 6.1, 6.3_
- _Boundary: scripts/ci/tests/_
- [x] 6.2 Run the guard locally to confirm reproducibility against the committed baseline
- From a clean working tree at `main` (or a worktree at `origin/main`
+ this branch's new files merged on top), invoke the guard with no
arguments and confirm exit code 0 and the success summary.
- Confirm the same command is the documented developer entry point
referenced from the failure-message refresh hint.
- Observable: terminal session shows exit code 0 and the documented
one-line per-check success summary; the same script path (`scripts/ci/i18n_cjk_guard.py`)
appears verbatim in the regression-failure refresh hint.
- _Requirements: 6.1, 6.2, 6.3_
- _Boundary: i18n_cjk_guard.py, baseline.txt_

2924
.kiro/specs/i18n-e2e-english-verification/audit/9dcaecd2d27e6325bae0c53b9ab41eb86d0269cd/cjk-grep-bucketed.txt Normal file
View File

File diff suppressed because it is too large Load Diff

2916
.kiro/specs/i18n-e2e-english-verification/audit/9dcaecd2d27e6325bae0c53b9ab41eb86d0269cd/cjk-grep.txt Normal file
View File

File diff suppressed because it is too large Load Diff

2917
.kiro/specs/i18n-e2e-english-verification/audit/9dcaecd2d27e6325bae0c53b9ab41eb86d0269cd/classified.csv Normal file
View File

File diff suppressed because it is too large Load Diff

60
.kiro/specs/i18n-e2e-english-verification/audit/9dcaecd2d27e6325bae0c53b9ab41eb86d0269cd/comment-body.md Normal file
View File

@ -0,0 +1,60 @@
### Verification report - run on commit `9dcaecd2d27e6325bae0c53b9ab41eb86d0269cd`
This run was produced by `.kiro/specs/i18n-e2e-english-verification/audit/scripts/run_audit.sh`.
Captured artefacts live under `.kiro/specs/i18n-e2e-english-verification/audit/<commit-sha>/`.
**Audit summary:** 2916 CJK matches across the auditable paths.
- 237 `gap` (actionable, see follow-ups)
- 380 `review-needed` (soft signal; needs human eyeball)
- 2299 `deliberate` (mostly backend docstrings/comments - covered by issue #7)
- 0 `non-applicable` (binary file false positives - excluded)
**Gap-category breakdown:** backend-prompt-label=143, frontend-ui-string=49, frontend-regex-parser=36, backend-log=9
---
#### Issue checklist mapping
## Section 5 - Issue #10 checklist mapping
Each line below is taken from the ticket body, with an explicit status.
- [ ] **GAP** - **Frontend UI** — every label, button, modal, error toast, and tooltip in EN. No Chinese strings on screen. - 29 hard-coded CJK literal(s) in `frontend/src/views|components/`
- [ ] **GAP** - **Step 1 — Graph Build** - 5 gap(s) classified, see Section 1/3
- MANUAL-PENDING: Status messages in EN - not verifiable statically; awaiting live run
- GAP: Ontology JSON descriptions in EN (depends on #2) - 14 gap(s) classified, see Section 1/3
- GAP: Backend logs in EN (depends on #6) - 9 gap(s) classified, see Section 1/3
- [ ] **GAP** - **Step 2 — Env Setup** - 61 gap(s) classified, see Section 1/3
- GAP: Generated agent profiles (`bio`, `persona`, `profession`, `interested_topics`) in EN (depends on #3) - 61 gap(s) classified, see Section 1/3
- MANUAL-PENDING: `gender` still the English enum (`male` / `female` / `other`) - not verifiable statically; awaiting live run
- [ ] **GAP** - **Step 3 — Simulation** - 14 gap(s) classified, see Section 1/3
- GAP: Sim config `content`, `narrative_direction`, `hot_topics`, `reasoning` in EN (depends on #4) - 14 gap(s) classified, see Section 1/3
- MANUAL-PENDING: `poster_type` still PascalCase English - not verifiable statically; awaiting live run
- MANUAL-PENDING: `stance` still one of `supportive` / `opposing` / `neutral` / `observer` - not verifiable statically; awaiting live run
- GAP: Generated tweets / Reddit posts in EN (depends on #3 personas + #4 sim config) - 14 gap(s) classified, see Section 1/3
- [ ] **GAP** - **Step 4 — Report** - 70 gap(s) classified, see Section 1/3
- GAP: Report sections, headings, prose in EN (depends on #5) - 70 gap(s) classified, see Section 1/3
- MANUAL-PENDING: ReACT thinking trace in EN - requires live walkthrough
- MANUAL-PENDING: Tool-call results render correctly - requires live walkthrough
- [ ] **GAP** - **Step 5 — Interaction** - 2 gap(s) classified, see Section 1/3
- GAP: Interview chat replies in EN (depends on #3) - 2 gap(s) classified, see Section 1/3
- GAP: Report Agent chat replies in EN (depends on #5) - 72 gap(s) classified, see Section 1/3
- [ ] **GAP** - **Backend logs** — full pipeline-run logs in EN (depends on #6) - 9 gap(s) classified, see Section 1/3
- [ ] **GAP** - **Locale propagation** — confirm `Accept-Language: en` (or thread-local locale set via `set_locale`) reaches background tasks and survives the OASIS subprocess boundary. - 9 CJK log strings on EN code path
- [ ] **MANUAL-PENDING** - Every touchpoint above renders in Chinese; no English regressions. - requires live walkthrough
- [ ] **MANUAL-PENDING** - zh.json backfill (#8) covered: Step 3, Step 4, Step 5, and graph panel labels are all Chinese. - not verifiable statically; awaiting live run
---
#### How to re-run
```bash
# from the repository root, on any commit:
bash .kiro/specs/i18n-e2e-english-verification/audit/scripts/run_audit.sh
# artefacts at .kiro/specs/i18n-e2e-english-verification/audit/<HEAD-sha>/
```
If `gh` is not authenticated when re-running, the comment body and follow-up bodies are written to `PENDING-issue-10-comment.md` / `PENDING-followups/` for a human to post.
Out of scope for this run (per R5.3 / R7.3): live UI walkthrough, full Docker-Compose pipeline run, and any inline gap fixes.

1
.kiro/specs/i18n-e2e-english-verification/audit/9dcaecd2d27e6325bae0c53b9ab41eb86d0269cd/comment-url.txt Normal file
View File

@ -0,0 +1 @@
https://github.com/salestech-group/MiroFish/issues/10#issuecomment-4400060417

4
.kiro/specs/i18n-e2e-english-verification/audit/9dcaecd2d27e6325bae0c53b9ab41eb86d0269cd/followup-urls.txt Normal file
View File

@ -0,0 +1,4 @@
https://github.com/salestech-group/MiroFish/issues/23
https://github.com/salestech-group/MiroFish/issues/24
https://github.com/salestech-group/MiroFish/issues/25
https://github.com/salestech-group/MiroFish/issues/26

143
.kiro/specs/i18n-e2e-english-verification/audit/9dcaecd2d27e6325bae0c53b9ab41eb86d0269cd/gap-report.md Normal file
View File

@ -0,0 +1,143 @@
# Verification gap report - i18n-e2e-english-verification
**Commit:** `9dcaecd2d27e6325bae0c53b9ab41eb86d0269cd`
## Overview
- Total CJK matches audited: **2916**
- Class distribution: deliberate=2299, review-needed=380, gap=237
- Gap categories: backend-prompt-label=143, frontend-ui-string=49, frontend-regex-parser=36, backend-log=9
- Gap pipeline steps: Report=70, Env Setup=61, n/a=47, UI=29, Simulation=14, Logs=9, Graph Build=5, Interaction=2
## Section 1 - Static CJK audit
Canonical command (PCRE):
```
git grep -nIP "[\x{4e00}-\x{9fff}]" -- backend/app frontend/src locales/en.json
```
Raw output captured at `audit/9dcaecd2d27e6325bae0c53b9ab41eb86d0269cd/cjk-grep.txt` and bucketed at `audit/9dcaecd2d27e6325bae0c53b9ab41eb86d0269cd/cjk-grep-bucketed.txt`.
`locales/en.json` CJK matches: **0** (acceptance: zero).
Top files by gap count:
| File | Gap count |
|------|-----------|
| `backend/app/services/oasis_profile_generator.py` | 60 |
| `frontend/src/components/Step4Report.vue` | 50 |
| `backend/app/services/zep_graph_memory_updater.py` | 47 |
| `frontend/src/views/Process.vue` | 29 |
| `backend/app/services/report_agent.py` | 20 |
| `backend/app/services/simulation_config_generator.py` | 13 |
| `backend/app/services/ontology_generator.py` | 5 |
| `backend/app/utils/retry.py` | 4 |
| `backend/app/api/graph.py` | 3 |
| `frontend/src/components/Step2EnvSetup.vue` | 3 |
| `frontend/src/components/Step5Interaction.vue` | 2 |
| `frontend/src/components/Step3Simulation.vue` | 1 |
## Section 2 - Locale catalogue parity
```
# Locale parity for HEAD
# en keys: 953
# zh keys: 953
[missing-keys]
# (none)
[cjk-in-en]
# (none)
[identical-values]
# (none)
```
## Section 3 - LLM-prompt locale verification
Backend prompt-label gaps (CJK string literals inside services that compose LLM prompts): **143**
First 10 examples (file:line - match):
- `backend/app/services/oasis_profile_generator.py:65` - "username": self.user_name, # OASIS 库要求字段名为 username（无下划线）
- `backend/app/services/oasis_profile_generator.py:93` - "username": self.user_name, # OASIS 库要求字段名为 username（无下划线）
- `backend/app/services/oasis_profile_generator.py:194` - raise ValueError("LLM_API_KEY 未配置")
- `backend/app/services/oasis_profile_generator.py:384` - all_summaries.add(f"相关实体: {node.name}")
- `backend/app/services/oasis_profile_generator.py:390` - context_parts.append("事实信息:\n" + "\n".join(f"- {f}" for f in results["facts"][:20]))
- `backend/app/services/oasis_profile_generator.py:392` - context_parts.append("相关实体:\n" + "\n".join(f"- {s}" for s in results["node_summaries"][:10]))
- `backend/app/services/oasis_profile_generator.py:422` - context_parts.append("### 实体属性\n" + "\n".join(attrs))
- `backend/app/services/oasis_profile_generator.py:438` - relationships.append(f"- {entity.name} --[{edge_name}]--> (相关实体)")
- `backend/app/services/oasis_profile_generator.py:440` - relationships.append(f"- (相关实体) --[{edge_name}]--> {entity.name}")
- `backend/app/services/oasis_profile_generator.py:443` - context_parts.append("### 相关事实和关系\n" + "\n".join(relationships))
- ... and 133 more (see `classified.csv`)
These prompts feed the LLM verbatim; CJK labels bias the model toward Chinese output even when the requested locale is English.
## Section 4 - Locale propagation surface
| Boundary | Status | Evidence |
|----------|--------|----------|
| HTTP -> Flask handler | manual-pending | runtime not exercised in sandbox; static review showed no per-request locale carrier |
| Flask handler -> Task worker | manual-pending | thread-local `set_locale` referenced in CLAUDE.md but not statically verified end-to-end |
| Task worker -> OASIS subprocess | manual-pending | subprocess boundary requires live run |
| Backend logger | gap | 9 hard-coded CJK log line(s) on EN code path |
First 10 backend-log gap examples:
- `backend/app/api/graph.py:385` - build_logger.info(f"[{task_id}] 开始构建图谱...")
- `backend/app/api/graph.py:494` - build_logger.info(f"[{task_id}] 图谱构建完成: graph_id={graph_id}, 节点={node_count}, 边={edge_count}")
- `backend/app/api/graph.py:513` - build_logger.error(f"[{task_id}] 图谱构建失败: {str(e)}")
- `backend/app/services/oasis_profile_generator.py:945` - print(f"开始生成Agent人设 - 共 {total} 个实体，并行数: {parallel_count}")
- `backend/app/services/oasis_profile_generator.py:1001` - print(f"人设生成完成！共生成 {len([p for p in profiles if p])} 个Agent")
- `backend/app/utils/retry.py:55` - logger.error(f"函数 {func.__name__} 在 {max_retries} 次重试后仍失败: {str(e)}")
- `backend/app/utils/retry.py:108` - logger.error(f"异步函数 {func.__name__} 在 {max_retries} 次重试后仍失败: {str(e)}")
- `backend/app/utils/retry.py:179` - logger.error(f"API调用在 {self.max_retries} 次重试后仍失败: {str(e)}")
- `backend/app/utils/retry.py:227` - logger.error(f"处理第 {idx + 1} 项失败: {str(e)}")
## Section 5 - Issue #10 checklist mapping
Each line below is taken from the ticket body, with an explicit status.
- [ ] **GAP** - **Frontend UI** — every label, button, modal, error toast, and tooltip in EN. No Chinese strings on screen. - 29 hard-coded CJK literal(s) in `frontend/src/views|components/`
- [ ] **GAP** - **Step 1 — Graph Build** - 5 gap(s) classified, see Section 1/3
- MANUAL-PENDING: Status messages in EN - not verifiable statically; awaiting live run
- GAP: Ontology JSON descriptions in EN (depends on #2) - 14 gap(s) classified, see Section 1/3
- GAP: Backend logs in EN (depends on #6) - 9 gap(s) classified, see Section 1/3
- [ ] **GAP** - **Step 2 — Env Setup** - 61 gap(s) classified, see Section 1/3
- GAP: Generated agent profiles (`bio`, `persona`, `profession`, `interested_topics`) in EN (depends on #3) - 61 gap(s) classified, see Section 1/3
- MANUAL-PENDING: `gender` still the English enum (`male` / `female` / `other`) - not verifiable statically; awaiting live run
- [ ] **GAP** - **Step 3 — Simulation** - 14 gap(s) classified, see Section 1/3
- GAP: Sim config `content`, `narrative_direction`, `hot_topics`, `reasoning` in EN (depends on #4) - 14 gap(s) classified, see Section 1/3
- MANUAL-PENDING: `poster_type` still PascalCase English - not verifiable statically; awaiting live run
- MANUAL-PENDING: `stance` still one of `supportive` / `opposing` / `neutral` / `observer` - not verifiable statically; awaiting live run
- GAP: Generated tweets / Reddit posts in EN (depends on #3 personas + #4 sim config) - 14 gap(s) classified, see Section 1/3
- [ ] **GAP** - **Step 4 — Report** - 70 gap(s) classified, see Section 1/3
- GAP: Report sections, headings, prose in EN (depends on #5) - 70 gap(s) classified, see Section 1/3
- MANUAL-PENDING: ReACT thinking trace in EN - requires live walkthrough
- MANUAL-PENDING: Tool-call results render correctly - requires live walkthrough
- [ ] **GAP** - **Step 5 — Interaction** - 2 gap(s) classified, see Section 1/3
- GAP: Interview chat replies in EN (depends on #3) - 2 gap(s) classified, see Section 1/3
- GAP: Report Agent chat replies in EN (depends on #5) - 72 gap(s) classified, see Section 1/3
- [ ] **GAP** - **Backend logs** — full pipeline-run logs in EN (depends on #6) - 9 gap(s) classified, see Section 1/3
- [ ] **GAP** - **Locale propagation** — confirm `Accept-Language: en` (or thread-local locale set via `set_locale`) reaches background tasks and survives the OASIS subprocess boundary. - 9 CJK log strings on EN code path
- [ ] **MANUAL-PENDING** - Every touchpoint above renders in Chinese; no English regressions. - requires live walkthrough
- [ ] **MANUAL-PENDING** - zh.json backfill (#8) covered: Step 3, Step 4, Step 5, and graph panel labels are all Chinese. - not verifiable statically; awaiting live run
## Section 6 - ZH regression check
- Locale catalogues at full key parity (953 EN keys / 953 ZH keys, symmetric difference 0 - see Section 2).
- No ZH-specific regression detected in static review. Live ZH walkthrough is `manual-pending`.
## Section 7 - Follow-up plan
Per R7.2, gaps are grouped into the following follow-up issues (placeholder bodies in `PENDING-followups/`):
1. **Frontend hard-coded UI strings** (49 matches + 36 regex parsers depending on CJK backend output).
2. **Backend log strings** (9 matches).
3. **Backend LLM-prompt context labels** (143 matches).
4. **Permanent CI guard** (preventative - re-run this audit on every PR).
Backend docstring/comment matches (the bulk of `deliberate` rows) are covered by the existing issue #7 and are not re-filed here.

13
.kiro/specs/i18n-e2e-english-verification/audit/9dcaecd2d27e6325bae0c53b9ab41eb86d0269cd/parity.txt Normal file
View File

@ -0,0 +1,13 @@
# Locale parity for HEAD
# en keys: 953
# zh keys: 953
[missing-keys]
# (none)
[cjk-in-en]
# (none)
[identical-values]
# (none)

62
.kiro/specs/i18n-e2e-english-verification/audit/scripts/audit_cjk.sh Executable file
View File

@ -0,0 +1,62 @@
#!/usr/bin/env bash
# Run the canonical CJK grep with PCRE, then write the raw output and a
# bucketed summary partitioned by top-level path. Excludes binary file
# matches (e.g. .jpeg) since ripgrep / git grep can otherwise score them.
set -euo pipefail
if [ "$#" -ne 1 ]; then
printf 'usage: %s <sha-dir>\n' "$0" >&2
exit 64
fi
sha_dir="$1"
mkdir -p "${sha_dir}"
raw="${sha_dir}/cjk-grep.txt"
bucketed="${sha_dir}/cjk-grep-bucketed.txt"
# Canonical PCRE grep against the three top-level paths owned by this audit.
# git grep -P uses PCRE2 - ranges like \x{4e00}-\x{9fff} are valid here.
# `-I` (--no-binary) excludes binary-file matches outright so the audit
# reports only text content.
git grep -nIP '[\x{4e00}-\x{9fff}]' \
-- backend/app frontend/src locales/en.json \
> "${raw}" \
|| true
awk_script='
function bucket(path) {
if (path ~ /^backend\/app\//) return "backend/app"
if (path ~ /^frontend\/src\//) return "frontend/src"
if (path ~ /^locales\/en\.json/) return "locales/en.json"
return "other"
}
{
split($0, parts, ":")
path = parts[1]
b = bucket(path)
counts[b]++
lines[b] = (b in lines ? lines[b] "\n" : "") $0
}
END {
order[1] = "backend/app"
order[2] = "frontend/src"
order[3] = "locales/en.json"
order[4] = "other"
for (i = 1; i <= 4; i++) {
b = order[i]
c = (b in counts ? counts[b] : 0)
printf("[%s] (%d lines)\n", b, c)
if (c > 0) {
print lines[b]
}
print ""
}
}
'
awk "${awk_script}" "${raw}" > "${bucketed}"
raw_lines=$(wc -l < "${raw}" | tr -d ' ')
printf ' cjk-grep.txt: %s lines\n' "${raw_lines}"
printf ' cjk-grep-bucketed.txt: written\n'

128
.kiro/specs/i18n-e2e-english-verification/audit/scripts/check_parity.py Executable file
View File

@ -0,0 +1,128 @@
#!/usr/bin/env python3
"""Diff locales/en.json against locales/zh.json and emit parity.txt.
Three labelled blocks are written:
* `[missing-keys]` - keys present on one side but not the other.
* `[cjk-in-en]` - EN catalogue values that contain CJK characters.
* `[identical-values]` - keys whose EN and ZH value are identical AND the
value is non-empty AND has more than two ASCII words.
These are review-needed signals, not gaps.
Run from the repository root.
"""
from __future__ import annotations
import json
import re
import sys
from pathlib import Path
from typing import Dict, Iterator, Tuple
CJK_RANGE = re.compile(r"[一-鿿]")
def flatten(d: Dict[str, object], prefix: str = "") -> Iterator[Tuple[str, object]]:
"""Recursively yield (dotted-key, value) pairs from a nested dict."""
for key, value in d.items():
path = f"{prefix}.{key}" if prefix else key
if isinstance(value, dict):
yield from flatten(value, path)
else:
yield path, value
def is_non_trivial_english_prose(value: object) -> bool:
"""Heuristic for the identical-value 'review-needed' signal.
True when:
* value is a string,
* value is non-empty after strip,
* value contains more than two whitespace-separated tokens,
* value contains no CJK characters (otherwise it's just an untranslated
ZH original which is not a review-needed signal here).
"""
if not isinstance(value, str):
return False
text = value.strip()
if not text:
return False
if CJK_RANGE.search(text):
return False
return len(text.split()) > 2
def main(argv: list[str]) -> int:
if len(argv) != 2:
print(f"usage: {argv[0]} <sha-dir>", file=sys.stderr)
return 64
sha_dir = Path(argv[1])
sha_dir.mkdir(parents=True, exist_ok=True)
out_path = sha_dir / "parity.txt"
en_path = Path("locales/en.json")
zh_path = Path("locales/zh.json")
if not en_path.exists() or not zh_path.exists():
print(f"missing locale files: {en_path}, {zh_path}", file=sys.stderr)
return 1
en = json.loads(en_path.read_text(encoding="utf-8"))
zh = json.loads(zh_path.read_text(encoding="utf-8"))
en_flat = dict(flatten(en))
zh_flat = dict(flatten(zh))
en_only = sorted(set(en_flat) - set(zh_flat))
zh_only = sorted(set(zh_flat) - set(en_flat))
cjk_in_en = []
for key, value in sorted(en_flat.items()):
if isinstance(value, str) and CJK_RANGE.search(value):
cjk_in_en.append((key, value))
identical = []
for key in sorted(set(en_flat) & set(zh_flat)):
en_val = en_flat[key]
zh_val = zh_flat[key]
if en_val == zh_val and is_non_trivial_english_prose(en_val):
identical.append((key, en_val))
lines: list[str] = []
lines.append(f"# Locale parity for HEAD")
lines.append(f"# en keys: {len(en_flat)}")
lines.append(f"# zh keys: {len(zh_flat)}")
lines.append("")
lines.append("[missing-keys]")
if not en_only and not zh_only:
lines.append("# (none)")
for key in en_only:
lines.append(f"en-only: {key}")
for key in zh_only:
lines.append(f"zh-only: {key}")
lines.append("")
lines.append("[cjk-in-en]")
if not cjk_in_en:
lines.append("# (none)")
for key, value in cjk_in_en:
snippet = value if len(value) <= 80 else value[:77] + "..."
lines.append(f"{key}: {snippet}")
lines.append("")
lines.append("[identical-values]")
if not identical:
lines.append("# (none)")
for key, value in identical:
snippet = value if len(value) <= 80 else value[:77] + "..."
lines.append(f"{key}: {snippet}")
lines.append("")
out_path.write_text("\n".join(lines) + "\n", encoding="utf-8")
print(
f" parity.txt written: missing={len(en_only) + len(zh_only)}, "
f"cjk-in-en={len(cjk_in_en)}, identical-values={len(identical)}"
)
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))

182
.kiro/specs/i18n-e2e-english-verification/audit/scripts/classify.py Executable file
View File

@ -0,0 +1,182 @@
#!/usr/bin/env python3
"""Classify each CJK match into a 4-class label and a category tag.
Inputs (read from <sha-dir>):
cjk-grep.txt - raw `git grep -nP` output, one match per line.
parity.txt - output of check_parity.py (used to harvest cjk-in-en gaps).
Output (written to <sha-dir>/classified.csv):
CSV columns: file, line, match, class, category, pipeline_step
Classes are a closed set: deliberate / gap / non-applicable / review-needed.
Categories and pipeline-step tags are likewise closed sets - see classify_match.
Run from the repository root.
"""
from __future__ import annotations
import csv
import re
import sys
from pathlib import Path
from typing import Iterable, Tuple
CJK_RANGE = re.compile(r"[一-鿿]")
PROMPT_FILES = (
"backend/app/services/ontology_generator.py",
"backend/app/services/oasis_profile_generator.py",
"backend/app/services/simulation_config_generator.py",
"backend/app/services/report_agent.py",
"backend/app/services/zep_graph_memory_updater.py",
)
LOG_HINTS = ("logger.", "log.", "print(", "build_logger.", "logging.")
BINARY_EXTS = (
".jpg", ".jpeg", ".png", ".gif", ".pdf",
".woff", ".woff2", ".ttf", ".eot", ".ico",
)
def classify_match(file: str, raw_line: str) -> Tuple[str, str, str]:
"""Return (class, category, pipeline_step) for one grep match line."""
if any(file.lower().endswith(ext) for ext in BINARY_EXTS):
return ("non-applicable", "binary-false-positive", "n/a")
if file == "locales/en.json":
return ("gap", "catalogue-parity", "UI")
stripped = raw_line.lstrip()
pipeline_step = pipeline_step_for(file)
if file.endswith(".vue"):
if re.search(r"\.match\s*\(\s*/", raw_line):
return ("gap", "frontend-regex-parser", pipeline_step)
if re.search(r"['\"`].*[一-鿿].*['\"`]", raw_line):
return ("gap", "frontend-ui-string", pipeline_step)
if stripped.startswith("//") or stripped.startswith("/*") or stripped.startswith("*"):
return ("deliberate", "frontend-comment", pipeline_step)
return ("review-needed", "frontend-other", pipeline_step)
if file.endswith(".py"):
if stripped.startswith("#"):
return ("deliberate", "backend-comment", pipeline_step)
if stripped.startswith('"""') or stripped.startswith("'''"):
return ("deliberate", "backend-docstring", pipeline_step)
if not re.search(r"['\"]", raw_line):
# bare CJK on a non-string line: most likely an unterminated docstring
# body. Treat as a docstring continuation.
return ("deliberate", "backend-docstring", pipeline_step)
if any(hint in raw_line for hint in LOG_HINTS):
return ("gap", "backend-log", "Logs")
if file in PROMPT_FILES:
return ("gap", "backend-prompt-label", pipeline_step)
return ("review-needed", "backend-string", pipeline_step)
if file.endswith(".js") or file.endswith(".ts"):
if stripped.startswith("//") or stripped.startswith("*"):
return ("deliberate", "frontend-comment", pipeline_step)
return ("review-needed", "frontend-other", pipeline_step)
return ("review-needed", "uncategorised", pipeline_step)
def pipeline_step_for(file: str) -> str:
"""Map a path to one of the closed-set pipeline-step tags."""
if "ontology_generator" in file or "graph_builder" in file or "graph.py" in file:
return "Graph Build"
if "oasis_profile_generator" in file or "Step2" in file:
return "Env Setup"
if "simulation_config_generator" in file or "simulation" in file or "Step3" in file:
return "Simulation"
if "report_agent" in file or "Step4" in file:
return "Report"
if "Step5" in file or "interaction" in file.lower() or "interview" in file.lower():
return "Interaction"
if "logger" in file or "retry" in file:
return "Logs"
if file.startswith("frontend/src/views/") or file.startswith("frontend/src/components/"):
return "UI"
return "n/a"
def parse_grep_line(line: str) -> Tuple[str, str, str]:
"""Split a `git grep -n` line into (file, line-number, match-text)."""
parts = line.split(":", 2)
if len(parts) < 3:
return ("", "", line)
return (parts[0], parts[1], parts[2])
def parity_to_rows(parity_path: Path) -> Iterable[Tuple[str, str, str, str, str, str]]:
"""Promote `[cjk-in-en]` block entries from parity.txt into classified rows."""
if not parity_path.exists():
return
in_block = False
for raw in parity_path.read_text(encoding="utf-8").splitlines():
if raw.startswith("["):
in_block = raw.strip() == "[cjk-in-en]"
continue
if not in_block:
continue
if not raw or raw.startswith("#"):
continue
yield (
"locales/en.json",
"0",
raw,
"gap",
"catalogue-parity",
"UI",
)
def main(argv: list[str]) -> int:
if len(argv) != 2:
print(f"usage: {argv[0]} <sha-dir>", file=sys.stderr)
return 64
sha_dir = Path(argv[1])
grep_path = sha_dir / "cjk-grep.txt"
parity_path = sha_dir / "parity.txt"
out_path = sha_dir / "classified.csv"
if not grep_path.exists():
print(f"missing input: {grep_path}", file=sys.stderr)
return 1
rows: list[Tuple[str, str, str, str, str, str]] = []
grep_lines = grep_path.read_text(encoding="utf-8").splitlines()
for raw_line in grep_lines:
if not raw_line:
continue
file, lineno, match = parse_grep_line(raw_line)
if not file:
continue
cls, category, step = classify_match(file, match)
rows.append((file, lineno, match.strip(), cls, category, step))
rows.extend(parity_to_rows(parity_path))
raw_count = sum(1 for line in grep_lines if line.strip())
grep_rows = [r for r in rows if r[0] != "locales/en.json" or r[1] != "0"]
if len(grep_rows) != raw_count:
print(
f"row-count drift: input={raw_count}, classified={len(grep_rows)}",
file=sys.stderr,
)
return 1
with out_path.open("w", encoding="utf-8", newline="") as fh:
writer = csv.writer(fh)
writer.writerow(["file", "line", "match", "class", "category", "pipeline_step"])
writer.writerows(rows)
summary: dict[str, int] = {}
for row in rows:
summary[row[3]] = summary.get(row[3], 0) + 1
summary_str = ", ".join(f"{cls}={n}" for cls, n in sorted(summary.items()))
print(f" classified.csv: {len(rows)} rows ({summary_str})")
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))

79
.kiro/specs/i18n-e2e-english-verification/audit/scripts/file_followups.sh Executable file
View File

@ -0,0 +1,79 @@
#!/usr/bin/env bash
# Iterate <sha-dir>/PENDING-followups/*.md and file each non-empty body
# as a GitHub issue. The first markdown heading line (`# title`) becomes
# the issue title; any `<!-- labels: a,b,c -->` line at the bottom of the
# body becomes the --label argument.
#
# On per-category failure the body is left in place and the script exits
# non-zero at the end (after attempting all categories).
set -uo pipefail
if [ "$#" -ne 1 ]; then
printf 'usage: %s <sha-dir>\n' "$0" >&2
exit 64
fi
sha_dir="$1"
pending_dir="${sha_dir}/PENDING-followups"
urls_path="${sha_dir}/followup-urls.txt"
if [ ! -d "${pending_dir}" ]; then
printf 'missing PENDING-followups dir: %s\n' "${pending_dir}" >&2
exit 1
fi
# Append-only URL log so retries on the same sha-dir preserve previous filings.
touch "${urls_path}"
if ! command -v gh >/dev/null 2>&1; then
printf ' gh not available; leaving all bodies in PENDING-followups/\n'
exit 2
fi
if ! gh auth status >/dev/null 2>&1; then
printf ' gh not authenticated; leaving all bodies in PENDING-followups/\n'
exit 2
fi
partial=0
for body in "${pending_dir}"/[0-9]*-*.md; do
[ -f "${body}" ] || continue
if [ ! -s "${body}" ]; then
# Empty placeholder - the corresponding category had zero gaps in this run.
continue
fi
title="$(awk 'NR==1 && /^# /{sub(/^# /, ""); print; exit}' "${body}")"
if [ -z "${title}" ]; then
title="i18n: follow-up from issue #10 verification ($(basename "${body}" .md))"
fi
label_line="$(grep -oE '<!-- labels: [^>]+-->' "${body}" | head -1 || true)"
labels="$(printf '%s' "${label_line}" | sed -E 's/<!-- labels: //; s/ *-->//' || true)"
label_args=()
if [ -n "${labels}" ]; then
IFS=',' read -ra parts <<< "${labels}"
for label in "${parts[@]}"; do
label_args+=( --label "$(echo "${label}" | tr -d ' ')" )
done
fi
printf ' filing: %s\n' "${title}"
if url="$(gh issue create --repo salestech-group/MiroFish \
--title "${title}" \
--body-file "${body}" \
"${label_args[@]}" 2>&1)"; then
printf '%s\n' "${url}" >> "${urls_path}"
printf ' -> %s\n' "${url}"
rm -f "${body}"
else
printf ' !! gh issue create failed: %s\n' "${url}" >&2
partial=1
fi
done
if [ "${partial}" -eq 1 ]; then
exit 2
fi
exit 0

42
.kiro/specs/i18n-e2e-english-verification/audit/scripts/post_comment.sh Executable file
View File

@ -0,0 +1,42 @@
#!/usr/bin/env bash
# Post comment-body.md as a comment on issue #10.
#
# Falls back to writing PENDING-issue-10-comment.md when gh is unavailable
# or the post fails - exits non-zero in that case so the orchestrator can
# downgrade its overall status.
set -euo pipefail
if [ "$#" -ne 1 ]; then
printf 'usage: %s <sha-dir>\n' "$0" >&2
exit 64
fi
sha_dir="$1"
body="${sha_dir}/comment-body.md"
if [ ! -f "${body}" ]; then
printf 'missing comment body: %s\n' "${body}" >&2
exit 1
fi
if ! command -v gh >/dev/null 2>&1; then
printf ' gh not available; writing PENDING-issue-10-comment.md\n'
cp "${body}" "${sha_dir}/PENDING-issue-10-comment.md"
exit 2
fi
if ! gh auth status >/dev/null 2>&1; then
printf ' gh not authenticated; writing PENDING-issue-10-comment.md\n'
cp "${body}" "${sha_dir}/PENDING-issue-10-comment.md"
exit 2
fi
if url="$(gh issue comment 10 --repo salestech-group/MiroFish --body-file "${body}" 2>&1)"; then
printf '%s\n' "${url}" > "${sha_dir}/comment-url.txt"
printf ' posted: %s\n' "${url}"
rm -f "${sha_dir}/PENDING-issue-10-comment.md"
exit 0
fi
printf ' gh post failed; writing PENDING-issue-10-comment.md\n'
cp "${body}" "${sha_dir}/PENDING-issue-10-comment.md"
exit 2

419
.kiro/specs/i18n-e2e-english-verification/audit/scripts/render_report.py Executable file
View File

@ -0,0 +1,419 @@
#!/usr/bin/env python3
"""Render the gap report and the issue-#10 comment body.
Inputs (from <sha-dir>):
classified.csv - per-match classification rows.
parity.txt - en/zh catalogue parity output.
cjk-grep-bucketed.txt - human-readable bucketed grep output.
Inputs (from repo):
.ticket/10.md - snapshot of issue #10's body (used to mirror its checklist).
Outputs (to <sha-dir>):
gap-report.md - full structured report (seven sections).
comment-body.md - markdown comment to be posted on issue #10.
PENDING-followups/01..04-*.md - one body per gap category (placeholders allowed).
Usage:
python3 render_report.py <sha-dir> <commit-sha>
"""
from __future__ import annotations
import csv
import re
import sys
from collections import Counter, defaultdict
from pathlib import Path
from typing import Dict, List
ISSUE_NUMBER = 10
REPO_SLUG = "salestech-group/MiroFish"
def load_rows(csv_path: Path) -> list[dict]:
with csv_path.open(encoding="utf-8", newline="") as fh:
return list(csv.DictReader(fh))
def load_ticket_body(ticket_path: Path) -> str:
"""Strip the YAML frontmatter and return the markdown body."""
text = ticket_path.read_text(encoding="utf-8")
if text.startswith("---\n"):
end = text.find("\n---\n", 4)
if end != -1:
return text[end + 5 :]
return text
CHECKBOX_RE = re.compile(r"^(\s*)- \[ \] (.+)$")
SUBBULLET_RE = re.compile(r"^(\s+)- (.+)$")
def evidence_for_step(rows: list[dict], step: str) -> list[dict]:
"""Return gap rows whose pipeline_step matches the given UI tag."""
return [r for r in rows if r["class"] == "gap" and r["pipeline_step"] == step]
def render_section_5(ticket_body: str, rows: list[dict]) -> str:
"""Map every checklist item from the ticket body to a status."""
gaps_by_step = defaultdict(list)
for row in rows:
if row["class"] == "gap":
gaps_by_step[row["pipeline_step"]].append(row)
out: list[str] = []
out.append("## Section 5 - Issue #10 checklist mapping\n")
out.append("Each line below is taken from the ticket body, with an explicit status.\n")
in_checklist = False
for line in ticket_body.splitlines():
match = CHECKBOX_RE.match(line)
if match:
in_checklist = True
indent, text = match.group(1), match.group(2)
status, note = status_for_checklist_item(text, gaps_by_step)
out.append(f"{indent}- [{('x' if status == 'pass' else ' ')}] **{status.upper()}** - {text}{note}")
continue
sub = SUBBULLET_RE.match(line)
if in_checklist and sub:
indent, text = sub.group(1), sub.group(2)
status, note = status_for_checklist_item(text, gaps_by_step)
out.append(f"{indent}- {status.upper()}: {text}{note}")
continue
if line.startswith("##") or line.startswith("---"):
in_checklist = False
return "\n".join(out) + "\n"
def status_for_checklist_item(text: str, gaps_by_step: Dict[str, list]) -> tuple[str, str]:
"""Return (status, suffix-note) for one checklist line.
Pure-UI items default to manual-pending in this run; items with a
backing pipeline-step that has gaps are reported as gap with a count.
"""
lower = text.lower()
candidates: list[str] = []
if "graph build" in lower or "ontology" in lower:
candidates.append("Graph Build")
if "env setup" in lower or "agent profile" in lower or "profession" in lower:
candidates.append("Env Setup")
if "simulation" in lower or "tweet" in lower or "reddit" in lower or "sim config" in lower:
candidates.append("Simulation")
if "report" in lower:
candidates.append("Report")
if "interaction" in lower or "interview" in lower or "chat repl" in lower:
candidates.append("Interaction")
if "log" in lower:
candidates.append("Logs")
relevant_gaps = []
for step in candidates:
relevant_gaps.extend(gaps_by_step.get(step, []))
if "frontend ui" in lower or "no chinese strings on screen" in lower or "every label" in lower:
ui_gaps = gaps_by_step.get("UI", [])
if ui_gaps:
return ("gap", f" - {len(ui_gaps)} hard-coded CJK literal(s) in `frontend/src/views|components/`")
return ("manual-pending", " - live UI walkthrough not run in this sandbox")
if "locale propagation" in lower or "set_locale" in lower:
prop = gaps_by_step.get("Logs", [])
if prop:
return ("gap", f" - {len(prop)} CJK log strings on EN code path")
return ("manual-pending", " - locale-propagation runtime check not run in this sandbox")
if relevant_gaps:
return ("gap", f" - {len(relevant_gaps)} gap(s) classified, see Section 1/3")
if any(c in lower for c in ("ui", "screenshot", "chat", "modal", "tooltip", "render", "trace", "thinking")):
return ("manual-pending", " - requires live walkthrough")
return ("manual-pending", " - not verifiable statically; awaiting live run")
def render_gap_report(rows: list[dict], ticket_body: str, parity_text: str, sha: str) -> str:
classes = Counter(r["class"] for r in rows)
gap_rows = [r for r in rows if r["class"] == "gap"]
gap_categories = Counter(r["category"] for r in gap_rows)
gap_steps = Counter(r["pipeline_step"] for r in gap_rows)
out: list[str] = []
out.append(f"# Verification gap report - i18n-e2e-english-verification\n")
out.append(f"**Commit:** `{sha}`\n")
out.append("")
out.append("## Overview\n")
out.append(f"- Total CJK matches audited: **{len(rows)}**")
out.append(f"- Class distribution: {format_counter(classes)}")
out.append(f"- Gap categories: {format_counter(gap_categories)}")
out.append(f"- Gap pipeline steps: {format_counter(gap_steps)}")
out.append("")
out.append("## Section 1 - Static CJK audit\n")
out.append("Canonical command (PCRE):\n")
out.append("```")
out.append('git grep -nIP "[\\x{4e00}-\\x{9fff}]" -- backend/app frontend/src locales/en.json')
out.append("```")
out.append("")
out.append(f"Raw output captured at `audit/{sha}/cjk-grep.txt` and bucketed at `audit/{sha}/cjk-grep-bucketed.txt`.")
out.append("")
out.append(f"`locales/en.json` CJK matches: **{sum(1 for r in rows if r['file'] == 'locales/en.json')}** (acceptance: zero).")
out.append("")
out.append("Top files by gap count:")
out.append("")
out.append("| File | Gap count |")
out.append("|------|-----------|")
by_file = Counter(r["file"] for r in gap_rows)
for file, count in by_file.most_common(15):
out.append(f"| `{file}` | {count} |")
out.append("")
out.append("## Section 2 - Locale catalogue parity\n")
out.append("```")
out.append(parity_text.strip())
out.append("```")
out.append("")
out.append("## Section 3 - LLM-prompt locale verification\n")
prompt_gaps = [r for r in gap_rows if r["category"] == "backend-prompt-label"]
out.append(f"Backend prompt-label gaps (CJK string literals inside services that compose LLM prompts): **{len(prompt_gaps)}**")
out.append("")
if prompt_gaps:
out.append("First 10 examples (file:line - match):")
out.append("")
for row in prompt_gaps[:10]:
out.append(f"- `{row['file']}:{row['line']}` - {row['match']}")
if len(prompt_gaps) > 10:
out.append(f"- ... and {len(prompt_gaps) - 10} more (see `classified.csv`)")
out.append("")
out.append(
"These prompts feed the LLM verbatim; CJK labels bias the model toward Chinese output even when "
"the requested locale is English."
)
out.append("")
out.append("## Section 4 - Locale propagation surface\n")
log_gaps = [r for r in gap_rows if r["category"] == "backend-log"]
out.append("| Boundary | Status | Evidence |")
out.append("|----------|--------|----------|")
out.append(
"| HTTP -> Flask handler | manual-pending | runtime not exercised in sandbox; static review showed no per-request locale carrier |"
)
out.append(
"| Flask handler -> Task worker | manual-pending | thread-local `set_locale` referenced in CLAUDE.md but not statically verified end-to-end |"
)
out.append(
f"| Task worker -> OASIS subprocess | manual-pending | subprocess boundary requires live run |"
)
out.append(
f"| Backend logger | {'gap' if log_gaps else 'pass'} | {len(log_gaps)} hard-coded CJK log line(s) on EN code path |"
)
out.append("")
if log_gaps:
out.append("First 10 backend-log gap examples:")
out.append("")
for row in log_gaps[:10]:
out.append(f"- `{row['file']}:{row['line']}` - {row['match']}")
out.append("")
out.append(render_section_5(ticket_body, rows))
out.append("## Section 6 - ZH regression check\n")
out.append(
"- Locale catalogues at full key parity (953 EN keys / 953 ZH keys, symmetric difference 0 - "
"see Section 2).\n"
"- No ZH-specific regression detected in static review. Live ZH walkthrough is `manual-pending`.\n"
)
out.append("## Section 7 - Follow-up plan\n")
out.append("Per R7.2, gaps are grouped into the following follow-up issues (placeholder bodies in `PENDING-followups/`):")
out.append("")
out.append(
f"1. **Frontend hard-coded UI strings** ({len(by_category(rows, 'frontend-ui-string'))} matches + "
f"{len(by_category(rows, 'frontend-regex-parser'))} regex parsers depending on CJK backend output)."
)
out.append(f"2. **Backend log strings** ({len(by_category(rows, 'backend-log'))} matches).")
out.append(f"3. **Backend LLM-prompt context labels** ({len(by_category(rows, 'backend-prompt-label'))} matches).")
out.append("4. **Permanent CI guard** (preventative - re-run this audit on every PR).")
out.append("")
out.append(
"Backend docstring/comment matches (the bulk of `deliberate` rows) are covered by the existing issue #7 and are not re-filed here."
)
return "\n".join(out) + "\n"
def by_category(rows: list[dict], category: str) -> list[dict]:
return [r for r in rows if r["category"] == category and r["class"] == "gap"]
def format_counter(c: Counter) -> str:
return ", ".join(f"{k}={v}" for k, v in c.most_common())
def render_comment_body(rows: list[dict], ticket_body: str, sha: str) -> str:
classes = Counter(r["class"] for r in rows)
gap_rows = [r for r in rows if r["class"] == "gap"]
gap_categories = Counter(r["category"] for r in gap_rows)
out: list[str] = []
out.append(f"### Verification report - run on commit `{sha}`\n")
out.append("This run was produced by `.kiro/specs/i18n-e2e-english-verification/audit/scripts/run_audit.sh`.")
out.append("Captured artefacts live under `.kiro/specs/i18n-e2e-english-verification/audit/<commit-sha>/`.\n")
out.append("")
out.append(f"**Audit summary:** {sum(classes.values())} CJK matches across the auditable paths.")
out.append(f"- {classes.get('gap', 0)} `gap` (actionable, see follow-ups)")
out.append(f"- {classes.get('review-needed', 0)} `review-needed` (soft signal; needs human eyeball)")
out.append(f"- {classes.get('deliberate', 0)} `deliberate` (mostly backend docstrings/comments - covered by issue #7)")
out.append(
f"- {classes.get('non-applicable', 0)} `non-applicable` (binary file false positives - excluded)"
)
out.append("")
out.append(f"**Gap-category breakdown:** {format_counter(gap_categories)}")
out.append("")
out.append("---")
out.append("")
out.append("#### Issue checklist mapping")
out.append("")
out.append(render_section_5(ticket_body, rows))
out.append("---")
out.append("")
out.append("#### How to re-run")
out.append("")
out.append("```bash")
out.append("# from the repository root, on any commit:")
out.append("bash .kiro/specs/i18n-e2e-english-verification/audit/scripts/run_audit.sh")
out.append("# artefacts at .kiro/specs/i18n-e2e-english-verification/audit/<HEAD-sha>/")
out.append("```")
out.append("")
out.append(
"If `gh` is not authenticated when re-running, the comment body and follow-up bodies are written to "
"`PENDING-issue-10-comment.md` / `PENDING-followups/` for a human to post."
)
out.append("")
out.append("Out of scope for this run (per R5.3 / R7.3): live UI walkthrough, full Docker-Compose pipeline run, and any inline gap fixes.")
return "\n".join(out) + "\n"
def render_followup_bodies(rows: list[dict], sha_dir: Path, sha: str) -> None:
pending_dir = sha_dir / "PENDING-followups"
pending_dir.mkdir(parents=True, exist_ok=True)
ui_gaps = by_category(rows, "frontend-ui-string") + by_category(rows, "frontend-regex-parser")
log_gaps = by_category(rows, "backend-log")
prompt_gaps = by_category(rows, "backend-prompt-label")
files = [
(
"01-frontend-ui-strings.md",
"i18n: replace hard-coded chinese ui strings in process and step components with i18n keys",
ui_gaps,
(
"Several `.vue` templates in `frontend/src/views/` and `frontend/src/components/` still emit "
"Chinese strings directly instead of routing them through `vue-i18n` keys. Some `Step4Report.vue` "
"regex parsers also rely on Chinese tokens emitted by the backend (so they will silently break "
"once the backend prompts are translated)."
),
["i18n", "bug"],
),
(
"02-backend-log-strings.md",
"i18n: externalise remaining chinese log strings in flask api and utils",
log_gaps,
(
"After issue #6 externalised most backend log messages, a handful of `logger.info` / "
"`logger.error` call sites in `backend/app/api/graph.py` and `backend/app/utils/retry.py` "
"still hard-code Chinese strings, so backend logs leak Chinese under EN locale."
),
["i18n"],
),
(
"03-backend-prompt-labels.md",
"i18n: translate chinese context labels inside llm-prompt assembly in backend services",
prompt_gaps,
(
"Several `services/*_generator.py` files compose LLM prompts that still embed Chinese "
"context labels (e.g. `\"事实信息:\"`, `\"相关实体:\"`) into the prompt string verbatim. These "
"labels bias the LLM toward Chinese output even when the requested locale is English."
),
["i18n"],
),
(
"04-permanent-ci-guard.md",
"i18n: add a permanent ci guard that runs the e2e cjk audit on every pr",
[],
(
"Promote the audit pipeline at `.kiro/specs/i18n-e2e-english-verification/audit/scripts/` to "
"a permanent CI check. The guard should fail when `locales/en.json` contains any CJK character "
"and when the gap count regresses against a committed baseline."
),
["i18n", "enhancement"],
),
]
for name, title, gaps, summary, labels in files:
if not gaps and not name.startswith("04-"):
(pending_dir / name).write_text("", encoding="utf-8")
continue
body = [
f"# {title}",
"",
"## Summary",
"",
summary,
"",
"## Linked from",
"",
f"- Issue #{ISSUE_NUMBER} (verification report comment).",
f"- Spec: `.kiro/specs/i18n-e2e-english-verification/` at commit `{sha}`.",
"",
"## Evidence",
"",
]
if gaps:
for row in gaps[:50]:
body.append(f"- `{row['file']}:{row['line']}` - {row['match']}")
if len(gaps) > 50:
body.append(f"- ... and {len(gaps) - 50} more (see `classified.csv` in the spec dir)")
else:
body.append("- (No gaps in this run; this is a preventative follow-up only.)")
body.append("")
body.append("## Acceptance")
body.append("")
body.append("- [ ] Each `file:line` above is fixed (or explicitly classified as `deliberate`).")
body.append("- [ ] Re-running `bash .kiro/specs/i18n-e2e-english-verification/audit/scripts/run_audit.sh` shows zero gaps in this category.")
body.append("")
body.append(f"<!-- labels: {','.join(labels)} -->")
body.append("")
(pending_dir / name).write_text("\n".join(body), encoding="utf-8")
def main(argv: list[str]) -> int:
if len(argv) != 3:
print(f"usage: {argv[0]} <sha-dir> <commit-sha>", file=sys.stderr)
return 64
sha_dir = Path(argv[1])
sha = argv[2]
rows = load_rows(sha_dir / "classified.csv")
parity_text = (sha_dir / "parity.txt").read_text(encoding="utf-8")
ticket_body = load_ticket_body(Path(".ticket/10.md"))
gap_report = render_gap_report(rows, ticket_body, parity_text, sha)
(sha_dir / "gap-report.md").write_text(gap_report, encoding="utf-8")
comment_body = render_comment_body(rows, ticket_body, sha)
(sha_dir / "comment-body.md").write_text(comment_body, encoding="utf-8")
render_followup_bodies(rows, sha_dir, sha)
print(f" gap-report.md, comment-body.md, PENDING-followups/ written under {sha_dir}")
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))

71
.kiro/specs/i18n-e2e-english-verification/audit/scripts/run_audit.sh Executable file
View File

@ -0,0 +1,71 @@
#!/usr/bin/env bash
# Orchestrate the i18n end-to-end verification audit.
#
# Reads working-tree state via git (no production-source modifications),
# captures classified output under audit/<commit-sha>/, and posts the
# verification report comment + follow-up issues via gh when available.
#
# Exit codes:
# 0 - audit succeeded and all GitHub side effects applied
# 1 - audit step failed (read-only producer aborted)
# 2 - audit succeeded but at least one GitHub side effect was deferred to PENDING
set -euo pipefail
repo_root="$(git rev-parse --show-toplevel)"
cd "$repo_root"
spec_root=".kiro/specs/i18n-e2e-english-verification"
scripts_dir="${spec_root}/audit/scripts"
sha="$(git rev-parse HEAD)"
sha_dir="${spec_root}/audit/${sha}"
mkdir -p "${sha_dir}"
printf 'Verification audit\n repo: %s\n sha: %s\n out: %s\n\n' \
"${repo_root}" "${sha}" "${sha_dir}"
ghs_exit=0
step() {
local label="$1"
shift
printf '== %s ==\n' "${label}"
"$@"
}
step "audit_cjk.sh" bash "${scripts_dir}/audit_cjk.sh" "${sha_dir}"
step "check_parity.py" python3 "${scripts_dir}/check_parity.py" "${sha_dir}"
step "classify.py" python3 "${scripts_dir}/classify.py" "${sha_dir}"
step "render_report.py" python3 "${scripts_dir}/render_report.py" "${sha_dir}" "${sha}"
# GitHub side effects: failures here downgrade the run to exit 2 but
# do not abort the rest of the side effects.
set +e
step "post_comment.sh" bash "${scripts_dir}/post_comment.sh" "${sha_dir}"
[ $? -ne 0 ] && ghs_exit=2
step "file_followups.sh" bash "${scripts_dir}/file_followups.sh" "${sha_dir}"
[ $? -ne 0 ] && ghs_exit=2
set -e
printf '\n== summary ==\n'
printf 'sha-dir: %s\n' "${sha_dir}"
if [ -f "${sha_dir}/comment-url.txt" ]; then
printf 'comment: %s\n' "$(cat "${sha_dir}/comment-url.txt")"
else
printf 'comment: PENDING (see %s/PENDING-issue-10-comment.md)\n' "${sha_dir}"
fi
if [ -f "${sha_dir}/followup-urls.txt" ]; then
printf 'follow-ups posted:\n'
sed 's/^/ /' "${sha_dir}/followup-urls.txt"
fi
if compgen -G "${sha_dir}/PENDING-followups/[0-9]*-*.md" > /dev/null; then
printf 'follow-ups PENDING:\n'
for body in "${sha_dir}"/PENDING-followups/[0-9]*-*.md; do
if [ -s "${body}" ]; then
printf ' %s\n' "${body}"
fi
done
fi
exit "${ghs_exit}"

560
.kiro/specs/i18n-e2e-english-verification/design.md Normal file
View File

@ -0,0 +1,560 @@
# Design — i18n-e2e-english-verification
## Overview
**Purpose**: This spec produces a deterministic, re-runnable verification pass that proves (or disproves) the MiroFish 5-step pipeline runs cleanly in English, and posts a structured report on issue #10 with a `pass` / `gap` / `manual-pending` status per checklist item.
**Users**: i18n maintainers reviewing the epic (#11), and any future verifier re-running the audit after subsequent merges. The deliverable is read by humans on GitHub (issue comment) and re-run by humans (or CI in a future iteration) to confirm parity.
**Impact**: No production code is modified. The repository gains one new directory tree (`.kiro/specs/i18n-e2e-english-verification/`) containing the spec, the audit scripts, and the captured outputs. One GitHub comment is posted on #10. Up to four follow-up issues are filed.
### Goals
- Static-audit `backend/app`, `frontend/src`, `locales/en.json` for CJK characters; classify every match.
- Verify EN / ZH locale catalogue parity and flag suspect untranslated entries.
- Verify LLM-prompt assets respect the requested locale.
- Document locale-propagation gaps across Flask → `Task` → OASIS subprocess → ReACT agent.
- Post a single canonical comment on issue #10 with per-checklist statuses.
- File follow-up issues for every gap (no inline fixes).
- Make the audit re-runnable by capturing artefacts under `.kiro/specs/.../audit/<commit-sha>/`.
### Non-Goals
- Patching any `gap` discovered (R7.3 — strictly verification).
- Performance / load testing.
- Adding new locales beyond EN / ZH.
- Building a permanent CI guard (filed as a follow-up issue, not implemented here).
- Live UI / Docker walkthrough — captured as `manual-pending` in this run's report.
## Boundary Commitments
### This Spec Owns
- The audit scripts and the captured audit outputs under `.kiro/specs/i18n-e2e-english-verification/audit/`.
- The `gap-report.md` artefact and the comment body posted on issue #10.
- The grouping rule for follow-up issues (one per category — UI strings, backend log strings, backend LLM-prompt labels, suggested CI guard).
- The `pass` / `gap` / `manual-pending` / `review-needed` classification scheme.
### Out of Boundary
- Any modification of files under `backend/app/`, `frontend/src/`, or `locales/`.
- Fixing the gaps the audit discovers — those land in their own follow-up issues.
- Live UI walkthrough, Docker run, or LLM execution.
- A permanent CI check — filed as a separate follow-up issue.
### Allowed Dependencies
- `git` (for `git grep`, capturing HEAD sha).
- `gh` CLI (for the comment + follow-up issues; with documented fallback when unavailable).
- `python3` (for the catalogue parity diff).
- The repo working tree at HEAD of the working branch.
### Revalidation Triggers
- Any merge to `main` that touches `locales/`, `backend/app/`, or `frontend/src/` invalidates the captured audit; a re-run should produce a new `audit/<commit-sha>/` directory.
- A change to issue #10's checklist body (e.g. a new sub-item) requires re-mapping in `gap-report.md`.
- A change to the four follow-up categories (e.g. project decides to file one issue per file) requires re-running the issue-filing script with new grouping.
## Architecture
### Existing Architecture Analysis
- The MiroFish backend is Flask + Python `Task` workers + an OASIS subprocess (per CLAUDE.md). i18n surfaces are: `vue-i18n` for the SPA, `locales/*.json` shared by both ends, a backend logger that resolves keys per locale, and inline LLM prompts in `backend/app/services/*.py`.
- The verification pass does **not** hook into any of these — it reads files only. No Flask blueprint, no `Task` model, no Neo4j query.
### Architecture Pattern & Boundary Map
```mermaid
graph TB
Verifier[Verifier shell entrypoint]
Audit[audit_cjk.sh]
Parity[check_parity.py]
Classify[classify.py]
Report[render_report.py]
Comment[post_comment.sh]
FollowUp[file_followups.sh]
Repo[Working tree]
Captures[audit slash sha slash]
GH[GitHub via gh CLI]
Verifier --> Audit
Verifier --> Parity
Audit --> Classify
Parity --> Classify
Classify --> Report
Report --> Captures
Report --> Comment
Report --> FollowUp
Audit --> Repo
Parity --> Repo
Comment --> GH
FollowUp --> GH
```
**Architecture Integration**:
- **Selected pattern**: Linear pipeline of read-only scripts that each emit a single artefact, composed by a thin shell entrypoint. No mutable state outside `audit/<sha>/`.
- **Domain boundaries**: `audit_cjk.sh` owns the raw grep; `check_parity.py` owns the catalogue diff; `classify.py` owns the four-class labels; `render_report.py` owns the comment body; `post_comment.sh` and `file_followups.sh` own GitHub side effects.
- **Existing patterns preserved**: Shell + Python script pair (matches the project's existing `setup`/`run` style); no new test runner, no new linter.
- **New components rationale**: Each script is single-purpose so failures (e.g. `gh` permission issues) are isolated and the pipeline can resume from the failed step.
- **Steering compliance**: No production-code touch (R7.3); 4-space indent in any committed Python; double quotes; `snake_case`; reserved Bash exits with a non-zero status on any uncaught error.
### Technology Stack
| Layer | Choice / Version | Role in Feature | Notes |
|-------|------------------|-----------------|-------|
| CLI / Audit runner | Bash 5+, `git grep -P` (PCRE) | Run the canonical CJK audit | `\x{...}` ranges require PCRE — `git grep -E` will fail on this regex (verified). |
| Static checks | Python 3.11 (project minimum per CLAUDE.md) | Catalogue parity + classification + report rendering | Standard library only — no new deps. |
| GitHub integration | `gh` CLI | Post the comment, file follow-ups | Falls back to `audit/<sha>/PENDING-*` files when missing. |
| Output formats | Plain text + Markdown | Captures + comment body | No HTML, no JSON beyond `gh`'s own. |
## File Structure Plan
### Directory Structure
```
.kiro/specs/i18n-e2e-english-verification/
├── spec.json
├── requirements.md
├── gap-analysis.md
├── research.md
├── design.md
├── tasks.md
├── HANDOFF.md # only if implementation hits the 3-cycle remediation cap
└── audit/
├── scripts/
│ ├── run_audit.sh # entrypoint - chains the steps below
│ ├── audit_cjk.sh # git grep PCRE + bucket counts
│ ├── check_parity.py # locales/en.json vs zh.json key + identical-value diff
│ ├── classify.py # apply 4-class labels to grep matches
│ ├── render_report.py # produce gap-report.md + comment-body.md
│ ├── post_comment.sh # gh issue comment 10 with comment-body.md (or PENDING-*)
│ └── file_followups.sh # gh issue create per category (or PENDING-*)
└── <commit-sha>/ # captured outputs of one verification run
├── cjk-grep.txt # raw `git grep -nP ...` output
├── cjk-grep-bucketed.txt # the same, partitioned by top-level path
├── parity.txt # en/zh diff summary
├── classified.csv # match-by-match label
├── gap-report.md # the canonical structured report
├── comment-body.md # the markdown posted to issue #10
├── PENDING-issue-10-comment.md # only if gh comment failed
└── PENDING-followups/ # only if gh issue create failed
├── 01-frontend-ui-strings.md
├── 02-backend-log-strings.md
├── 03-backend-prompt-labels.md
└── 04-permanent-ci-guard.md
```
### Modified Files
- *(None.)* The spec explicitly forbids touching production source.
## System Flows
```mermaid
sequenceDiagram
participant V as Verifier
participant Run as run_audit.sh
participant FS as Working tree
participant GH as GitHub
V->>Run: bash run_audit.sh
Run->>FS: git grep -nP, git rev-parse HEAD
FS-->>Run: cjk-grep.txt + sha
Run->>FS: read locales json
FS-->>Run: en/zh dicts
Run->>Run: classify
Run->>FS: write audit slash sha slash artefacts
Run->>GH: gh issue comment 10
alt gh succeeds
GH-->>Run: comment URL
Run->>GH: gh issue create x N follow-ups
GH-->>Run: issue URLs
else gh fails
Run->>FS: write PENDING markdown to audit slash sha slash
end
Run-->>V: exit 0 success or exit 2 PENDING
```
**Key flow decisions**:
- The audit always writes the captured artefacts to disk first (idempotent, re-runnable). The GitHub side effects are the *last* steps so any earlier failure leaves a complete capture for inspection.
- A non-zero `gh` exit shifts the pipeline to PENDING mode rather than failing the whole run; the script exits `2` to flag "audit ran but GitHub side-effects didn't apply".
## Requirements Traceability
| Requirement | Summary | Components | Interfaces / Artefacts | Flows |
|-------------|---------|------------|------------------------|-------|
| 1.1 | Run canonical `git grep` | audit_cjk.sh | `cjk-grep.txt` | Audit step |
| 1.2 | Classify each match | classify.py | `classified.csv` | Audit step |
| 1.3 | Record file:line + step tag for `gap` | classify.py | `classified.csv` (`step` column) | Audit step |
| 1.4 | No file modifications during audit | run_audit.sh | scripts are read-only | — |
| 1.5 | `en.json` CJK = always `gap` | classify.py | hard rule in classifier | Audit step |
| 2.1 | Enumerate keys recursively | check_parity.py | `parity.txt` | Audit step |
| 2.2 | Missing-key gaps recorded | check_parity.py | `parity.txt` (missing-key block) | Audit step |
| 2.3 | EN catalogue CJK = `gap` | check_parity.py | `parity.txt` (cjk-in-en block) | Audit step |
| 2.4 | EN/ZH identical = `review-needed` | check_parity.py | `parity.txt` (identical-value block) | Audit step |
| 2.5 | No catalogue edits | check_parity.py | read-only stdlib JSON load | — |
| 3.1 | Enumerate prompt files | classify.py (heuristic — known files list) | `gap-report.md` Section 3 | — |
| 3.2 | Confirm locale-aware or EN-only | classify.py | `gap-report.md` Section 3 | — |
| 3.3 | Hard-coded ZH directive = `gap` | classify.py | `classified.csv` (`category=prompt-label`) | — |
| 3.4 | #3, #4, #5 prompts post-merge check | classify.py | `gap-report.md` Section 3 | — |
| 4.1 | Identify handoff boundaries | render_report.py | `gap-report.md` Section 4 | — |
| 4.2 | Confirm explicit or re-derived locale | render_report.py | `gap-report.md` Section 4 | — |
| 4.3 | Silent default = `gap` | classify.py | `classified.csv` (`category=propagation`) | — |
| 4.4 | Backend logger EN under EN | classify.py | `classified.csv` (`category=backend-log`) | — |
| 5.1 | Comment lists every checklist item | render_report.py | `comment-body.md` | Comment-post |
| 5.2 | Each `gap` includes file:line + follow-up link | render_report.py | `comment-body.md` | Comment-post |
| 5.3 | `manual-pending` items state repro steps | render_report.py | `comment-body.md` | Comment-post |
| 5.4 | Comment includes raw audit (or path) | render_report.py | `comment-body.md` (path reference) | Comment-post |
| 5.5 | Post via `gh issue comment 10` | post_comment.sh | `comment-body.md` | Comment-post |
| 6.1 | ZH covers every EN key | check_parity.py | (already passes per gap-analysis) | — |
| 6.2 | Locale-aware prompts symmetric | render_report.py | `gap-report.md` Section 6 | — |
| 6.3 | EN-only ZH value = `review-needed` | check_parity.py | `parity.txt` (identical-value block) | — |
| 6.4 | ZH regression filed as gap | classify.py | `classified.csv` | — |
| 7.1 | File issue per gap | file_followups.sh | `gh issue create` | Follow-up |
| 7.2 | Group by category | file_followups.sh | one body per category in `PENDING-followups/` | Follow-up |
| 7.3 | No production-code edits | run_audit.sh | only writes under `.kiro/specs/.../` | — |
| 7.4 | Label follow-ups `i18n` | file_followups.sh | `gh issue create --label i18n` | Follow-up |
| 7.5 | Fallback inline list when no `gh` | file_followups.sh | `PENDING-followups/*.md` | Follow-up |
| 8.1 | Capture raw output | run_audit.sh | `audit/<sha>/` directory | Audit step |
| 8.2 | Preserve previous run | run_audit.sh | `<sha>` subdirectory naming | Audit step |
| 8.3 | Record HEAD sha | run_audit.sh | `git rev-parse HEAD` | Audit step |
| 8.4 | Idempotent re-run | run_audit.sh | re-running on same sha overwrites that sha's dir | Audit step |
## Components and Interfaces
| Component | Domain | Intent | Req Coverage | Key Dependencies (P0/P1) | Contracts |
|-----------|--------|--------|--------------|--------------------------|-----------|
| run_audit.sh | Verification pipeline | Compose the audit and route artefacts | 1.4, 7.3, 8.1, 8.2, 8.3, 8.4 | git (P0), python3 (P0), gh (P1) | Batch |
| audit_cjk.sh | Static audit | Run `git grep -nP` and bucket | 1.1, 1.5 | git (P0) | Batch |
| check_parity.py | Catalogue diff | Diff en/zh + identical-value heuristic | 2.1, 2.2, 2.3, 2.4, 2.5, 6.1, 6.3 | python3 stdlib (P0) | Batch |
| classify.py | Classification | Apply the 4-class label per match | 1.2, 1.3, 1.5, 3.1, 3.2, 3.3, 3.4, 4.3, 4.4, 6.4 | cjk-grep.txt (P0), parity.txt (P0) | Batch |
| render_report.py | Report assembly | Produce gap-report.md + comment-body.md | 4.1, 4.2, 5.1, 5.2, 5.3, 5.4, 6.2 | classified.csv (P0) | Batch |
| post_comment.sh | GitHub side-effect | Post the comment on #10 | 5.5 | gh (P0), comment-body.md (P0) | Service |
| file_followups.sh | GitHub side-effect | Open follow-up issues | 7.1, 7.2, 7.4, 7.5 | gh (P0), PENDING-followups/* (P0) | Service |
### Verification pipeline
#### `run_audit.sh`
| Field | Detail |
|-------|--------|
| Intent | Single shell entrypoint that runs every step in order and persists artefacts under `audit/<commit-sha>/` |
| Requirements | 1.4, 7.3, 8.1, 8.2, 8.3, 8.4 |
**Responsibilities & Constraints**
- Must NOT modify any file outside `.kiro/specs/i18n-e2e-english-verification/`.
- Must capture HEAD sha before any other step (so the artefact path is set).
- Must exit `0` on full success (audit + GitHub side effects) and `2` on PENDING (audit succeeded, side effects didn't).
- Must be safely re-runnable on the same sha (overwriting that sha's directory is acceptable).
**Dependencies**
- Inbound: invoked manually by the verifier (`bash run_audit.sh`) — Criticality: P0.
- Outbound: `audit_cjk.sh`, `check_parity.py`, `classify.py`, `render_report.py`, `post_comment.sh`, `file_followups.sh` — Criticality: P0 each.
- External: `git`, `python3`, `gh` (P1 — fallback supported).
**Contracts**: Service [ ] / API [ ] / Event [ ] / Batch [x] / State [ ]
##### Batch / Job Contract
- **Trigger**: manual `bash .kiro/specs/i18n-e2e-english-verification/audit/scripts/run_audit.sh`.
- **Input / validation**: working tree at any commit; rejects detached non-clean trees? — no, the audit reads tracked files only via `git grep`, so unstaged edits are ignored deliberately.
- **Output / destination**: `.kiro/specs/i18n-e2e-english-verification/audit/<commit-sha>/`.
- **Idempotency & recovery**: Re-running on the same sha overwrites that sha's directory. PENDING outputs survive across runs until a `gh`-enabled run replaces them.
**Implementation Notes**
- Integration: invoked by humans only — no CI hookup in this spec.
- Validation: confirm `gh auth status` before attempting comment/issue posts; on failure, branch to PENDING.
- Risks: shell quoting around the PCRE pattern (`[\x{4e00}-\x{9fff}]`) — use single-quoted argument to `git grep -P`.
#### `audit_cjk.sh`
| Field | Detail |
|-------|--------|
| Intent | Run the canonical PCRE grep + per-bucket counts |
| Requirements | 1.1, 1.5 |
**Responsibilities & Constraints**
- Output: `cjk-grep.txt` (raw `git grep -nP` lines) and `cjk-grep-bucketed.txt` (one section per top-level path: `backend/app`, `frontend/src`, `locales/en.json`).
- Excludes binary file matches (e.g. `.jpeg` false positives).
**Dependencies**
- Inbound: `run_audit.sh` (P0).
- External: `git` 2.x (P0 — must support `-P` for PCRE).
**Contracts**: Batch [x]
##### Batch / Job Contract
- **Trigger**: invoked by `run_audit.sh`.
- **Input / validation**: receives the target output directory as argv[1]; aborts if missing.
- **Output / destination**: `cjk-grep.txt`, `cjk-grep-bucketed.txt` in `<sha>/`.
- **Idempotency & recovery**: deterministic — same tree → same output.
**Implementation Notes**
- Integration: pure read-only against `git`.
- Validation: `git --version` precondition; abort with a clear error if PCRE unsupported.
- Risks: ripgrep is NOT used (avoids a hard `rg` dependency); `git grep -P` is built-in to git's PCRE2 binding.
#### `check_parity.py`
| Field | Detail |
|-------|--------|
| Intent | Compare `locales/en.json` and `locales/zh.json`: key parity, CJK in EN, identical-value heuristic |
| Requirements | 2.1, 2.2, 2.3, 2.4, 2.5, 6.1, 6.3 |
**Responsibilities & Constraints**
- Recursively flattens nested-dict keys with dotted paths.
- Reports three blocks: `missing-keys`, `cjk-in-en`, `identical-values`.
- Treats values as `review-needed` only if (a) en value == zh value, (b) value is non-empty, (c) value is more than two ASCII words.
**Dependencies**
- Inbound: `run_audit.sh` (P0).
- External: `json` from Python stdlib (P0).
**Contracts**: Batch [x]
##### Batch / Job Contract
- **Trigger**: invoked by `run_audit.sh` with the `<sha>` directory as argv[1].
- **Input / validation**: reads `locales/en.json` and `locales/zh.json` from cwd (must be invoked from repo root); fails fast on JSON parse error.
- **Output / destination**: `parity.txt` in `<sha>/`.
- **Idempotency & recovery**: pure function of catalogue contents.
**Implementation Notes**
- Integration: invoked from repo root so relative paths resolve.
- Validation: parse-on-load, both files must be objects.
- Risks: the "more than two ASCII words" heuristic may produce noise — `review-needed` is intentionally a soft label not a `gap`.
#### `classify.py`
| Field | Detail |
|-------|--------|
| Intent | Apply the 4-class label (`deliberate` / `gap` / `non-applicable` / `review-needed`) and a category tag per match |
| Requirements | 1.2, 1.3, 1.5, 3.1, 3.2, 3.3, 3.4, 4.3, 4.4, 6.4 |
**Responsibilities & Constraints**
- Reads `cjk-grep.txt` and `parity.txt`; emits `classified.csv` with columns: `file`, `line`, `match`, `class`, `category`, `pipeline_step`.
- Categories (closed set): `frontend-ui-string`, `frontend-regex-parser`, `backend-docstring`, `backend-comment`, `backend-log`, `backend-prompt-label`, `propagation`, `catalogue-parity`, `binary-false-positive`.
- Pipeline-step tags (closed set): `Graph Build`, `Env Setup`, `Simulation`, `Report`, `Interaction`, `Logs`, `UI`, `n/a`.
- Classification rules:
- `locales/en.json` CJK → always `gap` / `catalogue-parity` / `n/a` (R1.5).
- File path under `frontend/src/views/` or `frontend/src/components/` AND match is inside a string literal (heuristic: enclosed in `'…'`/`"…"`/`` `…` ``) → `gap` / `frontend-ui-string`.
- Match inside a `text.match(/.../)` call in a `.vue` file → `frontend-regex-parser` / `gap` (cause: backend emits CJK).
- Backend `.py` file, line starts with `#` or appears inside a triple-quoted docstring → `deliberate-blocked-by-#7` / `backend-docstring` (or `backend-comment`) — counted but not filed as a fresh follow-up since #7 already covers it.
- Backend `.py` file, line contains `logger.`, `log.`, `print(` and CJK in a string literal → `gap` / `backend-log` / appropriate step tag.
- Backend `.py` file in `services/{ontology,oasis_profile,simulation_config,report_agent}_generator.py` and CJK appears inside an LLM-prompt context label (heuristic: a string literal not preceded by `#`) → `gap` / `backend-prompt-label`.
- Binary files (e.g. `.jpeg` ripgrep matches): `non-applicable` / `binary-false-positive`.
- Anything else: `review-needed` (forces a human look).
**Dependencies**
- Inbound: `audit_cjk.sh`, `check_parity.py` (P0).
- External: `csv` from Python stdlib.
**Contracts**: Batch [x]
##### Batch / Job Contract
- **Trigger**: invoked by `run_audit.sh` after the two preceding steps.
- **Input / validation**: `cjk-grep.txt` and `parity.txt` must exist in `<sha>/`.
- **Output / destination**: `classified.csv`.
- **Idempotency & recovery**: deterministic — same inputs → same csv.
**Implementation Notes**
- Integration: classification rules are heuristics, not a parser; correctness is bounded by careful regexes and an explicit "fallthrough = `review-needed`" rule.
- Validation: every input row produces an output row (no silent drops); a count-equality assertion runs at the end.
- Risks: false negatives (e.g. a Chinese log string that doesn't contain `logger.` on the same line) — `review-needed` fallthrough catches these.
#### `render_report.py`
| Field | Detail |
|-------|--------|
| Intent | Produce `gap-report.md` and `comment-body.md` |
| Requirements | 4.1, 4.2, 5.1, 5.2, 5.3, 5.4, 6.2 |
**Responsibilities & Constraints**
- `gap-report.md`: Sections: Overview, Section 1 (static audit), Section 2 (parity), Section 3 (prompt verification), Section 4 (propagation), Section 5 (issue-#10 checklist mapping), Section 6 (ZH regression), Section 7 (follow-up plan).
- `comment-body.md`: Markdown comment for issue #10 — mirrors the issue's checklist with `pass` / `gap` / `manual-pending` for each line, plus a "How to re-run" footer.
- Reads `classified.csv` and the issue body (snapshot at `.ticket/10.md`).
**Dependencies**
- Inbound: `classify.py` (P0), `.ticket/10.md` (P0).
- External: Python stdlib only.
**Contracts**: Batch [x]
##### Batch / Job Contract
- **Trigger**: `run_audit.sh` after `classify.py`.
- **Input / validation**: `classified.csv` and `.ticket/10.md` must exist.
- **Output / destination**: `gap-report.md`, `comment-body.md` in `<sha>/`.
- **Idempotency & recovery**: deterministic.
**Implementation Notes**
- Integration: the comment body must include a `Run on commit <sha>` header so the comment is traceable.
- Validation: confirm every issue-body checkbox has been mapped (count check).
- Risks: rendering CJK characters in markdown — Python writes UTF-8 by default; comment body is verified to round-trip via `gh`.
#### `post_comment.sh`
| Field | Detail |
|-------|--------|
| Intent | Post `comment-body.md` as a comment on issue #10 |
| Requirements | 5.5 |
**Responsibilities & Constraints**
- `gh issue comment 10 --repo salestech-group/MiroFish --body-file <sha>/comment-body.md`.
- On non-zero exit, copies the body to `<sha>/PENDING-issue-10-comment.md` and exits non-zero.
**Dependencies**
- External: `gh` (P0; degrades to PENDING when missing).
**Contracts**: Service [x]
##### Service Interface
```text
post_comment.sh <sha-dir>
precondition: <sha-dir>/comment-body.md exists
postcondition (success): comment posted; URL printed to stdout
postcondition (failure): <sha-dir>/PENDING-issue-10-comment.md present; exit code 2
```
**Implementation Notes**
- Integration: must be the second-to-last step (so failures don't block the issue-filing fallback).
- Validation: parses `gh`'s URL output and writes it to `<sha>/comment-url.txt` on success.
- Risks: PR-time rate limits — unlikely for a single comment.
#### `file_followups.sh`
| Field | Detail |
|-------|--------|
| Intent | Open one follow-up issue per gap category |
| Requirements | 7.1, 7.2, 7.4, 7.5 |
**Responsibilities & Constraints**
- Iterates `<sha>/PENDING-followups/*.md` (which `render_report.py` always writes; the ones whose category had zero gaps stay empty placeholders).
- For each non-empty body, runs `gh issue create --repo salestech-group/MiroFish --title <title> --body-file <body> --label i18n`.
- On `gh` failure for any single category, leaves the corresponding `PENDING-followups/<n>-*.md` in place and exits non-zero at the end (after attempting all categories).
**Dependencies**
- External: `gh` (P0; degrades to PENDING).
**Contracts**: Service [x]
##### Service Interface
```text
file_followups.sh <sha-dir>
precondition: <sha-dir>/PENDING-followups/*.md exist (possibly empty placeholders)
postcondition (success): all non-empty bodies posted; URLs appended to <sha-dir>/followup-urls.txt; bodies removed from PENDING-followups/
postcondition (partial): URLs in followup-urls.txt for the ones that posted; the rest stay in PENDING-followups/; exit code 2
```
**Implementation Notes**
- Integration: must be the last step.
- Validation: post-hoc count check (`gh` URLs + remaining PENDING bodies = total categories).
- Risks: a category that the spec already considers covered (e.g. backend docstrings → blocked by #7) is not re-filed; the spec's category list is closed and excludes that case.
## Data Models
### Domain Model
The audit operates on three logical concepts:
- **Match** — a single line of `git grep` output. `(file, line, raw_text)`.
- **Classification** — `(match, class ∈ {deliberate, gap, non-applicable, review-needed}, category ∈ closed-set, pipeline_step ∈ closed-set)`.
- **Follow-up** — `(category, title, body, status ∈ {posted, pending}, url?)`.
Invariant: every `Match` produces exactly one `Classification`; every `Classification` with `class == gap` belongs to exactly one `Follow-up` category (which may aggregate multiple gaps).
### Logical Data Model
**`classified.csv` schema** (CSV, UTF-8, header row):
| Column | Type | Notes |
|--------|------|-------|
| `file` | string | repo-relative path |
| `line` | int | 1-indexed |
| `match` | string | trimmed grep line |
| `class` | enum | `deliberate` / `gap` / `non-applicable` / `review-needed` |
| `category` | enum | closed set listed in classify.py rules |
| `pipeline_step` | enum | closed set listed in classify.py rules |
Natural key: `(file, line)`.
**`parity.txt` structure** (text, three labelled blocks):
```
[missing-keys]
en-only: <key.path>
zh-only: <key.path>
[cjk-in-en]
<key.path>: <value snippet>
[identical-values]
<key.path>: <value> # review-needed if non-trivial English prose
```
### Data Contracts & Integration
- **`comment-body.md`** must be valid GitHub-flavoured Markdown; checkbox lines preserve the issue's original ordering.
- **Follow-up issue body** must be valid GitHub-flavoured Markdown; first line is a one-sentence summary; subsequent sections are: `## Evidence` (file:line list), `## Linked from` (#10 + comment URL), `## Acceptance` (a small checklist).
## Error Handling
### Error Strategy
- **Read-only operations** (steps 1–4): on any uncaught error (missing file, JSON parse error), the script aborts with a non-zero exit before any artefact is half-written. The orchestrator uses `set -euo pipefail`.
- **GitHub side effects** (steps 5–6): wrapped — failure routes to PENDING outputs and the orchestrator exits `2`.
### Error Categories and Responses
- **User errors**: invoked from wrong directory → fail fast with "must be run from repo root".
- **System errors**: `git`/`python3`/`gh` missing → fail fast with "install <tool>"; `gh auth status` not OK → branch to PENDING.
- **Business errors**: classification produces 0 matches but `cjk-grep.txt` non-empty → assertion failure (count-equality bug).
### Monitoring
- The orchestrator prints a one-line status per step.
- Final summary block to stdout: total matches, gaps, `manual-pending`, follow-ups posted vs PENDING.
## Testing Strategy
- **Unit tests**: not introduced — the scripts are simple enough that a one-shot dry run on the live tree is the canonical validation.
- **Integration test**: a single `bash run_audit.sh` against the working tree; success criteria below.
- **Validation checklist** (run during implementation):
- The audit produces a non-empty `cjk-grep.txt`.
- `parity.txt` reports 0 missing keys (matches the live state at HEAD).
- `classified.csv` row count == `cjk-grep.txt` line count.
- `gap-report.md` and `comment-body.md` parse as valid markdown (manual eyeball — no toolchain required).
- The classifier marks every `locales/en.json` CJK as `gap` (currently zero such matches, so this asserts the negative).
- With `gh` available: a comment is posted on #10 and follow-up issues are created.
- With `gh` simulated as absent (e.g. `PATH=/dev/null`): PENDING outputs appear under `<sha>/`.
### Out of scope for testing
- The live UI walkthrough is `manual-pending` (R5.3) and not part of the test plan.
- Performance, scalability, security: nothing to test — read-only single-shot scripts.

136
.kiro/specs/i18n-e2e-english-verification/gap-analysis.md Normal file
View File

@ -0,0 +1,136 @@
# Gap Analysis — i18n-e2e-english-verification
## 1. Current state investigation
### Domain-relevant assets in the repo
| Concern | Location | Notes |
|---|---|---|
| Locale catalogues | `locales/en.json`, `locales/zh.json`, `locales/languages.json` | Flat-namespaced JSON, loaded by `vue-i18n` and the backend logger. |
| Frontend i18n loader | `frontend/src/i18n/` | Provides `useI18n()` to components. |
| Frontend UI surface | `frontend/src/views/`, `frontend/src/components/` | Step1–5 components + `Process.vue` orchestrator. |
| Backend logger | `backend/app/utils/logger.py` (per CLAUDE.md) | Externalised log messages (#6 work). |
| Locale helpers | `backend/app/utils/` | Per CLAUDE.md, locale propagation lives here. |
| Prompt assets that emit user-visible text | `backend/app/services/ontology_generator.py` (#2, #3?), `oasis_profile_generator.py` (#3), `simulation_config_generator.py` (#4), `report_agent.py` (#5) | Prompts are inline Python strings, not separate files. |
| Pipeline boundaries | `backend/app/api/*.py` (Flask), `services/simulation_runner.py` + `simulation_ipc.py` (subprocess), `services/report_agent.py` (ReACT) | Locale must propagate across all of these. |
### Project conventions surfaced
- `Task` model used for any long-running operation (CLAUDE.md). Verification doesn't introduce one — it is a one-shot batch.
- Reasoning-model output stripping convention exists, irrelevant here.
- Per-project `group_id` isolation in Neo4j — verification queries should NOT touch Neo4j; we run a static audit only.
- "Match the surrounding file's style" (no enforced formatter).
### Live audit baseline (commit `9dcaecd`)
```
git grep -nP "[\x{4e00}-\x{9fff}]" -- backend/app frontend/src locales/en.json | wc -l
→ 2918 lines across 36 files
```
Bucketed:
| Bucket | Files | Lines | Notes |
|---|---|---|---|
| `locales/en.json` | 0 | 0 | ✅ clean |
| `frontend/src/views/Process.vue` | 1 | 65 | hard-coded UI strings (template + JS literals), not i18n keys |
| `frontend/src/components/Step{2,3,4,5}*.vue` | 4 | ~50 (mostly Step4Report.vue regex parsers) | depends-on-backend regex parsers + a few literals |
| `backend/app/services/*.py` | 13 | majority | docstrings + comments + a few prompt assembly fragments + agent context labels (e.g. `"事实信息:"` in `oasis_profile_generator.py`) |
| `backend/app/api/*.py` | 4 | many | docstrings + comments + log-message Chinese (`build_logger.info(f"[{task_id}] 开始构建图谱...")` etc) |
| `backend/app/utils/*.py` | 7 | many | docstrings + comments + log strings (e.g. `retry.py` "函数 {func} 在 N 次重试后仍失败") |
| `backend/app/models/*.py` | 3 | docstrings | docstrings only (probably) |
### Locale catalogue parity (Python check)
```
en keys: 953
zh keys: 953
symmetric diff: 0
```
→ R2 (parity) passes. ZH backfill (#8) closed the gap and en/zh are now lock-step.
### Boundary review surface (R4)
- `backend/app/api/graph.py` `build_logger.info(f"[{task_id}] 开始构建图谱...")` shows the backend logger is still emitting Chinese on the build path — this is exactly the kind of leak #6 was supposed to externalise.
- `backend/app/utils/retry.py` `logger.error(f"函数 {func.__name__} 在 {max_retries} 次重试后仍失败...")` — same: log strings remain hard-coded Chinese.
- ReACT/agent context labels in `oasis_profile_generator.py` (`"事实信息:"`, `"相关实体:"`) feed directly into the LLM prompt — these will bias the model toward Chinese output.
## 2. Requirements feasibility
### Mapping requirements → existing assets
| Req | Need | Existing asset | Gap tag |
|---|---|---|---|
| R1 (static audit) | run `git grep` and capture output | git, ripgrep | None — straightforward |
| R1.5 (`en.json` CJK check) | inspect catalogue | already at 0 hits | None — passes |
| R2 (parity) | enumerate keys recursively, diff | small Python script | None — already passes |
| R3 (prompt verification) | read prompt strings in `services/*.py` | inline Python strings | **Constraint** — prompts are inline, not standalone files; verification must read source not assets |
| R4 (propagation) | trace locale across Flask → Task → OASIS → ReACT | source code review | **Research needed** in design phase: where exactly is locale stored today? CLAUDE.md hints `set_locale` thread-local exists but path not yet read |
| R5 (post comment) | `gh issue comment 10` | `gh` CLI | None |
| R6 (ZH regression) | confirm zh values are non-English | small Python script | None |
| R7 (file follow-ups) | `gh issue create` | `gh` CLI | None |
| R8 (capture & idempotence) | write under `.kiro/specs/.../audit/` | filesystem | None |
### Complexity signals
- Algorithmic: trivial — grep + count + diff.
- Workflow: post a comment + open follow-up issues — one-shot.
- External integrations: GitHub via `gh`. No DB, no Neo4j, no LLM calls.
### Constraints from existing architecture
- **No code edits to `backend/app/`, `frontend/src/`, `locales/`** — the spec is verification-only. The change-set is confined to `.kiro/specs/i18n-e2e-english-verification/` (audit captures, gap report, follow-up issue list) and any commit message / PR description.
- Manual UI walkthrough is not feasible in a sandboxed CLI — must be marked `manual-pending` per R5.3.
- Live `docker-compose up` likewise unavailable — same handling.
## 3. Implementation approach options
### Option A — Pure shell + Python script kept under `.kiro/specs/.../audit/`
- A single Bash + Python pipeline that emits `audit/cjk-grep.txt`, `audit/parity.txt`, `audit/gap-report.md`.
- Posts the comment via `gh` and opens follow-ups via `gh issue create`.
- Scripts are read-only against production source.
✅ Simplest, no production-code touch.
✅ Easy to re-run.
❌ Scripts only relevant to this ticket — scoped to `.kiro/specs/.../audit/scripts/`, not promoted to a reusable `tools/`.
### Option B — Build a reusable `tools/i18n-audit/` checker
- Create a permanent CLI under `tools/` so future verifiers can re-run.
- Integrates with CI (could become a check that fails when `en.json` contains CJK).
❌ Adds a tool & directory the project doesn't have. Scope creep — the spec is for one verification pass, not a CI check.
❌ A reusable tool wants its own ticket; ramming it in here violates the "no inline fixes" rule.
### Option C — Hybrid: ad-hoc script for this run, plus open a follow-up issue requesting the reusable CI check
- Run the verification with disposable scripts (Option A) AND file a follow-up issue asking for the reusable CI check (Option B as a future ticket).
✅ Keeps current ticket scoped.
✅ Captures the value of B without bloating this PR.
## 4. Out-of-scope items deferred
- Any **production code edits** that would close gaps. R7 makes this explicit.
- Live UI walkthrough / dynamic verification — captured as `manual-pending` in the report.
## 5. Effort & risk
- **Effort**: S (1 day) — auditing scripts + report writing + issue filings.
- **Risk**: Low — read-only operations, no architectural change, the failure mode (`gh` lacking permissions) is handled by R7.5 (fallback inline list).
## 6. Recommendations for design phase
- **Preferred approach**: Option C (hybrid).
- **Key decisions to make in design**:
- Concrete script layout under `.kiro/specs/i18n-e2e-english-verification/audit/`.
- Format of `audit/gap-report.md` (the artefact echoed into the issue comment).
- Exact follow-up issue grouping rule (R7.2): one issue per pipeline step? per file? per category (UI / logs / prompts / docstrings)?
- Reproducibility (R8.2): do we keep `audit/<commit-sha>/` per run, or `audit/latest/` + `audit/previous/`?
- Whether the scripts are committed to the repo (they live under `.kiro/specs/...` — yes by default) or only the captured outputs.
- **Research items to carry forward**:
- Read `backend/app/utils/` to confirm whether a locale helper / `set_locale` exists today (R4 detail).
- Read `backend/app/utils/logger.py` to confirm where externalised log keys live and how the locale is selected at log time (R4 + Step-1 logs checklist item).
- Confirm whether any `services/*.py` Chinese match is part of an LLM **prompt** vs a comment — only prompt matches block R3.

122
.kiro/specs/i18n-e2e-english-verification/requirements.md Normal file
View File

@ -0,0 +1,122 @@
# Requirements Document
## Project Description (Input)
Issue #10: i18n end-to-end verification of full pipeline. Run a verification pass to prove the entire 5-step pipeline (Graph Build, Env Setup, Simulation, Report, Interaction) works cleanly in English, with locale propagating across Flask routes, background tasks, OASIS subprocess, Graphiti/Neo4j, and the ReACT report agent. Produce a verification report (posted as a comment on issue #10) summarising pass/fail per checklist item and listing any leftover Chinese strings as `file:line` refs. Run the static audit `git grep -nE "[\\x{4e00}-\\x{9fff}]" -- backend/app frontend/src locales/en.json` and confirm only deliberately-kept Chinese remains. File any newly discovered gaps as follow-up issues (do NOT patch silently in this ticket). Acceptance: all checklist items pass for both EN and ZH; report posted; no surprise Chinese in EN paths. Out of scope: fixing newly discovered gaps inline; perf/load testing; new locales beyond EN/ZH.
## Introduction
This spec covers the final verification pass for the i18n epic (#11). After issues #2–#9, #12 land, the entire 5-step MiroFish pipeline must demonstrably run in English — UI, background work, LLM-generated artifacts (ontologies, agent profiles, sim configs, reports, chat replies), and backend logs — without any unintended Chinese leaking into English-locale paths. The pass also regression-checks that switching locale back to Chinese still produces fully Chinese output. Because the pipeline crosses a Flask app, background `Task` workers, an OASIS subprocess, Graphiti/Neo4j, and a ReACT report agent, the verification has both a static (grep + locale-file) component and a dynamic (live walkthrough of Step 1 → 5) component.
The deliverables are: (a) a static audit + categorization of any remaining Chinese strings under English paths, (b) a verification report posted as a comment on issue #10 summarising pass/fail per checklist item with `file:line` evidence, and (c) follow-up GitHub issues for every gap found — fixes are explicitly **out of scope** here.
## Boundary Context
- **In scope**:
- Static audit (`git grep` for CJK Unified Ideographs) of `backend/app/`, `frontend/src/`, and `locales/en.json`.
- Inspection of locale catalogues (`locales/en.json`, `locales/zh.json`) for parity, key coverage, and accidental Chinese in the EN catalogue.
- Inspection of LLM-prompt assets that drive Step 1–5 outputs (ontology, profile, sim-config, report-agent prompts) to confirm they emit English under EN locale.
- Inspection of locale propagation paths: HTTP request → Flask handler → `Task` background worker → OASIS subprocess → ReACT agent.
- Verification report posted as a comment on issue #10.
- Follow-up issues filed for every gap found.
- **Out of scope**:
- Fixing any newly discovered gaps inline in this ticket — they are filed as separate issues.
- Performance or load testing.
- Adding new locales beyond EN/ZH.
- The live UI walkthrough with screenshots, when no human or browser is available — the static audit results plus prompt/locale-catalogue evidence stand in. The verification report explicitly marks UI-only checklist items as "manual-pending" if not run live.
- **Adjacent expectations**:
- Closes the i18n epic #11 once #12 also lands.
- Depends on (and re-verifies) the work in #2, #3, #4, #5, #6, #8, #9, #12.
## Requirements
### Requirement 1: Static CJK audit of English code paths
**Objective:** As an i18n verifier, I want a deterministic grep-based audit of files that should be English-only, so that any Chinese leaking into the EN-locale code path is detected and recorded.
#### Acceptance Criteria
1. The Verification System shall execute `git grep -nE "[\x{4e00}-\x{9fff}]" -- backend/app frontend/src locales/en.json` and capture every match with `file:line` precision.
2. The Verification System shall classify each match as one of: (a) `deliberate` (e.g. test fixture demonstrating ZH input, doc example, comment explicitly retained per project convention), (b) `gap` (unintended Chinese in EN-facing code), or (c) `non-applicable` (false positive such as a regex character class).
3. When a match is classified as `gap`, the Verification System shall record `file:line`, the Chinese substring, and the affected pipeline step (Graph Build / Env Setup / Simulation / Report / Interaction / Logs / UI).
4. The Verification System shall not modify any matched file as part of this audit; remediation is filed as a follow-up issue per Requirement 7.
5. While the audit is running, the Verification System shall additionally inspect `locales/en.json` for entries whose value contains CJK characters and report those separately (an EN catalogue value containing Chinese is always a `gap`).
### Requirement 2: Locale catalogue parity check
**Objective:** As an i18n verifier, I want to confirm that the EN and ZH catalogues stay in lockstep, so that switching locale never falls back to a missing key or leaks the other locale.
#### Acceptance Criteria
1. The Verification System shall enumerate the key set of `locales/en.json` and `locales/zh.json` (recursively across nested objects) and compute the symmetric difference.
2. If a key is present in `en.json` but missing from `zh.json` (or vice versa), the Verification System shall record the missing key path and treat it as a `gap`.
3. If any value in `en.json` contains a CJK character, the Verification System shall record it as a `gap` (as in Requirement 1.5).
4. If any value in `zh.json` is identical to its `en.json` counterpart and the EN value is non-trivial English prose (more than two ASCII words), the Verification System shall flag it as a candidate untranslated entry — these are reported as `review-needed`, not auto-classified `gap`, since some technical terms (URLs, identifiers, single tokens) legitimately stay identical.
5. The Verification System shall not edit either catalogue file as part of this check.
### Requirement 3: LLM-prompt locale verification
**Objective:** As an i18n verifier, I want to confirm that every LLM prompt that drives a Step 1–5 output respects the requested locale, so that ontology entries, agent profiles, simulation configs, report prose, and chat replies render in the user's selected language.
#### Acceptance Criteria
1. The Verification System shall enumerate the prompt files that produce user-visible output for Steps 1–5 (e.g. ontology generator, OASIS profile generator, simulation-config generator, report agent prompts, interview chat).
2. For each prompt file, the Verification System shall confirm that it either (a) is fully English with an explicit "respond in ${locale}" directive, or (b) is rendered through a locale-aware template that injects the active locale.
3. If a prompt file hard-codes a Chinese-only directive (e.g. "请用中文回答") on the EN code path, the Verification System shall record it as a `gap`.
4. The Verification System shall confirm that the prompt files referenced by issues #3, #4, #5 are no longer Chinese-only post-merge; if any still are, they are recorded as `gap` blocking #10.
### Requirement 4: Locale propagation surface review
**Objective:** As an i18n verifier, I want to confirm that the active locale survives every process boundary, so that an EN request still produces EN output after it crosses into a `Task` worker, the OASIS subprocess, or the ReACT agent.
#### Acceptance Criteria
1. The Verification System shall identify each handoff boundary: HTTP → Flask handler, Flask handler → `Task` worker, `Task` worker → OASIS subprocess, ReACT agent → tool calls.
2. For each handoff, the Verification System shall confirm that the locale is either (a) carried explicitly in the call payload / kwargs, or (b) re-derived deterministically (e.g. from per-project config, `Accept-Language` header, or `set_locale` thread-local equivalent) on the receiving side.
3. If a boundary discards the locale and the receiving side defaults silently to Chinese (or any non-EN locale) under an EN request, the Verification System shall record the boundary as a `gap`.
4. The Verification System shall examine the backend logger to confirm that log messages on the EN code path resolve to English templates (depends on #6).
### Requirement 5: Verification report comment on issue #10
**Objective:** As the issue owner, I want a single canonical verification report posted as a comment on issue #10, so that reviewers can see pass/fail per checklist item and trace every `gap` to a `file:line` and a follow-up issue.
#### Acceptance Criteria
1. When the static audit, parity check, prompt verification, and propagation review are complete, the Verification System shall compose a markdown comment on issue #10 that lists every checklist item from the ticket body with one of the statuses `pass` / `gap` / `manual-pending`.
2. For each `gap` status, the comment shall include `file:line` references and a link to the follow-up issue filed per Requirement 7.
3. For each `manual-pending` status, the comment shall state explicitly that the item requires a live UI walkthrough (or full-stack run) which was not performed in this verification environment, and shall list the exact reproduction steps the next reviewer needs to run.
4. The comment shall include the raw output (or a path to the captured output) of the `git grep` audit so future verifiers can diff against the baseline.
5. The Verification System shall post the comment using `gh issue comment 10 --repo salestech-group/MiroFish` and shall record the resulting comment URL in the spec / commit message.
### Requirement 6: ZH regression check
**Objective:** As an i18n verifier, I want to confirm that the ZH locale still renders fully Chinese, so that the EN work has not regressed the original-language experience.
#### Acceptance Criteria
1. The Verification System shall confirm that `locales/zh.json` covers every key present in `locales/en.json` (Requirement 2) so that no UI string falls back to English under ZH.
2. The Verification System shall confirm that prompts rendered through locale-aware templates produce a Chinese variant when locale=zh (i.e. the templating mechanism is symmetric between EN and ZH).
3. If a UI string is English-only under ZH (i.e. `zh.json` value is identical to the EN value and the value is non-trivial English prose), the Verification System shall flag it per Requirement 2.4 as `review-needed`.
4. The Verification System shall record any ZH-specific regression as a separate `gap` and file a follow-up issue per Requirement 7.
### Requirement 7: Follow-up issues for every discovered gap
**Objective:** As the project owner, I want every gap discovered in this verification pass tracked as its own GitHub issue, so that fixes are sequenced separately and #10 stays scoped to verification only.
#### Acceptance Criteria
1. When a `gap` is recorded by Requirements 1–6, the Verification System shall file a GitHub issue against `salestech-group/MiroFish` containing: a one-sentence summary, the affected pipeline step, the `file:line` evidence, and a link back to issue #10 and to the verification report comment.
2. If grouping is sensible (e.g. five `gap`s in a single locale-catalogue file), the Verification System shall consolidate them into a single follow-up issue with a checklist body, instead of filing five micro-issues.
3. The Verification System shall not patch any gap inline in this ticket; the spec change-set must be limited to the verification artefacts (spec docs + report capture under `.kiro/specs/i18n-e2e-english-verification/`) and must not modify production source files under `backend/app/`, `frontend/src/`, or `locales/`.
4. The Verification System shall label every follow-up issue with the `i18n` label (and `bug` if the gap is regressing existing behaviour) so they aggregate under the i18n epic.
5. If the verification environment cannot file issues (e.g. no `gh` permissions), the Verification System shall list the would-be issues inline in the verification report as a fallback so a human can file them, and shall mark the corresponding checklist item `gap-pending-issue` instead of `gap`.
### Requirement 8: Reproducibility and idempotence
**Objective:** As a future verifier, I want this verification pass to be re-runnable, so that we can re-baseline after each subsequent merge to the i18n epic.
#### Acceptance Criteria
1. The Verification System shall capture the raw audit output to `.kiro/specs/i18n-e2e-english-verification/audit/` so the next verifier can diff against the previous run.
2. While a previous capture exists, the Verification System shall preserve it (timestamped or under a `previous/` subdirectory) rather than overwriting it silently.
3. The Verification System shall record the commit SHA at the time of the audit so the report comment can be tied to a specific tree state.
4. If the audit is re-run and the gap set is unchanged, the Verification System shall produce a no-op report comment that confirms parity rather than spamming a new gap list.

112
.kiro/specs/i18n-e2e-english-verification/research.md Normal file
View File

@ -0,0 +1,112 @@
# Research & Design Decisions — i18n-e2e-english-verification
## Summary
- **Feature**: `i18n-e2e-english-verification`
- **Discovery Scope**: Extension (verification-only against existing i18n surface)
- **Key Findings**:
- `locales/en.json` is already CJK-clean (0 hits) and `locales/zh.json` is at perfect parity (953/953 keys).
- Bulk of remaining CJK is in backend Python source (~26 files across `services/`, `api/`, `utils/`, `models/`) — overwhelmingly docstrings, comments, and a non-trivial number of log strings + LLM-prompt context labels. This is blocked by issue #7 (translate Chinese docstrings/comments).
- Frontend `Process.vue` still has ~65 hard-coded Chinese strings in template/JS literals (not routed through `t()` keys); 4 step components have a smaller surface (mainly Step4Report's regex parsers that match Chinese backend output).
- Live UI/full-stack walkthrough is not feasible in this sandboxed CLI environment — that portion of the verification will be reported as `manual-pending` with reproduction steps.
## Research Log
### Audit baseline
- **Context**: R1 requires running the canonical `git grep` audit and bucketing the matches.
- **Sources consulted**: ripgrep / `git grep -P` against the working tree at `9dcaecd` (HEAD of `docs/i18n-9-translate-frontend-comments`).
- **Findings**:
- Total CJK lines: **2918** across **36** files (counting 2 binary `.jpeg` false positives that ripgrep matches when scanning the assets folder).
- Bucket distribution: `locales/en.json` 0 / `frontend/src` 7 files (5 source + 2 binary) / `backend/app` 29 files.
- The shell-style regex `[\x{4e00}-\x{9fff}]` in the issue body must be passed to `git grep` with `-P` (PCRE) — POSIX ERE rejects `\x{...}` ranges. The verification scripts must use `-P` or document the deviation.
- **Implications**: The audit script must use PCRE; binary files should be excluded explicitly so the `.jpeg` false positives do not pollute the gap report.
### Locale-catalogue parity
- **Context**: R2 demands key-set parity between `en.json` and `zh.json`.
- **Sources consulted**: small Python diff over the catalogues (recursive nested-dict key flattening).
- **Findings**: 953 keys each, symmetric difference 0. Already passing.
- **Implications**: R2.1, R2.2 will trivially pass; R2.4 (untranslated-but-identical entries) still needs running.
### Locale propagation surface
- **Context**: R4 requires confirming that locale survives Flask handler → `Task` → OASIS subprocess → ReACT agent.
- **Sources consulted**: `backend/app/api/graph.py`, `backend/app/services/` skim, CLAUDE.md (mentions `set_locale` thread-local).
- **Findings**:
- `backend/app/api/graph.py` line 385 etc still emit Chinese log strings inline (`build_logger.info(f"[{task_id}] 开始构建图谱...")`) — the log externalisation work (#6) didn't reach these call sites.
- `backend/app/utils/retry.py` log strings are still hard-coded Chinese (`logger.error(f"函数 {func.__name__} ...")`).
- `oasis_profile_generator.py` LLM-prompt context labels (`"事实信息:"`, `"相关实体:"`) feed into the agent prompt verbatim — these will bias the LLM toward Chinese output even under EN locale.
- **Implications**: R4.3 (locale discarded silently → defaults non-EN) has live evidence; multiple `gap` items will be filed.
## Architecture Pattern Evaluation
| Option | Description | Strengths | Risks / Limitations | Notes |
|--------|-------------|-----------|---------------------|-------|
| Pure shell + Python script (Option A) | One-shot scripts in `.kiro/specs/.../audit/scripts/` produce `audit/<sha>/*.txt` and `audit/<sha>/gap-report.md` | Simplest; no production-code touch; easy to re-run; fits R8 capture format | Scoped to this ticket — not a permanent CI guard | Selected |
| Reusable `tools/i18n-audit/` CLI (Option B) | Promote the audit to a permanent project tool wired into CI | Long-term safety net; future PRs would fail on regressions | Out of scope per R7.3 (verification-only); adds new top-level directory | Filed as a follow-up issue, not implemented here |
| Hybrid (Option C) | Run Option A now; file an issue requesting Option B as future work | Captures B's value without bloating this PR | None material | Adopted |
## Design Decisions
### Decision: Audit lives entirely under `.kiro/specs/i18n-e2e-english-verification/`
- **Context**: R7.3 forbids modifying production source in this ticket; the verification artefacts (scripts and captures) need a home.
- **Alternatives considered**:
1. Top-level `tools/i18n-audit/` — rejected (creates a long-lived asset out of a one-shot ticket).
2. `scripts/` next to existing project scripts — rejected (project has no convention for verification scripts; `.kiro/specs/` is the canonical home for spec-scoped work).
3. `.kiro/specs/.../audit/` — selected.
- **Selected approach**: Scripts at `.kiro/specs/i18n-e2e-english-verification/audit/scripts/` and outputs at `.kiro/specs/.../audit/<commit-sha>/`.
- **Rationale**: Co-locates spec, requirements, design, and the artefacts a future verifier needs to re-run the pass. Honours the steering rule that the spec dir is the source of truth for spec-scoped state.
- **Trade-offs**: Scripts aren't reused beyond this ticket. Re-runs require checking out the spec dir (which is committed).
- **Follow-up**: File a follow-up issue suggesting Option B (a permanent CI guard) for the next iteration of the i18n epic.
### Decision: Manual UI walkthrough → `manual-pending`, not `gap`
- **Context**: R5.3 already permits `manual-pending` when a checklist item requires running the live stack. This run is sandboxed CLI — no browser, no Docker.
- **Alternatives considered**:
1. Mark UI items `gap` because they weren't proven — rejected (a `gap` is a *known* failure; UI items are simply untested in this run).
2. Skip them silently — rejected (R5.1 requires every checklist item to have a status).
3. Mark `manual-pending` with reproduction steps — selected.
- **Rationale**: Honest about the verification environment's limits. Future verifiers can flip `manual-pending` to `pass` or `gap` after running the live walkthrough.
- **Trade-offs**: Issue #10 cannot be fully closed by this run alone; the verification-pass comment will say so explicitly.
### Decision: Gap classification = (deliberate / gap / non-applicable / review-needed)
- **Context**: R1.2 lists three classes; R2.4 introduces a fourth (`review-needed`).
- **Alternatives considered**:
1. Three-class only — rejected (forces premature decisions on identical en/zh values).
2. Four-class with explicit semantics — selected.
- **Rationale**: A four-class scheme keeps the `gap` count truthful (it counts only known-bad lines), and `review-needed` is a soft signal that a human should re-check.
- **Trade-offs**: Slightly more complex schema; mitigated by documenting the four labels at the top of `gap-report.md`.
### Decision: Follow-up grouping by category, not by file
- **Context**: R7.2 allows consolidation. There are too many CJK-bearing files (29) to file one issue each.
- **Alternatives considered**:
1. One issue per file — rejected (29 micro-issues).
2. One issue per pipeline step (R1.3 step tag) — feasible but cross-cuts existing per-component issues like #7.
3. One issue per **gap category** — selected: (a) frontend hard-coded UI strings, (b) backend log strings, (c) backend LLM-prompt context labels, (d) recommend a permanent CI check.
- **Rationale**: Categories already align with how the i18n epic broke down work (#3, #4, #5, #6 = LLM-prompts; #7 = docstrings/comments; #9 = frontend comments). Categories also map cleanly to single PRs, which is how subsequent fixes will land.
- **Trade-offs**: Some files appear in multiple categories. Mitigated by listing `file:line` evidence inside each category issue.
### Decision: Issue-comment fallback when `gh` is unavailable
- **Context**: R7.5 mandates a fallback if `gh` permissions are missing.
- **Selected approach**: If `gh` posts fail, the script writes the comment body to `audit/<sha>/PENDING-issue-10-comment.md` and the would-be follow-up issue bodies to `audit/<sha>/PENDING-followups/*.md` so a human can paste them.
- **Rationale**: Keeps the audit re-runnable offline; keeps the artefact set faithful to what *would* have been posted.
- **Trade-offs**: Verification doesn't truly close until a human posts. Surfaced loudly in the run-summary.
## Risks & Mitigations
- **Risk**: A `gap` is mis-classified as `non-applicable` (e.g. a regex character class versus a real Chinese label) → Mitigation: classification tracked in a small CSV alongside the raw grep, so re-classification is auditable.
- **Risk**: `gh` rate limits hit when filing follow-ups → Mitigation: file at most 4 follow-ups (one per category) — far below any rate limit.
- **Risk**: Re-running the audit on a divergent branch produces a noisy diff → Mitigation: `audit/<commit-sha>/` directories preserve history; comparison is opt-in via `diff -ru`.
- **Risk**: Live walkthrough never happens, leaving #10 in `manual-pending` indefinitely → Mitigation: the verification report comment names a concrete "next reviewer" reproduction script; `manual-pending` items have explicit acceptance criteria.
## References
- Issue #10 — https://github.com/salestech-group/MiroFish/issues/10
- Epic #11 — https://github.com/salestech-group/MiroFish/issues/11
- `gap-analysis.md` — bucketed audit baseline
- `requirements.md` — EARS acceptance criteria for this spec

24
.kiro/specs/i18n-e2e-english-verification/spec.json Normal file
View File

@ -0,0 +1,24 @@
{
"feature_name": "i18n-e2e-english-verification",
"created_at": "2026-05-07T18:25:18Z",
"updated_at": "2026-05-07T18:25:18Z",
"language": "en",
"phase": "tasks-generated",
"ticket": 10,
"ticket_url": "https://github.com/salestech-group/MiroFish/issues/10",
"approvals": {
"requirements": {
"generated": true,
"approved": true
},
"design": {
"generated": true,
"approved": true
},
"tasks": {
"generated": true,
"approved": true
}
},
"ready_for_implementation": true
}

87
.kiro/specs/i18n-e2e-english-verification/tasks.md Normal file
View File

@ -0,0 +1,87 @@
# Tasks — i18n-e2e-english-verification
## 1. Foundation — audit workspace and entrypoint
- [x] 1.1 Create the audit script directory and the read-only orchestrator skeleton
- Establish `.kiro/specs/i18n-e2e-english-verification/audit/scripts/` with a `run_audit.sh` skeleton that uses `set -euo pipefail`.
- The orchestrator captures HEAD sha (`git rev-parse HEAD`) and creates `.kiro/specs/i18n-e2e-english-verification/audit/<sha>/` as the artefact root.
- Observable completion: running `bash .kiro/specs/i18n-e2e-english-verification/audit/scripts/run_audit.sh` from repo root creates an empty `audit/<sha>/` directory and exits `0`.
- _Requirements: 1.4, 7.3, 8.1, 8.2, 8.3, 8.4_
- _Boundary: run_audit.sh_
## 2. Core — read-only audit producers
- [x] 2.1 (P) Implement the canonical CJK grep with PCRE
- `audit_cjk.sh` runs `git grep -nP '[\x{4e00}-\x{9fff}]' -- backend/app frontend/src locales/en.json` and writes the raw output to `<sha>/cjk-grep.txt`.
- Produces a partitioned `<sha>/cjk-grep-bucketed.txt` with one section per top-level path (`backend/app`, `frontend/src`, `locales/en.json`).
- Excludes binary file matches (e.g. `.jpeg`) by skipping paths whose `git check-attr` reports `binary` (or by file-extension allowlist if check-attr is unset).
- Observable completion: `<sha>/cjk-grep.txt` contains exactly the same lines as a manual `git grep -nP …` run, and `<sha>/cjk-grep-bucketed.txt` has the three labelled sections with line counts.
- _Requirements: 1.1, 1.5_
- _Boundary: audit_cjk.sh_
- [x] 2.2 (P) Implement the locale-catalogue parity diff
- `check_parity.py` loads `locales/en.json` and `locales/zh.json`, recursively flattens nested-dict keys with dotted paths, and writes `<sha>/parity.txt` with three labelled blocks: `[missing-keys]`, `[cjk-in-en]`, `[identical-values]`.
- The `[identical-values]` block flags entries only when EN value equals ZH value AND the value is non-empty AND has more than two ASCII words.
- Observable completion: `<sha>/parity.txt` exists; on the current tree `[missing-keys]` is empty and `[cjk-in-en]` is empty (matching the gap-analysis baseline).
- _Requirements: 2.1, 2.2, 2.3, 2.4, 2.5, 6.1, 6.3_
- _Boundary: check_parity.py_
- [x] 2.3 Implement the four-class classifier
- `classify.py` consumes `<sha>/cjk-grep.txt` and `<sha>/parity.txt` and writes `<sha>/classified.csv` with columns `file,line,match,class,category,pipeline_step`.
- Implements the closed-set rules from design.md "classify.py": `locales/en.json` CJK → `gap`/`catalogue-parity`; `frontend/src/{views,components}/*.vue` string literal → `gap`/`frontend-ui-string`; `text.match(/.../)` regex pattern with CJK → `gap`/`frontend-regex-parser`; `.py` line starting with `#` or inside a triple-quoted block → `deliberate`/`backend-{comment,docstring}`; `.py` `logger.|log.|print(` line with CJK in a string literal → `gap`/`backend-log` with appropriate step tag; `.py` LLM-prompt label in `services/{ontology,oasis_profile,simulation_config,report_agent}_generator.py` → `gap`/`backend-prompt-label`; binary file → `non-applicable`/`binary-false-positive`; everything else → `review-needed`.
- Asserts row-count equality with the input grep (no silent drops).
- Observable completion: `<sha>/classified.csv` row count == `cjk-grep.txt` line count, and at least one row of each non-empty class is present (verified by counting per-class rows in stdout summary).
- _Requirements: 1.2, 1.3, 1.5, 3.1, 3.2, 3.3, 3.4, 4.3, 4.4, 6.4_
- _Boundary: classify.py_
- _Depends: 2.1, 2.2_
## 3. Core — report assembly
- [x] 3.1 Render the gap report and the issue-#10 comment body
- `render_report.py` reads `<sha>/classified.csv` and `.ticket/10.md`; writes `<sha>/gap-report.md` (with the seven sections from design.md) and `<sha>/comment-body.md` (mirroring the issue's checklist with `pass`/`gap`/`manual-pending` per line + a "How to re-run" footer + a `Run on commit <sha>` header).
- Section 4 of `gap-report.md` enumerates the four propagation boundaries and reports each as `pass`/`gap`/`unknown`, with file:line evidence drawn from `classified.csv`.
- Section 5 maps every checklist item from `.ticket/10.md` to a `pass` / `gap` / `manual-pending` status. UI-checklist items default to `manual-pending` (live walkthrough not feasible in sandbox) and include a concrete reproduction script.
- Always writes the four follow-up issue body templates to `<sha>/PENDING-followups/`: `01-frontend-ui-strings.md`, `02-backend-log-strings.md`, `03-backend-prompt-labels.md`, `04-permanent-ci-guard.md` — empty placeholder if the corresponding category had zero `gap` rows.
- Observable completion: `<sha>/gap-report.md`, `<sha>/comment-body.md`, and `<sha>/PENDING-followups/01..04-*.md` all exist; opening `<sha>/comment-body.md` shows every checkbox from `.ticket/10.md` mapped to a status.
- _Requirements: 4.1, 4.2, 5.1, 5.2, 5.3, 5.4, 6.2_
- _Boundary: render_report.py_
## 4. Integration — orchestrator and GitHub side effects
- [x] 4.1 Wire run_audit.sh to the four producer steps and add the GitHub posting hooks
- `run_audit.sh` invokes (in order) `audit_cjk.sh`, `check_parity.py`, `classify.py`, `render_report.py`, then `post_comment.sh` and `file_followups.sh`.
- On any error in steps 1-4 the orchestrator aborts (`set -euo pipefail`) before any subsequent step runs.
- On `gh` failure in steps 5 or 6, the orchestrator continues to the next step but exits `2` at the end (audit succeeded, side effects didn't fully apply).
- Observable completion: a clean run on the current tree creates a complete `<sha>/` directory; if `gh` is forced absent (e.g. `PATH=$(pwd)/empty bash run_audit.sh`), the orchestrator still produces all four producer artefacts and the `PENDING-followups/` and exits with `2`.
- _Requirements: 1.4, 7.3, 8.1, 8.2, 8.3, 8.4_
- _Boundary: run_audit.sh_
- _Depends: 2.3, 3.1_
- [x] 4.2 Implement post_comment.sh and file_followups.sh with PENDING fallback
- `post_comment.sh` calls `gh issue comment 10 --repo salestech-group/MiroFish --body-file <sha>/comment-body.md`; on failure it copies the body to `<sha>/PENDING-issue-10-comment.md` and exits non-zero. On success it writes the resulting URL to `<sha>/comment-url.txt`.
- `file_followups.sh` iterates `<sha>/PENDING-followups/*.md`; for each non-empty body it calls `gh issue create --repo salestech-group/MiroFish --title <title-from-body-first-line> --body-file <body> --label i18n` (and `--label bug` when the body's frontmatter declares regression). On per-category failure it leaves that body in place; on success it removes the body and appends the issue URL to `<sha>/followup-urls.txt`.
- Observable completion: with `gh` available, the comment URL appears in `<sha>/comment-url.txt` and any non-empty follow-up body produces an issue URL in `<sha>/followup-urls.txt`; with `gh` absent, both bodies stay under `<sha>/PENDING-*` and exit codes are non-zero.
- _Requirements: 5.5, 7.1, 7.2, 7.4, 7.5_
- _Boundary: post_comment.sh, file_followups.sh_
- _Depends: 3.1_
## 5. Validation — execute the verification pass
- [x] 5.1 Execute the audit on the current tree and capture a baseline run
- Run `bash .kiro/specs/i18n-e2e-english-verification/audit/scripts/run_audit.sh` from repo root.
- Confirm `<sha>/cjk-grep.txt`, `cjk-grep-bucketed.txt`, `parity.txt`, `classified.csv`, `gap-report.md`, `comment-body.md`, and `PENDING-followups/01..04-*.md` all exist and are non-empty (the placeholders for empty categories may be empty by design).
- Confirm `parity.txt` `[missing-keys]` and `[cjk-in-en]` blocks are empty (matches the gap-analysis baseline).
- Confirm `classified.csv` row count matches `cjk-grep.txt` line count exactly.
- Observable completion: the baseline `<sha>/` directory is committed under `.kiro/specs/i18n-e2e-english-verification/audit/`.
- _Requirements: 1.1, 1.2, 1.3, 2.1, 2.2, 2.3, 8.1, 8.3_
- _Boundary: run_audit.sh and producer scripts_
- _Depends: 4.1_
- [x] 5.2 Post the comment on issue #10 and file the follow-up issues
- Run `post_comment.sh <sha-dir>` and `file_followups.sh <sha-dir>` (or rely on `run_audit.sh` to invoke them) so the verification report comment is posted and follow-up issues are filed for non-empty categories.
- Capture `comment-url.txt` and `followup-urls.txt` under `<sha>/` so the PR description can link to them.
- If `gh` lacks permissions for any of the calls, the corresponding `PENDING-*` file is left in place per R7.5; the run summary surfaces the partial state.
- Observable completion: a comment appears on https://github.com/salestech-group/MiroFish/issues/10 mirroring `comment-body.md`; follow-up issues for non-empty categories exist and carry the `i18n` label.
- _Requirements: 5.1, 5.2, 5.3, 5.4, 5.5, 6.4, 7.1, 7.2, 7.4, 7.5_
- _Boundary: post_comment.sh, file_followups.sh_
- _Depends: 4.2, 5.1_

286
.kiro/specs/i18n-externalize-remaining-backend-logs/design.md Normal file
View File

@ -0,0 +1,286 @@
# Design Document
## Overview
**Purpose**: Replace the last nine hard-coded Chinese log/print strings in three backend modules (`backend/app/api/graph.py`, `backend/app/services/oasis_profile_generator.py`, `backend/app/utils/retry.py`) with calls to the existing `t("log.<domain>.<key>", **kwargs)` helper, and add the corresponding entries to `locales/en.json` and `locales/zh.json`. The result is locale-correct backend logs with zero behavioural drift.
**Users**: Backend operators reading logs in English deployments; existing Chinese-locale operators (preserved verbatim).
**Impact**: Removes the last sources of Chinese-text leakage in backend logs under the `en` locale, completing the i18n coverage started by ticket #6.
### Goals
- Replace the nine f-string arguments listed in ticket #24 with `t("log.<domain>.<key>", **kwargs)` calls.
- Add eleven new locale entries (3 in `log.graph_api`, 2 in `log.profile_generator`, 4 in new `log.retry`) to both `locales/en.json` and `locales/zh.json` with key parity.
- Preserve all interpolated values, all log levels, all control flow, and all `print(...)` console banners.
### Non-Goals
- Translating other Chinese strings in the same files (docstrings, comments, `update_task` messages, `progress_callback` messages, `logger.warning` retry messages) — out of scope for ticket #24.
- Modifying the `t()` helper, the locale resolution logic, or the locale dictionary structure (other than adding the listed keys).
- Frontend `vue-i18n` translation work or schema changes to `locales/{en,zh}.json`.
- Adding test infrastructure, the `run_audit.sh` script, or any new dev dependency.
## Boundary Commitments
### This Spec Owns
- The string-literal contents of nine specific `logger.{info,error}` and `print(...)` call sites (exact `file:line` listed in Requirement 1).
- Eleven new translation entries in `locales/en.json` and `locales/zh.json`.
- The new `log.retry` sub-namespace under the existing top-level `log` key.
### Out of Boundary
- Other Chinese strings in the three modified files.
- Any change to public API contracts, log levels, or response payloads.
- Any change to the `t()` helper or the per-thread / per-request locale resolution logic.
- Frontend `zh.json` entries beyond the ones this spec must add for backend parity (i.e., none — frontend keys are untouched).
### Allowed Dependencies
- `backend/app/utils/locale.py` (`t`) — already in use, just import it where needed.
- The existing locale dictionaries `locales/{en,zh}.json` — extend, don't re-organise.
- `get_logger` from `backend/app/utils/logger.py` — already imported by `retry.py`.
### Revalidation Triggers
- Renaming `t()` or moving it to a different module.
- Changing the placeholder syntax in `t()` from `{name}` to anything else.
- Restructuring `locales/en.json` / `zh.json` (e.g., flattening `log.<domain>.m###` into a flat key tree).
## Architecture
### Existing Architecture Analysis
This spec extends a pattern already established by ticket #6 (`i18n-externalize-backend-logs`). The convention is:
1. Source-code call sites use `t("log.<domain>.m###", placeholder=value, …)` instead of `f"…{value}…"`.
2. Each `t()` key has matching entries in `locales/en.json` (English copy) and `locales/zh.json` (verbatim original Chinese).
3. Placeholders use `{name}` (replaced via `str.replace` inside `t()`).
4. The locale is resolved per request (`Accept-Language`) or per thread (`set_locale`); `'zh'` is the default fallback; missing keys return the key string and emit a deduped warning.
The constraint: only the nine listed call sites change. No new architecture, no new component, no new integration point.
### Architecture Pattern & Boundary Map
The change is a **pure string-externalisation extension** of the existing localisation pattern. No new components, no new flows, no new dependencies. The only structural addition is a new `log.retry` sub-namespace inside the existing top-level `log` key in the locale dictionaries.
```mermaid
flowchart LR
A[graph.py:385/494/513<br/>build_logger.{info,error}] -->|t("log.graph_api.mNNN", ...)| L[t() helper<br/>backend/app/utils/locale.py]
B[oasis_profile_generator.py:945/1001<br/>print(...)] -->|t("log.profile_generator.mNNN", ...)| L
C[retry.py:55/108/179/227<br/>logger.error] -->|t("log.retry.mNNN", ...)| L
L --> EN[locales/en.json<br/>log.graph_api.m027-m029<br/>log.profile_generator.m024-m025<br/>log.retry.m001-m004]
L --> ZH[locales/zh.json<br/>same key paths<br/>verbatim Chinese values]
```
### Technology Stack
| Layer | Choice / Version | Role in Feature | Notes |
|-------|------------------|-----------------|-------|
| Backend / Services | Python ≥3.11 | Source-language change site | No version change |
| Backend / Services | `backend/app/utils/locale.py` (project-internal) | Provides `t(key, **kwargs)` | Reused as-is |
| Data / Storage | `locales/en.json`, `locales/zh.json` | Adds 11 new key/value pairs | Flat JSON, UTF-8 |
| Infrastructure / Runtime | Flask 3.0 / asyncio | Locale resolution context | No runtime change |
## File Structure Plan
### Modified Files
- `backend/app/api/graph.py` — Replace the f-string argument of three `build_logger.{info,error}` calls (lines 385, 494, 513) with `t("log.graph_api.<key>", **kwargs)`. No new imports (already imports `t` on line 21).
- `backend/app/services/oasis_profile_generator.py` — Replace the f-string argument of two `print(...)` calls (lines 945, 1001) with `t("log.profile_generator.<key>", **kwargs)`. No new imports (already imports `t` on line 23).
- `backend/app/utils/retry.py` — Add `from .locale import t` (or `from ..utils.locale import t`, matching the project's existing relative-import style). Replace the f-string argument of four `logger.error` calls (lines 55, 108, 179, 227) with `t("log.retry.<key>", **kwargs)`.
- `locales/en.json` — Append three keys to `log.graph_api`, two to `log.profile_generator`, and a new `log.retry` sub-namespace with four keys.
- `locales/zh.json` — Mirror the same key paths with verbatim original Chinese strings.
No new files. No deleted files.
## Requirements Traceability
| Requirement | Summary | Components | Interfaces | Flows |
|-------------|---------|------------|------------|-------|
| 1.1 | Replace `graph.py` log strings via `t()` | `graph.py` build-task closure | `t("log.graph_api.<key>", ...)` | Build pipeline log emission |
| 1.2 | Replace `oasis_profile_generator.py` banner prints via `t()` | `OasisProfileGenerator.generate_profiles_parallel` | `t("log.profile_generator.<key>", ...)` | Profile-generation banner |
| 1.3 | Replace `retry.py` errors via `t()` (new `log.retry` namespace) | `retry_with_backoff`, `retry_with_backoff_async`, `RetryableAPIClient` | `t("log.retry.<key>", ...)` | Retry-failure path |
| 1.4 | Preserve interpolated values via kwargs | All three modules | `t(key, name=value, ...)` with `{name}` placeholders | All log emission |
| 1.5 | Zero CJK in the listed lines after change | Same as 1.1–1.3 | n/a | n/a |
| 2.1, 2.2 | Add 11 new keys to `en.json` and `zh.json` | Locale dictionaries | JSON file edits | n/a |
| 2.3 | Use next available `m###` slot per namespace | Locale dictionaries | n/a | n/a |
| 2.4 | Structural parity across both files | Locale dictionaries | Verification script | n/a |
| 2.5 | No new top-level keys; no existing keys touched | Locale dictionaries | n/a | n/a |
| 3.1 | Graph build pipeline behaves identically | `graph.py` build-task closure | n/a | Build pipeline |
| 3.2 | Profile generator continues to print exactly two banners | `oasis_profile_generator.py` | n/a | Banner emission |
| 3.3 | Retry semantics unchanged (raise, sleep, level, position) | `retry.py` | n/a | Retry path |
| 3.4 | HTTP responses unchanged | All API endpoints | n/a | n/a |
| 4.1, 4.2, 4.3, 4.4 | Locale resolution works in all contexts | `t()` helper (unchanged) | n/a | n/a |
| 5.1 | CJK regex audit on the nine lines passes | Verification procedure | `grep -P "[一-鿿]"` | n/a |
| 5.2 | Key-parity audit passes | Verification procedure | Python `json.load` walk | n/a |
| 5.3 | Placeholder-integrity audit passes | Verification procedure | Python regex check | n/a |
| 5.4 | Only stock tooling | Verification procedure | `grep`, `python3` | n/a |
| 5.5 | `pytest` continues to pass | Backend test suite | `uv run python -m pytest` | n/a |
## Components and Interfaces
| Component | Domain/Layer | Intent | Req Coverage | Key Dependencies (P0/P1) | Contracts |
|-----------|--------------|--------|--------------|--------------------------|-----------|
| `graph.py` build-task closure | Backend / API | Log graph-build start/complete/fail in active locale | 1.1, 1.4, 1.5, 3.1 | `t()` (P0), `build_logger` (P0) | Behaviour-only |
| OASIS banner prints | Backend / Services | Print banner around parallel profile generation | 1.2, 1.4, 1.5, 3.2 | `t()` (P0) | Console-output |
| Retry error logs | Backend / Utils | Log final-failure errors after retry exhaustion | 1.3, 1.4, 1.5, 3.3 | `t()` (P0), `logger` (P0) | Behaviour-only |
| Locale dictionaries | Backend / Data | Provide en/zh strings for new keys | 2.1–2.5 | JSON parse (P0) | Data |
### Backend / Services
#### `graph.py` build-task closure
| Field | Detail |
|-------|--------|
| Intent | Emit "build started", "build completed", "build failed" log records using `t()` |
| Requirements | 1.1, 1.4, 1.5, 3.1 |
**Responsibilities & Constraints**
- Replace three f-string log arguments only.
- Do not change log level, log handler, control flow, or surrounding `task_manager.update_task(...)` calls.
**Dependencies**
- Inbound: called from `task_manager.run_task` (P0)
- Outbound: `t()` (P0), `build_logger.{info,error}` (P0)
**Contracts**: Service [ ] / API [ ] / Event [ ] / Batch [ ] / State [ ] ← (none — purely behavioural)
**Key Mapping**
| Line | Existing source | New key | EN translation | ZH translation |
|------|-----------------|---------|----------------|----------------|
| 385 | `f"[{task_id}] 开始构建图谱..."` | `log.graph_api.m027` | `[{task_id}] Starting graph build...` | `[{task_id}] 开始构建图谱...` |
| 494 | `f"[{task_id}] 图谱构建完成: graph_id={graph_id}, 节点={node_count}, 边={edge_count}"` | `log.graph_api.m028` | `[{task_id}] Graph build completed: graph_id={graph_id}, nodes={node_count}, edges={edge_count}` | `[{task_id}] 图谱构建完成: graph_id={graph_id}, 节点={node_count}, 边={edge_count}` |
| 513 | `f"[{task_id}] 图谱构建失败: {str(e)}"` | `log.graph_api.m029` | `[{task_id}] Graph build failed: {e}` | `[{task_id}] 图谱构建失败: {e}` |
**Implementation Notes**
- `t` is already imported at `graph.py:21`.
- Use `e=str(e)` to maintain the existing exception-string semantics.
#### OASIS banner prints (`oasis_profile_generator.py`)
| Field | Detail |
|-------|--------|
| Intent | Wrap the two banner-print arguments in `t()` while leaving the surrounding `'='*60` separator prints intact |
| Requirements | 1.2, 1.4, 1.5, 3.2 |
**Responsibilities & Constraints**
- Replace only the *content* line of each banner (the line at 945 and the line at 1001). The two `'='*60` separator prints around them (lines 944/946 and 1000/1002) contain only ASCII and stay verbatim.
- Do not remove either `print(...)` call.
- Do not modify the existing `logger.info(t("log.profile_generator.m017", …))` at line 943.
**Key Mapping**
| Line | Existing source | New key | EN translation | ZH translation |
|------|-----------------|---------|----------------|----------------|
| 945 | `f"开始生成Agent人设 - 共 {total} 个实体，并行数: {parallel_count}"` | `log.profile_generator.m024` | `Starting agent profile generation — {total} entities, parallelism: {parallel_count}` | `开始生成Agent人设 - 共 {total} 个实体，并行数: {parallel_count}` |
| 1001 | `f"人设生成完成！共生成 {len([p for p in profiles if p])} 个Agent"` | `log.profile_generator.m025` | `Profile generation complete — generated {count} agents` | `人设生成完成！共生成 {count} 个Agent` |
**Implementation Notes**
- The expression `len([p for p in profiles if p])` becomes a kwarg: `count=len([p for p in profiles if p])`. This is a single name, easier for the locale dictionaries.
- `t` is already imported at `oasis_profile_generator.py:23`.
#### Retry error logs (`retry.py`)
| Field | Detail |
|-------|--------|
| Intent | Localise the four "final-failure" `logger.error` strings; introduce `log.retry` sub-namespace |
| Requirements | 1.3, 1.4, 1.5, 3.3, 4.1–4.4 |
**Responsibilities & Constraints**
- Add `from ..utils.locale import t` at the top of `retry.py` (matching the relative-import depth used by other `backend/app/utils/*` files).
- Replace four f-string `logger.error(...)` arguments only.
- Do not touch the `logger.warning(...)` retry-attempt messages (out of scope per ticket #24).
- Do not change exception handling, control flow, or the public decorator/class signatures.
**Key Mapping**
| Line | Existing source | New key | EN translation | ZH translation |
|------|-----------------|---------|----------------|----------------|
| 55 | `f"函数 {func.__name__} 在 {max_retries} 次重试后仍失败: {str(e)}"` | `log.retry.m001` | `Function {func_name} still failing after {max_retries} retries: {e}` | `函数 {func_name} 在 {max_retries} 次重试后仍失败: {e}` |
| 108 | `f"异步函数 {func.__name__} 在 {max_retries} 次重试后仍失败: {str(e)}"` | `log.retry.m002` | `Async function {func_name} still failing after {max_retries} retries: {e}` | `异步函数 {func_name} 在 {max_retries} 次重试后仍失败: {e}` |
| 179 | `f"API调用在 {self.max_retries} 次重试后仍失败: {str(e)}"` | `log.retry.m003` | `API call still failing after {max_retries} retries: {e}` | `API调用在 {max_retries} 次重试后仍失败: {e}` |
| 227 | `f"处理第 {idx + 1} 项失败: {str(e)}"` | `log.retry.m004` | `Failed processing item #{index}: {e}` | `处理第 {index} 项失败: {e}` |
**Implementation Notes**
- Use kwargs `func_name=func.__name__`, `max_retries=max_retries` (or `self.max_retries`), `index=idx + 1`, `e=str(e)`.
- Locale resolution at the call site: in Flask request scope → `Accept-Language`; in background tasks → `set_locale` per-thread; in async coroutines → per-thread (asyncio shares the OS thread). Default fallback is `'zh'`. No new wiring needed (Requirement 4).
### Backend / Data
#### Locale dictionaries
| Field | Detail |
|-------|--------|
| Intent | Provide en/zh strings for the eleven new keys with structural parity |
| Requirements | 2.1, 2.2, 2.3, 2.4, 2.5 |
**Responsibilities & Constraints**
- Append to existing `log.graph_api` and `log.profile_generator` sub-namespaces.
- Add a new `log.retry` sub-namespace as a sibling of the others.
- No top-level key additions; no modifications to any pre-existing key.
- Maintain UTF-8 encoding and the file's existing 2-space indent style.
**Implementation Notes**
- Use `python3 -m json.tool` (or equivalent) to round-trip the JSON files after editing, to ensure formatting consistency.
- Validate parity with a small Python script that recursively compares key paths.
## System Flows
(Skipped — no non-trivial flow change. The build / profile / retry call paths execute as before; only the message text source language differs.)
## Error Handling
### Error Strategy
This spec changes only message-string sources. Error-handling semantics in the touched code are preserved:
- `graph.py:513` continues to set `project.status = ProjectStatus.FAILED` and call `task_manager.update_task(..., status=TaskStatus.FAILED, ...)` after the `build_logger.error(...)` call.
- `retry.py` continues to `raise` the underlying exception after the final `logger.error(...)`.
- The `t()` helper does not raise on missing keys — it returns the key string and emits a deduped warning. This contract is unchanged.
### Error Categories and Responses
Out of scope — no new error category is introduced.
## Testing Strategy
### Unit / Integration Tests
The project does not currently maintain a comprehensive backend unit-test suite for these modules. The change is verified mechanically rather than via new pytest tests:
1. **CJK absence on the touched lines** — `grep -nP "[一-鿿]"` against the nine specific lines must return no matches.
2. **JSON parse + key parity** — a small inline Python check that loads `locales/{en,zh}.json` and asserts every newly-added key path exists in both files.
3. **Placeholder integrity** — for each new key, every `{name}` placeholder in the `zh` value must also appear in the `en` value (and vice versa).
4. **Existing test suite** — `uv run python -m pytest` continues to pass; ticket #6's tests at `backend/scripts/test_profile_format.py` are not affected by this work.
### Manual Smoke Test
After implementation:
- Set `Accept-Language: en` and run an end-to-end graph build via the local Flask app (`npm run dev`); confirm the start / complete / fail log lines render in English.
- Run a profile generation flow and observe the banner prints in English.
- Force a retry exhaustion (e.g., temporarily lower `max_retries=0` and trigger an error) and confirm the `log.retry` message renders in English.
(Manual smoke is documentation-only; not a blocker for merging.)
## Optional Sections
### Security Considerations
None. No auth, no PII, no external integration changes. The exception text in log messages was already exposed via the previous f-string formatting; routing it through `t()` does not change the surface.
### Performance & Scalability
Negligible. `t()` is an in-memory dict lookup with `str.replace` for placeholders; cost is below noise floor for log emission.

124
.kiro/specs/i18n-externalize-remaining-backend-logs/gap-analysis.md Normal file
View File

@ -0,0 +1,124 @@
# Implementation Gap Analysis
## 1. Codebase Findings
### 1.1 Existing infrastructure already covers the i18n mechanics
- `backend/app/utils/locale.py` already exports `t(key, **kwargs)` with:
- per-thread locale (`set_locale` writes `_thread_local.locale`)
- per-request locale (`get_locale` checks Flask `has_request_context()` then `Accept-Language`)
- `zh` fallback when the active locale is missing a key, then key-string fallback if `zh` is missing too
- dedup'd warning on missing keys (`_warn_missing_key_once`), no exceptions raised
- All wiring required by Requirement 4 is therefore already in place. **No `locale.py` change is needed for ticket #24.**
### 1.2 The two files we touch already use `t()`
- `backend/app/api/graph.py:21` — `from ..utils.locale import t`
- `backend/app/services/oasis_profile_generator.py:23` — `from ..utils.locale import get_language_instruction, get_locale, set_locale, t`
The third file does NOT yet import `t`:
- `backend/app/utils/retry.py` — no `from ..utils.locale import t`. Need to add the import.
### 1.3 Existing locale namespace shape (from `locales/en.json`)
- `log.graph_api` — populated `m006`–`m019, m026`. Next free slots that are *contiguous* would be `m027`, `m028`, `m029`. (Could also reuse `m009, m010, m012, m020–m025` since they are absent, but it is safer to append at the tail to avoid colliding with any unmerged work assuming a particular reservation.)
- `log.profile_generator` — populated `m001`–`m023` densely. Next free: `m024`, `m025`.
- `log.retry` — does NOT exist. Will be created with `m001`–`m004`.
The `log.profile_generator.m017` key already covers a *similar* message ("Starting parallel generation of {total} agent profiles (parallelism: {parallel_count})…"). The `print(...)` at `oasis_profile_generator.py:945` and the `logger.info(t("log.profile_generator.m017", ...))` at line 943 are emitting the same logical event in two channels — log + console banner. The cleanest move is **not** to reuse `m017` (which would lose the banner-style separator/centring) but to introduce dedicated `m024` / `m025` keys for the banner text, so the banner has its own copy decoupled from the log line.
### 1.4 Translation pattern already established by ticket #6
Per the prior spec at `.kiro/specs/i18n-externalize-backend-logs/`, the project's convention is:
- `t("log.<domain>.m###", placeholder=value, …)` inside `logger.{info,warning,error,debug,exception}` calls.
- Placeholders use `{name}` syntax (replaced via `str.replace` inside `t()`); positional `{0}`/`{}` are not supported.
- f-string formatting must be removed entirely from the call argument; values are passed as kwargs.
- The Chinese source string is preserved verbatim in `zh.json`, with `f"…{var}…"` rewritten as `"…{var}…"`.
This work strictly extends the existing pattern. **No new convention is introduced.**
### 1.5 `build_logger` vs. module logger
In `graph.py`, the affected calls use a locally-created `build_logger = get_logger('mirofish.build')` inside the `build_task` background function (lines 383). This is a different logger handle, but `t()` is logger-agnostic — it returns a string that any logger can format. No special handling needed.
### 1.6 `print(...)` calls in `oasis_profile_generator.py`
The two banner prints (lines 945 and 1001) are deliberate console-output decorations (visible on stdout for the Flask process), separate from the structured log emitted by `logger.info` on lines 943 and earlier. The task is to keep them as `print(...)` but route the message text through `t(...)`:
```python
print(t("log.profile_generator.m024", total=total, parallel_count=parallel_count))
```
This preserves the user-visible banner cosmetics (`'='*60` separators on lines 944, 946, 1000, 1002) and only changes the text content.
### 1.7 Locale resolution for `retry.py`
`retry.py` is invoked from three contexts:
1. **Flask request handlers (sync)** — `has_request_context()` is true; `get_locale()` reads `Accept-Language`. Works.
2. **Background tasks** — the existing background-task entry points (e.g., `task_manager.run_task`) already call `set_locale(...)` per `i18n-externalize-backend-logs` (verified by reading `oasis_profile_generator.py` which uses the same pattern with `set_locale` imported on line 23). Works.
3. **Async coroutines (`retry_with_backoff_async`)** — `get_locale()` falls back to `_thread_local.locale`. Asyncio runs coroutines on the same thread by default, so the per-thread locale propagates. If the coroutine is dispatched onto a fresh executor thread without `set_locale`, the helper falls back to `zh` (the default) — still a valid string, just defaulting to Chinese. The default-fallback is acceptable here because (a) the helper still returns a non-None string, and (b) the audit only requires the *source code* to be free of Chinese literals, not that every emitted log record be English regardless of caller context.
**Decision:** No new locale-propagation wiring needed. Document the async fallback in the design and tasks.
## 2. Out-of-scope items (encountered during research)
These were observed in the same files but are explicitly **not** part of ticket #24 and will not be addressed:
- `backend/app/api/graph.py` — Chinese in `task_manager.update_task(..., message="初始化图谱构建服务...")` and similar (#24 lists only the three log calls).
- `backend/app/utils/retry.py` — Chinese in `logger.warning(...)` retry messages (lines 63–66, 115–117, 185–187) and Chinese docstrings (lines 1–3, 25–35, 36–39, 90, 156–166, 200–212).
- `backend/app/services/oasis_profile_generator.py` — Chinese in `progress_callback(... f"已完成 …")` (line 976) and Chinese docstrings/comments throughout.
These are tracked under sibling tickets (#7 for docstrings/comments; the residual `logger.warning` in `retry.py` is a candidate for a future audit ticket).
## 3. Implementation Approaches Considered
### Approach A — Append-at-tail with new `log.retry` namespace (recommended)
- New keys: `log.graph_api.m027`, `m028`, `m029`; `log.profile_generator.m024`, `m025`; new `log.retry.m001`–`m004`.
- Add `from ..utils.locale import t` to `retry.py`.
- Replace each f-string in the nine call sites with a `t(...)` call.
- Update `locales/en.json` and `locales/zh.json` in lock-step.
- **Pros:** Mirrors the conventions of #6 exactly; no risk of overwriting existing keys; minimal diff.
- **Cons:** Numbering gaps under `log.graph_api` remain (cosmetic).
### Approach B — Fill numbering gaps in `log.graph_api`
- Reuse missing slots `m009`, `m010`, `m012`, `m020`–`m025`.
- **Pros:** Tighter numbering.
- **Cons:** Risk of colliding with reserved-but-not-yet-merged keys from another branch; harder to review (mixed insertion sites in JSON).
- **Verdict:** Reject. The cost of conflict review is not worth the cosmetic gain.
### Approach C — Consolidate the `print(...)` banners into the existing `log.profile_generator.m017`
- Remove the two `print(...)` calls; rely solely on `logger.info(t(...))`.
- **Pros:** One fewer key to add.
- **Cons:** Deletes user-visible console banner behaviour (a behaviour change), violates Requirement 3.2 ("continue to print exactly two banner messages"), and is out-of-scope per ticket #24 which says "fixed (or explicitly classified as `deliberate`)" — i.e., translate, don't remove.
- **Verdict:** Reject.
## 4. Recommendation
Proceed with **Approach A**.
Implementation will:
1. Add four entries to `log.retry` (new sub-namespace) — one per `logger.error` line in `retry.py`.
2. Add three entries to `log.graph_api` — one per `build_logger` line in `graph.py`.
3. Add two entries to `log.profile_generator` — one per `print(...)` banner in `oasis_profile_generator.py`.
4. Replace all nine f-strings with `t(...)` calls; pass interpolated values as kwargs.
5. Add `from ..utils.locale import t` to `retry.py`.
6. Mirror every new key in `zh.json` with the verbatim original Chinese text.
7. Run a regex / Python audit to confirm parity and absence of CJK on the touched lines.
## 5. Risks / open questions
| Risk | Severity | Mitigation |
|---|---|---|
| `retry.py` async path running on a fresh thread without `set_locale` returns Chinese | Low | Documented; not a blocker for #24 acceptance, which targets *source-code* CJK absence. Any improvement is a separate ticket. |
| Adding `from ..utils.locale import t` introduces a new module import into `retry.py` (low-level utility) | Low | The `locale` module has no transitive imports of `retry.py`, so no circular-import risk. Verified by reading `locale.py`. |
| Existing test that asserts Chinese log text breaks | Low | Searched for `"开始构建图谱"` / `"图谱构建完成"` / `"图谱构建失败"` / `"开始生成Agent人设"` / `"人设生成完成"` / `"重试后仍失败"` / `"处理第"` test fixtures — none found in `backend/`. |
## 6. Conclusion
**Ready to proceed to design.** The gap is small: nine string-literal replacements, eleven new locale entries, one new import. The mechanics are identical to the already-merged ticket #6 work. No design uncertainty remains; design phase will simply formalise the key-naming and the per-file edit plan.

91
.kiro/specs/i18n-externalize-remaining-backend-logs/requirements.md Normal file
View File

@ -0,0 +1,91 @@
# Requirements Document
## Introduction
After ticket #6 externalised most backend log/print messages into the project's `t()` localization helper, a small set of call sites in three modules still emit hard-coded Chinese strings. As a result, English operators reading backend logs under the `en` locale see Chinese text leaking from these residual sites. This spec finishes the job for ticket #24 by routing every remaining hard-coded Chinese log/print string in `backend/app/api/graph.py`, `backend/app/services/oasis_profile_generator.py`, and `backend/app/utils/retry.py` through `t("log.<domain>.<key>", **fmt)` and adding the corresponding entries to `locales/en.json` and `locales/zh.json`. The goal is locale-correct backend logs with zero behavioural drift in HTTP responses, control flow, or interpolated values.
## Boundary Context
- **In scope**:
- Replace the Chinese string literals in the nine call sites listed by ticket #24:
- `backend/app/api/graph.py:385` — `build_logger.info(f"[{task_id}] 开始构建图谱...")`
- `backend/app/api/graph.py:494` — `build_logger.info(f"[{task_id}] 图谱构建完成: graph_id={graph_id}, 节点={node_count}, 边={edge_count}")`
- `backend/app/api/graph.py:513` — `build_logger.error(f"[{task_id}] 图谱构建失败: {str(e)}")`
- `backend/app/services/oasis_profile_generator.py:945` — `print(f"开始生成Agent人设 - 共 {total} 个实体，并行数: {parallel_count}")`
- `backend/app/services/oasis_profile_generator.py:1001` — `print(f"人设生成完成！共生成 {len([p for p in profiles if p])} 个Agent")`
- `backend/app/utils/retry.py:55` — `logger.error(f"函数 {func.__name__} 在 {max_retries} 次重试后仍失败: {str(e)}")`
- `backend/app/utils/retry.py:108` — `logger.error(f"异步函数 {func.__name__} 在 {max_retries} 次重试后仍失败: {str(e)}")`
- `backend/app/utils/retry.py:179` — `logger.error(f"API调用在 {self.max_retries} 次重试后仍失败: {str(e)}")`
- `backend/app/utils/retry.py:227` — `logger.error(f"处理第 {idx + 1} 项失败: {str(e)}")`
- Add new locale keys for the externalised strings to both `locales/en.json` (English) and `locales/zh.json` (verbatim original Chinese) under the existing top-level `log.<domain>` namespaces (`log.graph_api`, `log.profile_generator`, and a new `log.retry`).
- Pass interpolated values (`task_id`, `graph_id`, `node_count`, `edge_count`, `total`, `parallel_count`, `func_name`, `max_retries`, `idx`, exception text, etc.) through `t()` keyword arguments using the helper's `{name}` placeholder syntax.
- **Out of scope**:
- Other Chinese strings in the same files that are not on the ticket's evidence list (Chinese docstrings, Chinese inline comments, the `task_manager.update_task(... message="...")` Chinese values in `graph.py`, the `logger.warning("…重试…")` calls in `retry.py`, and the in-loop `progress_callback(... f"已完成 …")` and `print(f"-" * 70 …)` decorations in `oasis_profile_generator.py`). Those are tracked elsewhere (#7 for docstrings/comments; #25 for prompt/context labels; future audit may pick up the remaining warning-level retry strings under a separate ticket).
- Any change to log levels, response status codes, control flow, public API surface, or to the `t()` helper itself.
- Adding a new locale or changing the per-thread / per-request locale resolution.
- Frontend `vue-i18n` files; this spec touches only backend usage of `t()` and the shared `locales/{en,zh}.json`.
- **Adjacent expectations**:
- The `t()` helper at `backend/app/utils/locale.py` already covers `set_locale`, `get_locale`, missing-key fallback, and per-thread locale (verified by ticket #6). New code reuses it without modification.
- The two top-level `log` sub-namespaces `log.graph_api` and `log.profile_generator` already exist in `locales/en.json` / `locales/zh.json` with `m###` numeric suffixes; new keys must use the next available `m###` slot in each existing namespace and must not collide with or overwrite any existing key.
- `retry.py` is module-level shared infrastructure used from request handlers, background tasks, and async coroutines — locale resolution must continue to work in each of these contexts without new wiring (Requirement 4 below documents this explicitly so behaviour is mechanically verified).
- Ticket #24's acceptance criterion mentions a verification script under `.kiro/specs/i18n-e2e-english-verification/audit/scripts/run_audit.sh`. That script is not present in the repository at this commit; this spec substitutes a deterministic regex audit (see Requirement 5) that is runnable from the repo root with `grep` + `python` only and that any future `run_audit.sh` can incorporate.
## Requirements
### Requirement 1: Externalise Remaining Chinese Log/Print Strings via `t()`
**Objective:** As a backend operator viewing logs under the `en` locale, I want every Chinese log/print string in the nine listed call sites to be emitted via the existing `t()` helper, so that backend logs no longer leak Chinese text in English deployments.
#### Acceptance Criteria
1. The Backend Logging Layer shall replace the f-string argument of each of the three `build_logger.{info,error}` calls in `backend/app/api/graph.py` at lines 385, 494, and 513 with `t("log.graph_api.<key>", task_id=task_id, ...)`, where the key is a new entry under the existing `log.graph_api` namespace.
2. The Backend Logging Layer shall replace the f-string argument of each of the two `print(...)` calls in `backend/app/services/oasis_profile_generator.py` at lines 945 and 1001 with `print(t("log.profile_generator.<key>", ...))`, keeping the `print` call (so console-output behaviour is preserved) but routing the message text through `t()` under the existing `log.profile_generator` namespace.
3. The Backend Logging Layer shall replace the f-string argument of each of the four `logger.error` calls in `backend/app/utils/retry.py` at lines 55, 108, 179, and 227 with `t("log.retry.<key>", **kwargs)`, introducing a new top-level sub-namespace `log.retry` that mirrors the structure of the other `log.<domain>` sub-namespaces.
4. The Backend Logging Layer shall preserve every interpolated value (`task_id`, `graph_id`, `node_count`, `edge_count`, `total`, `parallel_count`, `func.__name__`, `max_retries`, `idx`, exception text) by passing them as keyword arguments to `t(...)` and referencing them via `{name}` placeholders inside the locale dictionaries; no `f"..."` formatting, `%`-formatting, or string concatenation shall remain around the call.
5. The Backend Logging Layer shall not contain any Chinese character (Unicode range `U+4E00`–`U+9FFF`) inside the string-literal argument of any `logger.{info,warning,error,debug,exception}`, `build_logger.{info,warning,error,debug,exception}`, or `print(...)` call at the nine listed line locations after the change.
### Requirement 2: Locale Dictionary Parity for the New Keys
**Objective:** As a translator or developer adding a new locale, I want every newly externalised key to exist in both `locales/en.json` and `locales/zh.json` with identical nested structure, so that the locale files remain mechanically diffable.
#### Acceptance Criteria
1. The Locale Dictionary shall add, in `locales/en.json`, an English translation for every key introduced by Requirement 1, placed under the relevant `log.<domain>` sub-namespace (`log.graph_api`, `log.profile_generator`, or the new `log.retry`).
2. The Locale Dictionary shall add, in `locales/zh.json`, the original Chinese text (verbatim, with `{placeholder}` substitutions where the source had `f"…{var}…"`) for every key introduced by Requirement 1, under the same key path used in `en.json`.
3. The Locale Dictionary shall use the next available `m###` numeric suffix per existing sub-namespace (so it does not overwrite or shadow any pre-existing `log.graph_api.m###` or `log.profile_generator.m###` key); the new `log.retry` sub-namespace shall start its keys at `m001`.
4. The Locale Dictionary shall expose a structurally identical key tree across `locales/en.json` and `locales/zh.json` for every newly added key path: a recursive comparison of the two files' key paths (ignoring values) shall produce an empty difference for the keys this spec introduces.
5. The Locale Dictionary shall not introduce a new top-level key (the only addition is the new `log.retry` sub-key under the existing top-level `log` namespace) and shall not modify, remove, or re-order any existing key already present in `locales/{en,zh}.json`.
### Requirement 3: Behavioural and Functional Equivalence
**Objective:** As a reviewer, I want to confirm that swapping the message strings does not change runtime behaviour, so that this PR is purely a localisation change.
#### Acceptance Criteria
1. The Graph Build Pipeline shall, after the change, continue to: update `project.status` to `GRAPH_BUILDING` then `GRAPH_COMPLETED` (or `FAILED` on error), call `task_manager.update_task(...)` with the same status/progress/result payloads, and emit one log record at each of the three pre-existing log points (start, completion, failure) with identical level (`info`/`info`/`error`) and identical interpolated values; only the human-readable text and its language source shall differ.
2. The Profile Generator shall, after the change, continue to print exactly two banner messages around `concurrent.futures.ThreadPoolExecutor`-driven generation (one before, one after), retain the surrounding `'='*60` separator lines verbatim, and not emit additional log records or alter the order of `logger.info`/`logger.warning` calls.
3. The Retry Utility shall, after the change, continue to: raise the original exception after the final retry, sleep for the same backoff durations, and emit exactly one `logger.error` per call site at the same control-flow position; the helper's signature, decorator behaviour, and async/sync split shall be unchanged.
4. The Backend HTTP Layer shall return the same HTTP status code, response key set, and (for non-translated keys) value structure for `/api/graph/build` and any other endpoint that transitively triggers the touched code paths; no `jsonify(...)` payload shape shall change as a side-effect of this work.
### Requirement 4: Locale Resolution in Background and Async Contexts
**Objective:** As a backend service author, I want the new `t()` calls to resolve to the correct locale even when invoked from background threads or async coroutines, so that operators see consistent log language regardless of where the call originates.
#### Acceptance Criteria
1. When `t("log.graph_api.<key>", ...)` is called from the `build_task` background thread inside `backend/app/api/graph.py` (started via `task_manager.run_task`), the Locale Helper shall resolve to the locale that was established for that thread (per the existing per-thread / `set_locale` mechanism), not silently fall back to the default `zh`.
2. When `t("log.retry.<key>", ...)` is called from the synchronous `retry_with_backoff` decorator wrapping a Flask request handler, the Locale Helper shall resolve via the active Flask request context (`Accept-Language` header), consistent with how request-scoped `t()` calls behave elsewhere in the codebase.
3. When `t("log.retry.<key>", ...)` is called from the asynchronous `retry_with_backoff_async` decorator under `asyncio`, the Locale Helper shall resolve via whichever locale source is in scope for that coroutine (request context if present; otherwise the per-thread fallback set by the caller), without raising and without requiring any new locale-propagation wiring inside `retry.py`.
4. If a `t()` call introduced by this spec references a key that is missing from both the active locale and the `zh` fallback, the Locale Helper shall continue to behave per the existing contract: emit a single deduped warning naming the key and locale, and return the key string itself (never `None`, never raise).
### Requirement 5: Verification and Regression Guards
**Objective:** As a reviewer of this PR, I want repeatable mechanical checks that prove the in-scope files are clean of stray hard-coded Chinese log/print strings on those nine lines, so that the acceptance criteria can be re-validated on every future change.
#### Acceptance Criteria
1. The Verification Procedure shall, when run against the repository, report zero matches of any Unicode CJK character (range `U+4E00`–`U+9FFF`) on the nine specific lines covered by Requirement 1 in their post-change form (i.e., `grep -P "[一-鿿]"` against the replaced lines returns no hits).
2. The Verification Procedure shall, when run against `locales/en.json` and `locales/zh.json`, confirm via a Python `json.load` + recursive key walk that every newly introduced key path exists in both files, and exit non-zero if a key path is present in only one of them.
3. The Verification Procedure shall confirm via Python that for each new key in `locales/zh.json` whose source f-string contained an `{var}` placeholder, the same `{var}` placeholder appears in the new English translation in `locales/en.json` (so interpolation is not silently dropped during translation).
4. The Verification Procedure shall require only tools already available in the dev environment (`grep`, `python3`, optional `jq`) — no new runtime or dev dependencies shall be added by this spec.
5. The Backend Test Suite shall continue to pass (`uv run python -m pytest`) after the change, with no new failures introduced; in particular, any pre-existing tests that assert the prior Chinese log/print text shall be updated to assert via the same `t()` lookup or an English translation rather than removed.

91
.kiro/specs/i18n-externalize-remaining-backend-logs/research.md Normal file
View File

@ -0,0 +1,91 @@
# Research & Design Decisions
## Summary
- **Feature**: `i18n-externalize-remaining-backend-logs`
- **Discovery Scope**: Simple Addition (extending an established convention from ticket #6)
- **Key Findings**:
- The `t()` helper, per-thread locale, and missing-key fallback are already in place in `backend/app/utils/locale.py` and require no changes.
- The convention `t("log.<domain>.m###", **kwargs)` with `{name}` placeholders is already used by all sibling modules; this spec strictly extends it.
- No existing test fixtures reference any of the nine Chinese strings to be replaced.
## Research Log
### Existing locale namespace structure
- **Context**: Need to add new keys without colliding with existing entries.
- **Sources Consulted**: `locales/en.json`, `locales/zh.json`, `.kiro/specs/i18n-externalize-backend-logs/requirements.md`.
- **Findings**:
- `log.graph_api` is densely populated `m006`–`m019` plus `m026`. Free contiguous slots starting at the tail: `m027`, `m028`, `m029`.
- `log.profile_generator` is densely populated `m001`–`m023`. Free slots: `m024`, `m025`.
- `log.retry` does not exist; introducing it as a sibling to other `log.<domain>` namespaces matches the existing pattern.
- **Implications**: New keys append at the tail per existing namespace; `log.retry` is created fresh starting at `m001`.
### Locale resolution in async / background contexts
- **Context**: `retry.py` is shared infrastructure invoked from sync request handlers, background tasks, and async coroutines.
- **Sources Consulted**: `backend/app/utils/locale.py`, `backend/app/services/oasis_profile_generator.py` (uses `set_locale`), Flask docs (request-context behaviour).
- **Findings**:
- `get_locale()` returns the request-context `Accept-Language` header when a Flask request is active, the per-thread locale otherwise, and `'zh'` as the default.
- Asyncio coroutines run on the same OS thread by default, so the per-thread locale set by the parent function propagates into `await`-driven calls.
- Missing-key fallback returns the key string and emits a deduped warning — never raises.
- **Implications**: No new locale-propagation wiring needed inside `retry.py`. Adding `from ..utils.locale import t` is sufficient.
### `print(...)` vs `logger` for the OASIS banners
- **Context**: Two `print(...)` banner statements at `oasis_profile_generator.py:945` and `:1001` decorate stdout. Should we keep them as `print` or fold them into existing `logger.info` calls?
- **Sources Consulted**: `backend/app/services/oasis_profile_generator.py:943` (existing `logger.info(t("log.profile_generator.m017", …))`), ticket #24 acceptance ("each `file:line` is fixed").
- **Findings**:
- The existing `logger.info` and the `print(...)` are emitting the same logical event in two channels. The banner adds `'='*60` separators on the surrounding lines, which is purely a console-cosmetic; replacing the print with a logger call would lose the visual banner.
- Ticket #24 wants externalisation, not removal.
- **Implications**: Keep both calls. Wrap the `print(f"...")` argument with `t(...)`. Introduce dedicated keys (`m024`, `m025`) so the banner copy is decoupled from the structured log copy at `m017`.
## Architecture Pattern Evaluation
| Option | Description | Strengths | Risks / Limitations | Notes |
|--------|-------------|-----------|---------------------|-------|
| Append-at-tail (selected) | Add new `m###` keys at the next contiguous slot per namespace; create `log.retry` fresh | Mirrors #6 convention; minimal diff; no overwrite risk | Numbering gaps under `log.graph_api` remain | Aligns with steering principle of preserving established conventions |
| Fill numbering gaps | Reuse missing slots `m009`, `m010`, etc. | Tighter numbering | Risk of colliding with reserved-but-not-yet-merged keys; mixed insertion sites complicate review | Rejected |
| Consolidate banner prints into logger | Remove the `print(...)` calls; use only `logger.info(t(...))` | One fewer key | Behaviour change (loses console banner); violates Requirement 3.2 | Rejected |
## Design Decisions
### Decision: Add a new `log.retry` sub-namespace rather than reusing `log.bootstrap` or `log.graph_api`
- **Context**: `retry.py` is a generic utility used by many callers; it does not belong to a single domain.
- **Alternatives Considered**:
1. Place keys under `log.bootstrap` — wrong domain (bootstrap is for app startup logs).
2. Place keys under each caller's namespace — would require dynamic key resolution, adding complexity.
3. New `log.retry` sub-namespace — clean and self-describing.
- **Selected Approach**: Introduce `log.retry.m001`–`m004` as a peer of `log.graph_api`, `log.profile_generator`, etc.
- **Rationale**: Matches the per-domain naming scheme already in use; locates retry-specific copy in one place.
- **Trade-offs**: Adds one new sub-namespace under `log`, but does not change the top-level key set.
- **Follow-up**: Verify that no other module already defines `log.retry` (verified: it does not exist).
### Decision: Wrap `print(...)` arguments rather than removing the prints
- **Context**: Ticket #24 mandates externalisation of the listed call sites; behaviour preservation is in scope.
- **Alternatives Considered**:
1. Keep `print(t("..."))` — preserves console banner, externalises text.
2. Remove `print(...)`; rely on `logger.info` only — drops banner.
- **Selected Approach**: Option 1. The `'='*60` separator lines stay; only the message text routes through `t(...)`.
- **Rationale**: Minimum change; respects Requirement 3.2.
- **Trade-offs**: None significant.
- **Follow-up**: Confirm during validation that the surrounding separator prints (`print(f"\n{'='*60}")`) are not on the ticket's evidence list (they are not — they contain only ASCII).
### Decision: Pass exception text as a keyword argument named `e` (not `error`)
- **Context**: Existing `log.profile_generator` keys use `e=str(e)` and `error=...` inconsistently. Need to pick one convention to remain consistent.
- **Alternatives Considered**:
1. Use `e` — matches `log.profile_generator.m003`, `m005`, `m008`, `m012`.
2. Use `error` — matches `log.profile_generator.m018`.
- **Selected Approach**: Use `e` for raw exception strings (the more common pattern). Where a separate label is more readable, use a domain-specific name (e.g. `error` is fine when it carries semantic weight).
- **Rationale**: Match the dominant existing convention.
- **Trade-offs**: None.
- **Follow-up**: Use `e` throughout the new keys.
## Risks & Mitigations
- **Async retry on a fresh thread without `set_locale`** — Falls back to `'zh'`. Acceptable: ticket #24 acceptance targets *source-code* CJK absence. Documented for future ticket if needed.
- **Circular imports when adding `from ..utils.locale import t` to `retry.py`** — `locale.py` imports only `json`, `logging`, `os`, `threading`, and `flask` (no project modules). No circular risk.
- **Test-suite breakage from changed log text** — No fixtures match the Chinese strings. Verified by grep of `backend/`. Low risk.
## References
- Sibling spec: `.kiro/specs/i18n-externalize-backend-logs/requirements.md` — established convention.
- Ticket #6 (closed) and ticket #24 (this work).
- `backend/app/utils/locale.py` — `t()` contract.

24
.kiro/specs/i18n-externalize-remaining-backend-logs/spec.json Normal file
View File

@ -0,0 +1,24 @@
{
"feature_name": "i18n-externalize-remaining-backend-logs",
"created_at": "2026-05-07T22:24:20Z",
"updated_at": "2026-05-07T22:50:00Z",
"language": "en",
"phase": "tasks-generated",
"approvals": {
"requirements": {
"generated": true,
"approved": true
},
"design": {
"generated": true,
"approved": true
},
"tasks": {
"generated": true,
"approved": true
}
},
"ready_for_implementation": true,
"ticket": 24,
"related_tickets": [10, 6]
}

63
.kiro/specs/i18n-externalize-remaining-backend-logs/tasks.md Normal file
View File

@ -0,0 +1,63 @@
# Implementation Plan
- [x] 1. Add three new keys to `log.graph_api` in both locale files
- In `locales/en.json`, append `m027`, `m028`, `m029` under `log.graph_api` with the English translations from the design's key-mapping table
- In `locales/zh.json`, append the same three keys under `log.graph_api` with the verbatim original Chinese text (rewriting `f"...{var}..."` as `"...{var}..."`)
- Confirm via `python3 -m json.tool` that both files round-trip without reformatting other keys
- Observable completion: `python3 -c "import json; en=json.load(open('locales/en.json'))['log']['graph_api']; zh=json.load(open('locales/zh.json'))['log']['graph_api']; assert {'m027','m028','m029'} <= set(en) <= set(zh) | set(en); print('ok')"` exits zero
- _Requirements: 2.1, 2.2, 2.3, 2.5_
- [x] 2. Replace the three Chinese f-strings in `backend/app/api/graph.py` with `t()` calls
- Line 385: replace `f"[{task_id}] 开始构建图谱..."` with `t("log.graph_api.m027", task_id=task_id)`
- Line 494: replace the build-completion f-string with `t("log.graph_api.m028", task_id=task_id, graph_id=graph_id, node_count=node_count, edge_count=edge_count)`
- Line 513: replace the build-failure f-string with `t("log.graph_api.m029", task_id=task_id, e=str(e))`
- Do not change log levels, surrounding `task_manager.update_task` calls, or control flow
- Observable completion: `grep -nP "[一-鿿]" backend/app/api/graph.py | grep -E "^(385|494|513):"` returns no matches; `python3 -c "import ast; ast.parse(open('backend/app/api/graph.py').read())"` succeeds
- _Requirements: 1.1, 1.4, 1.5, 3.1, 3.4_
- _Depends: 1_
- [x] 3. Add two new keys to `log.profile_generator` in both locale files
- In `locales/en.json`, append `m024` and `m025` under `log.profile_generator` per the design table
- In `locales/zh.json`, mirror with the verbatim original Chinese banner text (using `{count}` placeholder where the source had `len([p for p in profiles if p])`)
- Observable completion: same key-presence assertion as Task 1 but for `m024`, `m025`
- _Requirements: 2.1, 2.2, 2.3, 2.5_
- [x] 4. Replace the two `print(...)` banner strings in `backend/app/services/oasis_profile_generator.py` with `t()` calls
- Line 945: replace `f"开始生成Agent人设 - 共 {total} 个实体，并行数: {parallel_count}"` with `t("log.profile_generator.m024", total=total, parallel_count=parallel_count)`
- Line 1001: replace `f"人设生成完成！共生成 {len([p for p in profiles if p])} 个Agent"` with `t("log.profile_generator.m025", count=len([p for p in profiles if p]))`
- Keep the surrounding `print(f"\n{'='*60}")` separator lines exactly as they are; keep both `print(...)` calls (do not collapse into the existing `logger.info` at line 943)
- Observable completion: `grep -nP "[一-鿿]" backend/app/services/oasis_profile_generator.py | grep -E "^(945|1001):"` returns no matches; the file still parses with `ast.parse`
- _Requirements: 1.2, 1.4, 1.5, 3.2_
- _Depends: 3_
- [x] 5. Add a new `log.retry` sub-namespace with four keys to both locale files
- In `locales/en.json`, add `log.retry` as a peer of the other `log.<domain>` sub-namespaces, with keys `m001`–`m004` per the design table
- In `locales/zh.json`, mirror the same `log.retry` sub-namespace with verbatim original Chinese
- Use placeholder names `func_name`, `max_retries`, `index`, `e` consistently across both files (note: the source `idx + 1` is bound to `index=idx + 1` at the call site — placeholder names cannot contain `+`)
- Observable completion: `python3 -c "import json; en=json.load(open('locales/en.json'))['log']['retry']; zh=json.load(open('locales/zh.json'))['log']['retry']; assert set(en)==set(zh)=={'m001','m002','m003','m004'}; print('ok')"` exits zero
- _Requirements: 2.1, 2.2, 2.3, 2.5_
- [x] 6. Externalise the four `logger.error` strings in `backend/app/utils/retry.py`
- Add `from .locale import t` at the top of `retry.py` (use the same relative-import depth as `from ..utils.logger import get_logger` already in the file — i.e., `from .locale import t`)
- Line 55: replace `f"函数 {func.__name__} 在 {max_retries} 次重试后仍失败: {str(e)}"` with `t("log.retry.m001", func_name=func.__name__, max_retries=max_retries, e=str(e))`
- Line 108: replace `f"异步函数 {func.__name__} 在 {max_retries} 次重试后仍失败: {str(e)}"` with `t("log.retry.m002", func_name=func.__name__, max_retries=max_retries, e=str(e))`
- Line 179: replace `f"API调用在 {self.max_retries} 次重试后仍失败: {str(e)}"` with `t("log.retry.m003", max_retries=self.max_retries, e=str(e))`
- Line 227: replace `f"处理第 {idx + 1} 项失败: {str(e)}"` with `t("log.retry.m004", index=idx + 1, e=str(e))`
- Do not modify the `logger.warning(...)` retry-attempt messages or the docstrings (out of scope for #24)
- Observable completion: `grep -nP "[一-鿿]" backend/app/utils/retry.py | grep -E "^(55|108|179|227):"` returns no matches; `python3 -c "import ast; ast.parse(open('backend/app/utils/retry.py').read())"` succeeds; `python3 -c "from backend.app.utils import retry; print(retry.t)"` resolves the import
- _Requirements: 1.3, 1.4, 1.5, 3.3, 4.1, 4.2, 4.3, 4.4_
- _Depends: 5_
- [x] 7. Run mechanical verification across the change
- From the repo root, verify zero CJK on the nine affected lines:
```
grep -nP "[一-鿿]" backend/app/api/graph.py | grep -E "^(385|494|513):" || echo OK_graph
grep -nP "[一-鿿]" backend/app/services/oasis_profile_generator.py | grep -E "^(945|1001):" || echo OK_profile
grep -nP "[一-鿿]" backend/app/utils/retry.py | grep -E "^(55|108|179|227):" || echo OK_retry
```
Each should print `OK_*`.
- Run a Python parity check that asserts every newly-added key path exists in both `locales/en.json` and `locales/zh.json` and that every `{name}` placeholder in the `zh` value also appears in the `en` value (and vice versa).
- Run `cd backend && uv run python -m pytest` and confirm no new failures relative to the pre-change baseline.
- Observable completion: all three grep assertions print `OK_*`; the parity Python check exits zero; the pytest run reports the same pass/fail count as on `main` for these files.
- _Requirements: 1.5, 2.4, 5.1, 5.2, 5.3, 5.4, 5.5_
- _Depends: 2, 4, 6_

229
.kiro/specs/i18n-frontend-comments/design.md Normal file
View File

@ -0,0 +1,229 @@
# Design Document — i18n-frontend-comments
## Overview
**Purpose**: Translate Chinese developer comments in `frontend/src/` to English so non-Chinese-reading maintainers can understand intent without translation tooling. Strictly documentation-only; no behavior change.
**Users**: Frontend maintainers and reviewers of MiroFish — developers who read and modify `frontend/src/` but do not read Chinese.
**Impact**: 20 files in `frontend/src/` change; the compiled bundle is byte-equivalent modulo source-map comment lines. The `vue-i18n` user-facing translation surface (`/locales/*.json`) is unaffected.
### Goals
- Eliminate Chinese characters (U+4E00–U+9FFF) from `frontend/src/` comments and dev-facing string literals (`console.*`).
- Preserve every comment's *why* (semantic intent) when translating; delete comments that merely restate the code per `dev-guidelines.md`.
- Append `(#9)` ticket reference to any TODO/FIXME marker that lacks one.
- Keep `npm run build` green and the rendered UI byte-equivalent on a smoke check.
### Non-Goals
- Translating user-facing strings (those live in `/locales/*.json`; tracked separately).
- Translating LLM prompt template strings (translation would change model input — retained and documented in PR per Requirement 1.5).
- Restructuring comments into JSDoc (only keep JSDoc when already JSDoc-shaped).
- Reformatting code, renaming identifiers, or any change to `<script>` / `<template>` semantics.
- Touching backend Python comments (covered by ticket #7) or repo-root configuration files.
## Boundary Commitments
### This Spec Owns
- All comment text inside files under `frontend/src/`: line comments (`//`), block comments (`/* */`), JSDoc (`/** */`), and Vue template comments (`<!-- -->`).
- The natural-language portion of JSDoc tags (`@param`, `@returns`, etc.) — not the tag syntax itself.
- Chinese-content string literals passed to `console.error`, `console.warn`, and `console.log` (developer-facing, not in i18n locales).
- The PR-level documentation listing any deliberately-retained bilingual content.
### Out of Boundary
- Any change inside `/locales/*.json` (covered by issues #8 and #11).
- Any change in `backend/`, `static/`, repo root, or anywhere outside `frontend/src/`.
- LLM prompt template string literals (e.g. `Step5Interaction.vue:725-727`) — retained as documented exceptions.
- New tooling (linters, formatters, translation scripts).
- Any executable change: identifier names, import paths, expression edits, Vue template structure outside `<!-- -->` text, or `<style>` selectors / values.
### Allowed Dependencies
- Existing Vite build (`npm run build`) and Vue dev server (`npm run dev`) for verification.
- `ripgrep` for the verification command.
- No runtime dependencies — this is text-only editing.
### Revalidation Triggers
- Discovery during implementation that a category of Chinese content beyond comments + `console.*` strings exists in `frontend/src/` → update the design's String-Literal Decision Matrix and add residuals to the PR description rather than silently expanding scope.
- Discovery that a JSDoc block carries semantically-load-bearing Chinese (e.g. an idiom that does not have a 1:1 English rendering) → keep both languages, document in PR per Req 1.5.
## Architecture
### Existing Architecture Analysis
Per `structure.md`, `frontend/src/` is layered into `views/`, `components/`, `api/`, `store/`, plus `App.vue`. This spec does not change the layering. Per `tech.md`, the project uses no enforced linter/formatter and existing files mix English and Chinese comments — this spec is the explicit ask to normalize the comment language to English in this directory.
### Architecture Pattern & Boundary Map
This is a documentation-only change — no architectural pattern to choose. The relevant boundary is purely *which textual regions of which files are eligible for edit*. The decision matrix below is the architecture for this spec.
#### Region eligibility matrix
| Region | Action |
| --- | --- |
| `//` line comment | Translate; delete if it restates the code per Req 2.1 |
| `/* */` block comment | Translate; delete if redundant per Req 2.1 |
| `/** */` JSDoc block | Translate the natural-language content; preserve tag syntax (`@param`, `@returns`, etc.) per Req 1.4 |
| `<!-- -->` Vue template comment | Translate per Req 1.3 |
| `console.error|warn|log('… 中文 …')` | Translate the string content (developer-facing, not in i18n locales) |
| LLM prompt template string literal | **Do not translate**; document in PR per Req 1.5 |
| Any other string literal containing Chinese | **Do not translate** (Req 4.4); document if non-empty |
| Identifiers, imports, exports, expressions | **Do not change** (Req 4.2) |
| Vue template structure (tags, attributes, bindings) | **Do not change** (Req 4.2) |
### Technology Stack
| Layer | Choice / Version | Role in Feature | Notes |
|---|---|---|---|
| Frontend | Vue 3.5 + Vite 7 (existing) | Build target — must continue to compile | No version change |
| Verification | `ripgrep` (already present in repo workflows) | Acceptance gate via `rg '[\x{4e00}-\x{9fff}]' frontend/src/` | No new dependency |
| No new tooling | — | — | Per `tech.md` steering: "No enforced linter or formatter… match the surrounding file's style" |
## File Structure Plan
No directory or file additions. All edits are in-place inside the 20 files identified by ripgrep:
```
frontend/src/
├── App.vue # 4 hits — translate
├── api/
│ ├── graph.js # 10 hits
│ ├── index.js # 8 hits (incl. JSDoc-light line comments)
│ ├── report.js # 8 hits
│ └── simulation.js # 29 hits (JSDoc-heavy)
├── components/
│ ├── GraphPanel.vue # 84 hits — D3 logic comments + template
│ ├── HistoryDatabase.vue # 124 hits
│ ├── Step1GraphBuild.vue # 5 hits + 3 console.error strings
│ ├── Step2EnvSetup.vue # 76 hits
│ ├── Step3Simulation.vue # 52 hits
│ ├── Step4Report.vue # 176 hits
│ └── Step5Interaction.vue # 34 hits + LLM prompt strings (RETAIN)
├── store/
│ └── pendingUpload.js # 2 hits
└── views/
├── Home.vue # 43 hits
├── InteractionView.vue # 6 hits
├── MainView.vue # 4 hits
├── Process.vue # 191 hits — largest file (2067 lines)
├── ReportView.vue # 6 hits
├── SimulationRunView.vue # 18 hits
└── SimulationView.vue # 22 hits
```
### Modified Files
All 20 files above receive comment translation (and, for `Step1GraphBuild.vue` and any others discovered during implementation, `console.*` string translation). No file is created, deleted, or moved.
## System Flows
### Per-file translation sequence
```mermaid
flowchart TD
A[Open file] --> B[Locate Chinese region with rg or editor]
B --> C{Region type?}
C -->|Comment| D{Restates the code?}
D -->|Yes| E[Delete comment]
D -->|No| F[Translate, preserve intent]
C -->|JSDoc| G[Translate natural-language content<br/>preserve tag syntax]
C -->|Vue template comment| H[Translate inside &lt;!-- --&gt;]
C -->|console.* string| I[Translate string content]
C -->|LLM prompt string| J[Skip; record for PR description]
C -->|Other string literal| K[Skip per Req 4.4]
E --> L[Next region]
F --> L
G --> L
H --> L
I --> L
J --> L
K --> L
L --> M{File done?}
M -->|No| B
M -->|Yes| N[Run rg on file: confirm zero remaining hits<br/>OR all remaining are intentional retentions]
N --> O[File complete]
```
### TODO/FIXME sweep
```mermaid
flowchart LR
A[rg 'TODO|FIXME' frontend/src/] --> B{Any hits?}
B -->|None| C[Document in PR: no markers found]
B -->|Has hits| D[For each hit]
D --> E{Already has #N reference?}
E -->|Yes| F[Leave unchanged]
E -->|No, was Chinese| G[Translate description AND append #9]
E -->|No, was already English| H[Out of scope; leave unchanged]
```
## Requirements Traceability
| Requirement | Summary | Realized by |
|---|---|---|
| 1.1 | Zero Chinese in `frontend/src/` per ripgrep | Per-file translation pass; verification command in PR |
| 1.2 | Preserve semantic intent | Translator judgment per region; Req 2.3 enforces conservative-on-ambiguity |
| 1.3 | Handle SFC blocks correctly | Region eligibility matrix (`<script>` / `<template>` / `<style>` rows) |
| 1.4 | Preserve JSDoc structure | Region matrix: "Translate the natural-language content; preserve tag syntax" |
| 1.5 | Document retained bilingual content | PR description lists `Step5Interaction.vue` LLM prompts (and any others) |
| 2.1 | Delete redundant comments | Per-file flowchart `D → E` branch |
| 2.2 | Translate intent-bearing comments | Per-file flowchart `D → F` branch |
| 2.3 | Conservative on ambiguity | Translator rule encoded in research.md Decision; default is *translate, not delete* |
| 2.4 | No new explanatory comments | Translation rule: never add comments not present in original (except `(#9)` ticket ref) |
| 3.1 | Keep TODO/FIXME marker, translate trailing text | TODO sweep flowchart `G` branch |
| 3.2 | Append `(#9)` ticket ref where missing | TODO sweep flowchart `G` branch |
| 3.3 | Preserve existing ticket refs | TODO sweep flowchart `E → F` branch |
| 4.1 | `npm run build` exit 0 | Build run as part of acceptance check |
| 4.2 | No executable change | Region matrix: identifiers/imports/expressions are *not eligible* |
| 4.3 | UI smoke-check identical | Manual smoke after build |
| 4.4 | Leave string literals untouched (except `console.*`) | Region matrix; documented exception for `console.*` is the sole carve-out |
| 5.1 | Verification command in PR | PR template hand-off |
| 5.2 | List retained bilingual files | PR template hand-off |
| 5.3 | Branch + commit naming | `docs/i18n-9-translate-frontend-comments` and `docs(i18n): translate chinese comments in frontend src to english` |
| 5.4 | No edits outside `frontend/src/` | `git diff --name-only main..HEAD` review at PR time |
## Components and Interfaces
This is a documentation-only change — there are no software components, services, or APIs to design. The "interfaces" of this spec are textual:
| Interface | Owner | Contract |
| --- | --- | --- |
| `frontend/src/**/*.{vue,js}` comments | This spec | All comment text is English. Chinese is permitted only when explicitly listed in the PR description as a deliberately-retained bilingual case. |
| `frontend/src/**/*.{vue,js}` `console.*` string literals | This spec | All `console.error|warn|log` argument strings are English. |
| `frontend/src/**/*.{vue,js}` non-`console` string literals | Out of scope | Unchanged from baseline. Any Chinese in these strings (e.g. LLM prompt templates) is documented in the PR. |
| `frontend/src/**/*.{vue,js}` executable code | Out of scope | Byte-identical except for surrounding comment lines. |
## Data Models
Not applicable — no data structures change.
## Error Handling
Not applicable — no runtime code path changes. The "errors" of this spec are reviewer-detectable issues:
| Issue | Detection | Response |
|---|---|---|
| Translation drift (wrong meaning) | Reviewer reads English comment against surrounding code | Reviewer flags; translator revises |
| Accidental edit to executable code | `git diff` review filtered to non-comment lines | Revert; restart that file |
| Residual Chinese in non-LLM string | Verification ripgrep returns unexpected file | Either translate (if `console.*`) or move LLM exception to PR description |
| Build failure on `npm run build` | CI / local build | Bisect: most likely accidental edit to a `<script>` or `<template>` block; revert |
## Testing Strategy
No automated tests added. The spec's verification surface is:
- **Acceptance ripgrep**: `rg '[\x{4e00}-\x{9fff}]' frontend/src/` returns no files (or only files listed as retained in the PR description).
- **Vite build**: `npm run build` exits 0.
- **Manual UI smoke**: `npm run dev`, navigate Home → Process → each Step component → Interaction → Report; confirm rendering matches pre-change baseline. (Cannot be fully proven; explicit acknowledgment of "manual smoke" per the steering note that "type-check/test passes do not prove feature correctness here".)
- **Diff hygiene check**: `git diff --stat main..HEAD` shows only `frontend/src/` files modified.
## Implementation Notes
- Per the project's manual-style ethos, do this in an editor with rg-driven navigation. No new scripts.
- For each file, do all edits in one pass, then re-run `rg '[\x{4e00}-\x{9fff}]' <file>` to confirm zero residual (or only the deliberately-retained string literals, which the implementer should know about ahead of time per the design's eligibility matrix).
- The largest 6 files (`Process.vue`, `Step4Report.vue`, `HistoryDatabase.vue`, `GraphPanel.vue`, `Step2EnvSetup.vue`, `Step3Simulation.vue`) account for ~80% of the work; budget time accordingly.
- Reviewer aid: the PR description should list, in order, the verification command, the verification result, the file count, and any retained-bilingual exceptions. Keep the description short — the diff itself carries the work.

133
.kiro/specs/i18n-frontend-comments/gap-analysis.md Normal file
View File

@ -0,0 +1,133 @@
# Gap Analysis — i18n-frontend-comments
## 1. Current State Investigation
### Scope discovery (ground truth)
Ripgrep `[\x{4e00}-\x{9fff}]` over `frontend/src/` returns **20 files, 902 occurrences**:
| File | Hits |
| --- | ---: |
| `views/Process.vue` | 191 |
| `components/Step4Report.vue` | 176 |
| `components/HistoryDatabase.vue` | 124 |
| `components/GraphPanel.vue` | 84 |
| `components/Step2EnvSetup.vue` | 76 |
| `components/Step3Simulation.vue` | 52 |
| `views/Home.vue` | 43 |
| `components/Step5Interaction.vue` | 34 |
| `api/simulation.js` | 29 |
| `views/SimulationView.vue` | 22 |
| `views/SimulationRunView.vue` | 18 |
| `api/graph.js` | 10 |
| `api/index.js` | 8 |
| `api/report.js` | 8 |
| `views/InteractionView.vue` | 6 |
| `views/ReportView.vue` | 6 |
| `components/Step1GraphBuild.vue` | 5 |
| `App.vue` | 4 |
| `views/MainView.vue` | 4 |
| `store/pendingUpload.js` | 2 |
No `.css` files exist under `frontend/src/`; styles live inside Vue SFC `<style>` blocks.
### Comment shapes encountered
Sampling representative files confirms three syntactic forms — all already English-syntax, only the natural-language content is Chinese:
- **JS line comments**: `// 创建axios实例`, `timeout: 300000, // 5分钟超时（本体生成可能需要较长时间）`
- **JSDoc blocks** in `api/simulation.js`: `/** * 创建模拟 */`, `* @returns {Promise} 返回配置信息，包含元数据和配置内容`
- **Vue template comments** in `views/Home.vue`: `<!-- 顶部导航栏 -->`, `<!-- 上半部分：Hero 区域 -->`
### String literals containing Chinese (NOT comments)
A naive regex for Chinese inside quoted strings flags **8 files**. Spot-checks reveal two distinct categories that the ticket body did not explicitly anticipate:
- **Developer-facing log strings** — e.g. `Step1GraphBuild.vue:216` `console.error('缺少项目或图谱信息')`. These print to the browser dev console and are not part of the i18n locale surface. Translating them does not change runtime behavior.
- **LLM prompt template strings** — e.g. `Step5Interaction.vue:725-727` `\`以下是我们之前的对话：\n${historyContext}\n\n现在我的新问题是：${message}\``. These are sent to a Chinese-tuned LLM (default Qwen). Translating them *would* change the model's input and could shift output behavior.
The ticket says **"no UI string changes (those are already in `locales/en.json`)"** and **"Out of scope: Translating user-facing strings"**. Neither category above is user-facing UI text — `locales/*.json` already covers user-facing strings via `vue-i18n`. The ticket's acceptance criterion #1 (`grep returns no files, or only files with deliberately-kept bilingual comments listed in PR`) leaves room to retain the LLM prompt strings as documented exceptions.
### Conventions to respect (from steering)
- `tech.md`: 4-space indent, no enforced linter, "match the surrounding file's style". Existing files mix English and Chinese in comments/docstrings — preserve both *unless asked*. **This ticket is the explicit ask.**
- `structure.md`: `frontend/src/api/*.js` services use Axios with 5-min timeout + exponential retry. The translation pass must not touch the retry/timeout logic.
- `dev-guidelines.md` (project-level): "Don't comment the obvious — comment the *why*." JSDoc on all exported functions, classes, interfaces (so JSDoc blocks must be **kept** in JSDoc form when translating, not deleted as redundant).
- `commits.md`: Conventional Commits, lowercase, imperative, max 72 chars, no `Co-Authored-By:` footer. Branch `<type>/<ticket>-<desc>` — ticket dictates `docs/i18n-9-translate-frontend-comments`.
### Existing i18n-related precedent
Recent merged PRs in the same epic (#11):
- `feat/i18n-2-translate-ontology-generator-prompts` → backend prompt translation, full content swap.
- `feat/i18n-4-translate-sim-config-prompts`, `feat/i18n-5-translate-report-agent-prompts` → similar backend prompt swaps.
- `feat/i18n-6-externalize-backend-logs` → moved log strings out of code into i18n keys.
- `fix/i18n-8-backfill-zh-json` (current branch base) → backfilled missing zh translations.
**Pattern**: prior i18n work changed both content *and* infrastructure (locale-keying logs). This ticket explicitly does not — it is a documentation-only pass without re-keying anything.
## 2. Requirements ↔ Asset Map
| Req | Asset to change | Gap tag | Note |
| --- | --- | --- | --- |
| 1.1–1.4 (translate comments incl. JSDoc) | All 20 files listed above | — (clear) | Largely mechanical; respect SFC block boundaries (`<script>` vs `<template>` vs `<style>`). |
| 1.5 (deliberately bilingual) | LLM prompt strings in `Step5Interaction.vue` (and any others discovered) | **Constraint** | Keep Chinese, document in PR. Behavior-risk if translated. |
| 2.x (drop redundant) | Files with `// 获取数据`-style restate-the-code comments | — | Apply per case during the pass; conservative when ambiguous. |
| 3.x (TODO/FIXME ticket refs) | Search `frontend/src/` for `TODO\|FIXME` | **Unknown** | No matches noted in spot checks; will sweep during implementation. If none found, requirement is satisfied vacuously. |
| 4.x (no behavior change) | Confirmed by `npm run build` exit 0 + manual smoke | — | Vite build is the reference; keep all string-literal content (other than developer-log strings) untouched; identifiers and imports are off-limits. |
| 5.x (PR hand-off) | PR description, branch name, commit message | — | Branch name from ticket: `docs/i18n-9-translate-frontend-comments`. |
### Discovered scope ambiguity → decision needed
Two boundary calls that the requirements should sharpen before design:
- **`console.error` / `console.warn` / `console.log` strings with Chinese content** — translate (developer-facing, not in locales) or leave (string-literal change risks scope creep)? Recommended: **translate**, since they are dev-facing comments-by-other-means and the ticket's spirit is "English-readable code". This is a design decision to be encoded in the design doc, not a new requirement.
- **LLM prompt template strings** — leave as-is and list in PR (per Req 1.5). This is the safer call: the LLM is Chinese-tuned by default and translating a system prompt is a behavior change.
Both decisions stay inside the requirements as currently written (specifically Req 1.5 + Req 4.4, which already excludes string literals from the translation pass except where developer-log strings are concerned). The design phase will document the rule explicitly.
## 3. Implementation Approach Options
### Option A — Single-pass translation per file, no tooling
**Approach**: Open each of the 20 files, translate every Chinese comment in place, drop redundant ones, append `(#9)` to bare TODO/FIXME, leave Chinese string literals (LLM prompts) and translate `console.*` Chinese strings. Verify with `rg [\x{4e00}-\x{9fff}] frontend/src/`.
- ✅ Lowest overhead, no new tools or scripts
- ✅ Fits a one-shot doc-only PR
- ✅ Maximally aligns with `dev-guidelines.md` "comment the *why*" — judgment per comment
- ❌ ~900 occurrences spread across 20 files — most concentrated in 6 files (>50 hits each) which are large (`Process.vue` is 2067 lines, `Step4Report.vue`, `HistoryDatabase.vue`)
- ❌ Manual judgment for redundant-vs-meaningful adds reviewer load
### Option B — Automated translation script + manual pass
**Approach**: Write a Node/Python script that walks files, extracts Chinese comments, runs them through an LLM, and writes back. Then a manual pass on the diff.
- ✅ Faster on long files
- ❌ Adds a dependency (LLM call) and a scratch script, neither delivered
- ❌ The translation needs *judgment* (drop vs translate per Req 2) — automation undercuts the "comment the *why*" rule
- ❌ Risk of touching string literals or identifiers if regex is loose
- ❌ Out of step with the steering "no enforced tooling without discussion" principle
### Option C — File-by-file with task batching
**Approach**: Group the 20 files into work units by size: (a) high-touch (Process, Step4Report, HistoryDatabase, GraphPanel, Step2EnvSetup, Step3Simulation), (b) mid-touch (Home, Step5Interaction, simulation.js, SimulationView, SimulationRunView), (c) light (api/{graph,index,report}.js, the 4–8 hit views, App.vue, store/pendingUpload.js, Step1GraphBuild.vue). Implementation tasks mirror these groups. Verify after each group with the ripgrep check.
- ✅ Same translation effort as A but with checkpointable progress (matches the project's task-tracking pattern from steering — "background tasks expose progress")
- ✅ Reviewer can read the PR file-group-by-file-group instead of all-at-once
- ✅ If the PR needs to land partial (rare), the light + mid groups still ship a valuable subset
- ❌ A few extra task headings in `tasks.md` vs Option A's "do the thing"
## 4. Effort & Risk
- **Effort**: **S (1–2 days)**. Mechanical translation, plus judgment calls. ~900 occurrences but no architectural work.
- **Risk**: **Low**. Doc-only change. The only real risks are (a) accidentally editing a string literal that affects the LLM prompt or a hardcoded user-visible string, and (b) deleting a comment whose intent the translator misread. Both are mitigated by Req 4.4 ("leave string literals unchanged") and Req 2.3 (conservative-when-ambiguous).
## 5. Recommendations for Design Phase
- **Preferred approach**: **Option C** — file-grouped translation pass, no tooling, no script. It matches the project's manual-style ethos and the existing pipeline-aligned task structure, and produces a reviewable PR.
- **Encode in design**:
- The translation rule for each comment shape (`//`, `/* */`, JSDoc, `<!-- -->`).
- The decision matrix for string literals: translate `console.*` Chinese strings; retain LLM prompt strings (in `Step5Interaction.vue`) and list them in the PR per Req 1.5.
- The TODO/FIXME sweep approach (single ripgrep pass before the file loop).
- The verification command and acceptance check sequence.
- **Research items carried forward**: none — the codebase has been inspected enough to commit to Option C without further investigation.

70
.kiro/specs/i18n-frontend-comments/requirements.md Normal file
View File

@ -0,0 +1,70 @@
# Requirements Document
## Introduction
This spec covers a pure-documentation cleanup pass: translate Chinese developer comments in `frontend/src/` to English so English-speaking maintainers can read the code. The change is documentation-only — no runtime behavior changes, no UI string changes (those live in `/locales/*.json`), and no architectural refactor. Tracked as GitHub issue #9, the lowest user-impact ticket in the i18n epic (#11).
The work targets developer-facing comments in 20 known files: 7 views, 7 components, 4 `api/*.js` modules, `App.vue`, and `store/pendingUpload.js`. The discovery method is `grep -rln '[一-鿿]' frontend/src/` (or the ripgrep equivalent), which must return zero matches at completion (or only files explicitly listed as deliberately bilingual in the PR description).
## Boundary Context
- **In scope**: Translating Chinese developer comments (line comments, block comments, JSDoc, and Vue `<!-- ... -->` template comments) to English in `frontend/src/`. Dropping comments that merely restate the code, per `dev-guidelines.md`. Appending ticket references to TODO/FIXME markers that lack one.
- **Out of scope**: Any user-facing string, label, placeholder, toast, or template-rendered text — these live in `/locales/en.json` and `/locales/zh.json` and are tracked separately (see #8). Restructuring comments into JSDoc unless they are already JSDoc-shaped. Reformatting code, renaming identifiers, or any non-comment change. Backend Python comments (covered by ticket #7).
- **Adjacent expectations**: The Vite build (`npm run build`) and the Vue dev server (`npm run dev`) must continue to compile and run. The `vue-i18n` translation surface in `/locales/*.json` is unaffected. The frontend `api/` services keep their existing behavior — the 5-min Axios timeout and exponential retry described in steering remain unchanged.
## Requirements
### Requirement 1: Comment Translation Coverage
**Objective:** As a frontend maintainer who does not read Chinese, I want every developer comment in `frontend/src/` to be in English, so that I can understand intent without translation tooling.
#### Acceptance Criteria
1. The Frontend Source Tree shall contain no Chinese characters (Unicode range U+4E00–U+9FFF) in any `.vue`, `.js`, or `.css` file under `frontend/src/`, as verified by ripgrep `[\x{4e00}-\x{9fff}]` returning zero matching files.
2. When a Chinese comment is translated, the Translation Pass shall preserve the original semantic intent (the *why* the comment was written) without paraphrasing into a different meaning.
3. Where a comment exists in `<script>`, `<template>`, or `<style>` blocks of a Single-File Component, the Translation Pass shall translate it in-place using the syntax appropriate to that block (`//` / `/* */` for script and style, `<!-- -->` for template).
4. If a Chinese comment is part of a JSDoc block (`/** ... */`), the Translation Pass shall keep the JSDoc structure intact and translate only the natural-language content.
5. Where a deliberately-bilingual comment must be retained (e.g. a quotation, a domain term needing the original), the Translation Pass shall list the file in the PR description and shall keep an English explanation alongside the Chinese.
### Requirement 2: Drop Redundant Comments
**Objective:** As a code reviewer, I want comments that merely restate the code to be removed during the translation pass, so that the codebase aligns with `dev-guidelines.md` ("comment the *why*, not the *what*").
#### Acceptance Criteria
1. When a Chinese comment only paraphrases the immediately following statement in different words (e.g. `// 获取数据` above `fetchData()`), the Translation Pass shall delete the comment rather than translate it.
2. When a Chinese comment encodes non-obvious intent (a constraint, an invariant, a workaround, a reason behind a magic number), the Translation Pass shall translate it rather than delete it.
3. If a comment's value cannot be judged from local context alone, the Translation Pass shall translate it conservatively (preserve rather than delete) and shall not delete a comment merely because the maintainer is unsure of its purpose.
4. The Translation Pass shall not introduce new comments beyond those required to translate or to add a TODO ticket reference; gratuitous explanatory comments are not added.
### Requirement 3: Preserve TODO/FIXME Markers and Add Ticket References
**Objective:** As a project maintainer tracking work-in-progress markers, I want every TODO and FIXME comment to carry a ticket reference, so that future cleanup can be triaged systematically.
#### Acceptance Criteria
1. When a Chinese TODO or FIXME comment is encountered, the Translation Pass shall keep the `TODO` / `FIXME` marker (uppercase, English) and translate the trailing description.
2. Where a TODO or FIXME marker lacks a ticket reference, the Translation Pass shall append a reference in the form `TODO(#<n>): …` or `FIXME(#<n>): …`, using `#9` if no more specific ticket exists for the underlying work.
3. If a TODO or FIXME marker already references a ticket (e.g. `TODO(#42)`), the Translation Pass shall preserve that reference unchanged.
### Requirement 4: No Runtime Behavior Change
**Objective:** As a release engineer, I want the translated branch to produce a behaviorally identical bundle, so that I can ship the change without retesting feature surfaces.
#### Acceptance Criteria
1. When `npm run build` runs against the translated branch, the Vite Build shall complete successfully with the same exit code (0) as the pre-translation baseline.
2. The Translation Pass shall not change any executable code: no identifier renames, no expression edits, no import or export changes, no Vue template structure changes outside `<!-- -->` comment text.
3. While the application is running in `npm run dev`, the User Interface shall render identically to the pre-translation baseline for the Home, Process, and each Step component flow on a manual smoke check.
4. If a translation pass risks ambiguity between a comment and a string literal (Chinese characters in a quoted string), the Translation Pass shall leave the string literal unchanged — string content is out of scope and belongs to `/locales/*.json`.
### Requirement 5: Verifiability and PR Hand-off
**Objective:** As a reviewer of this PR, I want a single command and a short checklist to confirm acceptance, so that review effort is bounded and reproducible.
#### Acceptance Criteria
1. The PR Description shall include the verification command and its expected output: `rg '[\x{4e00}-\x{9fff}]' frontend/src/` returns no matches (or only the deliberately-bilingual files listed in the PR).
2. The PR Description shall list any deliberately-retained bilingual comments with the file path and a one-line rationale.
3. The Branch Name shall be `docs/i18n-9-translate-frontend-comments` and the Commit Message shall start with `docs(i18n): translate chinese comments in frontend src to english` per the ticket's stated convention and the project's Conventional Commits rule.
4. The Translation Pass shall not modify files outside `frontend/src/` (notably no edits under `/locales/`, `/backend/`, or repo-root configuration).

97
.kiro/specs/i18n-frontend-comments/research.md Normal file
View File

@ -0,0 +1,97 @@
# Research & Design Decisions — i18n-frontend-comments
## Summary
- **Feature**: `i18n-frontend-comments`
- **Discovery Scope**: Simple Addition (documentation-only translation pass; no architectural change)
- **Key Findings**:
- 20 files in `frontend/src/` contain Chinese characters (902 total occurrences). Concentration follows file size: `Process.vue` (191), `Step4Report.vue` (176), `HistoryDatabase.vue` (124), `GraphPanel.vue` (84), `Step2EnvSetup.vue` (76), `Step3Simulation.vue` (52). The remaining 14 files have ≤43 hits each.
- Chinese appears in three comment shapes (JS line `//`, JSDoc `/** */`, Vue `<!-- -->`) and — unexpectedly — inside two flavors of string literal: `console.error('…')` developer logs (low risk to translate) and LLM prompt template strings in `Step5Interaction.vue` (behavior risk if translated, since the default LLM is Chinese-tuned).
- The codebase has no enforced linter/formatter (per `tech.md`) and `dev-guidelines.md` already states "comment the *why*, not the *what*". The existing comment density skews toward restating-the-code in Chinese; a meaningful share will be deleted rather than translated.
## Research Log
### Inventory and shape of Chinese content
- **Context**: Need to decide whether one pass can mechanically translate or whether per-file judgment is required.
- **Sources Consulted**: `rg [\x{4e00}-\x{9fff}] frontend/src/` (full count) and content-mode samples of `api/index.js`, `api/simulation.js`, `views/Home.vue`, `components/Step1GraphBuild.vue`, `components/Step5Interaction.vue`.
- **Findings**:
- Comments are syntactically standard (`//`, `/** */`, `<!-- -->`); no inline-Chinese identifiers.
- JSDoc blocks in `api/simulation.js` (and likely `api/graph.js`, `api/report.js`) include `@returns`, `@param` annotations with Chinese descriptions — translate only the natural-language portion, keep tag structure.
- `console.error` strings in `components/Step1GraphBuild.vue` (3 hits at lines 216, 237, 241) are dev-facing only, not user-facing.
- LLM prompt template strings in `components/Step5Interaction.vue` (lines 725–727) are sent to a Chinese-tuned model; translation is a behavior change.
- **Implications**: Per-file judgment pass is required. String literals are out of scope by default (Req 4.4); only `console.*` Chinese strings are in scope as a documented exception (developer-facing).
### Tooling decision: manual vs scripted
- **Context**: ~900 occurrences across 20 files — would automation help?
- **Sources Consulted**: Steering `tech.md` ("No enforced linter or formatter in this repo by design… Discuss with the user before introducing ESLint/Prettier/Ruff/Black"); `dev-guidelines.md` ("comment the *why*"); gap-analysis Option B trade-offs.
- **Findings**: Automation undercuts Req 2 (drop redundant comments requires human judgment). The project explicitly disallows new tooling without discussion. The work fits an S-effort manual pass.
- **Implications**: No new scripts; no new dependencies; manual translation file-by-file.
### Verification path
- **Context**: How does the reviewer confirm acceptance?
- **Sources Consulted**: Ticket body acceptance criteria; project's Vite build (`npm run build`).
- **Findings**: A single ripgrep command confirms Req 1.1; `npm run build` confirms Req 4.1; manual smoke confirms Req 4.3. No new test harness is justified for a doc-only change (per steering "Don't add a heavy test harness without discussing scope").
- **Implications**: PR description carries the verification one-liner; the build is the proof.
## Architecture Pattern Evaluation
| Option | Description | Strengths | Risks / Limitations | Notes |
|--------|-------------|-----------|---------------------|-------|
| A. Single-pass translation, no tooling | Translate all 20 files in one PR; manual judgment per comment | Simple, low overhead | Long diff for the largest 6 files | Matches "manual style" steering ethos |
| B. Automated LLM-driven script + manual review | Script extracts Chinese comments, LLM translates, dev reviews diff | Faster on long files | Adds dependency; undercuts judgment requirement; risk of touching strings/identifiers | Rejected — clashes with "no new tooling" steering |
| C. File-grouped manual pass (selected) | Same translation effort as A, but tasks split into file groups: high-touch / mid-touch / light | Reviewable progress, matches project's task-tracking pattern | A few extra task headings | Selected — pairs cleanly with `tasks.md` structure |
## Design Decisions
### Decision: Manual file-grouped translation, no tooling
- **Context**: 20 files, ~900 occurrences, mixed comment shapes plus a small set of in-scope dev-log strings.
- **Alternatives Considered**:
1. Single mass pass (Option A) — workable but reviewer-unfriendly for the largest files
2. Automated LLM translation script (Option B) — fast but loses per-comment judgment and adds tooling
3. File-grouped manual pass (Option C) — same effort as A with clearer task decomposition
- **Selected Approach**: Group files into three batches by occurrence count and translate each batch as one task. After each batch, run the verification ripgrep to check progress.
- **Rationale**: Aligns with `tech.md` steering ("match the surrounding file's style"), `dev-guidelines.md` ("comment the *why*"), and lets `tasks.md` mirror the existing project task-tracking pattern. The S-effort estimate fits one work session.
- **Trade-offs**: A few extra task headings vs. cleaner reviewability. No infrastructure cost.
- **Follow-up**: Confirm `console.*` Chinese strings are translated; confirm LLM prompts in `Step5Interaction.vue` are documented as retained in PR description.
### Decision: String-literal scope rule
- **Context**: Some Chinese appears in string literals, not just comments.
- **Alternatives Considered**:
1. Strict: comments only — leaves dev-facing `console.*` Chinese which any maintainer reading dev console would still see in Chinese
2. Permissive: all string literals — translates LLM prompt templates, changing model behavior
3. Targeted: comments + dev-facing log strings (`console.*`); retain LLM prompts as documented exceptions
- **Selected Approach**: Targeted (option 3). Translate `console.error`, `console.warn`, `console.log` strings whose content is Chinese. Leave LLM prompt template strings alone and list them in the PR description per Req 1.5.
- **Rationale**: Honors the spirit of the ticket ("English-readable code") while preserving Req 4 ("no runtime behavior change") for the LLM-bound strings. Matches Req 4.4 (string literals untouched *except* where dev-log translation is unambiguous).
- **Trade-offs**: Reviewer needs to verify the exception list in the PR description against the residual ripgrep matches. Mitigated by Req 5.1 (PR description must document residuals).
- **Follow-up**: During implementation, confirm there are no other categories of Chinese-string-literal beyond `console.*` and LLM prompts. If discovered, add to the documented exception list rather than expanding scope.
### Decision: TODO/FIXME ticket reference policy
- **Context**: Req 3 mandates ticket references on TODO/FIXME markers.
- **Alternatives Considered**:
1. Skip the sweep entirely if no markers exist
2. Sweep `frontend/src/` for `TODO|FIXME` once at the start; append `(#9)` only where missing
- **Selected Approach**: Run a single `rg 'TODO|FIXME' frontend/src/` sweep before the file-translation loop; record any matches; apply Req 3.1–3.3 inline with each file's translation.
- **Rationale**: Lightest-weight implementation of Req 3. If no markers exist (likely for `frontend/src/`), the requirement is satisfied vacuously and noted in the PR description.
- **Trade-offs**: None.
- **Follow-up**: If markers exist in non-Chinese form (English TODOs without ticket refs), the requirement says only to act on *Chinese* markers; out of scope to retrofit unrelated existing English TODOs.
## Risks & Mitigations
- **Risk**: Accidentally translating an LLM prompt string and shifting model behavior. **Mitigation**: Req 4.4 + Decision "String-literal scope rule"; document retained Chinese strings in PR.
- **Risk**: Misinterpreting a Chinese comment and translating to wrong meaning. **Mitigation**: Req 2.3 (conservative when ambiguous; keep + translate rather than delete).
- **Risk**: Reviewer churn over which comments to delete vs. translate. **Mitigation**: `dev-guidelines.md` is the rubric; Decision documents the rule (delete only when comment paraphrases the next statement; translate when the comment encodes intent).
- **Risk**: PR is too large to review (Process.vue alone has ~191 hits). **Mitigation**: File-grouped tasks + per-group ripgrep checkpoint; each group is reviewable as a unit.
## References
- `dev-guidelines.md` (project) — comment philosophy and Conventional Commits.
- `tech.md` (steering) — "No enforced linter or formatter… match the surrounding file's style."
- `structure.md` (steering) — `frontend/src/` directory layout (views/components/api/store).
- Ticket #9 body — acceptance criteria, branch and commit naming.
- Gap analysis (`gap-analysis.md`) — Option C trade-offs and effort/risk estimate.

23
.kiro/specs/i18n-frontend-comments/spec.json Normal file
View File

@ -0,0 +1,23 @@
{
"feature_name": "i18n-frontend-comments",
"created_at": "2026-05-07T16:24:12Z",
"updated_at": "2026-05-07T16:35:00Z",
"language": "en",
"phase": "tasks-generated",
"ticket": 9,
"approvals": {
"requirements": {
"generated": true,
"approved": true
},
"design": {
"generated": true,
"approved": true
},
"tasks": {
"generated": true,
"approved": false
}
},
"ready_for_implementation": false
}

53
.kiro/specs/i18n-frontend-comments/tasks.md Normal file
View File

@ -0,0 +1,53 @@
# Implementation Plan
## Foundation
- [x] 1. Sweep TODO/FIXME markers and capture pre-translation baseline
- Run `rg 'TODO|FIXME' frontend/src/` and record all matches with file/line; for each, note whether the description is in Chinese (in scope for translation) or already English (out of scope per Boundary Commitments).
- Capture the pre-translation ripgrep baseline so the verification command output can be compared after the translation pass.
- Observable completion: a working note (not committed) listing every TODO/FIXME in `frontend/src/`, classified as "translate-and-tag", "already-tagged", or "English-out-of-scope", and a recorded count of files matching `[\x{4e00}-\x{9fff}]` in `frontend/src/` (expected: 20 files, ~902 occurrences).
- _Requirements: 3.1, 3.2, 3.3, 5.1_
## Core Translation Pass
- [x] 2. Translate light-touch files (≤10 hits)
- Translate Chinese comments to English in `App.vue`, `store/pendingUpload.js`, `views/MainView.vue`, `views/InteractionView.vue`, `views/ReportView.vue`, `components/Step1GraphBuild.vue`, `api/index.js`, `api/graph.js`, `api/report.js`. Apply the region-eligibility matrix from design.md: translate line/block/JSDoc/template comments; preserve JSDoc tag syntax; delete comments that merely restate the next statement; keep comments that encode intent.
- Translate Chinese content inside `console.error|warn|log` string literals in `components/Step1GraphBuild.vue` (3 known hits at lines 216, 237, 241). Leave all other string literals unchanged.
- For any TODO/FIXME marker that was Chinese and lacked a ticket reference, append `(#9)`; preserve existing references.
- Observable completion: `rg '[\x{4e00}-\x{9fff}]' frontend/src/{App.vue,store,views/MainView.vue,views/InteractionView.vue,views/ReportView.vue,components/Step1GraphBuild.vue,api}` returns no matches (no retained-bilingual cases expected in this group).
- _Requirements: 1.1, 1.2, 1.3, 1.4, 2.1, 2.2, 2.3, 2.4, 3.1, 3.2, 3.3, 4.2, 4.4, 5.4_
- [x] 3. Translate mid-touch files (10–60 hits)
- Translate `api/simulation.js` (29 hits, JSDoc-heavy: keep `@param`, `@returns`, etc., translate only natural-language content), `views/SimulationRunView.vue` (18 hits), `views/SimulationView.vue` (22 hits), `views/Home.vue` (43 hits), `components/Step5Interaction.vue` (34 hits), `components/Step3Simulation.vue` (52 hits).
- In `components/Step5Interaction.vue`, retain Chinese inside the LLM prompt template strings (around lines 725–727) per Requirement 1.5; record file/line in a working note for the PR description. Translate all comments and any non-LLM-prompt Chinese content in this file as normal.
- For any other Chinese-content string literal encountered in this group, leave the literal unchanged and record file/line for the PR description.
- Observable completion: `rg '[\x{4e00}-\x{9fff}]' <files-in-group>` returns matches only for `components/Step5Interaction.vue` (LLM prompt strings) and any other documented retained-bilingual literals; no comment-region match remains in any of these files.
- _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 2.1, 2.2, 2.3, 2.4, 3.1, 3.2, 3.3, 4.2, 4.4, 5.4_
- [x] 4. Translate high-touch files (>60 hits)
- Translate `components/Step2EnvSetup.vue` (76 hits), `components/GraphPanel.vue` (84 hits, mixed D3 logic comments and `<template>` comments), `components/HistoryDatabase.vue` (124 hits), `components/Step4Report.vue` (176 hits), `views/Process.vue` (191 hits, the 2067-line workflow orchestrator).
- These files concentrate ~80% of total occurrences; budget time accordingly. Apply the same region-eligibility matrix as task 2: translate comments, preserve JSDoc tag syntax, delete redundant comments, keep intent-bearing ones.
- Translate `console.*` Chinese strings if encountered; leave LLM prompts and other string literals unchanged and record for the PR description.
- Observable completion: `rg '[\x{4e00}-\x{9fff}]' frontend/src/components/{Step2EnvSetup,GraphPanel,HistoryDatabase,Step4Report}.vue frontend/src/views/Process.vue` returns no comment-region matches; any residuals are documented retained-bilingual string literals.
- _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 2.1, 2.2, 2.3, 2.4, 3.1, 3.2, 3.3, 4.2, 4.4, 5.4_
## Integration & Validation
- [x] 5. Run final acceptance verification
- Run `rg '[\x{4e00}-\x{9fff}]' frontend/src/` on the full directory and confirm output is empty or contains only the pre-recorded retained-bilingual files (LLM prompt strings in `components/Step5Interaction.vue` and any others documented during tasks 2–4).
- Run `npm run build` and confirm exit code 0 and successful Vite build output.
- Run `git diff --stat main..HEAD` (or against the branch base) and confirm only `frontend/src/**` paths are modified — no edits under `/locales/`, `/backend/`, or repo root.
- Observable completion: all three checks pass; if any check fails, return to the relevant translation task before proceeding to PR.
- _Requirements: 1.1, 4.1, 4.2, 5.4_
- [x] 6. Manual UI smoke check
- Run `npm run dev`; in a browser, navigate Home → Process → each Step component (1–5) → Interaction → Report; confirm rendering matches the pre-translation baseline (no missing text, no broken bindings, no console errors that did not exist before).
- Per `tech.md` steering, the manual smoke is the only practical proof that no executable change crept in; type-check or build pass alone is not sufficient.
- Observable completion: every page renders identically to baseline; no new console errors; the implementer can confirm "UI unchanged" in the PR description.
- _Requirements: 4.3_
- [x] 7. Compose PR description with verification artifacts
- Draft the PR body listing: (a) the verification command `rg '[\x{4e00}-\x{9fff}]' frontend/src/` and its post-translation output, (b) the file count and any retained-bilingual files with one-line rationale per Requirement 5.2, (c) confirmation that the manual UI smoke passed, (d) confirmation that no files outside `frontend/src/` were modified.
- Use branch name `docs/i18n-9-translate-frontend-comments` and commit message `docs(i18n): translate chinese comments in frontend src to english` per Requirement 5.3 and the project's Conventional Commits rule.
- Observable completion: the PR description, branch name, and commit subject are ready to use by `/done`; all five Requirement 5 acceptance criteria are visibly satisfied in the PR body.
- _Requirements: 5.1, 5.2, 5.3, 5.4_

517
.kiro/specs/i18n-locale-parity-guard/design.md Normal file
View File

@ -0,0 +1,517 @@
# Design — i18n-locale-parity-guard
## Overview
This feature extends the project's PR-time i18n CI guard so that any pull request which introduces a key in only one of `locales/en.json` / `locales/zh.json` fails. It satisfies acceptance criterion #4 of epic #11 (locale-key parity) with a permanent automated check.
**Purpose**: Lock in locale-catalogue key parity as a permanent CI invariant so that AC #4 of epic #11 cannot regress as new strings are added.
**Users**: Project maintainers and PR authors. Maintainers gain a hard regression gate; PR authors gain a script they can run locally to confirm parity before pushing.
**Impact**: Adds a third check to the existing PR-time guard `scripts/ci/i18n_cjk_guard.py`. No production source under `backend/app/`, `frontend/src/`, or `locales/` is modified by this spec.
### Goals
- Fail any PR whose flattened-key set in `locales/en.json` differs from that of `locales/zh.json`.
- Print actionable failure lines (`<file>:<line>: parity-<en|zh>-only: <dotted-key>`) and a summary count.
- Compose with the existing CJK-clean and per-path-ratchet checks in a single CLI invocation, with a single exit code, no short-circuit.
- Run end-to-end in well under one second on the live catalogues; stdlib-only.
- Pass on `main` at the moment this spec ships (live catalogues are already parity-clean).
### Non-Goals
- Re-implementing the manual audit pipeline at `.kiro/specs/i18n-e2e-english-verification/audit/scripts/`. The new check is the CI extract; the audit retains its own copy of `check_parity.py`.
- Cross-locale value-equality, identical-value heuristics, or ICU-placeholder-shape checks.
- Auto-creating missing keys, suggesting translations, or reformatting the catalogues.
- Modifying the `locales/` schema, the `vue-i18n` runtime, or `backend/app/utils/locale.py`.
- Adding a new GitHub Actions workflow or workflow step.
## Boundary Commitments
### This Spec Owns
- The new parity-check helpers (`_flatten_keys`, `_locate_key_line`, `_format_parity_finding`, `run_parity_check`) and constants (`ZH_JSON_REL_PATH`) inside `scripts/ci/i18n_cjk_guard.py`.
- The new third block of `run_check` that invokes `run_parity_check` and integrates its result into the existing `failed` accumulator and `success_summary` collector.
- The pass/fail semantics of the locale-key parity check.
- New unit / integration tests under `scripts/ci/tests/` covering the parity check and its composition.
### Out of Boundary
- The audit pipeline at `.kiro/specs/i18n-e2e-english-verification/audit/scripts/check_parity.py` (independent, manual-only).
- The structure or format of the baseline file `.kiro/specs/i18n-ci-guard/baseline.txt` (parity is binary; no baseline needed).
- The workflow file `.github/workflows/i18n-cjk-guard.yml` (unchanged; same `python scripts/ci/i18n_cjk_guard.py` invocation already covers the new check).
- Any change to `locales/en.json` or `locales/zh.json` content.
- Open follow-up issues #7, #23, #25 (out-of-scope translation work).
### Allowed Dependencies
- Python ≥3.11 standard library (`json`, `os`, `pathlib`, `re`, `subprocess`, `sys`, `argparse`, `unittest`).
- The existing helpers `_flatten`, `_value_line_number`, `_truncate`, the `EN_JSON_REL_PATH` constant, and the `run_check`/`update_baseline` functions in `scripts/ci/i18n_cjk_guard.py`.
- `git` (for the existing CJK-counting block, untouched here).
### Revalidation Triggers
- Adding a third locale catalogue → parity becomes pairwise; design must be revisited.
- Changing the `flatten` contract (e.g. encoding non-dict containers like lists) → the parity check's "exact match with `check_parity.py`" clause must be re-asserted against the new contract.
- Splitting the guard into multiple CLI scripts → Requirement 3 ("one invocation") must be re-anchored.
## Architecture
### Existing Architecture Analysis
The guard is a single-file Python CLI: `scripts/ci/i18n_cjk_guard.py` (~393 lines, stdlib-only) invoked by one workflow step in `.github/workflows/i18n-cjk-guard.yml`. Its `run_check(repo_root, baseline_path) -> int` function is the orchestrator; today it composes two checks without short-circuit:
1. `scan_locale_cjk(en_json_path)` — fail when `locales/en.json` contains any CJK character.
2. Per-path baseline ratchet — fail when `count_path_cjk(repo_root, p)` exceeds `read_baseline(...)[p]` for any `p` in `("backend/app", "frontend/src")`.
A `failed: bool` accumulator is set independently by each block; a `success_summary: list[str]` collects "OK …" lines that print only on full success. This design extends it with a third block.
The audit pipeline at `.kiro/specs/i18n-e2e-english-verification/audit/scripts/check_parity.py` already implements the algorithm we need (recursive `flatten` + symmetric difference). Its logic is the canonical reference for Requirement 1.1.
### Architecture Pattern & Boundary Map
```mermaid
graph TB
Workflow[GitHub Actions step]
Main[main entry]
UpdateBaseline[update_baseline]
RunCheck[run_check orchestrator]
CjkClean[scan_locale_cjk]
Ratchet[count_path_cjk + read_baseline]
Parity[run_parity_check NEW]
EnJson[locales en.json]
ZhJson[locales zh.json]
BaselineFile[baseline.txt]
Workflow --> Main
Main -->|--update-baseline| UpdateBaseline
Main --> RunCheck
RunCheck --> CjkClean
RunCheck --> Ratchet
RunCheck --> Parity
CjkClean --> EnJson
Ratchet --> BaselineFile
Parity --> EnJson
Parity --> ZhJson
```
**Architecture Integration**:
- **Selected pattern**: Composed checks inside a single orchestrator (`run_check`). Each check is an independent function that returns a pass/fail signal and a list of human-readable lines; the orchestrator accumulates them.
- **Domain/feature boundaries**: Parity logic is internal to the guard module. It does not depend on the audit pipeline, the per-path ratchet, or the locale runtime.
- **Existing patterns preserved**: No-short-circuit composition, stderr-for-failure / stdout-for-success, lexicographic ordering for determinism, atomic-write / tmp-rename for any new persistence (none added here).
- **New components rationale**: `run_parity_check` is the only new orchestrator-level function; small private helpers (`_flatten_keys`, `_locate_key_line`, `_format_parity_finding`) keep `run_parity_check`'s body short and individually testable.
- **Steering compliance**: Stdlib-only; explicit type hints (PEP 604 union syntax already in use in this module); single-responsibility helpers; module dependency direction unchanged (still no imports from `backend/`, `frontend/`, or `locales/` runtime code).
### Technology Stack
| Layer | Choice / Version | Role in Feature | Notes |
|-------|------------------|-----------------|-------|
| Backend / Services | n/a | n/a | This is a CI tool; no backend or service code is touched. |
| Infrastructure / Runtime | Python 3.11 stdlib (`json`, `pathlib`, `re`, `subprocess`, `sys`, `argparse`); GitHub Actions `ubuntu-latest`; `actions/checkout@v4`; `actions/setup-python@v5` | Runtime for the guard script and its new parity check. | Versions match the existing guard. No new dependencies; `pyproject.toml` and CI image unchanged. |
| Test Tooling | Python `unittest` (stdlib) | Drives parity check unit + integration tests. | Same framework as existing tests in `scripts/ci/tests/test_i18n_cjk_guard.py`. |
## File Structure Plan
### Directory Structure
```
scripts/
└── ci/
├── i18n_cjk_guard.py # Extended: adds parity helpers + third block in run_check
└── tests/
└── test_i18n_cjk_guard.py # Extended: adds ParityCheckTests + composition test
```
### Modified Files
- `scripts/ci/i18n_cjk_guard.py`
- Add module-level constants: `ZH_JSON_REL_PATH = "locales/zh.json"`.
- Add private helpers: `_flatten_keys`, `_locate_key_line`, `_format_parity_finding`.
- Add public function: `run_parity_check(repo_root: Path) -> ParityResult`.
- Add a new `NamedTuple` (or `@dataclass(frozen=True, slots=True)`) `ParityResult` with fields `(passed: bool, failure_lines: list[str], success_summary: str | None)`.
- Edit `run_check`: insert the parity block after the per-path-ratchet block, before the final `if not failed: print(success_summary)` block. Match the existing accumulator idiom.
- Update the module docstring to list three checks.
- `scripts/ci/tests/test_i18n_cjk_guard.py`
- Extend `_make_full_repo` (or add a sibling `_make_full_repo_with_zh`) to write a `locales/zh.json` alongside the existing `locales/en.json`. Keep the default ZH a parity-clean mirror of the EN fixture so existing tests do not need to change semantically.
- Add new test class `ParityCheckTests` covering Requirements 1.1, 1.2, 1.3, 1.4, 1.5, 2.1, 2.2, 2.3, 2.5.
- Add one composition test (Requirement 5.1.f) inside `RunCheckEndToEndTests` (or a new `RunCheckCompositionTests` class) that plants a CJK string and a parity divergence in the same repo and asserts both failure lines + exit 1.
- Update existing `RunCheckEndToEndTests.test_*` to either commit a parity-clean `locales/zh.json` or assert the parity check now also runs but does not flip the test outcome.
### Files Not Created
- No new source file is created. Option C (separate `locale_parity.py` helper module) was rejected in `gap-analysis.md` and `research.md`.
- No new workflow file. The existing `.github/workflows/i18n-cjk-guard.yml` is invoked unchanged.
## Requirements Traceability
| Requirement | Summary | Components | Interfaces | Flows |
|-------------|---------|------------|------------|-------|
| 1.1 | Flatten EN/ZH into matching dotted-key sets | `i18n_cjk_guard._flatten_keys` (new), reuses `_flatten` | `_flatten_keys(data: dict) -> set[str]` | n/a |
| 1.2 | Pass on identical key sets, success line includes shared count | `run_parity_check`, `run_check` | `ParityResult.success_summary` | Run-Check Composition |
| 1.3 / 1.4 | Fail on en-only or zh-only keys | `run_parity_check` | `ParityResult.passed`, `ParityResult.failure_lines` | Run-Check Composition |
| 1.5 | Dict leaves are non-leaves; scalar leaves are leaves | `_flatten_keys` (no type narrowing) | n/a | n/a |
| 2.1 | `<file>:<line>: parity-<side>-only: <key>` lines | `_format_parity_finding`, `_locate_key_line` | `_format_parity_finding(file, line, key, side) -> str` | n/a |
| 2.2 | Line-1 fallback when key not located | `_locate_key_line` | `_locate_key_line(text_lines, key) -> int` (returns 1 on miss) | n/a |
| 2.3 | Final `parity: en-only=N, zh-only=M` summary | `run_parity_check` | Last entry of `ParityResult.failure_lines` on failure | n/a |
| 2.4 | All parity output to stderr | `run_check` integration block | `print(..., file=sys.stderr)` | Run-Check Composition |
| 2.5 | Lexicographic ordering | `run_parity_check` | `sorted(...)` over symmetric difference | n/a |
| 3.1 | All checks run, no short-circuit | `run_check` (existing accumulator pattern) | `failed: bool` accumulator | Run-Check Composition |
| 3.2 / 3.3 | Single exit code: 1 on any fail, 0 otherwise | `run_check` | Returns `1 if failed else 0` | Run-Check Composition |
| 3.4 / 3.5 | `--update-baseline`, `--baseline`, `--repo-root` flags unchanged | `main`, `_build_parser` | Existing argparse surface | n/a |
| 3.6 | Workflow file unchanged | `.github/workflows/i18n-cjk-guard.yml` | n/a (no edit) | n/a |
| 4.1 | Stdlib-only | `i18n_cjk_guard` imports | No new imports | n/a |
| 4.2 | Sub-second runtime | `_flatten_keys` is O(keys); set-diff is O(keys) | n/a | n/a |
| 4.3 | Deterministic output | All sorts lexicographic | n/a | n/a |
| 5.1 (a–f) | Tests for success, en-only, zh-only, both, scalar-leaf, composition | `scripts/ci/tests/test_i18n_cjk_guard.py:ParityCheckTests` + composition test | n/a | n/a |
| 5.2 / 5.3 / 5.4 | Match existing test style; isolated fixtures; clean run on parity-clean repo | Same test file | n/a | n/a |
| 6.1 | Guard passes on live catalogues at HEAD | Manual run at implementation time | `python scripts/ci/i18n_cjk_guard.py` exit 0 | n/a |
| 6.2 | If divergence found, document in tasks.md and fix | n/a (does not trigger; live parity holds) | n/a | n/a |
## System Flows
### Run-Check Composition
```mermaid
sequenceDiagram
participant CLI as main
participant Orch as run_check
participant CjkChk as scan_locale_cjk
participant RatChk as ratchet block
participant ParChk as run_parity_check
participant Out as stderr/stdout
CLI->>Orch: run_check repo baseline
Orch->>CjkChk: scan en.json
CjkChk-->>Orch: findings list
alt findings non-empty
Orch->>Out: stderr cjk-in-en lines
Note over Orch: failed = True
else
Note over Orch: success summary append
end
Orch->>RatChk: count + read baseline
RatChk-->>Orch: regressions list
alt regressions non-empty
Orch->>Out: stderr cjk-regression lines + refresh hint
Note over Orch: failed = True
else
Note over Orch: success summary append
end
Orch->>ParChk: run parity check
ParChk-->>Orch: ParityResult
alt parity failed
Orch->>Out: stderr parity lines + parity summary
Note over Orch: failed = True
else
Note over Orch: success summary append
end
alt failed false
Orch->>Out: stdout success lines
end
Orch-->>CLI: 1 if failed else 0
```
**Key decisions**:
- The parity block is appended last so its (potentially long) failure list is contiguous in the failure stream.
- The `failed` accumulator is shared with the prior two blocks; this is the only mechanism for cross-block signalling.
- The summary line `parity: en-only=N, zh-only=M` is appended to `ParityResult.failure_lines` (last entry) so the orchestrator can print all failure lines uniformly without a special-case branch.
## Components and Interfaces
| Component | Domain/Layer | Intent | Req Coverage | Key Dependencies (P0/P1) | Contracts |
|-----------|--------------|--------|--------------|--------------------------|-----------|
| `_flatten_keys` | Guard / helper | Return the dotted-key set of a parsed JSON catalogue, mirroring `check_parity.py.flatten`. | 1.1, 1.5 | `_flatten` (P0, existing) | Service |
| `_locate_key_line` | Guard / helper | Best-effort line-number resolution for a dotted key in raw JSON text, with line-1 fallback. | 2.1, 2.2 | none | Service |
| `_format_parity_finding` | Guard / helper | Format one failure line as `<file>:<line>: parity-<side>-only: <key>`. | 2.1 | none | Service |
| `ParityResult` | Guard / DTO | Carry parity-check outcome (passed flag, failure lines, success-summary line). | 1.2, 2.3, 2.5 | none | State |
| `run_parity_check` | Guard / orchestrator-leaf | Read both catalogues, compute symmetric difference, build `ParityResult`. | 1.1–1.5, 2.1–2.5 | `_flatten_keys` (P0), `_locate_key_line` (P0), `_format_parity_finding` (P0) | Service |
| `run_check` (modified) | Guard / orchestrator | Compose the three checks with a single `failed` accumulator and exit code. | 3.1–3.3 | All three checks (P0) | Service |
| `ParityCheckTests` (test) | Tests | Unit + integration coverage for parity. | 5.1 (a–f), 5.2–5.4 | `run_parity_check`, `run_check` (P0) | Service |
### Guard / helper layer
#### `_flatten_keys`
| Field | Detail |
|-------|--------|
| Intent | Return the set of dotted-key paths of a parsed JSON object, mirroring `check_parity.py.flatten`. |
| Requirements | 1.1, 1.5 |
**Responsibilities & Constraints**
- Iterate via the existing `_flatten(prefix, value, out)` helper to guarantee identical path semantics.
- Descend only into `dict`. Any non-dict (string, number, bool, null, list) at a leaf produces a key.
- Return a `set[str]` so the parity caller can compute symmetric differences without re-deduplicating.
**Dependencies**
- Inbound: `run_parity_check` (P0).
- Outbound: `_flatten` (P0, existing private helper in same module).
**Contracts**: Service [x]
##### Service Interface
```python
def _flatten_keys(data: dict[str, object]) -> set[str]:
...
```
- Preconditions: `data` is the result of `json.loads` over a catalogue file (i.e., a `dict` at the top level).
- Postconditions: every dotted path returned corresponds to a non-`dict` leaf in `data`. The set is unordered; callers must sort before formatting output (Requirement 2.5).
- Invariants: `_flatten_keys({}) == set()`. For any catalogue `c`, `_flatten_keys(c)` is identical to the set of keys produced by `check_parity.py.flatten(c)`.
**Implementation Notes**
- Integration: One call site (`run_parity_check`).
- Validation: Unit-test against a hand-rolled fixture with mixed leaf types (string, number, bool, null) and at least three nesting levels (Requirement 5.1.e).
- Risks: None. Reuses the existing flatten primitive verbatim.
#### `_locate_key_line`
| Field | Detail |
|-------|--------|
| Intent | Best-effort line-number resolution for a dotted key in the raw JSON source text, with a deterministic line-1 fallback. |
| Requirements | 2.1, 2.2 |
**Responsibilities & Constraints**
- Accept the splitlines view of a JSON file (`text_lines: list[str]`) and a dotted key (`dotted_key: str`).
- Search for the leaf segment of the dotted key (after the last `.`) wrapped in JSON quotes, e.g. `"missingKey"`. Return the 1-based line number of the first match.
- Fall back to `1` when no match is found (mirrors `_value_line_number`).
- Performance must remain linear in the number of lines.
**Dependencies**
- Inbound: `run_parity_check` (P0).
- Outbound: none.
**Contracts**: Service [x]
##### Service Interface
```python
def _locate_key_line(text_lines: list[str], dotted_key: str) -> int:
...
```
- Preconditions: `dotted_key` non-empty; `text_lines` is the result of `Path.read_text(...).splitlines()`.
- Postconditions: returns an integer ≥ 1.
- Invariants: When the leaf segment appears in `text_lines` wrapped in `"..."`, the return is the (1-based) line number of the first occurrence. Otherwise the return is `1`.
**Implementation Notes**
- Integration: One call site (`run_parity_check`).
- Validation: Unit-test the exact-match path, the multi-occurrence path (first match wins), and the not-found fallback.
- Risks: A leaf segment that also appears as part of another (unrelated) key or in a value text could yield a slightly misleading line number. Acceptable: the dotted key in the failure message is the source of truth; the line is a navigation aid. Documented in the docstring.
#### `_format_parity_finding`
| Field | Detail |
|-------|--------|
| Intent | Format a single parity-failure line in the canonical layout used by the guard. |
| Requirements | 2.1 |
**Responsibilities & Constraints**
- Produce strings of the exact form `<file>:<line>: parity-en-only: <dotted-key>` or `<file>:<line>: parity-zh-only: <dotted-key>`.
- Mirror the existing `_format_locale_finding` style (`<file>:<line>: <category>: <payload>`).
**Dependencies**
- Inbound: `run_parity_check` (P0).
- Outbound: none.
**Contracts**: Service [x]
##### Service Interface
```python
def _format_parity_finding(file_rel_path: str, line_no: int, dotted_key: str, side: str) -> str:
...
```
- Preconditions: `side in {"en-only", "zh-only"}`; `file_rel_path` is one of `EN_JSON_REL_PATH` / `ZH_JSON_REL_PATH`; `line_no >= 1`.
- Postconditions: returns a single line with no embedded newline.
- Invariants: The category token in the line is exactly `parity-en-only` or `parity-zh-only` so log greps match deterministically.
### Guard / DTO layer
#### `ParityResult`
| Field | Detail |
|-------|--------|
| Intent | Immutable carrier for parity-check outcome consumed by `run_check`. |
| Requirements | 1.2, 2.3, 2.5 |
**Contracts**: State [x]
##### State Management
- State model:
```python
class ParityResult(NamedTuple):
passed: bool
failure_lines: list[str] # already-formatted lines, including the trailing "parity: en-only=N, zh-only=M" summary on failure
success_summary: str | None # populated only when passed is True
```
- Persistence & consistency: in-memory only; constructed by `run_parity_check` and consumed by `run_check`.
- Concurrency strategy: n/a (single-process, single-call).
### Guard / orchestrator-leaf
#### `run_parity_check`
| Field | Detail |
|-------|--------|
| Intent | Compute the locale-key parity outcome and produce a `ParityResult`. |
| Requirements | 1.1–1.5, 2.1–2.5 |
**Responsibilities & Constraints**
- Read both `locales/en.json` and `locales/zh.json` from `repo_root`.
- Flatten each via `_flatten_keys` and compute the symmetric difference.
- For each en-only key (sorted lexicographically): resolve its line via `_locate_key_line` over the EN catalogue's source-text lines, and emit a `parity-en-only` line via `_format_parity_finding`.
- For each zh-only key (sorted lexicographically, after en-only): resolve its line via `_locate_key_line` over the ZH catalogue's source-text lines, and emit a `parity-zh-only` line.
- On failure, append a final `parity: en-only=N, zh-only=M` summary line to `failure_lines`.
- On success, build the success summary `OK locale-parity: <count> keys per side`.
- If either catalogue file is missing, return a `ParityResult(passed=False, failure_lines=[<single error line>], success_summary=None)` and let `run_check` fold the error into the global `failed` flag.
**Dependencies**
- Inbound: `run_check` (P0).
- Outbound: `_flatten_keys`, `_locate_key_line`, `_format_parity_finding` (all P0).
**Contracts**: Service [x]
##### Service Interface
```python
def run_parity_check(repo_root: Path) -> ParityResult:
...
```
- Preconditions: `repo_root` is a valid working-tree directory; `locales/en.json` and `locales/zh.json` are expected at the relative paths defined by `EN_JSON_REL_PATH` and `ZH_JSON_REL_PATH`.
- Postconditions: returns a `ParityResult`. When `passed`, `failure_lines == []` and `success_summary` is non-`None`. When not `passed`, `failure_lines` is non-empty and ends with a `parity: en-only=…` summary line; `success_summary` is `None`.
- Invariants: Flattened-key-set computation matches `check_parity.py.flatten` byte-for-byte for any input. Output is deterministic across runs for identical inputs.
**Implementation Notes**
- Integration: Called once per `run_check` invocation. Skipped entirely in `--update-baseline` mode (covered by Requirement 3.4 — `update_baseline` is invoked from `main` instead of `run_check`).
- Validation: Unit-test all required outcomes (Requirement 5.1 a–e); integration-test composition (5.1 f).
- Risks: A malformed JSON catalogue raises `json.JSONDecodeError`. The function should treat this the same as a missing file (return `ParityResult(passed=False, …)`), so the guard reports a clean failure rather than crashing CI with a Python traceback.
### Guard / orchestrator (modified)
#### `run_check` (modification)
| Field | Detail |
|-------|--------|
| Intent | Compose all three checks (CJK-clean, per-path ratchet, parity) into one exit code. |
| Requirements | 3.1, 3.2, 3.3 |
**Responsibilities & Constraints**
- After the existing per-path-ratchet block (existing line ~258–293) and before the final `if not failed` block (existing line ~295–298), call `run_parity_check(repo_root)`.
- If the result is not passed, set `failed = True`, print every entry of `result.failure_lines` to `sys.stderr`, one line per `print(...)` call.
- If passed, append `result.success_summary` to `success_summary`.
- Return `1 if failed else 0` (unchanged).
**Dependencies**
- Inbound: `main` (P0, via either standalone CLI or test invocation).
- Outbound: `scan_locale_cjk`, per-path ratchet helpers, `run_parity_check` (all P0).
**Contracts**: Service [x] / State [x]
##### Service Interface
Unchanged signature: `def run_check(repo_root: Path, baseline_path: Path) -> int`.
- Preconditions: unchanged.
- Postconditions: exit code reflects all three checks (was: two checks).
- Invariants: still no short-circuit between checks.
**Implementation Notes**
- Integration: One inserted block of ~10 lines in the existing function.
- Validation: Existing CLI smoke tests continue to pass; new `RunCheckEndToEndTests` cases assert correct fail/pass propagation when only the parity check fails, only an existing check fails, or both fail.
- Risks: A future maintainer could accidentally short-circuit by inserting an early `return` between blocks. Mitigated by the composition test (Requirement 5.1.f) which fails if any block is skipped.
### Tests
#### `ParityCheckTests`
| Field | Detail |
|-------|--------|
| Intent | Unit + integration coverage for the parity check, matching the style of existing `RunCheckEndToEndTests`. |
| Requirements | 5.1 (a–f), 5.2, 5.3, 5.4 |
**Responsibilities & Constraints**
- Use `unittest`, `tempfile.TemporaryDirectory`, and the existing `_make_repo` / `_commit_file` test helpers.
- Each test owns its own ephemeral repo. No reliance on the live `locales/` content for negative paths (Requirement 5.3).
- Assertions check exit code AND substring presence of the failure category tokens (`parity-en-only`, `parity-zh-only`) AND that the summary line is the last failure line.
**Dependencies**
- Inbound: `unittest.main`.
- Outbound: `i18n_cjk_guard.run_parity_check`, `i18n_cjk_guard.run_check` (both P0).
**Implementation Notes**
- Test cases (one per Requirement 5.1 sub-bullet):
- (a) `test_passes_when_keys_match` — both catalogues identical → `run_parity_check` returns `passed=True`; `run_check` returns 0.
- (b) `test_fails_on_en_only_key` — `en.json` has an extra key → `run_parity_check` returns `passed=False`, failure includes `parity-en-only`, summary is `parity: en-only=1, zh-only=0`.
- (c) `test_fails_on_zh_only_key` — symmetric of (b).
- (d) `test_fails_on_both_sided_divergence` — failure list contains both `parity-en-only` and `parity-zh-only` lines, ordered en-first then zh, each lex-sorted within its group.
- (e) `test_passes_with_scalar_leaves_at_same_path` — both catalogues have a scalar (e.g. `null`, `42`, `false`) at the same dotted path → parity passes (Requirement 1.5).
- (f) `test_run_check_no_short_circuit` — one repo plants both a CJK in `en.json` and a parity-divergent key. Expect: exit 1; stderr contains both `cjk-in-en` and `parity-en-only` (or `parity-zh-only`); the per-path-ratchet success summary is suppressed (since failed).
- Risks: Test fixtures must use `ensure_ascii=False` JSON to match the live catalogue style.
## Error Handling
### Error Strategy
- **Missing catalogue file** → `run_parity_check` returns `ParityResult(passed=False, failure_lines=[<missing-file-line>], success_summary=None)`. `run_check` flips `failed`, prints the line to stderr, returns 1.
- **Malformed JSON** → same path as missing catalogue. `json.JSONDecodeError` is caught inside `run_parity_check`; the line printed names the offending file and the parser's `msg`.
- **Parity divergence** (the expected unhappy path) → fail per Requirements 1.3 / 1.4 / 2.1–2.5.
- **`_locate_key_line` cannot find the key** → fall back to line 1 (Requirement 2.2). Not an error; the caller proceeds.
- **No-short-circuit invariant** → enforced by the orchestrator's accumulator pattern; covered by Requirement 5.1.f.
### Monitoring
CI workflow logs (GitHub Actions) are the sole observability surface. Failure lines are designed to be greppable: `parity-en-only`, `parity-zh-only`, `parity: en-only=`, `parity: zh-only=` are stable tokens.
## Testing Strategy
### Unit Tests
- `_flatten_keys`: empty input, flat input, mixed-type leaves, three-level nesting, `null` and scalar leaves.
- `_locate_key_line`: exact match, multi-occurrence (first wins), not found (line-1 fallback).
- `_format_parity_finding`: en-only and zh-only sides, embedded special characters in key names (e.g. underscores, digits).
- `ParityResult`: pass-shape and fail-shape construction.
### Integration Tests
- All six `ParityCheckTests` sub-cases listed above.
- The composition case (Requirement 5.1.f) inside `RunCheckCompositionTests` (or appended to `RunCheckEndToEndTests`).
- A regression of the existing `RunCheckEndToEndTests` cases after extending `_make_full_repo` to write a default parity-clean `locales/zh.json`.
### Performance / Load
- One sanity case: parity check on a synthetic 10 000-key catalogue completes in well under one second on the CI runner. Asserted by a `time.perf_counter()` budget of 1.0 s in the integration test.
## Performance & Scalability
- Catalogue size: ~1000 keys today; growth bounded by the number of UI strings + log keys. Even at 10× the current size, `_flatten` + set-diff remains negligible (<100 ms).
- The CI workflow timeout is 1 minute (`.github/workflows/i18n-cjk-guard.yml:timeout-minutes: 1`); the new check adds at most tens of milliseconds.
## Supporting References
- `gap-analysis.md` (this spec) — implementation-approach options A/B/C with rationale.
- `research.md` (this spec) — design decision records.
- `.kiro/specs/i18n-ci-guard/design.md` — prior CI guard's design doc (style and boundary precedents).
- `.kiro/specs/i18n-e2e-english-verification/audit/scripts/check_parity.py` — reference parity algorithm.

154
.kiro/specs/i18n-locale-parity-guard/gap-analysis.md Normal file
View File

@ -0,0 +1,154 @@
# Gap Analysis — i18n-locale-parity-guard
## Current State Investigation
### Domain assets
| Asset | Path | Role |
|------|------|------|
| Existing PR-time guard | `scripts/ci/i18n_cjk_guard.py` (393 lines) | Runs (a) zero-CJK-in-`en.json`, (b) per-path CJK ratchet on `backend/app` + `frontend/src`. CLI: `--update-baseline`, `--baseline`, `--repo-root`. Stdlib-only. |
| Workflow | `.github/workflows/i18n-cjk-guard.yml` | `pull_request` trigger; single step `python scripts/ci/i18n_cjk_guard.py`. 1-minute timeout. Python 3.11. |
| Existing tests | `scripts/ci/tests/test_i18n_cjk_guard.py` (358 lines) | `unittest`, stdlib-only. Per-function test classes (`ScanLocaleCjkTests`, `CountPathCjkTests`, `BaselineRoundTripTests`, `RunCheckEndToEndTests`, `UpdateBaselineTests`, `CliSmokeTests`). Synthetic git repos via `tempfile.TemporaryDirectory` + `git init`. |
| Reference parity logic | `.kiro/specs/i18n-e2e-english-verification/audit/scripts/check_parity.py` (128 lines) | Already implements `flatten()` (recursive dotted-key generator) and the EN/ZH symmetric-difference computation. Used only by the manual audit pipeline; not in CI. |
| Locale catalogues | `locales/en.json`, `locales/zh.json` | Two-space-indented JSON, `ensure_ascii=False`. 962 keys per side at HEAD; symmetric difference 0. Multi-level nesting (e.g. `common.confirm`, `step1.upload.title`, `log.api.graph.startBuild`). |
| Prior spec | `.kiro/specs/i18n-ci-guard/{design.md,baseline.txt}` | Documents the CJK-guard's design, format, and "scope ratchets only" rationale. The new check should compose, not replace. |
### Conventions extracted
- **Module layout**: One CLI script per check class; checks compose inside a `run_check(...)` orchestrator that returns 0/1.
- **Output discipline**: Stderr for failures, stdout for success summaries. Each failure line is self-contained (`<file>:<line>: <category>: <key/payload>`). Refresh hints (when applicable) printed once at the end.
- **No-short-circuit composition**: `run_check` evaluates every check before exiting (existing pattern at lines 230, 258, 271 in `i18n_cjk_guard.py`).
- **Stdlib-only, deterministic**: existing module imports only `argparse`, `json`, `os`, `re`, `subprocess`, `sys`, `pathlib`. All sorts use lexicographic order.
- **Test-fixture isolation**: Each test owns a `tempfile.TemporaryDirectory()` and writes its own JSON / source files. Negative-path tests never depend on the live `locales/`.
- **Atomic writes**: `write_baseline` uses tmp-file + `os.replace`; if any new persistence is added, mirror that pattern.
- **JSON line-resolution helper**: `_value_line_number(text_lines, value)` already implements the line-fallback semantics required by R2.2 (returns 1 when value not found). Reusable for parity reporting if we resolve by **key name** rather than by **value**.
### Integration surfaces
- The workflow file invokes the guard exactly once: `python scripts/ci/i18n_cjk_guard.py`. Anything done inside `run_check` is automatically picked up — **no workflow change needed** if we extend the existing script (R3.6).
- `--update-baseline` short-circuits inside `main()` *before* `run_check` is called; the new parity check naturally won't run in that mode (R3.4).
- The audit pipeline at `.kiro/specs/i18n-e2e-english-verification/audit/scripts/check_parity.py` is independent and stays untouched (R6's "spec for prior CI guard" boundary).
- Baseline file format is single-purpose (CJK counts) and does not need to grow to accommodate parity (parity has no baseline — divergence is binary).
## Requirement-to-Asset Map
| # | Requirement | Existing asset(s) | Gap tag | Notes |
|---|-------------|------------------|---------|-------|
| 1.1 | Flatten EN/ZH into dotted keys matching `check_parity.py` | `audit/scripts/check_parity.py:flatten` (reference); existing `_flatten` in guard also flattens but only collects (key, value) pairs into a list | **Constraint** | Two `_flatten` flavours exist. Need ONE canonical function inside the guard module that mirrors `check_parity.py.flatten` (recursive, descends into dicts only, emits leaf scalars). The existing private `_flatten(prefix, value, out)` in the guard is already key-value-emitting and will work; the parity check just consumes its keys. |
| 1.2 | Pass when key sets identical, emit success summary with key count | `success_summary` list in `run_check` | **Missing** | Add a parity success line in the same idiom: `"OK locale-parity: 962 keys per side"`. |
| 1.3 / 1.4 | Fail on en-only or zh-only keys | None — no parity check exists | **Missing** | Compute symmetric difference. |
| 1.5 | Treat dict leaves as non-leaves; treat scalar leaves the same as string leaves for parity | `_flatten` already descends only into dicts and emits any non-dict as a leaf; `scan_locale_cjk` then narrows to strings, but parity should NOT narrow | **Constraint** | Use `_flatten` directly (no narrowing). |
| 2.1 | Print `<file>:<line>: <key>: en-only|zh-only` | `_value_line_number` resolves a value's line; needs adaptation for keys | **Missing** | Search for the JSON key token (e.g. `"missingKey"`) in the source-text lines using a substring scan; reuse the line-1 fallback from `_value_line_number`. |
| 2.2 | Fall back to line 1 when location not found | `_value_line_number` already returns 1 in this case | **Reuse** | |
| 2.3 | Final summary `parity: en-only=<n>, zh-only=<m>` | None | **Missing** | One line, stderr. |
| 2.4 | All parity output to stderr | `print(..., file=sys.stderr)` pattern used everywhere | **Reuse** | |
| 2.5 | Lexicographic sort | Existing patterns use `sorted(...)` | **Reuse** | |
| 3.1 / 3.2 / 3.3 | Compose with existing checks; one exit code | `run_check` already composes (a) and (b) without short-circuit | **Constraint** | Insert (c) at the end of `run_check`, after the per-path block but before the final return. Each check toggles the same `failed` flag. |
| 3.4 | `--update-baseline` does not run parity | `main()` short-circuits to `update_baseline()` and never enters `run_check` | **Reuse** | Untouched. |
| 3.5 | `--baseline` and `--repo-root` semantics unchanged | `_build_parser` and `_detect_repo_root` | **Reuse** | Untouched. |
| 3.6 | Workflow file unchanged | `.github/workflows/i18n-cjk-guard.yml` | **Reuse** | No edit needed. |
| 4.1 | Stdlib-only | Existing module is stdlib-only | **Reuse** | `json` is the only library needed for ZH loading. |
| 4.2 | Sub-second runtime | ~1k keys; flatten + set diff is O(n) | **Constraint** | Trivially holds. |
| 4.3 | Deterministic output | All sorts lexicographic | **Reuse** | |
| 5.1–5.4 | Tests under `scripts/ci/tests/` for success / en-only / zh-only / both / scalar-leaves / no-short-circuit | `test_i18n_cjk_guard.py:RunCheckEndToEndTests` is the integration class | **Missing** | Add either a new `ParityCheckTests` class or extend `RunCheckEndToEndTests`. Reuse `_make_full_repo` style; need a `zh_json` argument or a new helper that writes both locale files. |
| 6.1 | Guard passes on live catalogues at merge target | EN/ZH parity verified manually (962/962, 0 diff) | **Reuse** | Manual run after implementation. |
| 6.2 | Document any blocking divergence in tasks.md | n/a | **Conditional** | Only relevant if 6.1 fails — currently does not. |
### Complexity signal
- **Algorithmic logic** only: load two JSON files, recursive flatten, set diff, sort, format, print. No external integrations, no I/O contention, no perf concerns at the catalogue size.
## Implementation Approach Options
### Option A — Extend `scripts/ci/i18n_cjk_guard.py` *(recommended)*
**What changes**:
- Add private helpers to the existing module:
- `_flatten_keys(data) -> set[str]` — wrapper over the existing `_flatten` that returns just the dotted-key set.
- `_locate_key_line(text_lines, dotted_key) -> int` — substring scan for the leaf segment (after the last `.`) wrapped in JSON quotes; returns 1 on miss (mirrors `_value_line_number`'s fallback).
- `_format_parity_finding(file_rel_path, line_no, dotted_key, side) -> str` — single-line formatter.
- Add a function `run_parity_check(repo_root) -> tuple[bool, list[str], str]` returning `(passed, failure_lines, success_summary_line)`. Callable independently for tests.
- In `run_check`, after the per-path baseline block and before the final return:
- Call `run_parity_check(repo_root)`.
- If failed, set `failed = True`, print all failure lines + the `parity: ...` summary to stderr.
- If passed, append the success line to `success_summary`.
- Add a `ZH_JSON_REL_PATH` constant alongside `EN_JSON_REL_PATH`.
**Compatibility assessment**:
- All existing CLI flags, exit codes, and stdout/stderr patterns preserved.
- No new top-level dependencies. `json` already imported.
- The module grows to ~470 lines, comparable to similar single-purpose CLI scripts in the repo (`oasis_profile_generator.py` is much larger). Single-responsibility is preserved: the responsibility is "PR-time i18n catalogue health," and parity is a sub-instance of that.
- Existing tests continue to pass unmodified (none of the changed functions break their contract).
**Trade-offs**:
- ✅ Zero workflow churn, single CI job, single CLI surface.
- ✅ Reuses `_flatten`, line-resolution fallback, sort/print idioms.
- ✅ All checks fail/pass together — easier to read in CI logs.
- ❌ Module name (`i18n_cjk_guard`) is now slightly misleading: it also enforces parity, not just CJK presence. Mitigated by docstring update.
### Option B — New parallel script `scripts/ci/i18n_locale_parity_guard.py` + new workflow step
**What changes**:
- New script that implements the parity check standalone.
- Either (i) add a second job to `.github/workflows/i18n-cjk-guard.yml`, or (ii) add a new workflow file `i18n-locale-parity-guard.yml`.
- New test file `scripts/ci/tests/test_i18n_locale_parity_guard.py`.
**Compatibility assessment**:
- Both scripts duplicate `_flatten`, line-resolution helper, JSON loader, repo-root detection, argparse boilerplate.
- Two CI runs (or two steps) to read and ack on every PR.
**Trade-offs**:
- ✅ Single-responsibility script per file (matches one literal reading of project conventions).
- ❌ Code duplication ~80 lines.
- ❌ Two CI surfaces; PR review fatigue.
- ❌ Violates the spirit of R3 ("compose with the existing checks") — composing across two scripts requires either `&&` or two-job aggregation.
### Option C — Hybrid: new helper module + extended guard
**What changes**:
- New module `scripts/ci/locale_parity.py` exposing `compute_parity_findings(en_path, zh_path) -> ParityResult`.
- The existing `i18n_cjk_guard.py` imports from it and integrates the call into `run_check`, identical to Option A's runtime behaviour.
- Tests split: `test_locale_parity.py` covers the helper in isolation; `test_i18n_cjk_guard.py` gains one composition test.
**Compatibility assessment**:
- Adds package-style imports inside `scripts/ci/` (currently flat — `scripts/ci/i18n_cjk_guard.py` adds `_GUARD_DIR` to `sys.path` via the test bootstrap, which works for sibling modules without further config).
- No workflow change.
**Trade-offs**:
- ✅ Clean separation, more reusable helper.
- ✅ Possible to import the helper from the audit pipeline later (collapsing the duplicate `check_parity.py`).
- ❌ More files for what is ~80 lines of new logic; over-engineering for current scope.
- ❌ Risks scope creep into "deduplicate `check_parity.py`," which is explicitly out of scope.
## Effort & Risk
- **Effort**: **S** (1–2 days). Existing module patterns are mature; the algorithmic logic is small and proven (`check_parity.py`); test scaffolding is already in place.
- **Risk**: **Low**. Stdlib-only; no external integrations; no shared mutable state; deterministic algorithm; existing CI workflow unchanged; live catalogues already pass.
## Recommendations for Design Phase
### Preferred approach: Option A (extend `scripts/ci/i18n_cjk_guard.py`)
Rationale:
1. The existing module's docstring already says "PR-time guard: fail when locales/en.json contains CJK or when backend/app + frontend/src CJK match counts exceed the committed baseline." Extending it to also fail on locale-key parity is the smallest possible delta that also reads naturally in the codebase.
2. R3 ("composes with the existing CJK and per-path checks; one CLI; no workflow edit") is satisfied trivially.
3. Reuses `_flatten`, line-fallback, sort/print idioms verbatim.
4. The module name remains accurate — "CJK Guard" is the canonical name of the i18n PR-time gate; we'll add a docstring note that parity is the third covered check.
### Key design decisions to settle in `design.md`
- **Function boundary**: should `run_parity_check` live in the same module or in a small helper module? *Suggest: same module, as a private function alongside `count_path_cjk` / `scan_locale_cjk` for symmetry.*
- **Failure line format**: exact string layout (file:line:key:side, ordering of the four pieces, separator characters). *Suggest mirroring `_format_locale_finding` exactly: `f"{file}:{line}: {category}: {key}"` where `category` is `parity-en-only` or `parity-zh-only`.*
- **Test fixture for `RunCheckEndToEndTests`**: extend `_make_full_repo` to accept an optional `zh_json` parameter, or add a sibling helper. *Suggest extending — keeps the integration test in one place and lets the existing tests opt out by passing `zh_json=None` (the helper writes a parity-clean default).*
- **Whether to expose a `--check=parity` selector**: *Out of scope per R3.1 (no short-circuit, all-or-nothing).*
### Research items to carry forward
None. All required information is in the existing repo and the cited reference scripts. No external dependencies, no new tech, no perf research, no security implications.

96
.kiro/specs/i18n-locale-parity-guard/requirements.md Normal file
View File

@ -0,0 +1,96 @@
# Requirements Document
## Introduction
Epic #11 ("complete english support across ui, agents, logs, and docs") states as acceptance criterion #4: *"For every externalized log message, matching `log.*` keys exist in both `locales/en.json` and `locales/zh.json`."* The wider intent is symmetric: any externalized string introduced into either locale catalogue must have a counterpart in the other, otherwise English users hit fallback keys at runtime (and the inverse for Chinese users).
Parity holds today (962 keys per side, symmetric difference 0), but no automated check enforces it. The existing CI guard at `scripts/ci/i18n_cjk_guard.py` (workflow `.github/workflows/i18n-cjk-guard.yml`, landed via #26) only enforces (1) zero CJK in `locales/en.json` and (2) a per-path CJK count ratchet for `backend/app` + `frontend/src`. The audit script at `.kiro/specs/i18n-e2e-english-verification/audit/scripts/check_parity.py` does compute the symmetric difference, but only as part of a manual audit — it never runs in CI.
This spec extends the existing PR-time CI guard to enforce locale-key parity permanently. Once shipped, any pull request that introduces a key on only one side will fail CI with a precise list of the offending keys, freezing AC #4 in place for the rest of the epic and beyond.
## Boundary Context
- **In scope**:
- Symmetric-difference check between flattened dotted-key sets of `locales/en.json` and `locales/zh.json`.
- Integration of the new check into the existing `scripts/ci/i18n_cjk_guard.py` so the existing workflow `.github/workflows/i18n-cjk-guard.yml` exercises it without any workflow edit beyond what's strictly necessary.
- Test coverage under `scripts/ci/tests/` matching the style of the existing CJK-guard tests.
- Failure output formatted so a developer can locate the offending key without further tooling.
- **Out of scope**:
- Translating any remaining hard-coded strings in `backend/app` or `frontend/src` (tracked under open assigned issues #7, #23, #25).
- Value-equality, identical-value, or "review-needed" heuristics from the audit script's `[identical-values]` block — only key presence is asserted here.
- Any change to the `locales/` directory layout, schemas, or to `vue-i18n` / `backend/app/utils/locale.py` consumers.
- Cross-locale value-shape checks (e.g. matching ICU placeholders).
- README, `.env.example`, or documentation updates beyond what's needed inside the spec / guard module itself.
- **Adjacent expectations**:
- The existing CJK-clean and per-path-ratchet checks in `scripts/ci/i18n_cjk_guard.py` continue to run unchanged and report independently of the new parity check.
- The audit pipeline at `.kiro/specs/i18n-e2e-english-verification/audit/scripts/` keeps its own copy of `check_parity.py` for manual deep-dive use; the new CI check does not depend on the audit pipeline being invoked.
- All four checks (CJK in en.json, per-path ratchet, en-only keys, zh-only keys) run in a single CI job and surface together; no short-circuit between them.
## Requirements
### Requirement 1: Locale-key parity check
**Objective:** As a maintainer of the i18n catalogues, I want a CI check that detects any key present on only one of `locales/en.json` / `locales/zh.json`, so that AC #4 of epic #11 stays satisfied as new strings are added.
#### Acceptance Criteria
1. The i18n CJK Guard shall load `locales/en.json` and `locales/zh.json` and flatten each into a set of dotted keys whose paths exactly match those produced by `flatten()` in `.kiro/specs/i18n-e2e-english-verification/audit/scripts/check_parity.py`.
2. When the flattened EN and ZH key sets are identical, the i18n CJK Guard shall pass the parity check and emit a single success summary line that includes the shared key count.
3. When the flattened EN key set contains any key that is absent from ZH, the i18n CJK Guard shall fail the parity check.
4. When the flattened ZH key set contains any key that is absent from EN, the i18n CJK Guard shall fail the parity check.
5. The i18n CJK Guard shall treat a leaf whose value is a nested object as a non-leaf (no key emitted) and shall treat a leaf whose value is a non-string scalar (number, boolean, null) the same way it treats a string leaf for parity purposes.
### Requirement 2: Actionable failure reporting
**Objective:** As a developer whose PR is failing on parity, I want the failure message to name every offending key and the side it is missing on, so that I can fix the divergence without re-running the audit pipeline.
#### Acceptance Criteria
1. If the parity check fails, then the i18n CJK Guard shall print one line per missing key in the form `<locales/en.json|locales/zh.json>:<line>: <dotted-key>: en-only` or `... zh-only`, with `<line>` being the 1-based line number of that key in the source JSON file.
2. If a missing key cannot be located in its source file (e.g. owing to JSON formatting), then the i18n CJK Guard shall fall back to line 1 and still print the offending key and side.
3. If the parity check fails, then the i18n CJK Guard shall print a final summary line of the form `parity: en-only=<n>, zh-only=<m>` where `<n>` and `<m>` are the counts of en-only and zh-only keys.
4. The i18n CJK Guard shall print all parity-related output to stderr.
5. The i18n CJK Guard shall sort each side's missing-key list lexicographically so that the failure output is deterministic across environments.
### Requirement 3: Integration with the existing guard
**Objective:** As a maintainer extending the CI guard, I want the new parity check to compose with the existing CJK-clean and per-path-ratchet checks rather than replace them, so that all four checks are visible in a single CI run.
#### Acceptance Criteria
1. The i18n CJK Guard shall execute all of (a) the CJK-clean check on `locales/en.json`, (b) the per-path baseline ratchet on `backend/app` and `frontend/src`, and (c) the new parity check on every invocation of `python scripts/ci/i18n_cjk_guard.py` without short-circuiting between checks.
2. When any of (a), (b), or (c) fail, the i18n CJK Guard shall exit with status code 1.
3. When all of (a), (b), and (c) pass, the i18n CJK Guard shall exit with status code 0.
4. The i18n CJK Guard shall continue to support the `--update-baseline` flag with its existing semantics (refresh per-path counts and exit 0); the parity check shall not run in `--update-baseline` mode.
5. The i18n CJK Guard shall continue to support the `--baseline` and `--repo-root` flags with their existing semantics.
6. The existing GitHub Actions workflow `.github/workflows/i18n-cjk-guard.yml` shall continue to invoke the guard via the same single command (`python scripts/ci/i18n_cjk_guard.py`), with no new workflow steps required.
### Requirement 4: Stdlib-only, deterministic, fast
**Objective:** As a CI operator, I want the parity check to run quickly and without new dependencies, so that the existing 1-minute job timeout still holds.
#### Acceptance Criteria
1. The i18n CJK Guard shall implement the parity check using only the Python standard library; no new package shall be added to `pyproject.toml`, `requirements*.txt`, or any other dependency manifest.
2. The i18n CJK Guard shall complete the parity check in well under one second on the current catalogue size (~1000 keys per side) under normal CI conditions.
3. The i18n CJK Guard shall produce identical output for identical inputs across runs (no timestamps, no run IDs, no nondeterministic ordering).
### Requirement 5: Test coverage
**Objective:** As a future contributor modifying the guard, I want automated tests for every parity behaviour, so that regressions in either check or in their composition are caught locally.
#### Acceptance Criteria
1. The repository shall contain unit tests under `scripts/ci/tests/` that cover at minimum: (a) the success path where EN and ZH have identical key sets, (b) an en-only-key failure, (c) a zh-only-key failure, (d) a both-sides-divergent failure, (e) a leaf-value-type-mismatch case (string vs scalar/null) that does NOT count as a parity failure, and (f) the integration case where the parity check runs alongside the existing CJK-clean and per-path-ratchet checks without short-circuiting.
2. The new tests shall use the same testing style and framework already used by the existing tests in `scripts/ci/tests/`.
3. When a new test fixture is required for a JSON file, the fixture shall live under `scripts/ci/tests/` in a self-contained form (no reliance on `locales/` content for negative-path tests).
4. When the test suite is run from the repository root, the i18n CJK Guard test module shall pass without warnings on a clean checkout where `locales/en.json` and `locales/zh.json` have full key parity.
### Requirement 6: Self-test against the live catalogues
**Objective:** As an epic-#11 closer, I want to know the moment this guard ships that it observes the live catalogues as parity-clean, so that the guard's first PR doesn't produce a false alarm.
#### Acceptance Criteria
1. While the live catalogues `locales/en.json` and `locales/zh.json` have a symmetric difference of zero on the merge target branch, the i18n CJK Guard shall pass the parity check on a manual run from the repository root.
2. If the merge target branch is found to have a non-zero symmetric difference at the time this spec is implemented, then the implementer shall (a) document the divergence in the spec's `tasks.md` as a blocking finding and (b) fix the divergence before completing the implementation tasks, rather than weakening the parity check.

104
.kiro/specs/i18n-locale-parity-guard/research.md Normal file
View File

@ -0,0 +1,104 @@
# Research & Design Decisions — i18n-locale-parity-guard
## Summary
- **Feature**: `i18n-locale-parity-guard`
- **Discovery Scope**: Extension (extends an existing single-script CI guard)
- **Key Findings**:
- The existing PR-time guard `scripts/ci/i18n_cjk_guard.py` already implements the no-short-circuit composition pattern, the JSON-flatten primitive, and the line-fallback line-resolution helper that the new parity check needs to reuse.
- The audit pipeline's `check_parity.py` (in `.kiro/specs/i18n-e2e-english-verification/audit/scripts/`) already proves the algorithm: flatten both catalogues into dotted-key sets and compute their symmetric difference. It runs only in the manual audit path; promoting it to CI is a pure plumbing exercise.
- The live catalogues at `HEAD` of `main` are parity-clean (962 keys per side, symmetric difference 0), so the new guard's first run will not produce a false alarm and Requirement 6.1 holds out of the gate.
## Research Log
### Composition with the existing guard
- **Context**: Requirement 3 mandates that all checks (CJK-clean, per-path ratchet, parity) run in a single invocation without short-circuit and surface a unified exit code.
- **Sources Consulted**: `scripts/ci/i18n_cjk_guard.py:run_check` (lines 220–299).
- **Findings**: `run_check` uses a `failed: bool` accumulator and a `success_summary: list[str]` collector, evaluating every block before deciding the exit code. The parity check fits trivially as a third block at the end of `run_check`, before the final `if not failed: print(success_summary)` block.
- **Implications**: No structural refactor is needed. The extension is additive.
### Flatten and key resolution semantics
- **Context**: Requirement 1.1 anchors the flatten contract to `check_parity.py.flatten`. Requirement 1.5 specifies that scalar leaves and string leaves are treated identically for parity (only dict leaves are skipped).
- **Sources Consulted**: `.kiro/specs/i18n-e2e-english-verification/audit/scripts/check_parity.py:flatten`; `scripts/ci/i18n_cjk_guard.py:_flatten`.
- **Findings**: The two implementations are byte-equivalent in behaviour: both descend only into `dict`, both yield `(dotted-path, value)` for any non-dict leaf, both build dotted paths with `.` separators. The guard's existing `_flatten` is suitable; the parity check just consumes its keys (set comprehension over the flattened pairs).
- **Implications**: No new flatten function is needed. Requirement 1.1's "exactly match" clause is satisfied by reusing `_flatten`. Add a thin `_flatten_keys(data) -> set[str]` wrapper to keep call sites readable.
### Line resolution for missing keys
- **Context**: Requirement 2.1 demands `<file>:<line>: <key>: <side>` output. Requirement 2.2 demands a line-1 fallback when location is unknown.
- **Sources Consulted**: `scripts/ci/i18n_cjk_guard.py:_value_line_number` (lines 70–87).
- **Findings**: `_value_line_number` resolves a value's line by substring scan with two candidates (raw + JSON-escaped), falling back to line 1. For parity we must resolve a key, not a value. The minimal adaptation is a `_locate_key_line(text_lines, dotted_key)` that searches for the leaf segment of the dotted key wrapped in JSON quotes (e.g. `"missingKey"`). Falling back to line 1 mirrors `_value_line_number`'s contract.
- **Implications**: A small new helper is needed; it follows the same code idiom as `_value_line_number`. Edge cases: leaf segments that appear elsewhere in the file (other keys, value text) — accepting a coarse first-match is acceptable because the *primary* signal (the dotted key + side) is unambiguous; the line number is a navigation aid.
### Stdlib-only enforcement
- **Context**: Requirement 4.1 prohibits new dependencies.
- **Sources Consulted**: `pyproject.toml`, `requirements*.txt` (none at repo root); existing guard imports.
- **Findings**: The existing guard imports `argparse`, `json`, `os`, `re`, `subprocess`, `sys`, `pathlib`. Parity needs none beyond `json` and `pathlib` — both already in use.
- **Implications**: No `pyproject.toml` change. CI runtime image needs no addition.
### Live catalogue parity at HEAD
- **Context**: Requirement 6.1 asserts the guard must pass on the merge target's current state.
- **Sources Consulted**: `locales/en.json`, `locales/zh.json` flattened via stdlib `json.loads` + recursive descent.
- **Findings**: 962 keys per side, symmetric difference 0. Pre-existing `log.*` namespace fully mirrored (373 keys per side).
- **Implications**: No remediation translation work is needed. Requirement 6.2's conditional ("if divergence is found, fix it before completing") does not trigger.
## Architecture Pattern Evaluation
| Option | Description | Strengths | Risks / Limitations | Notes |
|--------|-------------|-----------|---------------------|-------|
| Extend existing guard (Option A — selected) | Add parity helpers + a third block in `run_check` inside `scripts/ci/i18n_cjk_guard.py`; no workflow edit. | Single CI surface; reuses `_flatten`, line-fallback, sort/print idioms; trivially satisfies Requirement 3.6. | Module grows ~80 lines; module name no longer narrowly "CJK" — mitigated by docstring update. | Recommended in `gap-analysis.md`. |
| Parallel script + step (Option B) | New `scripts/ci/i18n_locale_parity_guard.py`; either second job in existing workflow or new workflow file. | Tightest single-responsibility per file. | Code duplication (~80 lines); two CI surfaces; violates the spirit of Requirement 3 ("compose with existing checks"). | Rejected. |
| Helper module + thin import (Option C) | New `scripts/ci/locale_parity.py`; the existing guard imports it and integrates the call. | Cleaner unit-test isolation; possible future de-duplication of audit `check_parity.py`. | Adds package-style imports for ~80 lines of logic; risks scope creep into "deduplicate audit script" (out of scope). | Rejected. |
## Design Decisions
### Decision: Extend `scripts/ci/i18n_cjk_guard.py` rather than create a new script
- **Context**: Requirement 3 mandates a single CLI invocation that runs all i18n CI checks together with no short-circuit and one exit code.
- **Alternatives Considered**:
1. New parallel script + workflow step — duplicates ~80 lines of plumbing.
2. New helper module imported by the guard — introduces package structure for trivial logic.
- **Selected Approach**: Add `_flatten_keys`, `_locate_key_line`, `_format_parity_finding`, and `run_parity_check` to the existing module; insert a third block into `run_check` after the per-path baseline block.
- **Rationale**: Smallest delta that fully satisfies Requirement 3; reuses the existing no-short-circuit accumulator pattern verbatim; no workflow edit (Requirement 3.6 holds for free); existing test scaffolding (`unittest`, synthetic git repos) extends naturally.
- **Trade-offs**: The module name (`i18n_cjk_guard`) becomes slightly broader than literal — mitigated by an updated module docstring listing all three checks. Module length grows from ~393 to ~470 lines, still well below the project's de facto threshold for splitting (`oasis_profile_generator.py` exceeds 1000).
- **Follow-up**: Update the module docstring; verify `--help` text and existing CLI smoke test still pass after the change.
### Decision: Treat scalar leaves identically to string leaves for parity
- **Context**: Requirement 1.5 — `_flatten` does not narrow by type; scalars (numbers, booleans, null) at a leaf must register as keys.
- **Alternatives Considered**:
1. Narrow to string leaves only (mirror `scan_locale_cjk`'s behaviour). Rejected because a numeric or null value on one side is still a string-on-the-other-side parity question, and the `log.*` namespace today is all strings — there's no payoff in narrowing.
2. Skip dict leaves; emit everything else. Selected.
- **Selected Approach**: `_flatten_keys(data) -> set[str]` returns every dotted path emitted by the existing `_flatten`, regardless of value type.
- **Rationale**: Aligns with the audit script's `flatten` contract (which also does not type-narrow). Catches accidental type drift across catalogues as a side benefit (any divergence at a key surfaces as a missing key).
- **Trade-offs**: None significant — the catalogues today are entirely string-typed at leaves; the choice is mostly future-proofing.
- **Follow-up**: Add a unit test (Requirement 5.1.e) that plants a scalar-typed leaf on both sides at the same path and asserts the parity check passes.
### Decision: Failure category strings — `parity-en-only` / `parity-zh-only`
- **Context**: Requirement 2.1 specifies the format `<file>:<line>: <key>: en-only` (or `... zh-only`). The existing CJK-clean check formats failures as `<file>:<line>: cjk-in-en: <key> = <snippet>`.
- **Alternatives Considered**:
1. Use bare `en-only` / `zh-only` as the category. Inconsistent with the CJK check's namespaced category (`cjk-in-en`).
2. Use namespaced categories `parity-en-only` / `parity-zh-only`. Selected.
- **Selected Approach**: Format failure lines as `<en.json|zh.json>:<line>: parity-en-only: <key>` and `... parity-zh-only: <key>` (file is whichever catalogue the missing key would belong to).
- **Rationale**: Mirrors the CJK check's `cjk-in-en` category naming, so a dev grepping CI logs for `parity-` finds all parity failures. The bare-side requirement of 2.1 is satisfied because the side appears verbatim after `parity-` (`parity-en-only` contains `en-only`).
- **Trade-offs**: Minor verbosity vs. consistency — favour consistency.
- **Follow-up**: Tests assert exact substring `parity-en-only` / `parity-zh-only` in failure lines.
## Risks & Mitigations
- **Risk**: A future maintainer renames the existing `_flatten` and the parity check silently breaks. **Mitigation**: A test in the new `ParityCheckTests` class asserts that flattening a known nested fixture produces the expected dotted-key set (locking in the contract).
- **Risk**: The `_locate_key_line` helper produces a misleading line number when the leaf segment also appears in another (unrelated) key or in a value. **Mitigation**: First-match on the JSON-quoted leaf is "good enough" for navigation; the dotted key in the message is the source of truth. Document this in the helper's docstring.
- **Risk**: Future test writers forget the no-short-circuit invariant when extending `run_check`. **Mitigation**: Requirement 5.1.f's composition test guards this — both the parity check and the existing CJK check fail in the same run, and the test asserts both failure lines appear together.
## References
- `scripts/ci/i18n_cjk_guard.py` — existing guard (extension target).
- `.kiro/specs/i18n-e2e-english-verification/audit/scripts/check_parity.py` — reference parity algorithm.
- `.kiro/specs/i18n-ci-guard/design.md` — prior CI guard design (style and boundary precedents).
- `scripts/ci/tests/test_i18n_cjk_guard.py` — existing test patterns (extension target).
- `.github/workflows/i18n-cjk-guard.yml` — workflow that runs the guard (no edit required).

23
.kiro/specs/i18n-locale-parity-guard/spec.json Normal file
View File

@ -0,0 +1,23 @@
{
"feature_name": "i18n-locale-parity-guard",
"created_at": "2026-05-09T00:29:21Z",
"updated_at": "2026-05-09T00:46:00Z",
"language": "en",
"phase": "tasks-generated",
"approvals": {
"requirements": {
"generated": true,
"approved": true
},
"design": {
"generated": true,
"approved": true
},
"tasks": {
"generated": true,
"approved": true
}
},
"ready_for_implementation": true,
"ticket": 11
}

63
.kiro/specs/i18n-locale-parity-guard/tasks.md Normal file
View File

@ -0,0 +1,63 @@
# Implementation Plan
- [x] 1. Add parity primitives to the i18n CJK Guard module
- Introduce a constant naming the Chinese catalogue path alongside the existing English-catalogue constant.
- Add a private helper that returns the dotted-key set of a parsed catalogue, mirroring the audit pipeline's `flatten` contract (descend into dicts only; treat scalar leaves and string leaves identically; type-narrow nothing).
- Add a private helper that resolves the 1-based line number of a dotted key in raw JSON source text by searching for the leaf segment wrapped in JSON quotes, and falls back to line 1 on any miss.
- Add a private helper that formats a single parity-failure line in the layout `<file>:<line>: parity-en-only: <key>` or `... parity-zh-only: <key>`, with the side parameter typed as a literal of the two allowed strings (improvement carried over from the design review).
- Add an immutable result carrier (named tuple or frozen dataclass) holding the parity outcome (passed flag, formatted failure lines including the trailing summary, optional success-summary line).
- All additions stay stdlib-only and import nothing new beyond what the existing module already imports.
- Observable completion: the module exports the new constant, helpers, and result carrier; importing the module from a Python REPL or test stays warning-free, and the helpers can be exercised in isolation.
- _Requirements: 1.1, 1.5, 2.1, 2.2, 4.1, 4.3_
- _Boundary: i18n_cjk_guard module — helper layer_
- [x] 2. Implement the parity-check orchestrator
- Read both locale catalogues from the working tree using the existing path constants.
- Flatten each catalogue and compute the symmetric difference of the dotted-key sets.
- On match, build the success-summary string of the form `OK locale-parity: <count> keys per side`.
- On mismatch, sort en-only keys lexicographically and emit one formatted failure line per key with the EN catalogue path and a best-effort line number; then sort zh-only keys lexicographically and emit one line per key with the ZH catalogue path and a best-effort line number.
- Append a final summary line of the form `parity: en-only=<n>, zh-only=<m>` to the failure list so the orchestrator can print all lines uniformly.
- Treat a missing or malformed catalogue file as a parity failure that returns a single descriptive failure line; if the EN catalogue is the unreadable side, attribute the error to the parity check without re-stating the en-only error already produced by the existing CJK-clean block (refinement carried over from the design review).
- All output strings are deterministic across runs for identical inputs.
- Observable completion: calling the orchestrator function with synthetic parity-clean and parity-divergent catalogues returns a result carrier whose passed flag, failure list, and success summary match the documented contracts; running it against the live `locales/` directory returns `passed=True`.
- _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 2.1, 2.3, 2.4, 2.5, 4.2, 4.3_
- _Boundary: i18n_cjk_guard module — orchestrator-leaf layer_
- [x] 3. Compose the parity check into the existing run-check orchestrator
- Insert a new block inside the existing `run_check` function, after the per-path-ratchet block and before the final all-success branch.
- Invoke the parity-check orchestrator with the working-tree root.
- When the result is not passed, set the existing `failed` accumulator to true and print every entry of the result's failure list to stderr, one per call, preserving order.
- When the result is passed, append the result's success-summary line to the existing `success_summary` collector so it prints alongside the other success summaries on a fully-clean run.
- Update the module docstring to list all three checks (CJK-clean, per-path ratchet, locale-parity).
- Leave the CLI argument parser, `--update-baseline`, `--baseline`, `--repo-root`, the workflow file, and the baseline file format untouched. Confirm by visual diff that no other functions or files are modified.
- Observable completion: invoking the guard script via its CLI produces a single exit code, and `--help` text plus the existing CLI smoke test continues to pass without modification.
- _Requirements: 3.1, 3.2, 3.3, 3.4, 3.5, 3.6_
- _Boundary: i18n_cjk_guard module — run_check orchestrator_
- _Depends: 2_
- [x] 4. Add unit and integration tests for the parity check
- Extend the existing test-fixture helper that builds synthetic git repositories so callers can supply a Chinese catalogue alongside the English one; default the Chinese catalogue to a parity-clean mirror of the English fixture so the existing test cases continue to pass without semantic change.
- Add unit-level tests for the dotted-key flattener (empty input, flat input, mixed scalar/string/null leaves, three-level nesting), the line-number resolver (exact match, multi-occurrence first-wins, not-found line-1 fallback), and the failure-line formatter (both sides, special characters in key names).
- Add integration tests against the parity-check orchestrator covering: identical key sets pass; an en-only divergence fails with the expected category token, summary, and line attributing the key to the EN catalogue; a zh-only divergence fails with the symmetric output; a both-sides divergence yields en-only lines first then zh-only lines, each lex-sorted within its group; same-path scalar leaves on both sides do not count as a parity failure; a missing or malformed catalogue file produces a single deterministic failure line.
- All new tests use the standard-library testing framework already used in the existing test module; negative-path fixtures are self-contained and do not depend on the live catalogues.
- Observable completion: running the test module from the repository root produces a passing run with at least the new test cases reported, and a manually-induced en-only or zh-only key reliably trips the relevant test.
- _Requirements: 5.1, 5.2, 5.3, 5.4_
- _Boundary: i18n_cjk_guard test module — parity unit + integration coverage_
- _Depends: 3_
- [x] 5. Add a no-short-circuit composition test covering all three guard checks
- Plant CJK content in a synthetic English catalogue AND a parity-divergent key (in either direction) inside the same synthetic repository.
- Assert that running the full composed guard returns exit code 1, that stderr contains both the existing CJK-related category token and the new parity category token, and that the order of these blocks is preserved (CJK first, then ratchet, then parity) so failure logs remain greppable.
- Assert that on a fully-clean repository (no CJK in EN, ratchet within baseline, parity holds) the composed guard prints all three success summaries on stdout and exits 0.
- Observable completion: the new test case fails if any future change short-circuits the orchestrator after the first failure or before invoking the parity check.
- _Requirements: 3.1, 3.2, 3.3, 5.1_
- _Boundary: i18n_cjk_guard test module — composition coverage_
- _Depends: 3, 4_
- [x] 6. Verify the guard against the live locale catalogues
- Run the guard once from the repository root against the live `locales/en.json` and `locales/zh.json` and confirm it exits 0 with three success-summary lines (CJK-clean, per-path ratchet, locale-parity).
- If the live catalogues turn out to have non-zero symmetric difference at the time of implementation, document the divergence in this `tasks.md` as a blocking finding and remediate the divergence before completing the task; do not weaken the parity check.
- Observable completion: the guard's CLI invocation against the live tree prints `OK locale-parity: <count> keys per side` and exits 0, demonstrating that the new check is satisfied by the merge target without any source change.
- _Requirements: 6.1, 6.2_
- _Boundary: live `locales/` content (read-only verification)_
- _Depends: 5_

617
.kiro/specs/i18n-oasis-profile-generator-prompts/design.md Normal file
View File

@ -0,0 +1,617 @@
# Design Document — i18n-oasis-profile-generator-prompts
## Overview
**Purpose**: Translate the Chinese prompt strings in
`backend/app/services/oasis_profile_generator.py` (the system prompt
inside `_get_system_prompt`, the individual-persona f-string template
inside `_build_individual_persona_prompt`, the group-persona f-string
template inside `_build_group_persona_prompt`, and the four
`attrs_str`/`context_str` fallback literals) to English while
preserving every functional contract — JSON output keys, the `gender`
English enum, the `age` integer rule, the `persona` no-newline rule,
all `{variable}` interpolations, and every `get_language_instruction()`
call site. The goal is to remove the Chinese-language base-prompt bias
that currently leaks Chinese structure and word choice into persona
output even when `Accept-Language: en`.
**Users**: MiroFish operators running the Step 2 environment-setup
pipeline under any locale; downstream Step 3 (CAMEL-OASIS subprocess)
which consumes the produced persona dictionaries.
**Impact**: Replaces approximately one one-line system prompt and two
large f-string templates with English equivalents inside one file. No
API change, no new dependencies, no new files. The two production
callers (`backend/app/services/simulation_manager.py:316` and
`backend/app/api/simulation.py:1413`) and the OASIS subprocess are
unaffected.
### Goals
- Zero CJK characters in any prompt string literal contributed by
`oasis_profile_generator.py` to the system prompt or the two
user-message bodies (including the `attrs_str`/`context_str`
fallback literals).
- English persona prose (`bio`, `persona`, `profession`,
`interested_topics`) under `Accept-Language: en`.
- Continued Chinese persona prose under `Accept-Language: zh`, of
equivalent quality to the pre-change behaviour.
- `gender` field stays exactly one of `"male"`/`"female"`/`"other"`
regardless of locale.
- No diff to public signatures, taxonomy lists, LLM-call parameters,
or call sites.
### Non-Goals
- Externalizing prompts to `/locales/*.json` (out of scope per ticket).
- Translating logger calls in this file (covered by issue #6).
- Translating module/class/method docstrings or inline comments
(covered by issue #7).
- Refactoring the `OasisAgentProfile` schema, `MBTI_TYPES` /
`COUNTRIES` lists, or the `INDIVIDUAL_ENTITY_TYPES` /
`GROUP_ENTITY_TYPES` taxonomies.
- Modifying the rule-based fallback (`_generate_profile_rule_based`)
including its Chinese country defaults.
- Modifying the resilience helpers `_fix_truncated_json` /
`_try_fix_json` and the Chinese persona fallback fragments inside
them (e.g. `f"{entity_name}是一个{entity_type}。"`).
- Modifying `backend/app/utils/locale.py`, the locale registries, or
any non-target file.
- Modifying `backend/scripts/test_profile_format.py`.
## Boundary Commitments
### This Spec Owns
- The English content of `_get_system_prompt`'s `base_prompt` literal.
- The English content of the f-string template body in
`_build_individual_persona_prompt`.
- The English content of the f-string template body in
`_build_group_persona_prompt`.
- The English replacements for the four `"无"` / `"无额外上下文"`
fallback literals (in both individual and group builders).
### Out of Boundary
- Locale resolution machinery (`backend/app/utils/locale.py`).
- Per-locale `llmInstruction` definitions
(`/locales/languages.json`).
- Reasoning-model output stripping inside `_fix_truncated_json` /
`_try_fix_json`.
- Logger calls and translation keys (`t("log.profile_generator.*")`)
inside `oasis_profile_generator.py` (issue #6, already merged).
- Module / class / method docstrings and inline comments inside
`oasis_profile_generator.py` (issue #7).
- Rule-based fallback (`_generate_profile_rule_based`) including its
Chinese country defaults `"中国"`.
- Chinese persona fragments inside the resilience helpers (e.g.
`f"{entity_name}是一个{entity_type}。"`) — those are runtime data
fallbacks, not LLM prompts.
- All callers of `OasisProfileGenerator`
(`simulation_manager.py`, `api/simulation.py`).
- Tests, scripts, and frontend code.
- The `print(...)` banner at line 945 (closely associated with logger
externalization #6).
### Allowed Dependencies
- Existing imports in the target file (no additions). Specifically:
`get_language_instruction`, `get_locale`, `set_locale`, `t` from
`..utils.locale` are already imported and remain unchanged.
- Existing LLM transport via `self.client.chat.completions.create`
(unchanged).
### Revalidation Triggers
The following changes elsewhere would invalidate this design:
- A change to the JSON contract emitted by the LLM (`bio`, `persona`,
`age`, `gender`, `mbti`, `country`, `profession`,
`interested_topics` keys).
- A change to the `OasisAgentProfile` dataclass field set or the
Reddit/Twitter serializers.
- A change to `get_language_instruction()` semantics or the per-locale
`llmInstruction` strings.
- A change to OASIS subprocess profile-format expectations (verified
via `backend/scripts/test_profile_format.py`).
## Architecture
### Existing Architecture Analysis
`OasisProfileGenerator` lives in `backend/app/services/`, follows the
in-process service pattern, and is invoked from a Flask handler inside
a background task. The relevant flow:
1. The Flask handler resolves the request locale via `Accept-Language`;
`set_locale()` is propagated into worker threads in
`generate_profiles_for_entities` (locale captured at line ~910 and
restored inside `generate_single_profile` at line ~914).
2. For each entity, `generate_profile_from_entity` decides between the
individual or group prompt builder via
`self._is_individual_entity(entity_type)`.
3. The chosen builder produces a user-message string; `_get_system_prompt`
produces a system-message string. Both are sent to the LLM via
`self.client.chat.completions.create(..., response_format={"type": "json_object"})`.
4. The LLM response is JSON-decoded; on failure, `_try_fix_json` and
`_fix_truncated_json` attempt recovery; on terminal failure,
`_generate_profile_rule_based` produces a rule-based persona.
5. The result is wrapped in an `OasisAgentProfile` dataclass and
serialized to Reddit JSON or Twitter CSV via `_save_reddit_json` /
`_save_twitter_csv`.
This design preserves all of the above. The change is purely lexical
inside three method bodies and four literal defaults.
### Architecture Pattern & Boundary Map
```mermaid
graph TB
Caller["simulation_manager.py / api/simulation.py"]
Generator["OasisProfileGenerator"]
Sys["_get_system_prompt"]
Ind["_build_individual_persona_prompt"]
Grp["_build_group_persona_prompt"]
Locale["locale.get_language_instruction"]
Client["openai.chat.completions.create"]
Parser["_try_fix_json / _fix_truncated_json"]
Fallback["_generate_profile_rule_based"]
Serializer["_save_reddit_json / _save_twitter_csv"]
Caller --> Generator
Generator --> Sys
Generator --> Ind
Generator --> Grp
Sys -. inline call .-> Locale
Ind -. inline call .-> Locale
Grp -. inline call .-> Locale
Sys --> Client
Ind --> Client
Grp --> Client
Client --> Parser
Parser --> Fallback
Generator --> Serializer
classDef change fill:#fff4ce,stroke:#a16207,color:#000
class Sys,Ind,Grp change
```
The three highlighted nodes (`_get_system_prompt`,
`_build_individual_persona_prompt`,
`_build_group_persona_prompt`) are the only nodes whose **string
contents** change. Every edge — including each call to
`get_language_instruction()` — remains intact.
**Architecture Integration**:
- **Selected pattern**: In-place lexical translation of the three
prompt builders (Option A from `gap-analysis.md` / `research.md`).
- **Domain/feature boundaries**: Same as today; `OasisProfileGenerator`
remains the sole owner of persona prompt content. `LocaleService`
remains the sole owner of locale-postfix steering.
- **Existing patterns preserved**: locale-thread propagation, retry
logic with temperature decay, JSON resilience helpers, rule-based
fallback, two-platform serialization.
- **New components rationale**: none — no new components.
- **Steering compliance**: aligns with `tech.md` ("LLM prompts use the
`get_language_instruction()` postfix mechanism, not key files") and
`structure.md` ("services own their own prompt strings").
### Technology Stack & Alignment
| Layer | Choice / Version | Role in Feature | Notes |
|-------|------------------|-----------------|-------|
| Backend / Services | Python ≥3.11 | Hosts the prompt builders | No version change |
| LLM transport | `openai` SDK against any OpenAI-compatible endpoint | Sends translated prompts | Unchanged |
| i18n | `backend/app/utils/locale.py` | Resolves locale and provides `get_language_instruction()` postfix | Unchanged |
| Storage | None | — | No persistence change |
No new dependencies. No version bumps. The locale infrastructure used
by the change is the same one used by every sibling i18n spec already
merged.
## File Structure Plan
### Modified Files
- `backend/app/services/oasis_profile_generator.py` — only file that
changes.
- `_get_system_prompt(self, is_individual: bool) -> str` — translate
`base_prompt` literal to English. Keep
`f"{base_prompt}\n\n{get_language_instruction()}"` shape.
- `_build_individual_persona_prompt(self, entity_name, entity_type,
entity_summary, entity_attributes, context) -> str` — translate
the f-string body to English; replace `"无"` and `"无额外上下文"`
defaults; keep every `{variable}` interpolation and the inline
`{get_language_instruction()}` call.
- `_build_group_persona_prompt(self, entity_name, entity_type,
entity_summary, entity_attributes, context) -> str` — same
treatment as the individual builder.
No other files in the repository are touched by this change.
## System Flows
The runtime flow does not change. The only way to demonstrate this is
to compare the call graph before and after — and the call graph is
already shown in the Architecture diagram above. Skipping a separate
sequence diagram.
## Requirements Traceability
| Requirement | Summary | Components | Interfaces | Flows |
|-------------|---------|------------|------------|-------|
| 1.1 | `base_prompt` contains zero Chinese characters | `_get_system_prompt` | `(self, is_individual: bool) -> str` | system-message construction |
| 1.2 | Preserve `f"{base_prompt}\n\n{get_language_instruction()}"` | `_get_system_prompt` | inline `get_language_instruction()` | system-message construction |
| 1.3 | Preserve role/intent semantics | `_get_system_prompt` | — | — |
| 1.4 | Preserve signature `_get_system_prompt(self, is_individual: bool) -> str` | `_get_system_prompt` | (signature) | — |
| 2.1 | Individual prompt body in English | `_build_individual_persona_prompt` | f-string body | user-message construction |
| 2.2 | Preserve `{entity_name}`, `{entity_type}`, `{entity_summary}`, `{attrs_str}`, `{context_str}`, `{get_language_instruction()}` | `_build_individual_persona_prompt` | f-string interpolations | — |
| 2.3 | Preserve JSON keys `bio, persona, age, gender, mbti, country, profession, interested_topics` | `_build_individual_persona_prompt` | prompt content | — |
| 2.4 | Preserve field-level constraints (lengths, MBTI, gender enum, age int) | `_build_individual_persona_prompt` | prompt content | — |
| 2.5 | Preserve trailing-rules block semantics | `_build_individual_persona_prompt` | prompt content | — |
| 2.6 | Preserve method signature | `_build_individual_persona_prompt` | (signature) | — |
| 2.7 | Translate `"无"` and `"无额外上下文"` defaults | `_build_individual_persona_prompt` | literal defaults | — |
| 2.8 | Zero Chinese in assembled body | `_build_individual_persona_prompt` | — | — |
| 3.1 | Group prompt body in English | `_build_group_persona_prompt` | f-string body | user-message construction |
| 3.2 | Preserve interpolations | `_build_group_persona_prompt` | f-string interpolations | — |
| 3.3 | Preserve JSON keys | `_build_group_persona_prompt` | prompt content | — |
| 3.4 | Preserve field-level constraints (age=30, gender="other", etc.) | `_build_group_persona_prompt` | prompt content | — |
| 3.5 | Preserve trailing-rules semantics | `_build_group_persona_prompt` | prompt content | — |
| 3.6 | Preserve method signature | `_build_group_persona_prompt` | (signature) | — |
| 3.7 | Translate `"无"` / `"无额外上下文"` defaults | `_build_group_persona_prompt` | literal defaults | — |
| 3.8 | Zero Chinese in assembled body | `_build_group_persona_prompt` | — | — |
| 4.1 | Preserve every `get_language_instruction()` call site | all three builders | inline call | system + user message construction |
| 4.2 | Preserve locale-thread plumbing | `generate_profiles_for_entities` (untouched) | `set_locale(current_locale)` | worker thread spawn |
| 4.3 | Locale=zh produces Chinese personas | runtime behaviour | locale postfix | LLM call |
| 4.4 | Locale=en produces English personas | runtime behaviour | locale postfix | LLM call |
| 4.5 | `gender` ∈ {male, female, other} regardless of locale | prompt content | — | — |
| 4.6 | Don't alter locale.py / locales/ | (none) | — | — |
| 5.1 | Preserve `OasisAgentProfile` dataclass | (untouched) | dataclass | — |
| 5.2 | Preserve method signatures | (untouched) | signatures | — |
| 5.3 | Preserve LLM invocation parameters | (untouched) | `chat.completions.create(...)` | — |
| 5.4 | Preserve `MBTI_TYPES`, `COUNTRIES`, taxonomy lists | (untouched) | class constants | — |
| 6.1 | Preserve `_fix_truncated_json` / `_try_fix_json` | (untouched) | helpers | — |
| 6.2 | Reasoning-model recovery still works | (untouched) | resilience helpers | — |
| 6.3 | No new prompt-language-dependent pre-processing | (none added) | — | — |
| 6.4 | Round-trip yields non-empty `bio` and `persona` | runtime behaviour | LLM call | — |
| 7.1 | `pytest test_profile_format.py` passes | runtime behaviour | serializers | — |
| 7.2 | Reddit format schema preserved | (untouched) | `to_reddit_format` | — |
| 7.3 | Twitter format schema preserved | (untouched) | `to_twitter_format` | — |
| 7.4 | `gender` enum preserved | prompt content | — | — |
| 8.1 | No logger edits | (untouched) | — | — |
| 8.2 | No docstring/comment edits | (untouched) | — | — |
| 8.3 | No rule-based fallback edits | (untouched) | — | — |
| 8.4 | No edits outside the target file | (none) | — | — |
| 8.5 | No new dependencies | (none) | `pyproject.toml` / `uv.lock` untouched | — |
| 8.6 | No edits to `test_profile_format.py` | (untouched) | — | — |
## Components and Interfaces
| Component | Domain/Layer | Intent | Req Coverage | Key Dependencies (P0/P1) | Contracts |
|-----------|--------------|--------|--------------|--------------------------|-----------|
| `_get_system_prompt` | backend service / prompt builder | Produce the system message (English base + locale postfix) | 1.1, 1.2, 1.3, 1.4, 4.1, 4.5 | `get_language_instruction` (P0) | Service |
| `_build_individual_persona_prompt` | backend service / prompt builder | Produce the individual-entity user message in English | 2.x, 4.1, 4.5 | `get_language_instruction` (P0); JSON encoder (P1) | Service |
| `_build_group_persona_prompt` | backend service / prompt builder | Produce the group/institution user message in English | 3.x, 4.1, 4.5 | `get_language_instruction` (P0); JSON encoder (P1) | Service |
Only the three prompt-builder methods change. They all live inside the
single class `OasisProfileGenerator` in
`backend/app/services/oasis_profile_generator.py`. No new components.
### Backend / Services
#### `_get_system_prompt`
| Field | Detail |
|-------|--------|
| Intent | Build the `system` message: a one-line English directive that frames the model as a social-media persona expert + the per-locale postfix. |
| Requirements | 1.1, 1.2, 1.3, 1.4, 4.1, 4.5 |
**Responsibilities & Constraints**
- Construct and return a single string of the form
`f"{base_prompt}\n\n{get_language_instruction()}"`.
- Preserve the signature
`_get_system_prompt(self, is_individual: bool) -> str`.
- The English `base_prompt` MUST convey: (a) expert role in
social-media persona generation; (b) intent to produce detailed,
realistic personas for opinion-simulation, faithful to existing
reality; (c) the JSON-output requirement and the no-unescaped-newline
rule.
- The English `base_prompt` MUST NOT contain any CJK codepoint.
**Dependencies**
- Outbound: `get_language_instruction()` from
`backend/app/utils/locale.py` (P0, criticality high — the entire
locale-steering chain depends on it).
**Contracts**: Service [x] / API [ ] / Event [ ] / Batch [ ] / State [ ]
##### Service Interface
```python
def _get_system_prompt(self, is_individual: bool) -> str:
"""Return the LLM system message: English base + locale postfix."""
...
```
- Preconditions: none.
- Postconditions: returns a non-empty string ending with the locale
postfix produced by `get_language_instruction()`.
- Invariants: contains zero CJK codepoints.
**Implementation Notes**
- Integration: called only from `_call_llm_with_retry` (line ~523)
with `is_individual` decided upstream. The `is_individual` flag is
reserved for future divergence between system prompts; the current
implementation does not branch on it, and this design preserves
that.
- Validation: a CJK regex audit on the method body after the edit must
match zero codepoints.
- Risks: dropping one of the three role/intent pieces (expert framing,
JSON output requirement, no-newline rule). Implementation task lists
all three explicitly.
#### `_build_individual_persona_prompt`
| Field | Detail |
|-------|--------|
| Intent | Build the user-message string for an individual entity in English. Preserve every `{variable}` interpolation, the inline `{get_language_instruction()}` call, every JSON-output key, and every locale-independent constraint. |
| Requirements | 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 4.1, 4.5 |
**Responsibilities & Constraints**
- Preserve signature
`_build_individual_persona_prompt(self, entity_name: str, entity_type: str, entity_summary: str, entity_attributes: Dict[str, Any], context: str) -> str`.
- Preserve `attrs_str = json.dumps(entity_attributes, ensure_ascii=False) if entity_attributes else <fallback>` with `<fallback>` translated to English (`"None"`).
- Preserve `context_str = context[:3000] if context else <fallback>` with `<fallback>` translated to English (`"No additional context"`).
- Translate the f-string body to English with these structural sections (mirror the original Chinese intent):
1. **Lead sentence** — instruct the model to generate a detailed
social-media persona for the entity, faithful to existing reality.
2. **Entity context block** — labelled lines for `entity_name`,
`entity_type`, `entity_summary`, `entity_attributes` (English
labels; values via `{...}` interpolation).
3. **Context information block** — `Context information:` heading
followed by `{context_str}`.
4. **JSON-fields enumeration** — `Generate JSON with the following
fields:` followed by the eight numbered items (`bio`, `persona`,
`age`, `gender`, `mbti`, `country`, `profession`,
`interested_topics`) with English descriptions matching
Requirement 2.4.
5. **Trailing rules block** — `Important:` followed by:
- `All field values must be strings or numbers; do not use newlines.`
- `persona must be a single coherent block of text.`
- `{get_language_instruction()} (gender field MUST use English values: "male" or "female")`
- `Content must remain consistent with the entity information.`
- `age must be a valid integer; gender must be exactly "male" or "female".`
- Preserve every `{variable}` interpolation present in the original by
name: `{entity_name}`, `{entity_type}`, `{entity_summary}`,
`{attrs_str}`, `{context_str}`, `{get_language_instruction()}`.
- The translated body MUST NOT contain any CJK codepoint.
**Dependencies**
- Outbound: `json.dumps(..., ensure_ascii=False)` (P1, formatting the
attributes dict) — unchanged.
- Outbound: `get_language_instruction()` (P0) — interpolated inline.
**Contracts**: Service [x] / API [ ] / Event [ ] / Batch [ ] / State [ ]
##### Service Interface
```python
def _build_individual_persona_prompt(
self,
entity_name: str,
entity_type: str,
entity_summary: str,
entity_attributes: Dict[str, Any],
context: str,
) -> str:
"""Return the LLM user message for an individual-entity persona."""
...
```
- Preconditions: `entity_name`, `entity_type`, `entity_summary`
are strings (may be empty); `entity_attributes` is a dict (may be
empty); `context` is a string (may be empty).
- Postconditions: returns a non-empty English string with all six
interpolations resolved.
- Invariants: contains zero CJK codepoints; preserves every
`{variable}` interpolation by name.
**Implementation Notes**
- Integration: called from `_call_llm_with_retry` (line ~506) when
`is_individual` is true.
- Validation: post-edit CJK regex audit; interpolation-set audit
(verify the multiset of `{...}` tokens equals the pre-change set);
smoke import + `pytest backend/scripts/test_profile_format.py`.
- Risks: dropping the `gender` enum lock when translating; dropping
the inline `{get_language_instruction()}` call. The implementation
task list calls these out as discrete checks.
#### `_build_group_persona_prompt`
| Field | Detail |
|-------|--------|
| Intent | Build the user-message string for a group/institution entity in English. Preserve every `{variable}` interpolation, the inline `{get_language_instruction()}` call, every JSON-output key, and every locale-independent constraint (notably `age == 30` and `gender == "other"`). |
| Requirements | 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 4.1, 4.5 |
**Responsibilities & Constraints**
- Preserve signature
`_build_group_persona_prompt(self, entity_name: str, entity_type: str, entity_summary: str, entity_attributes: Dict[str, Any], context: str) -> str`.
- Preserve the `attrs_str` and `context_str` fallback handling with
English defaults (`"None"`, `"No additional context"`), identical to
the individual builder.
- Translate the f-string body to English with these structural
sections (mirror the original Chinese intent for institutions):
1. **Lead sentence** — instruct the model to generate a detailed
social-media account profile for the institution/group, faithful
to existing reality.
2. **Entity context block** — labelled lines for `entity_name`,
`entity_type`, `entity_summary`, `entity_attributes`.
3. **Context information block** — `Context information:` heading
followed by `{context_str}`.
4. **JSON-fields enumeration** — `Generate JSON with the following
fields:` followed by the eight numbered items as defined in
Requirement 3.4: `bio` (~200 chars, official voice), `persona`
(~2000 chars, single coherent text covering institutional
basics, account positioning, voice, publishing pattern, stance,
special notes, institutional memory), `age` (= integer 30,
institutional virtual age), `gender` (= literal `"other"`),
`mbti` (e.g. ISTJ for strict/conservative), `country` (country
name string), `profession` (institutional function),
`interested_topics` (array).
5. **Trailing rules block** — `Important:` followed by:
- `All field values must be strings or numbers; null is not allowed.`
- `persona must be a single coherent block of text without newlines.`
- `{get_language_instruction()} (gender field MUST use English value "other")`
- `age must be the integer 30; gender must be the string "other".`
- `Account voice must match its identity positioning.`
- Preserve every `{variable}` interpolation present in the original.
- The translated body MUST NOT contain any CJK codepoint.
**Dependencies**
- Outbound: same as individual builder.
**Contracts**: Service [x] / API [ ] / Event [ ] / Batch [ ] / State [ ]
##### Service Interface
```python
def _build_group_persona_prompt(
self,
entity_name: str,
entity_type: str,
entity_summary: str,
entity_attributes: Dict[str, Any],
context: str,
) -> str:
"""Return the LLM user message for a group/institution persona."""
...
```
- Preconditions / Postconditions / Invariants: same shape as the
individual builder.
**Implementation Notes**
- Integration: called from `_call_llm_with_retry` (line ~510) when
`is_individual` is false.
- Validation: same checks as the individual builder, plus an explicit
audit that the institutional sentinels (`age == 30`,
`gender == "other"`) appear in English in the trailing-rules block.
- Risks: same as the individual builder; additionally, the `country`
language hint (`"使用中文，如\"中国\""`) is intentionally dropped
during translation — the validation task verifies that under
`Accept-Language: en` a sample run produces an English country
name.
## Data Models
No data-model changes. The persona JSON schema, the
`OasisAgentProfile` dataclass, the Reddit/Twitter serializers, and the
OASIS subprocess profile-format expectations are all preserved
verbatim.
## Error Handling
### Error Strategy
No new error paths. The existing flow is preserved:
- `json.JSONDecodeError` → `_try_fix_json` → `_fix_truncated_json` →
partial-extract via regex → `_generate_profile_rule_based`.
- LLM call failure → retry with temperature decay (`0.7 - attempt * 0.1`)
up to `max_attempts = 3`.
- Terminal failure → rule-based fallback persona.
- Per-entity worker exception → fallback `OasisAgentProfile` produced
inside `generate_single_profile` at line ~932.
The translated prompts do not introduce new failure modes. Translating
prompt language has no semantic effect on JSON parsing or on the
`response_format={"type": "json_object"}` constraint.
### Error Categories and Responses
- **User errors**: not applicable (this is an internal pipeline).
- **System errors**: LLM transport errors are retried; logger emits
`t("log.profile_generator.m011")` etc. Logger keys already exist in
`locales/{en,zh}.json`.
- **Business-logic errors**: `gender` not in the English enum, `age`
not an integer — the prompt explicitly mandates them; the validator
inside `_try_fix_json` does not enforce these but the OASIS
subprocess does. No change in either direction.
### Monitoring
Existing logger calls are unchanged. Logger keys already i18n-keyed via
`t("log.profile_generator.*")`.
## Testing Strategy
### Unit Tests
- **(Existing)**
`backend/scripts/test_profile_format.py::test_profile_formats` —
must continue to pass without modification.
- **(Manual)** Smoke import:
`cd backend && uv run python -c "from app.services.oasis_profile_generator import OasisProfileGenerator"`
— confirms no syntax errors after editing f-strings.
### Integration Tests
- **(Manual)** Run the prompt builders directly under each locale:
- `set_locale("en")` →
`OasisProfileGenerator()._build_individual_persona_prompt("Alice", "Student", "summary", {"k": "v"}, "ctx")`
— assert no CJK codepoints in the output, assert the English
locale postfix appears via `get_language_instruction()` (which is
`"Please respond in English."`).
- `set_locale("zh")` → same call → assert the locale postfix is
`"请使用中文回答。"`.
- These do not require an LLM call; they only verify the rendered
prompt string.
### E2E Tests
- **(Manual, optional, preferred but skippable when no LLM key
present)** Run `npm run dev` and trigger Step 2 profile generation
from the UI under English locale on a small entity set; spot-check
that bios and persona prose are in English. Skip if a live LLM key
is unavailable in CI; sibling specs #2/#4/#5 used the same manual
E2E approach.
### Performance / Load
Not applicable. Prompt translation has no measurable performance
impact.
## Optional Sections
### Security Considerations
No security implications. No new external surfaces; no new data
retention; no change to authentication or authorization.
### Migration Strategy
No migration required. The change is forward-compatible: a deployment
that picks up the translated prompts continues to serve users on the
`zh` locale via the unchanged
`get_language_instruction()` postfix mechanism.
## Supporting References
- `gap-analysis.md` — option evaluation and effort/risk sizing.
- `research.md` — discovery findings, design decisions (in particular
the "drop the country language hint" decision), and risk register.
- `requirements.md` — EARS requirements with numeric IDs.
- Sibling specs `i18n-ontology-generator-prompts`,
`i18n-simulation-config-generator-prompts`,
`i18n-report-agent-prompts` — same translation pattern, already
merged.

241
.kiro/specs/i18n-oasis-profile-generator-prompts/gap-analysis.md Normal file
View File

@ -0,0 +1,241 @@
# Gap Analysis — i18n-oasis-profile-generator-prompts
This document analyzes the gap between the requirements and the existing
codebase, lists implementation options, and recommends an approach for the
design phase.
## 1. Current State Investigation
### Target file
`backend/app/services/oasis_profile_generator.py` — 1195 lines. Defines:
- `OasisAgentProfile` dataclass with Reddit / Twitter serializers.
- `OasisProfileGenerator` class with the following public-API surface:
`__init__`, `generate_profile_from_entity`, `generate_profiles_from_entities`,
`set_graph_id`, plus private helpers `_call_llm_with_retry`,
`_generate_profile_rule_based`, `_get_system_prompt`,
`_build_individual_persona_prompt`, `_build_group_persona_prompt`,
`_print_generated_profile`, `_fix_truncated_json`, `_try_fix_json`,
`_save_twitter_csv`, `_save_reddit_json`, `_generate_username`.
### Chinese surfaces in the file (by category)
| Category | Lines | In scope this issue? |
| --- | --- | --- |
| Module / class / method docstrings | scattered | **No** — covered by #7 |
| Inline `#` comments | scattered | **No** — covered by #7 |
| `logger.{info,warning,error}` calls (translated via `t("log.profile_generator.*")`) | scattered | **No** — already done by #6 |
| `print(...)` banners (e.g. line 945) | a few | **No** — companion to #6 in spirit; not a prompt literal |
| **System prompt `base_prompt`** (line 664) | 1 line | **Yes** |
| **Individual-persona prompt body** (lines 680–714) | block | **Yes** |
| **Group-persona prompt body** (lines 729–762) | block | **Yes** |
| `attrs_str` / `context_str` defaults `"无"` / `"无额外上下文"` (lines 677, 678, 726, 727) | 4 lines | **Yes** — they substitute *into* the prompt body |
| Rule-based fallback (`_generate_profile_rule_based`, lines 764–835) including `"country": "中国"` and `"国家"` placeholders | block | **No** — runtime data, not a prompt |
| Resilience-helper Chinese fragments (`f"{entity_name}是一个{entity_type}。"` at lines 547, 644, 659) | a few | **No** — runtime data, not a prompt |
The file already imports `get_locale`, `set_locale`, `t`, and
`get_language_instruction` from `app.utils.locale`. The locale-capture /
restore plumbing inside `generate_profiles_for_entities` (lines ~910–916)
already propagates the request locale to background-thread workers — no
changes required.
### Locale infrastructure (already in place)
`backend/app/utils/locale.py`:
- `get_language_instruction()` returns the per-locale postfix from
`/locales/languages.json` (e.g. `Please respond in English.` for `en`,
`请使用中文回答。` for `zh`).
- `t(key, **kwargs)` resolves `log.*` keys for backend logger messages;
not used by this issue.
- `set_locale` / `get_locale` are thread-local, with restoration plumbed
into `generate_profiles_for_entities`.
### Sibling specs already shipped
- `i18n-ontology-generator-prompts` (#2 — merged)
- `i18n-simulation-config-generator-prompts` (#4 — merged)
- `i18n-report-agent-prompts` (#5 — merged)
- `i18n-externalize-backend-logs` (#6 — merged; logger keys for
`log.profile_generator.*` are already in `locales/{en,zh}.json`)
The translation pattern they established:
1. Translate the base prompt body (English narrative + headings).
2. Preserve every `get_language_instruction()` call site verbatim so
`Accept-Language: zh` still produces Chinese output.
3. Preserve all `{variable}` interpolations in f-strings.
4. Preserve all locale-independent "lock" rules (e.g. `gender` enum) in
English text within the prompt.
5. No new dependencies, no new files, single-file diff.
This is a direct sibling — same pattern applies.
### Test contract
`backend/scripts/test_profile_format.py`:
- Pytest-collectable function `test_profile_formats`.
- Constructs `OasisAgentProfile` instances directly (no LLM call) and
serializes them via `_save_twitter_csv` / `_save_reddit_json`.
- Verifies CSV header includes `user_id, user_name, name, bio,
friend_count, follower_count, statuses_count, created_at` and JSON
output includes `realname, username, bio, persona`.
- **Does not exercise the prompts.** A pure prompt translation cannot
break it; a refactor of dataclass field names or serializers would.
### Callers
- `backend/app/services/simulation_manager.py:316` —
`OasisProfileGenerator(graph_id=state.graph_id)`.
- `backend/app/api/simulation.py:1413` — `OasisProfileGenerator()`.
Neither caller looks at prompt language; both consume the persona dict
output. No call-site changes are needed.
## 2. Requirement-to-Asset Map
| Req. | Asset / file | Gap |
| --- | --- | --- |
| 1. System prompt → English | `_get_system_prompt` line 664 | **Missing** — Chinese literal needs to become English literal |
| 2. Individual-persona template → English | `_build_individual_persona_prompt` lines 680–714 | **Missing** — Chinese block needs translation; preserve `{...}` interpolations and inline `{get_language_instruction()}` |
| 3. Group-persona template → English | `_build_group_persona_prompt` lines 729–762 | **Missing** — Chinese block needs translation; preserve `{...}` interpolations and inline `{get_language_instruction()}` |
| 4. Locale switching unchanged | `app.utils.locale` + the three `get_language_instruction()` call sites | **Constraint** — code path must stay byte-identical at those call sites |
| 5. Public API stability | `OasisAgentProfile` dataclass + `OasisProfileGenerator` method signatures | **Constraint** — no signatures change |
| 6. Reasoning-model parsing unchanged | `_fix_truncated_json`, `_try_fix_json` | **Constraint** — no edits |
| 7. OASIS schema parity | `_save_twitter_csv`, `_save_reddit_json`, `to_*_format` serializers | **Constraint** — no edits; pytest must continue passing |
| 8. Out-of-scope guard | logger calls, docstrings, comments, rule-based fallback | **Constraint** — explicitly do not edit |
No requirement is blocked or unknown. Every requirement maps to a known
location with a clear, narrow change.
## 3. Implementation Approach Options
### Option A — In-place edit of the three prompt builders (extend existing)
Translate `base_prompt` (1 line), the individual-persona f-string body
(~35 lines), and the group-persona f-string body (~34 lines) directly,
plus the four `"无"` / `"无额外上下文"` fallback literals. Keep all method
bodies otherwise byte-identical.
- **Files touched**: `backend/app/services/oasis_profile_generator.py`
only.
- **Compatibility**: zero API change. All call sites unaffected. Locale
switching preserved by leaving the inline `{get_language_instruction()}`
placeholders untouched.
- **Complexity**: low. Pattern is identical to merged siblings #2, #4,
#5.
**Trade-offs**:
- ✅ Minimal diff, exactly the pattern reviewers expect.
- ✅ No risk to the unrelated rule-based fallback or serialization paths.
- ✅ Out-of-scope items (logger, docstrings, rule-based fallback) are not
touched, so #6/#7 remain clean.
- ❌ Leaves the file mixed-language in non-prompt parts (docstrings, rule
fallback) until #7 lands. Acceptable per scope split.
### Option B — Move prompt strings into module-level constants
Introduce `INDIVIDUAL_PERSONA_PROMPT_TEMPLATE` and
`GROUP_PERSONA_PROMPT_TEMPLATE` constants at module scope (mirroring
`ONTOLOGY_SYSTEM_PROMPT` style in `ontology_generator.py`), and have the
builders `.format(**kwargs)` against them.
- **Files touched**: same single file, but with structural refactor.
- **Compatibility**: still zero public API change, but the diff is
larger and reviewers must verify equivalent behaviour around
`{get_language_instruction()}` (which would need to become a runtime
substitution not an f-string interpolation, since constants don't
re-evaluate per call).
**Trade-offs**:
- ✅ Constants are easier to spot in `git grep`.
- ❌ Larger diff, more review surface.
- ❌ The inline `get_language_instruction()` call is currently captured at
f-string render time; moving to a `.format(...)` template requires
passing the resolved instruction in as a kwarg — a behavioural change
that exceeds "translate prompts only".
- ❌ Diverges from the sibling pattern just shipped (#4, #5 used in-place
edits, not module constants). #2 used module constants but only for the
system prompt — the user-message template was still built inside the
method.
### Option C — Externalize prompt text into `/locales/*.json`
Move every prompt sentence into `locales/en.json` and `locales/zh.json`,
keyed under `prompt.profile_generator.*`, and use `t(key, **vars)` to
resolve.
- **Compatibility**: would address `Accept-Language` purely via the
existing translation mechanism without depending on the
`get_language_instruction()` postfix.
**Trade-offs**:
- ✅ Most i18n-pure approach.
- ❌ Significantly larger diff (touches three repos: source file,
`en.json`, `zh.json`).
- ❌ Diverges from the established project pattern. The sibling specs
(#2, #4, #5) deliberately did **not** externalize prompts — the
project rationale (per `tech.md`) is that backend logger messages are
the i18n surface, while LLM prompts use the `get_language_instruction()`
postfix mechanism.
- ❌ Higher review and merge cost for no operational gain.
## 4. Recommended Approach
**Option A** — single-file in-place edit of the three prompt builders
plus the four `"无"` / `"无额外上下文"` fallback literals.
Rationale:
- Matches the merged sibling specs verbatim (#2, #4, #5) so reviewers
can apply the same mental checklist.
- Smallest possible diff that satisfies every acceptance criterion in
requirements.md.
- Leaves out-of-scope surfaces (logger, docstrings, rule-based
fallback) untouched — clean handoff to #7 and clean separation from
already-merged #6.
- Zero new dependencies, zero new files, zero API change, zero risk to
`test_profile_format.py`.
### Translation choices to lock in during design
1. The system prompt `base_prompt` becomes a single English sentence in
the spirit of the original (expert in social-media persona generation;
detailed and realistic personas for opinion simulation; faithful
reflection of real-world conditions; valid JSON, no unescaped
newlines).
2. The two persona prompt bodies adopt English section headings and
prose. The previously-Chinese hint
`country: 国家（使用中文，如"中国"）` is dropped — the
`get_language_instruction()` postfix already steers locale, and the
rule-based fallback (out of scope) handles its own country values.
3. The trailing rules block keeps the locale-independent "lock"
constraints inline (`gender` enum, `age` integer requirement,
`persona` newline rule) and continues to embed
`{get_language_instruction()}` verbatim.
## 5. Effort & Risk
- **Effort**: **S** (1–3 days; realistically <½ day). One-file diff,
established sibling pattern, no new test infrastructure.
- **Risk**: **Low**. The translated prompts touch only the LLM
`messages` payload. The locale-switching pathway, public API,
serializers, retry logic, fallback, and tests are all untouched. The
only failure mode is a mistranslated constraint (e.g. accidentally
dropping `gender ∈ {male, female, other}`), which the design checklist
enumerates and reviewers can verify by diff.
### Research items carried into design phase
- None blocking. The design phase will:
- Enumerate the exact final English text for each of the three blocks.
- Verify each translated block preserves every JSON-output key,
every `{variable}` interpolation, and the inline
`{get_language_instruction()}` call.
- Spot-check that the diff stays within
`backend/app/services/oasis_profile_generator.py`.

145
.kiro/specs/i18n-oasis-profile-generator-prompts/requirements.md Normal file
View File

@ -0,0 +1,145 @@
# Requirements Document
## Introduction
This specification covers the English translation of the prompt strings in `backend/app/services/oasis_profile_generator.py`. The file converts Graphiti graph entities into OASIS agent persona dictionaries that drive Step 2 (Environment Setup) of the MiroFish pipeline. Today, the system prompt and the two `_build_*_persona_prompt` user-message templates are written in Chinese; the language is steered at runtime by appending `get_language_instruction()` to the system prompt and inside the user prompt body. While that postfix instructs the model *which* language to respond in, the base-prompt language biases the model's structural and lexical output, so persona prose (bio, persona, profession, interested_topics) skews Chinese under `Accept-Language: en`. Translating the base prompts to English removes that bias while preserving the existing locale-switching mechanism for non-English locales (`get_language_instruction()` returns `请使用中文回答。` when locale is `zh`, so a Chinese model response remains achievable from an English base prompt).
This work tracks GitHub issue [#3](https://github.com/salestech-group/MiroFish/issues/3) and is sibling to the already-merged ontology-generator (#2), simulation-config-generator (#4), and report-agent (#5) prompt translation specs.
## Boundary Context
- **In scope**:
- Translating the system-prompt base string in `OasisProfileGenerator._get_system_prompt` (currently `"你是社交媒体用户画像生成专家。…"` at line ~664) from Chinese to English.
- Translating the individual-persona user-message template in `OasisProfileGenerator._build_individual_persona_prompt` (currently lines ~680–714) from Chinese to English.
- Translating the group/institution-persona user-message template in `OasisProfileGenerator._build_group_persona_prompt` (currently lines ~729–762) from Chinese to English.
- Translating the small `attrs_str` and `context_str` fallback default literals (`"无"`, `"无额外上下文"`) to English equivalents.
- Preserving all functional contracts: every `get_language_instruction()` call site, all variable interpolations, all JSON output keys, the `gender` enum constraint, the `age` integer constraint, and the institutional age=30 / gender="other" rule.
- **Out of scope**:
- Logger calls (`logger.info`, `logger.warning`, `logger.error`) and the printed banner text inside `oasis_profile_generator.py` — covered by issue #6.
- Module docstring, class docstrings, method docstrings, and inline comments — covered by issue #7.
- The fallback Chinese string literals embedded in non-prompt code paths (e.g. `f"{entity_name}是一个{entity_type}。"` inside `_try_fix_json` and the rule-based fallback) — those are runtime data fallbacks, not LLM prompts, and are out of scope for this issue (they are part of the fallback flow covered when comments/docstrings #7 lands or in a future cleanup; they are not user-visible while the LLM path succeeds).
- Refactoring the OASIS profile JSON schema, the `OasisAgentProfile` dataclass, the MBTI list, the `COMMON_COUNTRIES` list, the entity-type taxonomy splits (`PERSONAL_ENTITY_TYPES` vs `GROUP_ENTITY_TYPES`), or persona-generation flow control.
- Changing OASIS profile-format compatibility — verified by `backend/scripts/test_profile_format.py`.
- Editing the locale plumbing block (currently the `current_locale = get_locale()` capture and the `set_locale(current_locale)` call inside `generate_single_profile` around lines ~910–916).
- **Adjacent expectations**:
- The Step 2 environment-setup pipeline must continue to consume the OASIS profile output unchanged. The Reddit (`to_reddit_format`) and Twitter (`to_twitter_format`) serializers are not coupled to prompt language; this is verified via the JSON schema contract preservation.
- The locale resolution chain (`Accept-Language` header → `get_locale()` → `get_language_instruction()`) is owned by `backend/app/utils/locale.py` and is unchanged by this work.
- Companion i18n issues (#6 logs, #7 comments/docstrings, #9 frontend comments, #10 e2e verification, #12 README) operate on different files or scopes and must not be touched here.
## Requirements
### Requirement 1: English Translation of the System Prompt
**Objective:** As a MiroFish operator running the pipeline under `Accept-Language: en`, I want the persona-generation system prompt to be authored in English, so that the LLM's persona prose is not biased toward Chinese structure or word choice.
#### Acceptance Criteria
1. The OASIS Profile Generator shall set the `base_prompt` constant inside `_get_system_prompt` to an English string containing zero Chinese characters.
2. The OASIS Profile Generator shall preserve the system-prompt assembly contract verbatim: the format `f"{base_prompt}\n\n{get_language_instruction()}"` and the call to `get_language_instruction()` at exactly that site.
3. The OASIS Profile Generator shall preserve the role and intent semantics of the original prompt: identifying the model as an expert in social-media user-persona generation, requesting detailed and realistic personas for opinion simulation that reflect existing real-world conditions, and mandating valid JSON output where string values must not contain unescaped newlines.
4. The OASIS Profile Generator shall preserve the function signature `_get_system_prompt(self, is_individual: bool) -> str`.
### Requirement 2: English Translation of the Individual-Persona User-Message Template
**Objective:** As a MiroFish operator generating personas for individual entities under `Accept-Language: en`, I want the user-message template constructed by `_build_individual_persona_prompt` to be authored in English, so that the rendered prompt does not interleave English `get_language_instruction()` directives with Chinese section headings.
#### Acceptance Criteria
1. The OASIS Profile Generator shall render the individual-persona user message with English section headings and prose in place of the current Chinese (entity name, entity type, entity summary, entity attributes, context section, JSON-fields enumeration, "important" trailing block).
2. The OASIS Profile Generator shall preserve all variable interpolations verbatim by name: `{entity_name}`, `{entity_type}`, `{entity_summary}`, `{attrs_str}`, `{context_str}`, and the inline `{get_language_instruction()}` call inside the trailing rules block.
3. The OASIS Profile Generator shall preserve the JSON output contract enumerated in the prompt: the keys `bio`, `persona`, `age`, `gender`, `mbti`, `country`, `profession`, `interested_topics` (verbatim, English).
4. The OASIS Profile Generator shall preserve the field-level constraints in the prompt:
- `bio` ≈ 200 characters, social-media biography.
- `persona` ≈ 2000 characters, single coherent text covering: basic information (age, profession, education, location), background (notable experience, event association, social ties), personality (MBTI, core traits, emotional expression), social-media behavior (posting frequency, content preferences, interaction style, language traits), stance (attitudes toward the topic, emotional triggers), unique features (catchphrases, special experiences, hobbies), and personal memory (the entity's relation to the event and prior actions/reactions in it).
- `age` MUST be an integer.
- `gender` MUST be one of `"male"` or `"female"` (English enum value, locale-independent).
- `mbti` MUST be an MBTI four-letter type (e.g. INTJ, ENFP).
- `country` MUST be a country name string.
- `profession` MUST be a profession string.
- `interested_topics` MUST be an array.
5. The OASIS Profile Generator shall preserve the trailing-block rules verbatim in spirit: every value is a string or number, no newlines inside string values, `persona` is a single coherent text, `gender` must be the English `male`/`female` enum even when locale is `zh`, content must stay consistent with the source entity, `age` must be a valid integer.
6. The OASIS Profile Generator shall preserve the function signature `_build_individual_persona_prompt(self, entity_name: str, entity_type: str, entity_summary: str, entity_attributes: Dict[str, Any], context: str) -> str`.
7. The OASIS Profile Generator shall preserve the `context[:3000]` truncation behaviour and the conditional fallback (`"无额外上下文"` translated to `"No additional context"`) when `context` is empty/falsy. Likewise, `attrs_str` shall fall back to an English placeholder (`"None"`) when `entity_attributes` is empty/falsy, replacing the current `"无"` literal.
8. The OASIS Profile Generator shall return zero Chinese characters across all string literals contributed to the assembled individual-persona prompt body.
### Requirement 3: English Translation of the Group/Institution-Persona User-Message Template
**Objective:** As a MiroFish operator generating personas for institutional/group entities under `Accept-Language: en`, I want the user-message template constructed by `_build_group_persona_prompt` to be authored in English, so that the rendered prompt does not interleave English `get_language_instruction()` directives with Chinese section headings.
#### Acceptance Criteria
1. The OASIS Profile Generator shall render the group-persona user message with English section headings and prose in place of the current Chinese.
2. The OASIS Profile Generator shall preserve all variable interpolations verbatim by name: `{entity_name}`, `{entity_type}`, `{entity_summary}`, `{attrs_str}`, `{context_str}`, and the inline `{get_language_instruction()}` call inside the trailing rules block.
3. The OASIS Profile Generator shall preserve the JSON output contract enumerated in the prompt: the keys `bio`, `persona`, `age`, `gender`, `mbti`, `country`, `profession`, `interested_topics` (verbatim, English).
4. The OASIS Profile Generator shall preserve the field-level constraints in the prompt:
- `bio` ≈ 200 characters, an official-account biography that reads as professionally appropriate.
- `persona` ≈ 2000 characters, single coherent text covering: institutional basics (formal name, type, founding background, primary functions), account positioning (account type, target audience, core function), voice (language traits, common phrasing, taboo topics), publishing pattern (content types, publishing frequency, active hours), stance (official position on the core topic, controversy-handling style), special notes (group portrait represented, operational habits), and institutional memory (the institution's relation to the event and prior actions/reactions in it).
- `age` MUST be the integer `30` (the institutional virtual-age sentinel).
- `gender` MUST be the literal `"other"` (English enum value, locale-independent), indicating non-individual.
- `mbti` MUST be an MBTI four-letter type used to characterize account voice (e.g. ISTJ for strict/conservative).
- `country` MUST be a country name string.
- `profession` MUST describe institutional function.
- `interested_topics` MUST be an array of focus areas.
5. The OASIS Profile Generator shall preserve the trailing-block rules verbatim in spirit: every value is a string or number, no `null` values, no newlines in string values, `persona` is a single coherent text, `gender` must be the English `"other"` enum even when locale is `zh`, the institutional account voice must match its identity positioning, and `age` must be the integer `30`.
6. The OASIS Profile Generator shall preserve the function signature `_build_group_persona_prompt(self, entity_name: str, entity_type: str, entity_summary: str, entity_attributes: Dict[str, Any], context: str) -> str`.
7. The OASIS Profile Generator shall preserve the `context[:3000]` truncation behaviour and the conditional English-equivalent fallback for empty `context` and empty `entity_attributes`, mirroring Requirement 2.
8. The OASIS Profile Generator shall return zero Chinese characters across all string literals contributed to the assembled group-persona prompt body.
### Requirement 4: Locale Switching Continues to Work via `get_language_instruction()`
**Objective:** As a MiroFish operator running the pipeline under `Accept-Language: zh` (or any other configured non-English locale), I want generated personas to remain in the requested locale at equivalent quality, so that translating the base prompt does not regress non-English support.
#### Acceptance Criteria
1. The OASIS Profile Generator shall preserve every existing `get_language_instruction()` call site exactly: the system-prompt site in `_get_system_prompt`, the inline call inside the trailing rules block of `_build_individual_persona_prompt`, and the inline call inside the trailing rules block of `_build_group_persona_prompt`.
2. The OASIS Profile Generator shall preserve the locale-capture/restore plumbing inside `generate_profiles_for_entities` (currently the `current_locale = get_locale()` capture and the `set_locale(current_locale)` call inside `generate_single_profile`) — this code is not modified by the change.
3. While the locale is `zh`, the OASIS Profile Generator shall produce profiles whose `bio`, `persona`, `profession`, and `interested_topics` content is in Chinese, equivalent in quality to the pre-change behaviour.
4. While the locale is `en`, the OASIS Profile Generator shall produce profiles whose `bio`, `persona`, `profession`, and `interested_topics` content is in English.
5. While the locale is `en` or `zh`, the OASIS Profile Generator shall produce profiles whose `gender` field is one of the literal English values `"male"`, `"female"` (individual entities) or `"other"` (group entities), regardless of locale.
6. The OASIS Profile Generator shall not alter `backend/app/utils/locale.py`, the `_languages`, the `_translations` registries, or the locales under `/locales/`.
### Requirement 5: Public API and Call-Site Stability
**Objective:** As a developer maintaining the rest of the MiroFish backend pipeline, I want the public surface of `OasisProfileGenerator` and `OasisAgentProfile` to remain unchanged, so that the Step 2 environment-setup flow and existing callers continue to work without modification.
#### Acceptance Criteria
1. The OASIS Profile Generator shall preserve the dataclass `OasisAgentProfile`, including its field set (`user_id`, `user_name`, `name`, `bio`, `persona`, `karma`, `friend_count`, `follower_count`, `statuses_count`, `age`, `gender`, `mbti`, `country`, `profession`, `interested_topics`, `source_entity_uuid`, `source_entity_type`, `created_at`), default values, and the `to_reddit_format`, `to_twitter_format`, `to_full_dict` serializers.
2. The OASIS Profile Generator shall preserve the signatures and call semantics of `OasisProfileGenerator.__init__`, `generate_profile_from_entity`, `generate_profiles_for_entities`, `_call_llm_with_retry`, `_generate_profile_rule_based`, `_get_system_prompt`, `_build_individual_persona_prompt`, `_build_group_persona_prompt`, `_print_generated_profile`, `_fix_truncated_json`, `_try_fix_json`, and `_generate_username`.
3. The OASIS Profile Generator shall preserve the LLM invocation parameters (`temperature`, `max_tokens`, model selection, retry behaviour) at the call sites that consume the prompts produced by the translated builders.
4. The OASIS Profile Generator shall preserve the `PERSONAL_ENTITY_TYPES` and `GROUP_ENTITY_TYPES` taxonomies, the `MBTI_TYPES` list, and the `COMMON_COUNTRIES` list verbatim.
### Requirement 6: Reasoning-Model Output Compatibility
**Objective:** As a MiroFish operator using a reasoning-model provider (e.g. MiniMax, GLM with `<think>` tags or markdown code fences), I want JSON parsing of the persona response to continue working, so that translating the base prompt does not regress provider compatibility.
#### Acceptance Criteria
1. The OASIS Profile Generator shall preserve the existing `_fix_truncated_json` and `_try_fix_json` resilience helpers exactly, including their regex-based extraction of `bio` and `persona` from partial output.
2. If a reasoning-model provider returns truncated, `<think>`-tagged, or markdown-fenced output, then the existing parsing/recovery flow shall continue to apply unchanged.
3. The OASIS Profile Generator shall not introduce any new pre-processing of the LLM response that depends on prompt language.
4. After translation, the OASIS Profile Generator shall continue to round-trip a representative entity through `generate_profile_from_entity` and produce a JSON object with at minimum a non-empty `bio` and a non-empty `persona`, matching the pre-change behaviour.
### Requirement 7: Step 2 Environment-Setup Parity (OASIS Format Compatibility)
**Objective:** As a MiroFish operator validating the change, I want the OASIS subprocess to accept the generated profiles unchanged, so that the translation does not silently break Step 2 → Step 3 hand-off.
#### Acceptance Criteria
1. While `uv run python -m pytest backend/scripts/test_profile_format.py` runs against the changed code, the test suite shall pass with zero regressions versus the pre-change baseline.
2. While a representative Reddit-format profile dictionary is produced under locale `en`, every field name shall match the existing OASIS-required schema: `user_id`, `username`, `name`, `bio`, `persona`, `karma`, `created_at`, plus optional `age`, `gender`, `mbti`, `country`, `profession`, `interested_topics`.
3. While a representative Twitter-format profile dictionary is produced under locale `en`, every field name shall match the existing OASIS-required schema: `user_id`, `username`, `name`, `bio`, `persona`, `friend_count`, `follower_count`, `statuses_count`, `created_at`, plus optional `age`, `gender`, `mbti`, `country`, `profession`, `interested_topics`.
4. The OASIS Profile Generator shall produce `gender` values that are exactly one of `"male"`, `"female"`, `"other"` regardless of locale, satisfying the OASIS subprocess's expected enum.
### Requirement 8: Out-of-Scope Surfaces Remain Untouched
**Objective:** As a reviewer of this PR, I want the change to remain narrowly scoped to prompt strings, so that translation responsibilities for adjacent surfaces (issues #6, #7, and the rule-based fallback) are not absorbed into this change.
#### Acceptance Criteria
1. The change shall not modify any `logger.warning(...)`, `logger.info(...)`, `logger.error(...)`, or `logger.debug(...)` call in `oasis_profile_generator.py` (covered by issue #6).
2. The change shall not modify the module docstring, class docstrings, method docstrings, or inline comments in `oasis_profile_generator.py` (covered by issue #7).
3. The change shall not modify the rule-based fallback Chinese fragments inside `_try_fix_json` (e.g. `f"{entity_name}是一个{entity_type}。"`) and the rule-based path inside `_generate_profile_rule_based` — those are runtime data fallbacks, not LLM prompts, and remain out of scope here.
4. The change shall not edit any file outside `backend/app/services/oasis_profile_generator.py` for production code.
5. The change shall not introduce a new dependency or modify `backend/pyproject.toml` / `backend/uv.lock`.
6. The change shall not modify `backend/scripts/test_profile_format.py` (the test is the contract; the implementation must match it).

222
.kiro/specs/i18n-oasis-profile-generator-prompts/research.md Normal file
View File

@ -0,0 +1,222 @@
# Research & Design Decisions — i18n-oasis-profile-generator-prompts
## Summary
- **Feature**: `i18n-oasis-profile-generator-prompts`
- **Discovery Scope**: **Extension** (single-file translation in an existing
brownfield service; sibling pattern already merged in #2, #4, #5)
- **Key Findings**:
- The existing `get_language_instruction()` postfix mechanism (defined in
`backend/app/utils/locale.py`) is the project-canonical way to steer LLM
output language. Translating the base prompt does not interfere with it
and is the same approach taken in already-merged sibling specs.
- The only Chinese surfaces inside the prompt-rendering path are
`_get_system_prompt`, `_build_individual_persona_prompt`,
`_build_group_persona_prompt`, and the four `attrs_str`/`context_str`
fallback literals (`"无"`, `"无额外上下文"`). All other Chinese in the
file is logger keys (already done by #6), docstrings/comments
(out-of-scope, #7), or rule-based fallback data (out-of-scope).
- `backend/scripts/test_profile_format.py` does not exercise prompts; it
only constructs `OasisAgentProfile` and round-trips through
`_save_twitter_csv` / `_save_reddit_json`. A pure-translation diff
cannot break it.
## Research Log
### Locale steering mechanism
- **Context**: Confirm that translating the base prompt does not regress
Chinese output under `Accept-Language: zh`.
- **Sources Consulted**:
- `backend/app/utils/locale.py` (lines 50–96).
- `locales/languages.json` (entries for `en` and `zh` with
`llmInstruction` field).
- Sibling spec `i18n-ontology-generator-prompts/design.md` and the
merged commits referenced by it.
- **Findings**:
- `get_language_instruction()` returns `Please respond in English.`
for locale `en`, `请使用中文回答。` for locale `zh`.
- The function is called as an inline f-string interpolation in the
individual-persona and group-persona prompt bodies, and explicitly
appended in `_get_system_prompt`. All three sites must be preserved
byte-for-byte.
- The thread-local locale is captured in
`generate_profiles_for_entities` (line ~910) and restored inside the
worker via `set_locale(current_locale)` (line ~914). This plumbing is
untouched by the change.
- **Implications**:
- Design lock-in: the inline `{get_language_instruction()}` call must
remain in each of the three builders. Removing or renaming it would
silently regress non-English locales.
- The Chinese hint `country: 国家（使用中文，如"中国"）` in the original
prompt overrides the locale postfix and forces Chinese output for one
field. The English translation drops that hint so the locale postfix
decides the country language. The rule-based fallback (out of scope)
has its own (Chinese) defaults and is not affected.
### Test contract
- **Context**: Verify that `backend/scripts/test_profile_format.py`
remains green after a prompt-only translation.
- **Sources Consulted**: `backend/scripts/test_profile_format.py`,
`oasis_profile_generator.py:_save_twitter_csv`,
`oasis_profile_generator.py:_save_reddit_json`,
`oasis_profile_generator.py:to_reddit_format`,
`oasis_profile_generator.py:to_twitter_format`.
- **Findings**:
- The pytest function `test_profile_formats` constructs
`OasisAgentProfile` instances directly without invoking the LLM.
- It calls `_save_twitter_csv` and `_save_reddit_json` to verify CSV
and JSON shape. Required CSV header: `user_id, user_name, name, bio,
friend_count, follower_count, statuses_count, created_at`. Required
JSON keys: `realname, username, bio, persona`.
- **Implications**:
- Translating prompts cannot regress this test. The validation
requirement (Requirement 7) is satisfied automatically as long as
serializer code is not edited.
- No new tests are required for this change.
### Sibling specs already shipped
- **Context**: Confirm there is an established project pattern this work
must mirror.
- **Sources Consulted**:
- `.kiro/specs/i18n-ontology-generator-prompts/{design,tasks,requirements}.md`
- `.kiro/specs/i18n-report-agent-prompts/`
- `.kiro/specs/i18n-simulation-config-generator-prompts/`
- Recent merged commits referencing #2, #4, #5.
- **Findings**:
- All three siblings used a single-file in-place translation diff.
- All three preserved every `get_language_instruction()` call site.
- All three left logger calls and docstrings to companion issues
(#6 / #7).
- None externalized prompts to `/locales/*.json`.
- **Implications**:
- The same approach is correct here. Reviewer expectations are set by
the sibling diffs.
### OASIS profile schema
- **Context**: Verify that translated prompts continue to satisfy the
OASIS subprocess's expected schema (especially `gender` enum and
`age` integer).
- **Sources Consulted**: `OasisAgentProfile` dataclass,
`to_reddit_format`, `to_twitter_format`, sibling `_generate_profile_rule_based`.
- **Findings**:
- OASIS-required fields are produced by serializers, not by the
prompt: `user_id`, `username`, `name`, `bio`, `karma`/`friend_count`/`follower_count`/`statuses_count`, `created_at`.
- The prompt-defined fields land in optional positions: `age`,
`gender`, `mbti`, `country`, `profession`, `interested_topics`.
- The `gender` enum constraint (`"male"`/`"female"` for individuals,
`"other"` for groups) is locale-independent and must remain in
English text inside the translated prompt.
- **Implications**:
- The English prompt must explicitly call out `gender ∈ {male, female}`
(individual) and `gender == "other"` (group), independent of the
`get_language_instruction()` postfix.
## Architecture Pattern Evaluation
| Option | Description | Strengths | Risks / Limitations | Notes |
|--------|-------------|-----------|---------------------|-------|
| **A — In-place builder edit** | Translate three method bodies + four fallback literals directly | Smallest diff; matches sibling pattern; zero API change | None of note | **Selected** |
| B — Module-level constants | Hoist prompts to `INDIVIDUAL_PERSONA_PROMPT_TEMPLATE` etc. | Easier `git grep` | Larger diff; the inline `{get_language_instruction()}` call would need to become a `.format()` kwarg, which is a behavioural change beyond translation | Diverges from #4 / #5 |
| C — Externalize to `locales/*.json` | Move every prompt sentence into `t(...)` keys | Most i18n-pure | Three-file diff; diverges from project rationale (prompts use postfix mechanism, not key files) | Rejected |
## Design Decisions
### Decision: In-place edit of the three prompt builders (Option A)
- **Context**: Three methods build prompt strings; one of them is a
one-line system prompt, the other two are large f-string templates
with embedded `{variable}` interpolations and an inline
`{get_language_instruction()}` call.
- **Alternatives Considered**:
1. Option B — module-level constants.
2. Option C — externalize to `/locales/*.json` keys.
- **Selected Approach**: Translate each method body in place. Replace
the four `"无"` / `"无额外上下文"` fallbacks with English equivalents
(`"None"` and `"No additional context"`). Preserve all `{...}`
interpolations and the inline `{get_language_instruction()}` call.
- **Rationale**: Matches merged sibling specs verbatim. Smallest review
surface. Zero API change. Out-of-scope surfaces (logger, docstrings,
rule-based fallback) cleanly avoided.
- **Trade-offs**: Leaves the file mixed-language in non-prompt parts
(docstrings, rule fallback) until #7 lands. Acceptable per scope
split.
- **Follow-up**: During implementation, run a regex audit for any
Chinese codepoints inside the three method bodies after the edit and
confirm the diff stays within
`backend/app/services/oasis_profile_generator.py`.
### Decision: Drop the "use Chinese country names" hint
- **Context**: The current prompt at line 704 reads
`country: 国家（使用中文，如"中国"）` and at line 753
`country: 国家（使用中文，如"中国"）`. This forces Chinese for the
`country` field even under `Accept-Language: en`.
- **Alternatives Considered**:
1. Translate to English literally:
`country: country (use English, e.g. "China")`.
2. Drop the language hint entirely:
`country: country name string`.
- **Selected Approach**: Drop the language hint. Let
`get_language_instruction()` steer the country language alongside
every other free-text field.
- **Rationale**: Hard-coding a language in the prompt defeats the
locale-steering mechanism. The rule-based fallback (out of scope)
carries its own Chinese defaults; under the LLM path, locale should
decide.
- **Trade-offs**: Under `Accept-Language: zh`, the LLM may produce a
Chinese country name (e.g. `中国`) — this is the desired behaviour.
Under `Accept-Language: en`, the LLM produces English (`China`),
matching `COUNTRIES = ["China", "US", ...]` already in the file.
- **Follow-up**: Verify in the validation phase that a sample run under
locale `en` produces an English country name.
### Decision: Keep `gender` enum constraint in English inside the prompt
- **Context**: `gender` must be one of `"male"`/`"female"`/`"other"`
regardless of locale, because OASIS consumers and the
`_generate_profile_rule_based` fallback assume English values.
- **Alternatives Considered**: None — the constraint is a contract.
- **Selected Approach**: The translated prompt explicitly states the
enum in English, even when the locale postfix asks for Chinese
output: `gender MUST be one of "male" or "female" (English literal)`.
- **Rationale**: Same as the existing Chinese prompt (which already
states `必须是英文: "male" 或 "female"`). The translation preserves
the same lock-in.
- **Trade-offs**: None.
- **Follow-up**: Validation phase will check that under both locales
the produced `gender` is one of the three English literals.
## Risks & Mitigations
- **Risk**: Mistranslation drops a locale-independent constraint
(e.g. `gender` enum, `age` integer rule, `persona` no-newline rule).
- **Mitigation**: The implementation task list will enumerate every
constraint inline so reviewers can check by diff.
- **Risk**: Variable-name typo inside an f-string causes a `KeyError`
at runtime.
- **Mitigation**: Implementation task verifies that the set of
`{variable}` interpolations in each translated block matches the
pre-change set 1:1; a `python -c "import ..."` smoke import and a
`pytest backend/scripts/test_profile_format.py` run are mandatory.
- **Risk**: Accidentally leaving a CJK codepoint inside the three
builders.
- **Mitigation**: Final implementation step runs the project's
repo-level CJK guard regex (added by #26) constrained to the three
builders' line ranges.
## References
- `backend/app/services/oasis_profile_generator.py` — target file.
- `backend/app/utils/locale.py` — locale infrastructure.
- `locales/languages.json`, `locales/en.json`, `locales/zh.json` —
locale registries.
- `.kiro/specs/i18n-ontology-generator-prompts/` — sibling spec #2.
- `.kiro/specs/i18n-simulation-config-generator-prompts/` — sibling
spec #4.
- `.kiro/specs/i18n-report-agent-prompts/` — sibling spec #5.
- GitHub issue
[#3](https://github.com/salestech-group/MiroFish/issues/3).

23
.kiro/specs/i18n-oasis-profile-generator-prompts/spec.json Normal file
View File

@ -0,0 +1,23 @@
{
"feature_name": "i18n-oasis-profile-generator-prompts",
"created_at": "2026-05-08T05:26:06Z",
"updated_at": "2026-05-08T05:30:00Z",
"language": "en",
"phase": "tasks-generated",
"ticket": 3,
"approvals": {
"requirements": {
"generated": true,
"approved": true
},
"design": {
"generated": true,
"approved": true
},
"tasks": {
"generated": true,
"approved": true
}
},
"ready_for_implementation": true
}

66
.kiro/specs/i18n-oasis-profile-generator-prompts/tasks.md Normal file
View File

@ -0,0 +1,66 @@
# Implementation Plan
- [x] 1. Translate the system-prompt builder to English
- Replace the Chinese `base_prompt` literal inside `_get_system_prompt` (currently `"你是社交媒体用户画像生成专家。…"` at line ~664) with an English rendering that conveys the same role and intent: identifies the model as an expert in social-media user-persona generation, asks for detailed and realistic personas suitable for opinion-simulation that faithfully reflect existing real-world conditions, mandates valid JSON output, and forbids unescaped newlines inside string values
- Preserve the assembled return shape `f"{base_prompt}\n\n{get_language_instruction()}"` exactly — the call to `get_language_instruction()` is unchanged in name and position
- Preserve the method signature `_get_system_prompt(self, is_individual: bool) -> str`; do not branch on `is_individual` (current behaviour preserved)
- Observable completion: `_get_system_prompt(True)` and `_get_system_prompt(False)` both return non-empty English strings ending with the per-locale postfix from `get_language_instruction()`; the `base_prompt` body contains zero CJK characters
- _Requirements: 1.1, 1.2, 1.3, 1.4_
- [x] 2. Translate the individual-persona user-message builder to English
- Replace the Chinese f-string body inside `_build_individual_persona_prompt` (currently lines ~680–714) with an English rendering structured as: a lead sentence requesting a detailed social-media persona faithful to existing reality; an entity-context block with English labels for `entity_name`, `entity_type`, `entity_summary`, `entity_attributes`; a `Context information:` block; a `Generate JSON with the following fields:` enumeration of the eight output keys (`bio`, `persona`, `age`, `gender`, `mbti`, `country`, `profession`, `interested_topics`); and a trailing `Important:` rules block
- Translate the field-level descriptions verbatim in spirit: `bio` ≈ 200 chars; `persona` ≈ 2000 chars covering basic info (age, profession, education, location), background (notable experience, event association, social ties), personality (MBTI, core traits, emotional expression), social-media behaviour (posting frequency, content preferences, interaction style, language traits), stance (attitudes toward the topic, emotional triggers), unique features (catchphrases, special experiences, hobbies), and personal memory (the entity's relation to the event and prior actions/reactions); `age` integer; `gender` MUST be the literal `"male"` or `"female"`; `mbti` four-letter type; `country` country name; `profession`; `interested_topics` array
- Translate the trailing rules block to English while keeping every locale-independent constraint intact: all values are strings or numbers; `persona` is a single coherent text without unescaped newlines; the inline `{get_language_instruction()}` call remains followed by the parenthetical reminder that `gender` MUST use the English values `"male"` / `"female"`; content stays consistent with the entity; `age` MUST be a valid integer
- Replace the `attrs_str` and `context_str` Chinese fallback defaults with English: `"无"` → `"None"` (used when `entity_attributes` is empty/falsy) and `"无额外上下文"` → `"No additional context"` (used when `context` is empty/falsy)
- Drop the country-language hint `（使用中文，如"中国"）` so `get_language_instruction()` steers the country language; preserve the country line as a neutral `country: country name` entry
- Preserve every f-string interpolation by name and position: `{entity_name}`, `{entity_type}`, `{entity_summary}`, `{attrs_str}`, `{context_str}`, `{get_language_instruction()}`
- Preserve the `context[:3000]` truncation behaviour and the method signature `_build_individual_persona_prompt(self, entity_name: str, entity_type: str, entity_summary: str, entity_attributes: Dict[str, Any], context: str) -> str`
- Observable completion: calling `_build_individual_persona_prompt("Alice", "Student", "summary", {"k": "v"}, "ctx")` returns a non-empty English string with all six interpolations resolved, with zero CJK characters in any literal contributed by this method, and the string contains the `gender` enum lock-in `"male"` / `"female"` exactly once
- _Requirements: 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 4.1, 4.5_
- [x] 3. Translate the group/institution-persona user-message builder to English
- Replace the Chinese f-string body inside `_build_group_persona_prompt` (currently lines ~729–762) with an English rendering structured the same way as Task 2 but adapted for institutional voice: lead sentence requesting a detailed social-media account profile for an institution/group faithful to existing reality; entity-context block; `Context information:` block; `Generate JSON with the following fields:` enumeration of the eight output keys; trailing `Important:` rules block
- Translate the field-level descriptions verbatim in spirit: `bio` ≈ 200 chars in an official-account voice; `persona` ≈ 2000 chars covering institutional basics (formal name, type, founding background, primary functions), account positioning (account type, target audience, core function), voice (language traits, common phrasing, taboo topics), publishing pattern (content types, publishing frequency, active hours), stance (official position on the core topic, controversy-handling style), special notes (group portrait represented, operational habits), and institutional memory (the institution's relation to the event and prior actions/reactions); `age` MUST be the integer `30`; `gender` MUST be the literal `"other"`; `mbti` four-letter type characterizing account voice; `country`; `profession` describes institutional function; `interested_topics` array
- Translate the trailing rules block to English while keeping every locale-independent constraint intact: all values are strings or numbers, no `null` allowed; `persona` is a single coherent text without unescaped newlines; the inline `{get_language_instruction()}` call remains followed by the parenthetical reminder that `gender` MUST use the English value `"other"`; `age` MUST be the integer `30` and `gender` MUST be the string `"other"`; account voice must match identity positioning
- Replace the `attrs_str` and `context_str` Chinese fallback defaults with the same English replacements applied in Task 2 (`"None"` and `"No additional context"`)
- Drop the country-language hint as in Task 2
- Preserve every f-string interpolation by name and position: `{entity_name}`, `{entity_type}`, `{entity_summary}`, `{attrs_str}`, `{context_str}`, `{get_language_instruction()}`
- Preserve the `context[:3000]` truncation behaviour and the method signature `_build_group_persona_prompt(self, entity_name: str, entity_type: str, entity_summary: str, entity_attributes: Dict[str, Any], context: str) -> str`
- Observable completion: calling `_build_group_persona_prompt("ACME Corp", "Organization", "summary", {"k": "v"}, "ctx")` returns a non-empty English string with all six interpolations resolved, with zero CJK characters in any literal contributed by this method, and the string contains both the `age == 30` lock-in and the `gender == "other"` lock-in
- _Requirements: 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 4.1, 4.5_
- [x] 4. Confirm boundary commitments around the translation
- Confirm every existing `get_language_instruction()` call site is preserved verbatim: the system-prompt assembly inside `_get_system_prompt`, the inline call inside the trailing rules block of `_build_individual_persona_prompt`, and the inline call inside the trailing rules block of `_build_group_persona_prompt`
- Confirm the locale-thread plumbing in `generate_profiles_for_entities` (capture `current_locale = get_locale()` at line ~910 and `set_locale(current_locale)` inside the worker at line ~914) is byte-identical
- Confirm the public signatures of `OasisProfileGenerator.__init__`, `generate_profile_from_entity`, `generate_profiles_for_entities`, `set_graph_id`, and the private helpers `_call_llm_with_retry`, `_generate_profile_rule_based`, `_print_generated_profile`, `_fix_truncated_json`, `_try_fix_json`, `_save_twitter_csv`, `_save_reddit_json`, `_generate_username` are unchanged
- Confirm the `OasisAgentProfile` dataclass field set, default values, and the `to_reddit_format`, `to_twitter_format`, `to_full_dict` serializers are unchanged
- Confirm class constants `MBTI_TYPES`, `COUNTRIES`, `INDIVIDUAL_ENTITY_TYPES`, `GROUP_ENTITY_TYPES` are unchanged
- Confirm the LLM invocation parameters at the call site that consumes the translated prompts (`response_format={"type": "json_object"}`, `temperature=0.7 - (attempt * 0.1)`, `max_attempts=3`) are unchanged
- Confirm `_fix_truncated_json` and `_try_fix_json` (including their Chinese persona fragments such as `f"{entity_name}是一个{entity_type}。"`) are not modified — these are runtime data fallbacks, not prompts, and are out of scope
- Confirm `_generate_profile_rule_based` is not modified — including its Chinese country defaults `"中国"` at lines ~807 and ~819
- Confirm `backend/app/utils/locale.py`, `/locales/languages.json`, `/locales/en.json`, and `/locales/zh.json` are not modified
- Confirm `logger.warning(...)`, `logger.info(...)`, `logger.error(...)`, the print banner at line ~945, module / class / method docstrings, and inline comments in `oasis_profile_generator.py` are not modified (owned by issues #6 and #7)
- Confirm `backend/scripts/test_profile_format.py`, `backend/pyproject.toml`, `backend/uv.lock`, and any file outside `backend/app/services/oasis_profile_generator.py` are not modified
- Observable completion: a `git diff` review against `main` shows changes only inside `backend/app/services/oasis_profile_generator.py`, only inside `_get_system_prompt`, `_build_individual_persona_prompt`, `_build_group_persona_prompt`, and the surrounding lines (method headers, neighbouring methods) are byte-identical
- _Requirements: 1.4, 2.6, 3.6, 4.1, 4.2, 4.6, 5.1, 5.2, 5.3, 5.4, 6.1, 6.3, 8.1, 8.2, 8.3, 8.4, 8.5, 8.6_
- [x] 5. Verify smoke import and OASIS profile-format pytest
- Run `cd backend && uv run python -c "from app.services.oasis_profile_generator import OasisProfileGenerator, OasisAgentProfile"` and confirm it exits 0 (catches f-string syntax errors)
- Run `cd backend && uv run python -m pytest backend/scripts/test_profile_format.py` (or equivalent invocation per project convention) and confirm it passes — the test does not exercise prompts, so a pure-translation diff must keep it green
- Construct an instance of `OasisProfileGenerator` (using `OasisProfileGenerator.__new__(OasisProfileGenerator)` to skip `__init__` if the LLM key is unavailable, mirroring the pattern in `test_profile_format.py`) and confirm `_get_system_prompt(True)`, `_build_individual_persona_prompt("Alice", "Student", "summary", {"k": "v"}, "ctx")`, and `_build_group_persona_prompt("ACME", "Organization", "summary", {"k": "v"}, "ctx")` each return a string with zero CJK matches against the regex `[一-鿿]`
- Observable completion: smoke import exits 0; pytest passes with zero regressions; the three prompt-builder calls each produce English-only output under the default `zh` locale (the `get_language_instruction()` postfix at the end is the only place where Chinese is allowed to appear, and only when locale is `zh`)
- _Requirements: 6.4, 7.1, 7.2, 7.3, 7.4_
- [x] 6. Verify locale-driven output language under both `en` and `zh`
- With the thread-local locale forced via `set_locale("en")`, render each of the three builders against representative inputs and confirm: each output contains zero CJK characters; each ends with the English locale postfix `"Please respond in English."`; the `gender` enum constraint appears as English `"male"` / `"female"` (individual) or `"other"` (group)
- With `set_locale("zh")`, render the same three builders and confirm: the per-prompt body remains English-only (the translated base prompt does not depend on locale); each ends with the Chinese locale postfix `"请使用中文回答。"`; the `gender` enum constraint still appears as the English literal values
- Optionally, with a configured LLM key, run `OasisProfileGenerator().generate_profile_from_entity(...)` end-to-end under each locale against a synthetic `EntityNode` and spot-check that the produced `bio`, `persona`, `profession` are English under `en` and Chinese under `zh`, while `gender` is one of the three English enum literals under both
- Observable completion: the locale-`en` rendering is CJK-free in the prompt body and ends with the English locale postfix; the locale-`zh` rendering preserves the prompt body in English and ends with the Chinese locale postfix; if the LLM round-trip is exercised, results are recorded in the PR description
- _Requirements: 4.3, 4.4, 4.5_
- [x] 7. Final CJK regression sweep on the three builders
- Run a regex audit limited to the three method bodies (`_get_system_prompt`, `_build_individual_persona_prompt`, `_build_group_persona_prompt`) using the project-level CJK guard regex (`[一-鿿]`) and confirm zero matches inside their string literals
- Run a CJK audit on the rendered output of the three builders for representative inputs and confirm zero matches in the prompt body (the locale postfix is excluded — its Chinese form is a deliberate kept use under `zh`)
- Confirm the file-level `git grep -nE '[\\x{4e00}-\\x{9fff}]' -- backend/app/services/oasis_profile_generator.py` output still flags only known out-of-scope locations: docstrings, comments, logger keys, rule-based fallback country `"中国"` defaults, and resilience-helper Chinese fragments — and does not flag any line inside the three translated method bodies
- Observable completion: the targeted regex audit returns zero matches inside the three method bodies; the file-level audit's residual CJK lines all fall outside the three method bodies and match the out-of-scope inventory in `design.md` § Boundary Commitments → Out of Boundary
- _Requirements: 1.1, 2.8, 3.8, 8.1, 8.2, 8.3_

222
.kiro/specs/i18n-readme-tagline-and-assets/design.md Normal file
View File

@ -0,0 +1,222 @@
# Design Document — i18n-readme-tagline-and-assets
## Overview
**Purpose**: Eliminate the remaining Chinese surface text from the project's English-facing entry points (`README.md`, `README-EN.md`, `package.json`, `backend/pyproject.toml`) and replace Chinese-named image assets under `static/image/` with ASCII-only equivalents, so that visitors landing on the GitHub repo or installing the npm package see English-only metadata and so that asset URLs are tooling- and CDN-friendly.
**Users**: Non-Chinese-reading visitors arriving at the GitHub README, downstream consumers reading `package.json` / `backend/pyproject.toml` metadata, and any tool (CDNs, link-rotters, screenshot-rendering bots) that handles repo asset URLs.
**Impact**: Documentation surface and static image filenames change; no runtime, API, or pipeline behavior is affected. The Chinese-language entry point (`README-ZH.md`) keeps its Chinese body text but its asset references are updated to point at the renamed files.
### Goals
- Replace the Chinese tagline with English on `README.md`, `README-EN.md`, `package.json`, `backend/pyproject.toml`.
- Rename nine Chinese-named assets under `static/image/` to ASCII filenames, preserving byte content.
- Update every `<img src>` reference in `README.md`, `README-EN.md`, and `README-ZH.md` to the new ASCII paths.
- Verifiable acceptance: a Chinese-character scan over `README.md` and `README-EN.md` returns zero matches outside the language-switcher line.
### Non-Goals
- Translating the body of `README-ZH.md` (Chinese variant by design).
- Changing the Chinese tagline value in `locales/zh.json` (legitimate Chinese locale content).
- Re-encoding or re-cropping any image (rename only).
- Adding a CI guard that enforces ASCII filenames or no-Chinese-in-EN-README (tracked separately as #26).
## Boundary Commitments
### This Spec Owns
- The English-language tagline string used in `README.md`, `README-EN.md`, `package.json`, `backend/pyproject.toml`.
- The ASCII filenames for the nine renamed assets under `static/image/`.
- All `<img src>` references inside the three READMEs that point to the renamed files.
### Out of Boundary
- Any asset under `static/image/` that already uses an ASCII name (`MiroFish_logo*.jpeg`, `shanda_logo.png`).
- Code-level i18n initiatives (frontend strings, backend logs, agent prompts) — those are owned by sibling i18n specs.
- README content beyond the lines explicitly identified in §"Modified Files".
### Allowed Dependencies
- Git (`git mv` for rename-with-history).
- No new project dependencies.
### Revalidation Triggers
- Any future change that adds another Chinese-named asset under `static/image/` referenced from a README — the verification scan in this spec must be re-run.
- Any future change to the structure of the language-switcher line — the R4 verification regex tolerance for `[中文文档]` may need adjusting.
## Architecture
### Existing Architecture Analysis
This is a documentation- and asset-rename change. There is no architectural component to extend or replace. The relevant existing patterns to respect:
- **Per `.claude/rules/file-paths.md`**: shell commands that touch paths with non-ASCII characters must quote the paths.
- **Per `.kiro/steering/structure.md`**: `static/` is the project's image asset root; READMEs reference it via relative paths from repo root.
- **Per `.claude/rules/commits.md`**: Conventional Commits, lowercase, imperative, max 72 chars, no `Co-Authored-By:` watermark.
### Architecture Pattern & Boundary Map
No new architecture is introduced. The flow is a one-shot edit:
```mermaid
flowchart LR
A[Chinese-named<br/>asset files] -->|git mv| B[ASCII-named<br/>asset files]
C[README.md / README-EN.md /<br/>README-ZH.md / package.json /<br/>backend/pyproject.toml] -->|Edit tool| D[Updated text +<br/>updated img src paths]
B --> D
D --> E[Verify: rg Chinese-char scan<br/>returns only language-switcher line]
```
### Technology Stack
| Layer | Choice / Version | Role in Feature | Notes |
|-------|------------------|-----------------|-------|
| Frontend / CLI | — | n/a | No code changes. |
| Backend / Services | — | n/a | No code changes. |
| Data / Storage | — | n/a | No data model changes. |
| Messaging / Events | — | n/a | n/a |
| Infrastructure / Runtime | git ≥ 2.x | `git mv` for renames | Already a project prerequisite. |
| Documentation | Markdown / HTML-in-MD | Edit READMEs, `package.json`, `backend/pyproject.toml` | No new tooling. |
## File Structure Plan
### Directory Structure
No new files or directories are created. The existing layout is preserved:
```
static/image/
├── MiroFish_logo.jpeg (unchanged)
├── MiroFish_logo_compressed.jpeg (unchanged)
├── shanda_logo.png (unchanged)
├── qq-group.png (renamed from "QQ群.png")
├── wuhan-university-simulation-cover.png (renamed from "武大模拟演示封面.png")
├── dream-of-the-red-chamber-simulation-cover.jpg (renamed from "红楼梦模拟推演封面.jpg")
└── Screenshot/
├── screenshot1.png (renamed from "运行截图1.png")
├── screenshot2.png (renamed from "运行截图2.png")
├── screenshot3.png (renamed from "运行截图3.png")
├── screenshot4.png (renamed from "运行截图4.png")
├── screenshot5.png (renamed from "运行截图5.png")
└── screenshot6.png (renamed from "运行截图6.png")
```
### Modified Files
- `README.md`
- Lines 7–8: delete the Chinese tagline line and the `</br>` separator; the existing `<em>` line on (former) line 9 becomes the lone tagline.
- Lines 52, 53, 56, 57, 60, 61: replace `Screenshot/运行截图{N}.png` with `Screenshot/screenshot{N}.png`.
- Line 71: replace `武大模拟演示封面.png` with `wuhan-university-simulation-cover.png`.
- Line 79: replace `红楼梦模拟推演封面.jpg` with `dream-of-the-red-chamber-simulation-cover.jpg`.
- Line 220: replace `QQ群.png` with `qq-group.png`.
- `README-EN.md` — identical edit set as `README.md`.
- `README-ZH.md`
- Lines 52, 53, 56, 57, 60, 61, 71, 79, 220: same nine `<img src>` replacements as above. Tagline and Chinese body text unchanged.
- `package.json`
- Line 4: replace the `description` value with `MiroFish - A Simple and Universal Swarm Intelligence Engine, Predicting Anything`.
- `backend/pyproject.toml`
- Line 4: replace the `description` value with `MiroFish - A Simple and Universal Swarm Intelligence Engine, Predicting Anything`.
### Renamed Files (via `git mv`)
| Old (quoted) | New |
|---|---|
| `"static/image/QQ群.png"` | `static/image/qq-group.png` |
| `"static/image/武大模拟演示封面.png"` | `static/image/wuhan-university-simulation-cover.png` |
| `"static/image/红楼梦模拟推演封面.jpg"` | `static/image/dream-of-the-red-chamber-simulation-cover.jpg` |
| `"static/image/Screenshot/运行截图1.png"` | `static/image/Screenshot/screenshot1.png` |
| `"static/image/Screenshot/运行截图2.png"` | `static/image/Screenshot/screenshot2.png` |
| `"static/image/Screenshot/运行截图3.png"` | `static/image/Screenshot/screenshot3.png` |
| `"static/image/Screenshot/运行截图4.png"` | `static/image/Screenshot/screenshot4.png` |
| `"static/image/Screenshot/运行截图5.png"` | `static/image/Screenshot/screenshot5.png` |
| `"static/image/Screenshot/运行截图6.png"` | `static/image/Screenshot/screenshot6.png` |
## System Flows
Not applicable. No runtime flows are introduced or changed.
## Requirements Traceability
| Requirement | Summary | Components | Interfaces | Flows |
|-------------|---------|------------|------------|-------|
| 1.1 | English tagline in README.md | README.md L7–9 edit | n/a | n/a |
| 1.2 | English tagline in README-EN.md | README-EN.md L7–9 edit | n/a | n/a |
| 1.3 | English description in package.json | package.json L4 edit | n/a | n/a |
| 1.4 | English description in backend/pyproject.toml | backend/pyproject.toml L4 edit | n/a | n/a |
| 1.5 | README-ZH.md tagline preserved | README-ZH.md (no L7 edit) | n/a | n/a |
| 2.1 | Rename screenshot{1..6} | `git mv` of six files | n/a | n/a |
| 2.2 | Rename Wuhan video cover | `git mv` of one file | n/a | n/a |
| 2.3 | Rename Red Chamber video cover | `git mv` of one file | n/a | n/a |
| 2.4 | Rename QQ group image | `git mv` of one file | n/a | n/a |
| 2.5 | Byte-preserving rename | `git mv` mechanism choice | n/a | n/a |
| 2.6 | No duplicate copies | `git mv` (atomic rename) + `git status` verification | n/a | n/a |
| 3.1 | README.md image references updated | README.md L52–61, 71, 79, 220 edits | n/a | n/a |
| 3.2 | README-EN.md image references updated | README-EN.md L52–61, 71, 79, 220 edits | n/a | n/a |
| 3.3 | README-ZH.md image references updated | README-ZH.md L52–61, 71, 79, 220 edits | n/a | n/a |
| 3.4 | No broken images on render | Post-edit verification step | n/a | n/a |
| 4.1 | No Chinese chars in README.md body (excl. switcher) | Verification scan | n/a | n/a |
| 4.2 | No Chinese chars in README-EN.md body (excl. switcher) | Verification scan | n/a | n/a |
| 4.3 | Reviewer-runnable scan returns zero matches | `rg` command in design + commit message | n/a | n/a |
## Components and Interfaces
This spec has no software components, services, or APIs. The "components" reduce to two textual operations (translate + rename) and one verification.
| Operation | Layer | Intent | Req Coverage | Key Dependencies | Contracts |
|-----------|-------|--------|--------------|------------------|-----------|
| Tagline translation | Docs / Metadata | Replace Chinese tagline with English in 4 files | 1.1, 1.2, 1.3, 1.4 | Edit tool | n/a |
| Asset rename + reference update | Static assets / Docs | Rename 9 files; update `<img src>` in 3 READMEs | 2.1–2.6, 3.1–3.4 | `git mv`, Edit tool | n/a |
| Verification scan | Acceptance gate | Confirm no residual Chinese in EN READMEs body | 4.1, 4.2, 4.3 | ripgrep | Commit message records the scan command and result |
### Verification Contract
The acceptance gate is a single ripgrep invocation, runnable by any reviewer:
```
rg --pcre2 '[\x{4e00}-\x{9fff}]' README.md README-EN.md \
| rg -v 'README-ZH\.md'
```
**Preconditions**: All edits and renames committed.
**Postconditions**: The pipeline returns zero lines (the only Chinese characters left are in `[中文文档](./README-ZH.md)`, which the second `rg` filters out by matching the `README-ZH.md` substring on the same line).
**Invariants**: `README-ZH.md` body is not modified by this scan logic; the language-switcher line in the EN READMEs is the sole expected exemption.
## Data Models
Not applicable. No data structures are added or modified.
## Error Handling
### Error Strategy
Failure modes are limited to (a) a `git mv` failing because a path was mistyped (immediately visible at command-execution time) and (b) a `<img src>` left pointing at an old Chinese-named filename (caught by the verification scan).
### Error Categories and Responses
- **Mistyped rename target**: `git mv` fails with a clear error; re-run with the correct path.
- **Missed reference update**: Verification scan returns the offending file/line; fix and re-scan.
- **Accidental binary re-encoding**: `git diff --stat` of the asset file shows non-zero content delta; abandon the change and redo with `git mv`.
### Monitoring
Not applicable for a one-shot docs change. The PR diff plus the verification-scan output in the PR description serve as the audit trail.
## Testing Strategy
This is a documentation/asset change with no executable code. Testing is review-time:
- **Verification scan (mandatory)**: Run the ripgrep command in §"Verification Contract" against the working tree before commit; expect zero output. Re-run once more in CI / on the PR branch.
- **Rendered-preview check (mandatory)**: Open `README.md`, `README-EN.md`, `README-ZH.md` in GitHub's rendered-markdown view (or a local Markdown previewer) on the feature branch and confirm:
1. The tagline appears once, in English, on `README.md` and `README-EN.md`.
2. All six screenshot tiles render.
3. Both video-cover thumbnails render.
4. The QQ group image renders.
5. `README-ZH.md` still renders identically except for the new ASCII image URLs.
- **`git diff --stat` check (mandatory)**: For each of the nine asset files, the stat must show `0 insertions(+), 0 deletions(-)` (pure rename). If any asset shows a content delta, the rename was performed incorrectly.
## Optional Sections
### Migration Strategy
No data migration. The "migration" is a single PR containing all renames + edits. There is no rollback step beyond a normal `git revert` of the merge commit if a broken image is reported post-merge.

131
.kiro/specs/i18n-readme-tagline-and-assets/gap-analysis.md Normal file
View File

@ -0,0 +1,131 @@
# Gap Analysis — i18n-readme-tagline-and-assets
## 1. Current State Investigation
### Scope ground truth
Ripgrep `[\x{4e00}-\x{9fff}]` over `README.md`, `README-EN.md`, `package.json`, and `backend/pyproject.toml` returns the following Chinese-character lines that fall under this feature's mandate:
| File | Line | Content (excerpt) | Category |
| --- | ---: | --- | --- |
| `README.md` | 7 | `简洁通用的群体智能引擎，预测万物` | Tagline |
| `README.md` | 23 | `[English](./README.md) \| [中文文档](./README-ZH.md)` | Language switcher (allowed) |
| `README.md` | 52–61 | `./static/image/Screenshot/运行截图{1..6}.png` (×6) | Asset path |
| `README.md` | 71 | `./static/image/武大模拟演示封面.png` | Asset path |
| `README.md` | 79 | `./static/image/红楼梦模拟推演封面.jpg` | Asset path |
| `README.md` | 220 | `./static/image/QQ群.png` | Asset path (not listed in ticket scope, see Gap §3) |
| `README-EN.md` | 7, 23, 52–61, 71, 79, 220 | identical structure to README.md | Same categories |
| `package.json` | 4 | `"description": "MiroFish - 简洁通用的群体智能引擎，预测万物"` | Tagline |
| `backend/pyproject.toml` | 4 | `description = "MiroFish - 简洁通用的群体智能引擎，预测万物"` | Tagline (twin string, not in original ticket) |
`README-ZH.md` carries Chinese body text by design (out of scope) but its asset paths must still be updated to point at the renamed ASCII files.
### Tracked image files (`git ls-files static/image/`)
```
static/image/MiroFish_logo.jpeg
static/image/MiroFish_logo_compressed.jpeg
static/image/QQ群.png
static/image/Screenshot/运行截图{1..6}.png
static/image/shanda_logo.png
static/image/武大模拟演示封面.png
static/image/红楼梦模拟推演封面.jpg
```
Nine files have Chinese names: six screenshots + `QQ群.png` + `武大模拟演示封面.png` + `红楼梦模拟推演封面.jpg`.
### Tagline structure observation
`README.md` lines 7–9 currently read:
```
简洁通用的群体智能引擎，预测万物
</br>
<em>A Simple and Universal Swarm Intelligence Engine, Predicting Anything</em>
```
The English equivalent already exists immediately below the Chinese as italic subtitle. Naive replacement would produce a duplicate (English in plain text + the same English in italic). The natural i18n collapse is to delete the Chinese line plus the `</br>` separator and let the existing `<em>` line stand alone. `README-EN.md` has the identical structure.
### Conventions to respect (from steering)
- `tech.md`: 4-space indent, no enforced linter, "match the surrounding file's style". Shell scripts must quote paths with spaces / non-ASCII characters per `.claude/rules/file-paths.md`.
- `commits.md`: Conventional Commits, lowercase, imperative, max 72 chars, no `Co-Authored-By:` footer. Branch `<type>/<ticket>-<desc>` — ticket dictates `chore/i18n-12-readme-tagline-and-assets` (or similar).
- `dev-guidelines.md`: kebab-case filenames for assets is consistent with the project's frontend file conventions.
### Existing precedent in the same i18n epic
Recently merged child issues of epic #11 (`#7`, `#9`, `#3`, `#5`, `#6`) have all been small, focused docs/tooling PRs. This is consistent with treating #12 as an S-effort docs cleanup.
## 2. Requirements Feasibility Analysis
### Per-requirement asset map
| Req | What it needs | Where it lives | Gap |
| --- | --- | --- | --- |
| R1 (tagline) | English tagline | `README.md:7-9`, `README-EN.md:7-9`, `package.json:4`, `backend/pyproject.toml:4` | **Editorial** — straight string edit. No code paths affected. |
| R2 (asset rename) | Rename 8 files (6 screenshots + 2 video covers) | `static/image/Screenshot/`, `static/image/` | **`git mv`** — preserves history. No callers outside READMEs found by grep. |
| R3 (README references updated) | Update `<img src>` paths | `README.md`, `README-EN.md`, `README-ZH.md` | **Editorial** — straight string edits. |
| R4 (no residual Chinese in EN READMEs) | Verifiable scan | Both `README.md` and `README-EN.md` | **Constraint surfaces extra asset** — `QQ群.png` (line 220) is not in the explicit ticket asset list but its src path contains Chinese, which would fail R4's verification. See Gap §3. |
### Gaps tagged
- **Constraint:** `static/image/QQ群.png` is referenced by all three READMEs but is **not explicitly listed in the ticket's scope bullets**, while the ticket's own acceptance criterion ("No Chinese characters in `README.md`, `README-EN.md` body text") would still flag its src path. Either we (a) expand scope to rename it as well or (b) accept a deviation. Recommendation: expand scope — same shape of fix, trivial cost, satisfies the literal acceptance criterion.
- **Constraint:** `backend/pyproject.toml:4` carries the identical Chinese tagline string as `package.json:4`. Not in original ticket bullets but is the obvious twin and would surprise a reviewer reading the diff. Already incorporated into requirements.md R1 acceptance criterion 4.
- **Unknown / Research Needed (minor):** Confirm GitHub Pages, the live demo site, and any external link to the screenshots do not deep-link into Chinese-named asset URLs. Quick `gh` / web check during design phase will resolve.
## 3. Implementation Approach Options
This is a docs/asset-rename feature. There is no algorithm to design — the only real decision is whether the renames go through `git mv` (preserves history) or `git rm`/`git add` (loses history). And whether to expand scope to `QQ群.png`.
### Option A — Strict ticket scope (no QQ群.png rename)
- Rename only the eight assets explicitly listed: `运行截图{1..6}.png`, `武大模拟演示封面.png`, `红楼梦模拟推演封面.jpg`.
- Translate taglines in `README.md`, `README-EN.md`, `package.json`, `backend/pyproject.toml`.
- Skip `QQ群.png`.
**Trade-offs:**
- ✅ Smallest possible diff; no scope creep.
- ❌ Acceptance criterion R4 ("no Chinese characters in README body outside language switcher") fails because line 220 still contains `QQ群` in the src path.
### Option B — Expanded scope including QQ群.png (RECOMMENDED)
- Same as Option A, plus rename `static/image/QQ群.png` → `static/image/qq-group.png` (or similar) and update its three references.
**Trade-offs:**
- ✅ Satisfies the ticket's own R4 acceptance criterion literally.
- ✅ One additional `git mv` + 3 string edits — negligible cost.
- ❌ Slightly broader than the ticket bullets (but explicitly justified by the ticket's own acceptance criteria).
### Option C — Hybrid (rename listed + leave QQ群 + edit alt-only)
Not viable: there is no way to leave the file in place and still satisfy R4 without renaming.
### Decision direction
Recommend Option B. Update requirements R2/R3 to include `QQ群.png` explicitly so the spec is internally consistent with R4.
## 4. Out-of-Scope for Gap Analysis
- Choice of exact ASCII filename slugs (decided in design phase).
- Whether to re-encode any image (No — bytes-preserving rename only, per R2.4).
## 5. Implementation Complexity & Risk
- **Effort:** **S (≈ half-day).** All work is text edits + `git mv` of 9 files + 3 README string-substitution passes + 2 description-field edits. No code changes, no tests.
- **Risk:** **Low.** Single failure mode is broken image links; mitigated by a simple grep + rendered-preview check before commit. No runtime, dependency, or pipeline impact. `git mv` preserves history.
## 6. Recommendations for Design Phase
- Adopt **Option B** (expanded scope including `QQ群.png`).
- Use `git mv` for all renames so history follows.
- Pick deterministic ASCII slugs; propose:
- `Screenshot/screenshot{1..6}.png`
- `wuhan-university-simulation-cover.png`
- `dream-of-the-red-chamber-simulation-cover.jpg`
- `qq-group.png`
- Collapse the duplicated tagline lines in `README.md` / `README-EN.md`: delete the Chinese line + `</br>` separator and let the existing `<em>` English subtitle become the lone tagline (avoids a verbatim-duplicate line).
- Verification step: re-run `rg '[\x{4e00}-\x{9fff}]' README.md README-EN.md package.json backend/pyproject.toml` after edits and confirm only the language-switcher line on each README returns a hit.
## Research items to carry forward
- (Light) confirm no off-repo deep-link into the renamed assets (live demo site, social cards). If a deep link is found, decide whether to leave a redirect / note in the PR.

65
.kiro/specs/i18n-readme-tagline-and-assets/requirements.md Normal file
View File

@ -0,0 +1,65 @@
# Requirements Document
## Project Description (Input)
Translate the Chinese tagline in README.md, README-EN.md, and package.json to English, and rename Chinese-named image asset files in static/image/Screenshot/ to ASCII filenames (Option A from the ticket), updating all references in README.md and README-ZH.md. Acceptance: no Chinese characters in README.md or README-EN.md body text (except the language switcher link to README-ZH.md); package.json description in English; all image links work. Source: GitHub issue #12 (.ticket/12.md).
## Introduction
This feature removes the remaining Chinese surface text from the English documentation entry points (`README.md`, `README-EN.md`) and from the npm package metadata (`package.json`), and replaces Chinese-named image asset filenames under `static/image/` with ASCII equivalents so that asset URLs are CDN- and tooling-friendly. References to those assets are updated in all three READMEs (`README.md`, `README-EN.md`, `README-ZH.md`) so that the Chinese-language entry point continues to render correctly. The Chinese-language README (`README-ZH.md`) keeps its Chinese body text by design.
## Boundary Context
- **In scope**:
- English tagline replacing Chinese tagline in `README.md`, `README-EN.md`, and `package.json` `description`.
- Renaming `static/image/Screenshot/运行截图{1..6}.png` to ASCII filenames.
- Renaming `static/image/武大模拟演示封面.png` and `static/image/红楼梦模拟推演封面.jpg` to ASCII filenames.
- Renaming `static/image/QQ群.png` to an ASCII filename (added per gap-analysis: required by R4 because the existing src path on README.md:220 / README-EN.md:220 contains Chinese characters and would fail the "no Chinese characters in body text" check).
- Updating all `<img src="...">` references to those renamed files in `README.md`, `README-EN.md`, and `README-ZH.md`.
- Updating `backend/pyproject.toml` `description` field, which carries an identical Chinese tagline string (adjacent twin of `package.json`).
- **Out of scope**:
- Translating the body of `README-ZH.md` (Chinese variant by design).
- Translating the language switcher link label `[中文文档]` (allowed by acceptance criteria).
- Touching `locales/zh.json` Chinese tagline value (legitimate Chinese locale content).
- **Adjacent expectations**:
- The ticket recommends Option A (rename to ASCII). This spec adopts Option A.
- This work is a child of the i18n epic (#11) and follows the project's existing `i18n-*` spec naming.
## Requirements
### Requirement 1: English tagline in English-facing documentation
**Objective:** As a non-Chinese-reading visitor landing on the GitHub repo or installing the npm package, I want the tagline in the English README files and the npm package metadata to be in English, so that I am not surprised by untranslated Chinese strings on the entry surface.
#### Acceptance Criteria
1. The README.md file shall contain the English tagline `A Simple and Universal Swarm Intelligence Engine, Predicting Anything` in place of the Chinese tagline `简洁通用的群体智能引擎，预测万物` on the same line.
2. The README-EN.md file shall contain the same English tagline replacement on the corresponding line.
3. The package.json `description` field shall contain an English description (no Chinese characters).
4. The backend/pyproject.toml `description` field shall contain the same English description used in package.json.
5. The README-ZH.md file shall keep its Chinese tagline unchanged.
### Requirement 2: ASCII filenames for screenshot and video-cover assets
**Objective:** As a developer cloning the repo or a CDN serving these assets, I want all image filenames under `static/image/` referenced from the READMEs to be ASCII, so that paths are URL-safe, copy-pasteable, and friendly to tools that mishandle non-ASCII filenames.
#### Acceptance Criteria
1. The `static/image/Screenshot/运行截图{N}.png` files (for N from 1 to 6) shall be renamed to `static/image/Screenshot/screenshot{N}.png`.
2. The `static/image/武大模拟演示封面.png` file shall be renamed to `static/image/wuhan-university-simulation-cover.png`.
3. The `static/image/红楼梦模拟推演封面.jpg` file shall be renamed to `static/image/dream-of-the-red-chamber-simulation-cover.jpg`.
4. The `static/image/QQ群.png` file shall be renamed to `static/image/qq-group.png`.
5. The renamed asset files shall preserve the original byte content (rename only, no re-encoding).
6. The static/image/ directory shall not contain duplicate copies of the renamed files (the original Chinese-named files are removed, not kept alongside).
### Requirement 3: All README references updated to the ASCII filenames
**Objective:** As a reader of any README variant, I want the screenshot and video-cover images to render correctly, so that the documentation remains visually intact after the rename.
#### Acceptance Criteria
1. The README.md file shall reference each renamed image at its new ASCII path; no `<img src="...">` in the file shall point to a Chinese-named file under `static/image/`.
2. The README-EN.md file shall reference each renamed image at its new ASCII path; no `<img src="...">` in the file shall point to a Chinese-named file under `static/image/`.
3. The README-ZH.md file shall reference each renamed image at its new ASCII path; no `<img src="...">` in the file shall point to a Chinese-named file under `static/image/`.
4. When a reader views the rendered README on GitHub after the change, the system shall display every screenshot and video-cover image without a broken-image placeholder.
### Requirement 4: No residual Chinese in English README body text
**Objective:** As a reviewer verifying acceptance, I want a single objective check that confirms `README.md` and `README-EN.md` body text contains no Chinese characters (apart from the explicit allowance for the language-switcher link), so that the acceptance criteria from the ticket are unambiguously satisfied.
#### Acceptance Criteria
1. The README.md file shall contain no Chinese characters (Unicode CJK Unified Ideographs blocks U+4E00–U+9FFF and adjacent CJK punctuation) outside of the language-switcher link `[中文文档](./README-ZH.md)`.
2. The README-EN.md file shall contain no Chinese characters outside of the same language-switcher link.
3. If a reviewer runs a Chinese-character scan over `README.md` and `README-EN.md` excluding the language-switcher line, the scan shall report zero matches.

107
.kiro/specs/i18n-readme-tagline-and-assets/research.md Normal file
View File

@ -0,0 +1,107 @@
# Research & Design Decisions — i18n-readme-tagline-and-assets
## Summary
- **Feature**: `i18n-readme-tagline-and-assets`
- **Discovery Scope**: Simple Addition (docs cleanup + asset rename, no runtime code paths)
- **Key Findings**:
- The duplicate Chinese-tagline / English-`<em>` structure on lines 7–9 of `README.md` and `README-EN.md` means a verbatim translation produces a duplicate; a structural collapse is preferable.
- `git ls-files` shows nine Chinese-named assets under `static/image/`; only the eight visible in READMEs need renaming for this spec (the `MiroFish_logo` files and `shanda_logo.png` already use ASCII names).
- `backend/pyproject.toml:4` is a twin of `package.json:4` (identical Chinese tagline string); leaving it untranslated would visibly contradict the spec's intent.
## Research Log
### Topic — Inventory of Chinese-named assets and references
- **Context**: Confirm the full set of files and references the spec must touch so no broken-image regression slips in.
- **Sources Consulted**: `git ls-files static/image/`, `rg '[\x{4e00}-\x{9fff}]'` over `README.md`, `README-EN.md`, `README-ZH.md`, `package.json`, `backend/pyproject.toml`.
- **Findings**:
- Tracked Chinese-named files (9): `QQ群.png`, six `Screenshot/运行截图{N}.png`, `武大模拟演示封面.png`, `红楼梦模拟推演封面.jpg`.
- Each Chinese-named asset is referenced exactly three times — once in each README. No code path or test references them.
- `locales/zh.json:36` contains the tagline as a Chinese-locale value (legitimate, out of scope).
- **Implications**: The rename is a closed set: 9 file moves + (3 README × N references) edits. No runtime impact.
### Topic — Tagline structure on lines 7–9
- **Context**: Decide the cleanest replacement for the Chinese tagline on the English-facing READMEs.
- **Sources Consulted**: `README.md:7-9`, `README-EN.md:7-9`.
- **Findings**: The current structure is `<chinese tagline>\n</br>\n<em>English equivalent</em>`. The English subtitle already exists. Naive replacement (substitute Chinese with English on line 7) produces `<english>\n</br>\n<em>English</em>` — visible duplicate.
- **Implications**: Collapse to the single existing `<em>` line by deleting the Chinese tagline line and the `</br>` separator on both files.
### Topic — `git mv` vs. `rm`/`add` for renames
- **Context**: Choose a rename mechanism that preserves blame/history on the assets.
- **Sources Consulted**: Project commit history shows `git mv` usage for prior renames (no formal rule, but consistent practice).
- **Findings**: `git mv "old" "new"` records a rename in the index. Git's heuristic file-move detection also picks up `rm + add` of identical bytes, but `git mv` is unambiguous and preserves rename detection across thresholds.
- **Implications**: Use `git mv` for all nine renames. Quote source paths (rule from `.claude/rules/file-paths.md`) since they contain non-ASCII characters.
### Topic — Off-repo deep links to renamed assets (light check)
- **Context**: The ticket's gap analysis flagged a research item: confirm no external pages deep-link the Chinese-named files.
- **Sources Consulted**: `git grep` of repo (no off-repo references). The bilibili links in the READMEs point to videos, not to the cover images. The `mirofish-live-demo` site and `Trendshift` badge are independent assets hosted elsewhere.
- **Findings**: No in-repo references outside the READMEs. Out-of-repo deep links are not enumerable from inside the repo; the cost of a broken external deep link is low (a missing image on someone else's page) and accepted. If a deep link surfaces post-merge, a same-day re-add of a redirect symlink resolves it.
- **Implications**: Proceed with hard renames; no redirect/copy-on-rename needed.
## Architecture Pattern Evaluation
| Option | Description | Strengths | Risks / Limitations | Notes |
|--------|-------------|-----------|---------------------|-------|
| Strict ticket scope | Rename only the 8 explicitly listed assets; leave `QQ群.png` | Smallest diff | Fails the ticket's own R4 acceptance criterion | Rejected |
| Expanded scope (selected) | Also rename `QQ群.png` and update `backend/pyproject.toml` | Internally consistent with R4; trivial cost | Slightly broader than ticket bullets | Selected |
| Hybrid (allow exception in R4) | Rename the 8 listed, exempt `QQ群` in the verification scan | Preserves the ticket bullets exactly | Adds an explicit ad-hoc exception that future readers must decode | Rejected |
## Design Decisions
### Decision: Rename `static/image/QQ群.png` to ASCII despite not being in the ticket's bullet list
- **Context**: Acceptance criterion R4 ("no Chinese characters in `README.md` / `README-EN.md` body") would fail because `QQ群` appears in the `<img src>` path on line 220 of both files.
- **Alternatives Considered**:
1. Strict scope — leave `QQ群.png` and accept R4 fail.
2. Expand scope — rename and update.
3. Exempt `QQ群.png` in R4's verification scope with explicit allow-list.
- **Selected Approach**: Expand scope. Rename `static/image/QQ群.png` → `static/image/qq-group.png`, update three references.
- **Rationale**: Trivial cost; same fix shape as the listed assets; the ticket's own acceptance criterion is the source of truth.
- **Trade-offs**: One extra file move. None material.
- **Follow-up**: None.
### Decision: Translate `backend/pyproject.toml:4` description in the same PR
- **Context**: `backend/pyproject.toml` carries the identical Chinese tagline as `package.json`. Leaving it untranslated produces a half-finished diff.
- **Alternatives Considered**:
1. Leave it for a follow-up ticket.
2. Translate it now alongside `package.json`.
- **Selected Approach**: Translate now.
- **Rationale**: Identical string, identical fix, same review surface. Splitting would create needless coordination.
- **Trade-offs**: One additional one-line diff. None material.
- **Follow-up**: None.
### Decision: Collapse duplicate tagline structure rather than substitute in place
- **Context**: Lines 7–9 of `README.md` and `README-EN.md` would yield a verbatim duplicate after a one-for-one Chinese-to-English substitution.
- **Alternatives Considered**:
1. Substitute Chinese line in place (produces duplicate).
2. Delete Chinese line + `</br>` separator; let the existing `<em>` line stand alone.
3. Delete the existing `<em>` line; keep a single non-italic English tagline on line 7.
- **Selected Approach**: Option 2 — delete lines 7 and 8, keep line 9 (`<em>` English tagline).
- **Rationale**: Preserves the existing visual treatment (italic subtitle below the Trendshift badge). Avoids style drift on a docs-only PR.
- **Trade-offs**: Slightly different visual weight (italic only) vs. the prior bilingual stack (plain Chinese + italic English). Acceptable for an English-facing doc.
- **Follow-up**: None.
### Decision: Use `git mv` for all renames
- **Context**: Need to preserve rename detection.
- **Alternatives Considered**: `git mv` vs. shell `mv` + `git rm` / `git add`.
- **Selected Approach**: `git mv "old" "new"` with quoted paths.
- **Rationale**: Unambiguous record in the index; matches existing project practice.
- **Trade-offs**: None.
- **Follow-up**: None.
## Risks & Mitigations
- **Risk:** Broken images on rendered GitHub README after merge. **Mitigation:** Post-edit grep to confirm zero remaining Chinese-named asset references in any README; preview rendered markdown locally or on a branch before merge.
- **Risk:** Off-repo deep links to old asset URLs (Trendshift cards, social previews). **Mitigation:** Accepted; cost is a single missing image on an external page.
- **Risk:** Diff churn from accidentally re-encoding a binary on macOS or Windows checkout. **Mitigation:** Use `git mv` (no content transform); verify `git diff --stat` shows only renames for the asset files (no content delta).
## References
- Ticket source: `.ticket/12.md` / GitHub issue #12.
- Project rule on quoting paths: `.claude/rules/file-paths.md`.
- Project commit conventions: `.claude/rules/commits.md` and `.kiro/steering/structure.md`.

23
.kiro/specs/i18n-readme-tagline-and-assets/spec.json Normal file
View File

@ -0,0 +1,23 @@
{
"feature_name": "i18n-readme-tagline-and-assets",
"created_at": "2026-05-07T19:24:24Z",
"updated_at": "2026-05-07T19:32:00Z",
"language": "en",
"phase": "tasks-generated",
"ticket": "12",
"approvals": {
"requirements": {
"generated": true,
"approved": true
},
"design": {
"generated": true,
"approved": true
},
"tasks": {
"generated": true,
"approved": false
}
},
"ready_for_implementation": true
}

47
.kiro/specs/i18n-readme-tagline-and-assets/tasks.md Normal file
View File

@ -0,0 +1,47 @@
# Implementation Plan
- [x] 1. Translate Chinese taglines to English in the project's English-facing metadata
- In `README.md`, delete the Chinese tagline line and the immediately following `</br>` line so the existing italic English subtitle on the next line stands as the lone tagline; verify the result still renders with one tagline visible above the Shanda badge
- Apply the identical edit to `README-EN.md`
- In `package.json`, set the `description` value to `MiroFish - A Simple and Universal Swarm Intelligence Engine, Predicting Anything`
- In `backend/pyproject.toml`, set the `description` value to the same English string used in `package.json`
- Leave `README-ZH.md` line 7 (the Chinese tagline) untouched
- Observable completion: a ripgrep scan for `[\x{4e00}-\x{9fff}]` over `README.md`, `README-EN.md`, `package.json`, and `backend/pyproject.toml` returns hits **only** on the language-switcher line of the two READMEs
- _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5_
- [x] 2. (P) Rename Chinese-named static image assets to ASCII filenames using git mv
- Move the six screenshot files `static/image/Screenshot/运行截图{1..6}.png` to `static/image/Screenshot/screenshot{1..6}.png`
- Move `static/image/武大模拟演示封面.png` to `static/image/wuhan-university-simulation-cover.png`
- Move `static/image/红楼梦模拟推演封面.jpg` to `static/image/dream-of-the-red-chamber-simulation-cover.jpg`
- Move `static/image/QQ群.png` to `static/image/qq-group.png`
- Quote source paths in shell invocations because they contain non-ASCII characters
- Use `git mv` (not shell `mv` + `git add`) so rename detection is recorded directly in the index
- Observable completion: `git status` reports nine `renamed:` entries with no other file modifications; `git diff --stat -M` shows zero content-line delta for each asset
- _Requirements: 2.1, 2.2, 2.3, 2.4, 2.5, 2.6_
- _Boundary: static/image/_
- [x] 3. Update README image references to point at the renamed ASCII asset paths
- In `README.md`, rewrite the nine `<img src="...">` paths on lines 52–61, 71, 79, and 220 so each points at the corresponding ASCII filename from task 2
- Apply the identical nine edits to `README-EN.md`
- Apply the identical nine edits to `README-ZH.md` (asset path updates only — Chinese body text and Chinese alt attributes preserved)
- Observable completion: a ripgrep search for `运行截图|武大模拟演示封面|红楼梦模拟推演封面|QQ群` in `README.md`, `README-EN.md`, and `README-ZH.md` returns zero matches
- _Requirements: 3.1, 3.2, 3.3_
- _Depends: 2_
- [x] 4. Verify acceptance gates before commit
- [x] 4.1 Run the Chinese-character verification scan and confirm zero residual hits in the EN READMEs body
- Execute `rg --pcre2 '[\x{4e00}-\x{9fff}]' README.md README-EN.md | rg -v 'README-ZH\.md'` from the repo root
- Observable completion: the pipeline produces zero output lines, confirming the only Chinese characters left in the EN READMEs are inside the language-switcher link to `README-ZH.md`
- _Requirements: 4.1, 4.2, 4.3_
- [x] 4.2 Confirm asset renames are byte-preserving and unambiguous
- Run `git diff --stat -M` and verify each of the nine asset files appears as a pure rename (no `+` or `-` line counts)
- Run `git status` and confirm there are no untracked Chinese-named files left behind in `static/image/` or `static/image/Screenshot/`
- Observable completion: nine `renamed:` entries in `git status`; zero untracked Chinese-named asset files; zero content delta on the asset rows of `git diff --stat`
- _Requirements: 2.5, 2.6, 3.4_
- [x] 4.3 Confirm rendered images by spot-checking the README in a Markdown previewer
- Open `README.md`, `README-EN.md`, and `README-ZH.md` in a Markdown preview (GitHub preview on the feature branch or local previewer) and inspect the screenshot grid, the two video-cover thumbnails, and the QQ group image on each file
- Observable completion: every `<img>` element renders an actual image (no broken-image placeholder) on all three READMEs
- _Requirements: 3.4_
- **Note**: This task ran in an autonomous environment where no Markdown previewer was available; instead, every `<img src>` path in all three READMEs was cross-checked against the working tree and all 33 references resolved to existing files (zero broken paths). A reviewer should still spot-check on the GitHub-rendered PR preview.

78
.kiro/specs/i18n-translate-backend-comments/HANDOFF.md Normal file
View File

@ -0,0 +1,78 @@
# Handoff — `i18n-translate-backend-comments` (Issue #7)
## Status
**Complete.** All in-scope Chinese docstrings and `#` comments under `backend/` have been translated to English.
This second installment of the ticket-#7 cleanup builds on the first installment (PR #20) and finishes the remaining 12 files. Together, the two installments cover the full 35-file in-scope set.
## Completed across both installments (35 files)
### First installment (PR #20 — landed on `feat/i18n-6-externalize-backend-logs`, then merged here via `merge main` into this branch)
- **Root**: `backend/app/__init__.py`, `backend/app/config.py`, `backend/run.py`
- **API package init**: `backend/app/api/__init__.py`
- **Models** (full package): `backend/app/models/__init__.py`, `project.py`, `task.py`
- **Utils** (full package): `backend/app/utils/__init__.py`, `file_parser.py`, `llm_client.py`, `locale.py`, `logger.py`, `retry.py`, `zep_paging.py`
- **Services** (partial): `backend/app/services/__init__.py`, `graph_builder.py`, `ontology_generator.py`, `simulation_ipc.py`, `simulation_manager.py`, `text_processor.py`, `zep_entity_reader.py`
- **Scripts** (partial): `backend/scripts/action_logger.py`, `backend/scripts/test_profile_format.py`
### Second installment (this PR — finishes the ticket)
| File | Starting in-scope hits | Comment-the-obvious deletions |
| --- | --- | --- |
| `backend/app/api/graph.py` | 70 | 25 |
| `backend/app/api/report.py` | 104 | 11 |
| `backend/app/api/simulation.py` | 351 | ~25 |
| `backend/app/services/oasis_profile_generator.py` | 185 | ~14 |
| `backend/app/services/report_agent.py` | 335 | 8 |
| `backend/app/services/simulation_config_generator.py` | 148 | 0 |
| `backend/app/services/simulation_runner.py` | 277 | ~31 |
| `backend/app/services/zep_graph_memory_updater.py` | 97 | 5 |
| `backend/app/services/zep_tools.py` | 269 | 6 |
| `backend/scripts/run_parallel_simulation.py` | 227 | ~7 |
| `backend/scripts/run_reddit_simulation.py` | 75 | 12 |
| `backend/scripts/run_twitter_simulation.py` | 97 | 21 |
| **Total** | **2,235** | **~165** |
After the pass, every file in the table reports zero in-scope hits from the AST scanner.
## Remaining residuals (out of scope — owned by sibling tickets)
After this PR, the only files under `backend/` that still contain CJK characters do so exclusively inside string literals. These are owned by sibling tickets and are intentional residuals for this spec:
- LLM prompt template strings: `oasis_profile_generator.py`, `ontology_generator.py`, `simulation_config_generator.py`, `report_agent.py` — owned by tickets #2 / #3 / #4 / #5.
- Runtime log strings, API response messages, exception arguments, CLI prints: distributed across `api/`, `services/`, `scripts/`, `utils/retry.py`, `utils/locale.py`, `run.py`, `app/config.py` — owned by ticket #6 (with follow-up tickets #18, #24 for residuals).
- Sample-data values returned to clients: `services/zep_tools.py`, `services/zep_graph_memory_updater.py`, `services/zep_entity_reader.py`, etc.
The CJK CI guard (`scripts/ci/i18n_cjk_guard.py`) enforces that this set never grows; the per-path baseline at `.kiro/specs/i18n-ci-guard/baseline.txt` is updated as part of this PR to reflect the new (lower) count.
## Verification methodology
The AST-aware scanner at `.kiro/specs/i18n-translate-backend-comments/scan_chinese.py` (committed in this branch) classifies every CJK-bearing line into one of three buckets:
- `DOCSTRING` — line lies inside a module/class/function docstring (in scope).
- `COMMENT` — line contains a `#` and is not inside a docstring or string-literal span (in scope).
- `STRING` — line is part of a string-literal value (out of scope, owned by sibling tickets).
For every translated file in this installment:
1. `python3 -m py_compile <file>` succeeds.
2. The scanner reports `0` in-scope hits.
3. `git diff <file>` shows only docstring lines and `#` comment lines changed; no signature, import, decorator, expression, or string-literal byte changes.
For two of the largest files (`api/simulation.py`, `report_agent.py`), the implementing agent additionally ran an AST-equivalence check (parsing both before and after, stripping docstrings, and confirming structural equality) to validate that no executable surface changed.
## Test environment caveat
The repo's `uv sync` builds `tiktoken` from source, which requires a Rust toolchain. The sandbox running this implementation pass does not have Rust, so `cd backend && uv run python -m pytest scripts/test_profile_format.py` cannot be executed end-to-end here. Because the change set is comments-and-docstrings-only, runtime behavior cannot be affected; the syntactic-validity check (`py_compile` across all 12 files) stands in for the test run in this environment.
A developer with the project's normal dev environment (Rust toolchain installed, full `uv sync` succeeded) should re-run `cd backend && uv run python -m pytest scripts/test_profile_format.py` against this branch before merging to confirm.
## What is NOT changed
- No string literal anywhere in the touched files (verified by AST classification).
- No executable Python statement.
- No symbol renamed; `zep_*` legacy filenames preserved per steering rule.
- No file added or removed (other than the AST scanner inside `.kiro/specs/i18n-translate-backend-comments/`).
- No dependency added or version-bumped.
## Branch & PR
- Branch: `docs/i18n-7-translate-backend-comments` (re-used from PR #20; that PR was merged into `feat/i18n-6-externalize-backend-logs` after `feat/i18n-6` had already merged into `main`, which orphaned PR #20's content from `main`).
- This PR re-targets the branch at `main`, including: the four prior commits from PR #20, a `Merge branch 'main'` commit (one conflict resolved in `services/ontology_generator.py` to combine PR #20's translated comment with main's English prompt-string), and the new commits for the 12 files completed here.
- Commits follow Conventional Commits in the form `docs(i18n): translate chinese docstrings/comments in backend/<area>`.
- The PR description references issue #7 with `Closes #7`.
- No `Co-Authored-By:` watermarks.

316
.kiro/specs/i18n-translate-backend-comments/design.md Normal file
View File

@ -0,0 +1,316 @@
# Design Document — `i18n-translate-backend-comments`
## Overview
**Purpose**: Translate Chinese-language docstrings and `#` comments across `backend/` Python files into English, so that English-speaking maintainers can read and review the codebase without translation overhead.
**Users**: Backend maintainers and code reviewers who do not read Chinese.
**Impact**: Improves developer ergonomics and review throughput. No runtime, behavior, or interface change. Adjacent i18n tickets (#2/#3/#4/#5/#6), which own the string-literal Chinese, remain unaffected.
### Goals
- Eliminate Chinese characters from docstrings and `#` comments under the in-scope paths.
- Preserve Google-style docstring shape and project formatting rules (4-space indent, ≤120 chars/line, double-quoted strings).
- Keep the diff comments-and-docstrings-only — no executable, string-literal, or symbol changes.
### Non-Goals
- Translating Chinese inside string literals (prompt templates, `logger.{info,warning,error}` arguments, API responses, error messages). These are owned by issues #2/#3/#4/#5/#6.
- Refactoring code, reformatting style, or renaming symbols.
- Introducing new tooling, linters, or CI rules.
- Translating `backend/tests/test_locale*.py` (Chinese there is intentional test data inside string literals; outside ticket scope).
## Boundary Commitments
### This Spec Owns
- Comment and docstring text under: `backend/app/__init__.py`, `backend/app/config.py`, `backend/app/api/`, `backend/app/models/`, `backend/app/services/`, `backend/app/utils/`, `backend/run.py`, `backend/scripts/`.
- The decision rule for distinguishing docstrings from value strings (first-statement rule).
- The Chinese→English Google-style docstring key map.
- The verification workflow (residual `grep`, `pytest`, diff sanity check).
### Out of Boundary
- All string-literal content, including triple-quoted strings used as values.
- Files under `backend/tests/`, `backend/.venv/`, and any non-Python file.
- Refactors, renames, formatting changes, or new dependencies.
- Front-end localization, locale JSON files, or i18n runtime behavior.
### Allowed Dependencies
- The repository's Python source (read + write for in-scope files only).
- The existing test suite (`backend/scripts/test_profile_format.py`) for verification.
- The existing `grep`-based residual scan for verification.
### Revalidation Triggers
- A new in-scope file added under the listed paths (would expand the file list).
- A change to `dev-guidelines.md` regarding docstring style (would change the key map or quote/indent rule).
- A merge of any adjacent i18n ticket (#2/#3/#4/#5/#6) that turns a string literal into a docstring or vice versa.
## Architecture
### Existing Architecture Analysis
This change touches only commentary; no architectural element of the backend is modified. The work spans the following packages:
- `backend/app/__init__.py`, `backend/app/config.py` (Flask app and configuration entrypoint).
- `backend/app/api/` (Flask blueprints).
- `backend/app/models/` (`Project`, `Task` models).
- `backend/app/services/` (graph builder, simulation runner, report agent, etc.).
- `backend/app/utils/` (LLM client, file parser, retry, logger, locale, paging).
- `backend/run.py` (process entrypoint).
- `backend/scripts/` (simulation runners, profile-format test).
### Architecture Pattern & Boundary Map
```mermaid
graph TB
Discovery[Residual Grep Scan]
Plan[Per-Package Plan]
Translator[Translation Pass]
Verify[Verification Gate]
Commit[Per-Package Commit]
PR[Single PR to main]
Discovery --> Plan
Plan --> Translator
Translator --> Verify
Verify -->|all checks pass| Commit
Verify -->|any check fails| Translator
Commit --> Plan
Commit -->|all packages done| PR
```
**Architecture Integration**:
- Selected pattern: **Iterative pass per package** with a verification gate after each pass. Linear, deterministic, low-coordination.
- Domain/feature boundaries: One pass per backend package; commits are package-scoped to keep review chunks small.
- Existing patterns preserved: 4-space indent, double-quoted strings, Google-style docstrings, `snake_case`, project file layout.
- New components rationale: None — no new code, no new files.
- Steering compliance: Conforms to repo-level coding rules and the commits ruleset.
### Technology Stack
| Layer | Choice / Version | Role in Feature | Notes |
|-------|------------------|-----------------|-------|
| Backend / Services | Python ≥3.11 | Source language whose docstrings/comments are being translated | No version change; no dependency change |
| Tooling | `git`, `grep`, `pytest` (existing) | Discovery, verification, regression check | No new tools |
No frontend, data, messaging, or infrastructure layer is touched.
## File Structure Plan
### Directory Structure (no additions, no deletions)
```
backend/
├── app/
│ ├── __init__.py # docstrings/comments only
│ ├── config.py # docstrings/comments only
│ ├── api/ # all *.py: docstrings/comments only
│ ├── models/ # all *.py: docstrings/comments only
│ ├── services/ # all *.py: docstrings/comments only
│ └── utils/ # all *.py: docstrings/comments only
├── run.py # docstrings/comments only
└── scripts/ # all *.py: docstrings/comments only
```
### Modified Files
The 37 in-scope files identified in `gap-analysis.md` are modified — comment and docstring lines only. No other paths are touched.
## Translation Rules
These rules drive the translation pass and the verification gate. They are normative; the implementation must follow them exactly.
### Rule 1 — Docstring vs Value String Disambiguation
A triple-quoted string is treated as a **docstring** (in scope) iff it is the first statement of a module, class, or function body. All other triple-quoted strings are **values** (out of scope) and must not be modified.
### Rule 2 — Translate Docstrings to English Google-style
- Translate Chinese narrative text to faithful English.
- Convert the following Chinese section keys to canonical English Google-style keys when present:
| Chinese key | English key |
| --- | --- |
| `参数:` | `Args:` |
| `返回:` | `Returns:` |
| `异常:` | `Raises:` |
| `产生:` / `生成:` | `Yields:` |
| `示例:` | `Examples:` |
| `注意:` / `备注:` | `Note:` |
- Preserve double-quoted triple-quoted form (`"""..."""`).
- Preserve indentation matching the surrounding scope.
### Rule 3 — Translate Inline `#` Comments to English
- Translate the comment text to English.
- If the translated comment would merely restate the immediately following executable line (a redundant verb-phrase paraphrase), delete the comment.
- Preserve `TODO:` / `FIXME:` markers and any embedded ticket reference verbatim.
- Preserve trailing in-line comments on the same line as code (e.g. `PENDING = "pending" # waiting`).
### Rule 4 — Style Compliance
- Keep every translated line ≤120 characters.
- Do not introduce trailing whitespace.
- Preserve the original indentation of each comment/docstring.
- Use double quotes for any docstring rewritten.
### Rule 5 — Preservation
- Do not modify any executable Python statement.
- Do not modify any string literal (single-, double-, triple-quoted, f-string, raw, byte) that is not a docstring under Rule 1. The single exception is the docstring being rewritten under Rule 2: quote-style normalization to triple double-quoted form (`"""..."""`) is permitted on the docstring only, since it is the artifact under translation.
- Do not rename any symbol.
## System Flows
### Per-package iteration
```mermaid
sequenceDiagram
participant Dev as Translator
participant Repo as Repo
participant Tests as Test Suite
Dev->>Repo: git checkout docs/i18n-7-translate-backend-comments
loop For each package in [models, utils, services, api, scripts, root]
Dev->>Repo: Translate docstrings/comments
Dev->>Repo: git diff --stat (sanity check)
Dev->>Tests: cd backend then uv run python -m pytest scripts/test_profile_format.py
Tests-->>Dev: pass / fail
Dev->>Repo: Re-run residual grep
Repo-->>Dev: residual hits (string-literal only)
Dev->>Repo: git commit -m "docs(i18n): translate chinese docstrings/comments in backend/<area>"
end
Dev->>Repo: gh pr create -> single PR closing #7
```
## Requirements Traceability
| Requirement | Summary | Components | Interfaces | Flows |
|-------------|---------|------------|------------|-------|
| 1.1 | No Chinese in docstrings under in-scope paths | Translation Pass | Rule 1, Rule 2 | Per-package iteration |
| 1.2 | No Chinese in `#` comments under in-scope paths | Translation Pass | Rule 3 | Per-package iteration |
| 1.3 | Residual grep returns only string-literal Chinese | Verification Gate | Residual grep workflow | Per-package iteration |
| 1.4 | Google-style docstring shape preserved | Translation Pass | Rule 2 (key map) | — |
| 2.1 | No executable statement modified | Verification Gate | Rule 5 | Per-package iteration |
| 2.2 | No string literal modified | Verification Gate | Rule 1 (first-statement rule), Rule 5 | Per-package iteration |
| 2.3 | No symbol renamed | Verification Gate | Rule 5 | Per-package iteration |
| 2.4 | `pytest` passes | Verification Gate | Test suite invocation | Per-package iteration |
| 2.5 | Hunks touching code rejected | Verification Gate | `git diff --stat` review | Per-package iteration |
| 3.1 | Drop redundant comments | Translation Pass | Rule 3 | — |
| 3.2 | Translate the *why* faithfully | Translation Pass | Rule 3 | — |
| 3.3 | Preserve `TODO:`/`FIXME:` and ticket refs | Translation Pass | Rule 3 | — |
| 3.4 | No new comments introduced | Translation Pass | Rule 3 | — |
| 4.1 | ≤120 chars/line | Verification Gate | Rule 4 | — |
| 4.2 | No trailing whitespace | Verification Gate | Rule 4 | — |
| 4.3 | Preserve indentation | Translation Pass | Rule 4 | — |
| 4.4 | Double quotes on rewritten docstrings | Translation Pass | Rule 4 | — |
| 4.5 | Preserve 4-space indentation | Translation Pass | Rule 4 | — |
| 5.1 | Use grep for discovery | Verification Gate | Discovery scan | — |
| 5.2 | Re-run grep after each batch | Verification Gate | Residual grep workflow | Per-package iteration |
| 5.3 | Continue until non-string-literal residual cleared | Verification Gate | Rule 1 disambiguation | Per-package iteration |
| 5.4 | `git diff --stat` only in-scope paths | Verification Gate | Diff sanity check | Per-package iteration |
| 6.1 | Branch `docs/i18n-7-translate-backend-comments` | Tracking & Branching | `/done` skill | — |
| 6.2 | Reference issue #7 | Tracking & Branching | Commit/PR template | — |
| 6.3 | Conventional Commits `docs(i18n)` | Tracking & Branching | `.claude/rules/commits.md` | — |
| 6.4 | No unrelated changes | Verification Gate | Diff sanity check | — |
## Components and Interfaces
| Component | Domain/Layer | Intent | Req Coverage | Key Dependencies (P0/P1) | Contracts |
|-----------|--------------|--------|--------------|--------------------------|-----------|
| Translation Pass | Process | Apply Rules 1–5 to one package's `*.py` | 1.1, 1.2, 1.4, 3.1, 3.2, 3.3, 3.4, 4.3, 4.4, 4.5 | None (manual + AI-assisted) | Process |
| Verification Gate | Process | Run residual grep, `pytest`, and diff sanity check after each package | 1.3, 2.1, 2.2, 2.3, 2.4, 2.5, 4.1, 4.2, 5.1, 5.2, 5.3, 5.4, 6.4 | `git`, `grep`, `pytest` (P0) | Process |
| Tracking & Branching | Process | Branching, commit messages, PR | 6.1, 6.2, 6.3 | `/done` skill, `gh` CLI (P0) | Process |
### Process
#### Translation Pass
| Field | Detail |
|-------|--------|
| Intent | Translate docstrings and `#` comments in one package without touching code or string literals |
| Requirements | 1.1, 1.2, 1.4, 3.1, 3.2, 3.3, 3.4, 4.3, 4.4, 4.5 |
**Responsibilities & Constraints**
- Apply Rule 1 (first-statement disambiguation) before editing any triple-quoted string.
- Apply Rule 2 (key map) for any Chinese Google-style key encountered.
- Apply Rule 3 to inline comments; delete redundant ones.
- Operate on one package at a time; do not interleave packages.
**Dependencies**
- Inbound: Verification Gate (provides feedback if a previous batch failed).
- Outbound: Verification Gate (hands off post-pass).
- External: None.
**Contracts**: Process [x] / Service [ ] / API [ ] / Event [ ] / Batch [ ] / State [ ]
**Implementation Notes**
- Integration: Operates directly on the working tree on branch `docs/i18n-7-translate-backend-comments`.
- Validation: After each file is rewritten, sanity-check that the diff for that file shows changes only on comment/docstring lines.
- Risks: Accidental edit to a string-literal triple-quoted value — mitigated by Rule 1 + diff review.
#### Verification Gate
| Field | Detail |
|-------|--------|
| Intent | Confirm a package's translation pass left runtime behavior intact |
| Requirements | 1.3, 2.1, 2.2, 2.3, 2.4, 2.5, 4.1, 4.2, 5.1, 5.2, 5.3, 5.4, 6.4 |
**Responsibilities & Constraints**
- Re-run `grep -rln '[一-鿿]' backend/ --include='*.py'` after each package and confirm residual hits are limited to string-literal Chinese owned by adjacent tickets.
- Run `uv run python -m pytest backend/scripts/test_profile_format.py` and confirm exit 0.
- Run `git diff --stat` and confirm only in-scope file paths are listed.
- Spot-check a sample of changed files to confirm only comment/docstring lines changed.
**Dependencies**
- Inbound: Translation Pass.
- Outbound: Tracking & Branching (commits) when all checks pass; loops back to Translation Pass otherwise.
- External: `git`, `grep`, `pytest` (P0 — required for verification).
**Contracts**: Process [x] / Service [ ] / API [ ] / Event [ ] / Batch [ ] / State [ ]
**Implementation Notes**
- Integration: Run from the repo root; no environment variables required beyond what `uv run` already provides.
- Validation: All four checks (grep / pytest / diff scope / spot diff) must pass before committing.
- Risks: A flaky `pytest` run unrelated to this change would block progress — mitigated by reading the failure and re-running once.
#### Tracking & Branching
| Field | Detail |
|-------|--------|
| Intent | Branch, commit, push, and open PR per project conventions |
| Requirements | 6.1, 6.2, 6.3 |
**Responsibilities & Constraints**
- Branch name: `docs/i18n-7-translate-backend-comments`.
- Commit messages follow Conventional Commits with `docs(i18n)` scope (e.g. `docs(i18n): translate chinese docstrings/comments in backend/services`).
- PR closes #7 and references the spec.
**Dependencies**
- Inbound: Verification Gate (only commits when all checks pass).
- External: `gh` CLI (P0), `/done` skill (P0).
**Contracts**: Process [x] / Service [ ] / API [ ] / Event [ ] / Batch [ ] / State [ ]
**Implementation Notes**
- Integration: Use `/done` skill at the end to handle branch/push/PR uniformly.
- Validation: Confirm PR body references issue #7 with `Closes #7` and lists each commit.
- Risks: None.
## Error Handling
### Error Strategy
This is a build-time / source-edit task — there is no runtime error path. Errors are caught by the Verification Gate.
### Error Categories and Responses
- **Translation slipped into a string literal**: caught by `git diff --stat` + spot diff. Response: revert that hunk, re-apply translation against the docstring/comment only.
- **Test suite fails after a pass**: caught by `pytest`. Response: read failure, identify which line was incorrectly modified (likely a string the translator misclassified as a docstring), revert that hunk, re-apply.
- **Residual grep returns non-string-literal Chinese**: caught by post-pass grep. Response: classify those hits as in-scope and translate them in the next sub-pass.
- **Line exceeds 120 chars after translation**: caught by spot diff. Response: reflow the comment/docstring without changing executable code.
### Monitoring
None — this is a one-shot change. No production observability required.
## Testing Strategy
The repository's existing tests are the safety net. No new tests are added.
### Default sections
- **Unit Tests**: Not applicable; nothing executable changes.
- **Integration Tests**: `uv run python -m pytest backend/scripts/test_profile_format.py` must continue to pass after each commit.
- **E2E/UI Tests**: Not applicable.
- **Verification checks (per package commit)**:
1. Residual `grep -rln '[一-鿿]' backend/ --include='*.py'` (run from repo root) returns only files whose remaining Chinese is in string literals owned by adjacent tickets.
2. `cd backend && uv run python -m pytest scripts/test_profile_format.py` exits 0.
3. `git diff --stat HEAD~..HEAD` shows only in-scope file paths.
4. Spot diff on three random changed files confirms only comment/docstring lines changed.
## Supporting References (Optional)
- `gap-analysis.md` — full file enumeration and pattern survey.
- `research.md` — discovery log, alternatives, and decisions.

92
.kiro/specs/i18n-translate-backend-comments/gap-analysis.md Normal file
View File

@ -0,0 +1,92 @@
# Gap Analysis — `i18n-translate-backend-comments`
## Scope Recap
- **Ticket**: salestech-group/MiroFish#7
- **Goal**: Translate Chinese docstrings and `#` comments in `backend/` to English without behavior changes.
- **Blast radius**: Comments and docstrings only; runtime semantics preserved.
## Current State Investigation
### Discovered files
A scan with the regex `[一-鿿]` across `backend/**/*.py` (excluding `.venv`) returns **37 in-app files** plus 2 test files:
| Area | Count | Files |
| --- | --- | --- |
| `backend/app/__init__.py` | 1 | `__init__.py` |
| `backend/app/config.py` | 1 | `config.py` |
| `backend/app/api/` | 4 | `__init__.py`, `graph.py`, `report.py`, `simulation.py` |
| `backend/app/models/` | 3 | `__init__.py`, `project.py`, `task.py` |
| `backend/app/services/` | 12 | `__init__.py`, `graph_builder.py`, `oasis_profile_generator.py`, `ontology_generator.py`, `report_agent.py`, `simulation_config_generator.py`, `simulation_ipc.py`, `simulation_manager.py`, `simulation_runner.py`, `text_processor.py`, `zep_entity_reader.py`, `zep_graph_memory_updater.py`, `zep_tools.py` |
| `backend/app/utils/` | 7 | `__init__.py`, `file_parser.py`, `llm_client.py`, `locale.py`, `logger.py`, `retry.py`, `zep_paging.py` |
| `backend/run.py` | 1 | `run.py` |
| `backend/scripts/` | 5 | `action_logger.py`, `run_parallel_simulation.py`, `run_reddit_simulation.py`, `run_twitter_simulation.py`, `test_profile_format.py` |
| `backend/tests/` (extra, not in ticket file list) | 2 | `test_locale.py`, `test_locale_request_resolution.py` |
Spot checks (`models/task.py`, `models/project.py`, `services/text_processor.py`, `utils/locale.py`):
- Module-level docstrings in Chinese (e.g. `"""任务状态管理"""`).
- Class/method docstrings in Chinese, often Google-shaped (`Args:` translated as `参数:`).
- Inline `#` comments tagging fields, sections, or restating obvious code (e.g. `# 标准化换行` above an `\n` normalization call).
- Status-enum trailing comments (e.g. `PENDING = "pending" # 等待中`).
### Conventions to preserve
- Project guideline: 4-space indent, max 120 char/line, double-quoted strings (Python).
- Docstring style: Google-style per `dev-guidelines.md`. Existing files mix English-shape `Args:`/`Returns:` keys with Chinese descriptions, or use Chinese keys (`参数:`, `返回:`). Translate both to canonical Google-style English.
- File-level convention: `snake_case` filenames, Python `__init__.py` modules typically have a one-line module docstring.
### Integration surfaces
None. This work touches only commentary; no API contracts, schemas, or imports change.
## Requirements Feasibility
| Requirement | Status | Notes |
| --- | --- | --- |
| R1 (coverage) | Feasible — straightforward | Files identified by `grep` rule. |
| R2 (behavior preservation) | Feasible | Achieved by limiting diffs to comment/docstring lines. Need to be careful with multi-line triple-quoted docstrings vs string literals (they are syntactically identical to strings — disambiguation: docstring is the *first* statement of a module/class/function body). |
| R3 (comment hygiene) | Feasible | Some judgment required; will adopt heuristic: drop comments whose translated form would be a single verb-phrase paraphrase of the next executable line. |
| R4 (style compliance) | Feasible | Watch line-length when translating dense Chinese to English (English is typically longer); rewrap as needed without changing executable code. |
| R5 (verification) | Feasible | The `grep -rln '[一-鿿]'` rule is reliable. Residual hits should land only in: prompt template strings (#2/#3/#4/#5), logger/API string literals (#6), and the `tests/test_locale*` files (intentional Chinese test data). |
| R6 (tracking/branching) | Feasible | Branch + commit conventions are standard for this repo; `/done` skill enforces them. |
### Gaps and constraints
- **Constraint**: Triple-quoted strings used as values (not as docstrings) must NOT be edited if their content is in scope of issues #2–#6 (prompts/log messages/error messages). Disambiguation matters.
- **Constraint**: Chinese characters appearing inside f-string literal segments must remain. They are out of scope.
- **Unknown / Research Needed**: None — task is mechanical and well-bounded.
### Adjacent specs / overlap with other tickets
- `i18n-externalize-backend-logs` (#6) owns translating `logger.{info,warning,error}` Chinese arguments and API response strings.
- `i18n-report-agent-prompts` (#5), and tickets #2/#3/#4 own prompt template strings.
- We must NOT touch any string literal that those tickets own. After this PR, residual `grep` hits should reduce by exactly the count of comments and docstrings translated and nothing else.
- The two `backend/tests/test_locale*.py` files are **not in the ticket's listed file scope**, and inspection shows their Chinese is exclusively in string literals (test data and a Unicode range check). They are out of scope by R1's enumerated paths and remain untouched.
## Implementation Approach Options
### Option A — Single-pass file-by-file translation (recommended)
- Walk the 37 in-scope files in a deterministic order (alphabetical), translating docstrings/comments per file, running the residual grep after each batch.
- Group commit by area (models, utils, services, api, scripts, root) to keep PR diff readable.
- ✅ Simple, low risk, easy to revert per-area.
- ✅ Maps directly to the requirements; easy to verify.
- ❌ Larger PR than option B, but ticket explicitly allows a single PR.
### Option B — Multi-PR per package
- Split into one PR per package (`models/`, `utils/`, …). The ticket allows this.
- ✅ Smaller diffs to review.
- ❌ More overhead (multiple branches/PRs); not necessary for a mechanical change of this size.
### Option C — Tooling-assisted bulk script
- Build a one-shot translation script (LLM-driven) that rewrites docstrings/comments.
- ✅ Could scale to other repos.
- ❌ Out of proportion for a single-ticket task; risk of errant edits to string literals; tooling itself becomes a deliverable to test and maintain.
## Effort and Risk
- **Effort**: **M (3–7 days of focused work)** — 37 files, hundreds of comments. In an interactive AI-assisted run, this collapses to a few hours.
- **Risk**: **Low** — comments-only diff; covered by mechanical verification (grep + pytest); easy to rollback per file/area.
## Recommendations for Design Phase
- **Preferred approach**: Option A (single-pass file-by-file, package-grouped commits, single PR).
- **Key decisions to capture in design**:
- Order of traversal (proposed: `models/` → `utils/` → `services/` → `api/` → `scripts/` → root files `__init__.py`, `config.py`, `run.py`).
- Heuristic for "drops the obvious comment" (one-line rule).
- How to handle Google-style docstring keys: always translate `参数:` → `Args:`, `返回:` → `Returns:`, `异常:` → `Raises:`.
- Verification cadence: re-run the grep after each package batch.
- **Research items to carry forward**: None.

67
.kiro/specs/i18n-translate-backend-comments/requirements.md Normal file
View File

@ -0,0 +1,67 @@
# Requirements Document
## Introduction
This specification covers the developer-facing internationalization of `backend/` Python source: translating Chinese docstrings and inline comments to English so that English-speaking maintainers can read and review the code without translation overhead. The change is mechanical — no behavior, no public strings, no symbol names are modified. It is one of several i18n tickets (#2, #3, #4, #5, #6, #7); this spec covers ticket #7 only.
## Boundary Context
- **In scope**: Translation of Chinese-language characters that appear in Python docstrings (module/class/function) and inline `#` comments under `backend/`. Removal of comments that merely restate the code. Preservation of `TODO:` / `FIXME:` markers and embedded ticket references.
- **Out of scope**: Chinese characters inside string literals (prompt templates, `logger.{info,warning,error}` arguments, API response bodies, error messages returned to clients) — these are tracked separately by issues #2/#3/#4/#5/#6. No refactoring, reformatting, renaming, or behavior changes.
- **Adjacent expectations**: Spec `i18n-externalize-backend-logs` (issue #6) and the prompt-translation specs handle string-literal Chinese; this spec must leave those untouched so the other tickets remain mergeable.
## Requirements
### Requirement 1: Translation Coverage of In-Scope Files
**Objective:** As a maintainer, I want every Chinese docstring and inline comment in the in-scope backend files translated to English, so that I can read and review the code without translation tools.
#### Acceptance Criteria
1. The Backend Codebase shall contain no Chinese characters (Unicode range U+4E00–U+9FFF) inside Python docstrings under `backend/app/__init__.py`, `backend/app/config.py`, `backend/app/models/`, `backend/app/services/`, `backend/app/api/`, `backend/app/utils/`, `backend/run.py`, and `backend/scripts/`.
2. The Backend Codebase shall contain no Chinese characters inside Python `#` inline comments under the same paths.
3. When `grep -rln '[一-鿿]' backend/ --include='*.py'` is run after this change, the Backend Codebase shall return only files whose remaining Chinese is contained within string literals owned by issues #2/#3/#4/#5/#6.
4. When a docstring is translated, the Translator shall preserve Google-style docstring shape (`Args:`, `Returns:`, `Raises:`, `Yields:` sections) per `dev-guidelines.md`.
### Requirement 2: Preservation of Code Behavior
**Objective:** As a maintainer, I want the translation to be comments-and-docstrings-only, so that runtime behavior is provably unchanged.
#### Acceptance Criteria
1. The Translator shall not modify any executable Python statement (assignments, function calls, control flow, decorators, imports).
2. The Translator shall not modify any Python string literal (single-, double-, triple-quoted, f-string, raw, byte) regardless of whether it contains Chinese characters.
3. The Translator shall not rename any symbol (variable, function, class, module, parameter).
4. When `uv run python -m pytest backend/scripts/test_profile_format.py` is run after the change, the Backend Codebase shall exit with status 0.
5. If a diff line touches any non-comment, non-docstring code, the Translator shall reject that diff hunk and revise.
### Requirement 3: Comment Quality Hygiene
**Objective:** As a maintainer, I want translated comments to add value, so that the codebase remains easy to read after the migration.
#### Acceptance Criteria
1. When a Chinese comment merely restates the immediately following code (e.g. `# 初始化客户端` above `client = Client()`), the Translator shall delete the comment rather than translate it.
2. When a Chinese comment captures non-obvious *why* (constraints, workarounds, invariants), the Translator shall translate it to a faithful English equivalent.
3. The Translator shall preserve any `TODO:` / `FIXME:` marker and any embedded ticket reference (e.g. `#1234`, `PROJ-456`) verbatim within the translated comment.
4. The Translator shall not introduce new comments that did not exist (or had no Chinese equivalent) in the original source.
### Requirement 4: Style and Format Compliance
**Objective:** As a maintainer, I want the translated output to comply with project style rules, so that no follow-up cleanup PR is needed.
#### Acceptance Criteria
1. The Translator shall keep all translated docstrings and comments at or below 120 characters per line.
2. The Translator shall not introduce trailing whitespace on any line.
3. The Translator shall preserve the original indentation (tabs/spaces) of every comment and docstring.
4. The Translator shall use double quotes for any docstring it rewrites, matching the existing Python convention in the file.
5. Where a file already uses 4-space indentation, the Translator shall preserve that indentation.
### Requirement 5: Discovery and Verification Workflow
**Objective:** As a reviewer, I want a reproducible discovery and verification workflow, so that I can confirm coverage and absence of regressions in CI or locally.
#### Acceptance Criteria
1. The Translator shall enumerate candidate files using `grep -rln '[一-鿿]' backend/ --include='*.py'` before beginning work.
2. The Translator shall re-run the same `grep` after each batch and confirm the residual hits are limited to string-literal Chinese owned by adjacent tickets (#2/#3/#4/#5/#6).
3. When the residual `grep` hits include any non-string-literal Chinese, the Translator shall classify those hits as in-scope and continue translation until they are gone.
4. The Translator shall verify that `git diff --stat` only reports changes inside the in-scope file paths listed in Requirement 1.
### Requirement 6: Tracking and Branching
**Objective:** As a release manager, I want the work tracked against ticket #7 on a dedicated branch, so that the PR remains scoped and traceable.
#### Acceptance Criteria
1. The Translator shall produce changes on a branch named `docs/i18n-7-translate-backend-comments`.
2. The Translator shall reference issue `salestech-group/MiroFish#7` in commit messages or PR description.
3. When committing, the Translator shall use Conventional Commits with type `docs` and scope `i18n` (e.g. `docs(i18n): translate chinese docstrings/comments in backend/<area>`).
4. The Translator shall not include unrelated changes (e.g. dependency bumps, config changes, refactors) in the resulting PR.

80
.kiro/specs/i18n-translate-backend-comments/research.md Normal file
View File

@ -0,0 +1,80 @@
# Research & Design Decisions — `i18n-translate-backend-comments`
## Summary
- **Feature**: `i18n-translate-backend-comments`
- **Discovery Scope**: Simple Addition (mechanical translation, no architectural change)
- **Key Findings**:
- 37 in-scope `backend/` Python files contain Chinese characters in docstrings or `#` comments. The full list is in `gap-analysis.md`.
- Existing docstrings mix English-shape Google-style keys (`Args:`/`Returns:`) with Chinese descriptions, and a smaller subset uses Chinese keys (`参数:`/`返回:`/`异常:`). Both patterns must converge to canonical English Google-style.
- Several `tests/test_locale*.py` files contain Chinese only inside string literals (intentional test data) and are out of scope by the ticket's enumerated paths.
## Research Log
### Discovery scan: where is Chinese in `backend/`?
- **Context**: Need a deterministic enumeration of files to translate.
- **Sources Consulted**: `grep`/Python-driven scan against `backend/**/*.py`.
- **Findings**:
- 37 in-app files (under `backend/app/`, `backend/run.py`, `backend/scripts/`).
- 2 additional test files in `backend/tests/` whose Chinese is only in string literals; not in ticket scope.
- `.venv/` matches are noise and excluded.
- **Implications**: The ticket-listed paths are exhaustive; no unexpected location. Order of traversal can be alphabetical within package groups.
### Disambiguation: docstring vs string literal
- **Context**: A triple-quoted string is a docstring iff it is the first statement of a module, class, or function body. Otherwise it is a value (e.g. a prompt template) owned by adjacent tickets.
- **Sources Consulted**: Python language reference; spot inspection of `services/ontology_generator.py`, `services/report_agent.py`.
- **Findings**:
- In-scope files contain both kinds of triple-quoted strings.
- Translating only the *first-statement* triple-quoted string per scope keeps the change comments-and-docstrings-only.
- **Implications**: Translation pass must visually verify each triple-quoted string is the first statement before rewriting; otherwise leave it alone.
### Google-style docstring conversions
- **Context**: `dev-guidelines.md` requires Google-style docstrings; existing Chinese docstrings sometimes use Chinese keys.
- **Findings**: The following key map applies:
- `参数:` → `Args:`
- `返回:` → `Returns:`
- `异常:` → `Raises:`
- `产生:` / `生成:` → `Yields:`
- `示例:` → `Example:` (or `Examples:`)
- `注意:` / `备注:` → `Note:` (or `Notes:`)
- **Implications**: Document this mapping in design.md so the implementation pass is mechanical.
## Architecture Pattern Evaluation
| Option | Description | Strengths | Risks / Limitations | Notes |
|--------|-------------|-----------|---------------------|-------|
| Manual file-by-file pass | Walk in alphabetical order, package-grouped commits | Predictable, easy to review per package | Human time required | Selected approach |
| Multi-PR per package | One PR per backend package | Smaller diffs to review | Higher overhead, more PR churn | Allowed by ticket but not required |
| Tooling-assisted bulk script | LLM-driven find-and-replace tool | Reusable | Risk of touching string literals; tool itself becomes a deliverable | Out of proportion |
## Design Decisions
### Decision: Single-pass, package-grouped commits, single PR
- **Context**: 37 files, mechanical change, ticket allows either single or split PRs.
- **Alternatives Considered**:
1. Multi-PR per package — more granular review but higher overhead.
2. Tooling-assisted bulk script — overkill for one ticket.
- **Selected Approach**: Single PR with one or more commits, grouped by package (`models/`, `utils/`, `services/`, `api/`, `scripts/`, root) so reviewers can read the diff one package at a time.
- **Rationale**: Mechanical change with low risk; ticket explicitly allows it; reduces PR overhead; `/done` produces one PR per branch by default.
- **Trade-offs**: One large PR, but partitioned by commit. Reviewer can use commit history to navigate.
- **Follow-up**: After each package commit, re-run residual `grep` and `pytest` to maintain the invariant.
### Decision: First-statement disambiguation rule
- **Context**: Distinguish docstrings (in scope) from value strings (out of scope).
- **Selected Approach**: A triple-quoted string is treated as a docstring (in scope) only if it is the first statement of a module / class / function body. All other triple-quoted strings are values (out of scope).
- **Rationale**: Matches Python's own definition; keeps boundary with adjacent tickets unambiguous.
### Decision: Drop comments that restate code
- **Context**: R3 requires deletion of comments whose translated form would merely paraphrase the next line.
- **Selected Approach**: Apply a one-line heuristic: if the translated comment would be a verb phrase that mirrors the immediately following executable line, delete the comment instead of writing it.
- **Rationale**: Aligns with project rule "comment the why, not the what".
## Risks & Mitigations
- **Risk**: Accidental edit to a string literal (would belong to ticket #2/#3/#4/#5/#6) — **Mitigation**: After each package commit, run `git diff --stat` and a per-file diff sanity check; verify only `#` lines and docstring lines change.
- **Risk**: Tests failing because a string-shape changed — **Mitigation**: Run `uv run python -m pytest backend/scripts/test_profile_format.py` after each commit.
- **Risk**: Line length violations after English expansion — **Mitigation**: Reflow long English at <= 120 chars within the docstring/comment only; never reflow code.
## References
- `dev-guidelines.md` — repo-level coding standards, Google-style docstring requirement.
- `.claude/rules/commits.md` — Conventional Commits standard for the commit message.
- Issue #7 — salestech-group/MiroFish: source ticket.
- Issues #2/#3/#4/#5/#6 — adjacent i18n tickets that own the string-literal Chinese.

92
.kiro/specs/i18n-translate-backend-comments/scan_chinese.py Normal file
View File

@ -0,0 +1,92 @@
#!/usr/bin/env python3
"""AST-aware classifier of Chinese characters in a Python source file.
Usage::
python3 .kiro/specs/i18n-translate-backend-comments/scan_chinese.py <path>
Classifies every line containing CJK Unified Ideographs (U+4E00..U+9FFF)
into one of three buckets:
* ``DOCSTRING`` — line lies within a module/class/function docstring (in
scope for ticket #7).
* ``COMMENT`` — line contains a ``#`` and is not inside a docstring or
a string literal span (in scope for ticket #7).
* ``STRING`` — line is part of a string literal value (out of scope —
owned by sibling tickets #2/#3/#4/#5/#6).
Exit code is the count of in-scope hits (DOCSTRING + COMMENT). Stdout
lists each in-scope hit as ``<line> <bucket>: <content>`` so callers can
inspect them.
"""
from __future__ import annotations
import ast
import pathlib
import re
import sys
CJK_RE = re.compile(r"[一-鿿]")
def classify(path: pathlib.Path) -> int:
text = path.read_text(encoding="utf-8")
lines = text.split("\n")
tree = ast.parse(text)
docstring_lines: set[int] = set()
for node in ast.walk(tree):
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef, ast.Module)):
ds = ast.get_docstring(node, clean=False)
if ds is None:
continue
body = node.body
if not body or not isinstance(body[0], ast.Expr):
continue
const = body[0].value
if isinstance(const, ast.Constant) and isinstance(const.value, str):
start = const.lineno
end = getattr(const, "end_lineno", start)
for ln in range(start, end + 1):
docstring_lines.add(ln)
string_value_lines: set[int] = set()
for node in ast.walk(tree):
if isinstance(node, ast.Constant) and isinstance(node.value, str):
start = node.lineno
end = getattr(node, "end_lineno", start)
for ln in range(start, end + 1):
string_value_lines.add(ln)
in_scope_count = 0
for i, line in enumerate(lines, start=1):
if not CJK_RE.search(line):
continue
if i in docstring_lines:
print(f"{i:5d} DOCSTRING: {line.rstrip()[:120]}")
in_scope_count += 1
elif i in string_value_lines:
# Out of scope: owned by sibling tickets.
pass
elif "#" in line:
print(f"{i:5d} COMMENT : {line.rstrip()[:120]}")
in_scope_count += 1
# else: unclassified — treat as out of scope (STRING value spanning).
return in_scope_count
def main(argv: list[str]) -> int:
if len(argv) < 2:
print("usage: scan_chinese.py <path>", file=sys.stderr)
return 2
path = pathlib.Path(argv[1])
in_scope = classify(path)
print(f"---", file=sys.stderr)
print(f"in-scope CJK hits in {path}: {in_scope}", file=sys.stderr)
return 0 if in_scope == 0 else 1
if __name__ == "__main__":
raise SystemExit(main(sys.argv))

24
.kiro/specs/i18n-translate-backend-comments/spec.json Normal file
View File

@ -0,0 +1,24 @@
{
"feature_name": "i18n-translate-backend-comments",
"created_at": "2026-05-07T14:24:17Z",
"updated_at": "2026-05-07T14:26:00Z",
"language": "en",
"phase": "tasks-generated",
"ticket": 7,
"ticket_url": "https://github.com/salestech-group/MiroFish/issues/7",
"approvals": {
"requirements": {
"generated": true,
"approved": true
},
"design": {
"generated": true,
"approved": true
},
"tasks": {
"generated": true,
"approved": true
}
},
"ready_for_implementation": true
}

97
.kiro/specs/i18n-translate-backend-comments/tasks.md Normal file
View File

@ -0,0 +1,97 @@
# Implementation Plan
## Foundation
- [x] 1. Establish baseline and working branch
- [x] 1.1 Create translation working branch and capture baseline state
- Create branch `docs/i18n-7-translate-backend-comments` from `main`.
- Capture the baseline residual hits by running the discovery scan (the regex `[一-鿿]` against `backend/**/*.py`, excluding `.venv`); record the file list as the work queue.
- Run `cd backend && uv run python -m pytest scripts/test_profile_format.py` and confirm a green baseline before any edits.
- Observable: a fresh branch exists, the baseline file list of 37 in-scope files is captured, and the baseline pytest run passes.
- _Requirements: 5.1, 6.1_
## Core — Per-Package Translation
- [x] 2. Translate Chinese docstrings and inline comments per package
- [x] 2.1 (P) Translate `backend/app/models/`
- Translate Chinese module/class/function docstrings and `#` comments in `backend/app/models/__init__.py`, `backend/app/models/project.py`, and `backend/app/models/task.py`.
- Apply the docstring-vs-value disambiguation rule (first-statement only) so that no string literal is touched.
- Apply the Google-style key map (`参数:` → `Args:`, `返回:` → `Returns:`, `异常:` → `Raises:`, `产生:`/`生成:` → `Yields:`, `示例:` → `Examples:`, `注意:`/`备注:` → `Note:`).
- Drop comments that merely restate the next executable line; preserve `TODO:`/`FIXME:` and any embedded ticket reference verbatim.
- Re-run the residual scan and confirm `backend/app/models/` no longer has Chinese in non-string-literal positions.
- Re-run `cd backend && uv run python -m pytest scripts/test_profile_format.py` and confirm exit 0.
- Observable: zero non-string-literal Chinese remains in `backend/app/models/*.py`, and the test command exits 0.
- _Requirements: 1.1, 1.2, 1.4, 2.1, 2.2, 2.3, 2.4, 3.1, 3.2, 3.3, 3.4, 4.1, 4.2, 4.3, 4.4, 4.5_
- _Boundary: backend/app/models/_
- [x] 2.2 (P) Translate `backend/app/utils/`
- Translate Chinese docstrings and `#` comments in `backend/app/utils/__init__.py`, `file_parser.py`, `llm_client.py`, `locale.py`, `logger.py`, `retry.py`, and `zep_paging.py`.
- Be especially careful with `locale.py` and `logger.py`: they intentionally route Chinese strings through their value paths; only docstrings and `#` comments are in scope.
- Apply Rules 1–5 from `design.md` (disambiguation, key map, comment hygiene, style, preservation).
- Re-run the residual scan and confirm `backend/app/utils/` no longer has Chinese in non-string-literal positions.
- Re-run the pytest command and confirm exit 0.
- Observable: zero non-string-literal Chinese remains in `backend/app/utils/*.py`, and the test command exits 0.
- _Requirements: 1.1, 1.2, 1.4, 2.1, 2.2, 2.3, 2.4, 3.1, 3.2, 3.3, 3.4, 4.1, 4.2, 4.3, 4.4, 4.5_
- _Boundary: backend/app/utils/_
- [x] 2.3 (P) Translate `backend/app/services/` — complete (all 12 files; finished in this installment)
- Translate Chinese docstrings and `#` comments across all 12 service files: `__init__.py`, `graph_builder.py`, `ontology_generator.py`, `oasis_profile_generator.py`, `report_agent.py`, `simulation_config_generator.py`, `simulation_ipc.py`, `simulation_manager.py`, `simulation_runner.py`, `text_processor.py`, `zep_entity_reader.py`, `zep_graph_memory_updater.py`, `zep_tools.py`.
- Treat all triple-quoted prompt templates and value strings as out of scope (owned by issues #2/#3/#4/#5/#6) — only the first-statement docstrings of modules/classes/functions are in scope.
- Apply Rules 1–5 from `design.md`.
- Re-run the residual scan and confirm `backend/app/services/` no longer has Chinese in non-string-literal positions.
- Re-run the pytest command and confirm exit 0.
- Observable: zero non-string-literal Chinese remains in `backend/app/services/*.py`, and the test command exits 0.
- _Requirements: 1.1, 1.2, 1.4, 2.1, 2.2, 2.3, 2.4, 3.1, 3.2, 3.3, 3.4, 4.1, 4.2, 4.3, 4.4, 4.5_
- _Boundary: backend/app/services/_
- [x] 2.4 (P) Translate `backend/app/api/` — complete (all 4 files; finished in this installment)
- Translate Chinese docstrings and `#` comments in `__init__.py`, `graph.py`, `report.py`, `simulation.py`.
- Treat any user-facing string-literal Chinese in API responses as out of scope (owned by issue #6).
- Apply Rules 1–5 from `design.md`.
- Re-run the residual scan and confirm `backend/app/api/` no longer has Chinese in non-string-literal positions.
- Re-run the pytest command and confirm exit 0.
- Observable: zero non-string-literal Chinese remains in `backend/app/api/*.py`, and the test command exits 0.
- _Requirements: 1.1, 1.2, 1.4, 2.1, 2.2, 2.3, 2.4, 3.1, 3.2, 3.3, 3.4, 4.1, 4.2, 4.3, 4.4, 4.5_
- _Boundary: backend/app/api/_
- [x] 2.5 (P) Translate `backend/scripts/` — complete (all 5 files; finished in this installment)
- Translate Chinese docstrings and `#` comments in `action_logger.py`, `run_parallel_simulation.py`, `run_reddit_simulation.py`, `run_twitter_simulation.py`, `test_profile_format.py`.
- Apply Rules 1–5 from `design.md`.
- Be especially careful with `test_profile_format.py`: any Chinese in test data string literals is out of scope; only docstrings and `#` comments are in scope.
- Re-run the residual scan and confirm `backend/scripts/` no longer has Chinese in non-string-literal positions.
- Re-run the pytest command and confirm exit 0.
- Observable: zero non-string-literal Chinese remains in `backend/scripts/*.py`, and the test command exits 0.
- _Requirements: 1.1, 1.2, 1.4, 2.1, 2.2, 2.3, 2.4, 3.1, 3.2, 3.3, 3.4, 4.1, 4.2, 4.3, 4.4, 4.5_
- _Boundary: backend/scripts/_
- [x] 2.6 (P) Translate root backend files
- Translate Chinese docstrings and `#` comments in `backend/app/__init__.py`, `backend/app/config.py`, and `backend/run.py`.
- Apply Rules 1–5 from `design.md`.
- Be especially careful with `backend/app/config.py`: any Chinese in default-value string literals is out of scope; only docstrings and `#` comments are in scope.
- Re-run the residual scan and confirm these three files no longer have Chinese in non-string-literal positions.
- Re-run the pytest command and confirm exit 0.
- Observable: zero non-string-literal Chinese remains in `backend/app/__init__.py`, `backend/app/config.py`, and `backend/run.py`, and the test command exits 0.
- _Requirements: 1.1, 1.2, 1.4, 2.1, 2.2, 2.3, 2.4, 3.1, 3.2, 3.3, 3.4, 4.1, 4.2, 4.3, 4.4, 4.5_
- _Boundary: backend/app (root), backend/run.py_
## Validation
- [x] 3. Final verification and PR preparation
- [x] 3.1 Run the final verification gate — scanner + py_compile pass on all 12 newly-translated files; CJK guard baseline updated (backend/app: 2792 → 307); pytest blocked by pre-existing env issues, see HANDOFF.md
- Run the residual scan one more time and confirm the only remaining hits are files where the Chinese is in string literals owned by issues #2/#3/#4/#5/#6, plus the intentional Chinese in `backend/tests/test_locale*.py`.
- Run `cd backend && uv run python -m pytest scripts/test_profile_format.py` and confirm exit 0.
- Run `git diff --stat origin/main...HEAD` and confirm only in-scope file paths under `backend/app/`, `backend/run.py`, and `backend/scripts/` are listed.
- Spot-check three random changed files with `git diff <path>` and confirm only `#` lines and docstring lines changed (no executable lines, no string-literal lines).
- Observable: residual scan, pytest, diff scope, and spot diff all pass.
- _Depends: 2.1, 2.2, 2.3, 2.4, 2.5, 2.6_
- _Requirements: 1.3, 2.5, 5.1, 5.2, 5.3, 5.4, 6.4_
- [ ] 3.2 Open PR and reference ticket #7
- Use `/done` to commit any remaining changes per Conventional Commits with type `docs` and scope `i18n` (e.g. `docs(i18n): translate chinese docstrings/comments in backend/<area>`), push the branch, and open a PR.
- The PR body must include `Closes #7` and reference the spec at `.kiro/specs/i18n-translate-backend-comments/`.
- Verify the PR contains no unrelated changes (no dependency bumps, no config changes, no refactors).
- Observable: a PR exists on GitHub from `docs/i18n-7-translate-backend-comments` to `main` that closes #7 and contains only docstring/comment translation diffs.
- _Depends: 3.1_
- _Requirements: 6.1, 6.2, 6.3, 6.4_

20
README-EN.md
View File

@ -4,8 +4,6 @@
<a href="https://trendshift.io/repositories/16144" target="_blank"><img src="https://trendshift.io/api/badge/repositories/16144" alt="666ghj%2FMiroFish | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a> <a href="https://trendshift.io/repositories/16144" target="_blank"><img src="https://trendshift.io/api/badge/repositories/16144" alt="666ghj%2FMiroFish | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
简洁通用的群体智能引擎，预测万物
</br>
<em>A Simple and Universal Swarm Intelligence Engine, Predicting Anything</em> <em>A Simple and Universal Swarm Intelligence Engine, Predicting Anything</em>
<a href="https://www.shanda.com/" target="_blank"><img src="./static/image/shanda_logo.png" alt="666ghj%2MiroFish | Shanda" height="40"/></a> <a href="https://www.shanda.com/" target="_blank"><img src="./static/image/shanda_logo.png" alt="666ghj%2MiroFish | Shanda" height="40"/></a>
@ -49,16 +47,16 @@ Welcome to visit our online demo environment and experience a prediction simulat
<div align="center"> <div align="center">
<table> <table>
<tr> <tr>
<td><img src="./static/image/Screenshot/运行截图1.png" alt="Screenshot 1" width="100%"/></td> <td><img src="./static/image/Screenshot/screenshot1.png" alt="Screenshot 1" width="100%"/></td>
<td><img src="./static/image/Screenshot/运行截图2.png" alt="Screenshot 2" width="100%"/></td> <td><img src="./static/image/Screenshot/screenshot2.png" alt="Screenshot 2" width="100%"/></td>
</tr> </tr>
<tr> <tr>
<td><img src="./static/image/Screenshot/运行截图3.png" alt="Screenshot 3" width="100%"/></td> <td><img src="./static/image/Screenshot/screenshot3.png" alt="Screenshot 3" width="100%"/></td>
<td><img src="./static/image/Screenshot/运行截图4.png" alt="Screenshot 4" width="100%"/></td> <td><img src="./static/image/Screenshot/screenshot4.png" alt="Screenshot 4" width="100%"/></td>
</tr> </tr>
<tr> <tr>
<td><img src="./static/image/Screenshot/运行截图5.png" alt="Screenshot 5" width="100%"/></td> <td><img src="./static/image/Screenshot/screenshot5.png" alt="Screenshot 5" width="100%"/></td>
<td><img src="./static/image/Screenshot/运行截图6.png" alt="Screenshot 6" width="100%"/></td> <td><img src="./static/image/Screenshot/screenshot6.png" alt="Screenshot 6" width="100%"/></td>
</tr> </tr>
</table> </table>
</div> </div>
@ -68,7 +66,7 @@ Welcome to visit our online demo environment and experience a prediction simulat
### 1. Wuhan University Public Opinion Simulation + MiroFish Project Introduction ### 1. Wuhan University Public Opinion Simulation + MiroFish Project Introduction
<div align="center"> <div align="center">
<a href="https://www.bilibili.com/video/BV1VYBsBHEMY/" target="_blank"><img src="./static/image/武大模拟演示封面.png" alt="MiroFish Demo Video" width="75%"/></a> <a href="https://www.bilibili.com/video/BV1VYBsBHEMY/" target="_blank"><img src="./static/image/wuhan-university-simulation-cover.png" alt="MiroFish Demo Video" width="75%"/></a>
Click the image to watch the complete demo video for prediction using BettaFish-generated "Wuhan University Public Opinion Report" Click the image to watch the complete demo video for prediction using BettaFish-generated "Wuhan University Public Opinion Report"
</div> </div>
@ -76,7 +74,7 @@ Click the image to watch the complete demo video for prediction using BettaFish-
### 2. Dream of the Red Chamber Lost Ending Simulation ### 2. Dream of the Red Chamber Lost Ending Simulation
<div align="center"> <div align="center">
<a href="https://www.bilibili.com/video/BV1cPk3BBExq" target="_blank"><img src="./static/image/红楼梦模拟推演封面.jpg" alt="MiroFish Demo Video" width="75%"/></a> <a href="https://www.bilibili.com/video/BV1cPk3BBExq" target="_blank"><img src="./static/image/dream-of-the-red-chamber-simulation-cover.jpg" alt="MiroFish Demo Video" width="75%"/></a>
Click the image to watch MiroFish's deep prediction of the lost ending based on hundreds of thousands of words from the first 80 chapters of "Dream of the Red Chamber" Click the image to watch MiroFish's deep prediction of the lost ending based on hundreds of thousands of words from the first 80 chapters of "Dream of the Red Chamber"
</div> </div>
@ -217,7 +215,7 @@ npm run frontend # Start frontend only
## 📬 Join the Conversation ## 📬 Join the Conversation
<div align="center"> <div align="center">
<img src="./static/image/QQ群.png" alt="QQ Group" width="60%"/> <img src="./static/image/qq-group.png" alt="QQ Group" width="60%"/>
</div> </div>
&nbsp; &nbsp;

18
README-ZH.md
View File

@ -49,16 +49,16 @@ MiroFish 致力于打造映射现实的群体智能镜像，通过捕捉个体
<div align="center"> <div align="center">
<table> <table>
<tr> <tr>
<td><img src="./static/image/Screenshot/运行截图1.png" alt="截图1" width="100%"/></td> <td><img src="./static/image/Screenshot/screenshot1.png" alt="截图1" width="100%"/></td>
<td><img src="./static/image/Screenshot/运行截图2.png" alt="截图2" width="100%"/></td> <td><img src="./static/image/Screenshot/screenshot2.png" alt="截图2" width="100%"/></td>
</tr> </tr>
<tr> <tr>
<td><img src="./static/image/Screenshot/运行截图3.png" alt="截图3" width="100%"/></td> <td><img src="./static/image/Screenshot/screenshot3.png" alt="截图3" width="100%"/></td>
<td><img src="./static/image/Screenshot/运行截图4.png" alt="截图4" width="100%"/></td> <td><img src="./static/image/Screenshot/screenshot4.png" alt="截图4" width="100%"/></td>
</tr> </tr>
<tr> <tr>
<td><img src="./static/image/Screenshot/运行截图5.png" alt="截图5" width="100%"/></td> <td><img src="./static/image/Screenshot/screenshot5.png" alt="截图5" width="100%"/></td>
<td><img src="./static/image/Screenshot/运行截图6.png" alt="截图6" width="100%"/></td> <td><img src="./static/image/Screenshot/screenshot6.png" alt="截图6" width="100%"/></td>
</tr> </tr>
</table> </table>
</div> </div>
@ -68,7 +68,7 @@ MiroFish 致力于打造映射现实的群体智能镜像，通过捕捉个体
### 1. 武汉大学舆情推演预测 + MiroFish项目讲解 ### 1. 武汉大学舆情推演预测 + MiroFish项目讲解
<div align="center"> <div align="center">
<a href="https://www.bilibili.com/video/BV1VYBsBHEMY/" target="_blank"><img src="./static/image/武大模拟演示封面.png" alt="MiroFish Demo Video" width="75%"/></a> <a href="https://www.bilibili.com/video/BV1VYBsBHEMY/" target="_blank"><img src="./static/image/wuhan-university-simulation-cover.png" alt="MiroFish Demo Video" width="75%"/></a>
点击图片查看使用微舆BettaFish生成的《武大舆情报告》进行预测的完整演示视频 点击图片查看使用微舆BettaFish生成的《武大舆情报告》进行预测的完整演示视频
</div> </div>
@ -76,7 +76,7 @@ MiroFish 致力于打造映射现实的群体智能镜像，通过捕捉个体
### 2. 《红楼梦》失传结局推演预测 ### 2. 《红楼梦》失传结局推演预测
<div align="center"> <div align="center">
<a href="https://www.bilibili.com/video/BV1cPk3BBExq" target="_blank"><img src="./static/image/红楼梦模拟推演封面.jpg" alt="MiroFish Demo Video" width="75%"/></a> <a href="https://www.bilibili.com/video/BV1cPk3BBExq" target="_blank"><img src="./static/image/dream-of-the-red-chamber-simulation-cover.jpg" alt="MiroFish Demo Video" width="75%"/></a>
点击图片查看基于《红楼梦》前80回数十万字，MiroFish深度预测失传结局 点击图片查看基于《红楼梦》前80回数十万字，MiroFish深度预测失传结局
</div> </div>
@ -217,7 +217,7 @@ npm run frontend # 仅启动前端
## 📬 更多交流 ## 📬 更多交流
<div align="center"> <div align="center">
<img src="./static/image/QQ群.png" alt="QQ交流群" width="60%"/> <img src="./static/image/qq-group.png" alt="QQ交流群" width="60%"/>
</div> </div>
&nbsp; &nbsp;

20
README.md
View File

@ -4,8 +4,6 @@
<a href="https://trendshift.io/repositories/16144" target="_blank"><img src="https://trendshift.io/api/badge/repositories/16144" alt="666ghj%2FMiroFish | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a> <a href="https://trendshift.io/repositories/16144" target="_blank"><img src="https://trendshift.io/api/badge/repositories/16144" alt="666ghj%2FMiroFish | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
简洁通用的群体智能引擎，预测万物
</br>
<em>A Simple and Universal Swarm Intelligence Engine, Predicting Anything</em> <em>A Simple and Universal Swarm Intelligence Engine, Predicting Anything</em>
<a href="https://www.shanda.com/" target="_blank"><img src="./static/image/shanda_logo.png" alt="666ghj%2MiroFish | Shanda" height="40"/></a> <a href="https://www.shanda.com/" target="_blank"><img src="./static/image/shanda_logo.png" alt="666ghj%2MiroFish | Shanda" height="40"/></a>
@ -49,16 +47,16 @@ Welcome to visit our online demo environment and experience a prediction simulat
<div align="center"> <div align="center">
<table> <table>
<tr> <tr>
<td><img src="./static/image/Screenshot/运行截图1.png" alt="Screenshot 1" width="100%"/></td> <td><img src="./static/image/Screenshot/screenshot1.png" alt="Screenshot 1" width="100%"/></td>
<td><img src="./static/image/Screenshot/运行截图2.png" alt="Screenshot 2" width="100%"/></td> <td><img src="./static/image/Screenshot/screenshot2.png" alt="Screenshot 2" width="100%"/></td>
</tr> </tr>
<tr> <tr>
<td><img src="./static/image/Screenshot/运行截图3.png" alt="Screenshot 3" width="100%"/></td> <td><img src="./static/image/Screenshot/screenshot3.png" alt="Screenshot 3" width="100%"/></td>
<td><img src="./static/image/Screenshot/运行截图4.png" alt="Screenshot 4" width="100%"/></td> <td><img src="./static/image/Screenshot/screenshot4.png" alt="Screenshot 4" width="100%"/></td>
</tr> </tr>
<tr> <tr>
<td><img src="./static/image/Screenshot/运行截图5.png" alt="Screenshot 5" width="100%"/></td> <td><img src="./static/image/Screenshot/screenshot5.png" alt="Screenshot 5" width="100%"/></td>
<td><img src="./static/image/Screenshot/运行截图6.png" alt="Screenshot 6" width="100%"/></td> <td><img src="./static/image/Screenshot/screenshot6.png" alt="Screenshot 6" width="100%"/></td>
</tr> </tr>
</table> </table>
</div> </div>
@ -68,7 +66,7 @@ Welcome to visit our online demo environment and experience a prediction simulat
### 1. Wuhan University Public Opinion Simulation + MiroFish Project Introduction ### 1. Wuhan University Public Opinion Simulation + MiroFish Project Introduction
<div align="center"> <div align="center">
<a href="https://www.bilibili.com/video/BV1VYBsBHEMY/" target="_blank"><img src="./static/image/武大模拟演示封面.png" alt="MiroFish Demo Video" width="75%"/></a> <a href="https://www.bilibili.com/video/BV1VYBsBHEMY/" target="_blank"><img src="./static/image/wuhan-university-simulation-cover.png" alt="MiroFish Demo Video" width="75%"/></a>
Click the image to watch the complete demo video for prediction using BettaFish-generated "Wuhan University Public Opinion Report" Click the image to watch the complete demo video for prediction using BettaFish-generated "Wuhan University Public Opinion Report"
</div> </div>
@ -76,7 +74,7 @@ Click the image to watch the complete demo video for prediction using BettaFish-
### 2. Dream of the Red Chamber Lost Ending Simulation ### 2. Dream of the Red Chamber Lost Ending Simulation
<div align="center"> <div align="center">
<a href="https://www.bilibili.com/video/BV1cPk3BBExq" target="_blank"><img src="./static/image/红楼梦模拟推演封面.jpg" alt="MiroFish Demo Video" width="75%"/></a> <a href="https://www.bilibili.com/video/BV1cPk3BBExq" target="_blank"><img src="./static/image/dream-of-the-red-chamber-simulation-cover.jpg" alt="MiroFish Demo Video" width="75%"/></a>
Click the image to watch MiroFish's deep prediction of the lost ending based on hundreds of thousands of words from the first 80 chapters of "Dream of the Red Chamber" Click the image to watch MiroFish's deep prediction of the lost ending based on hundreds of thousands of words from the first 80 chapters of "Dream of the Red Chamber"
</div> </div>
@ -236,7 +234,7 @@ npm run frontend # Start frontend only
## 📬 Join the Conversation ## 📬 Join the Conversation
<div align="center"> <div align="center">
<img src="./static/image/QQ群.png" alt="QQ Group" width="60%"/> <img src="./static/image/qq-group.png" alt="QQ Group" width="60%"/>
</div> </div>
&nbsp; &nbsp;

31
backend/app/__init__.py
View File

@ -1,12 +1,10 @@
""" """MiroFish backend Flask application factory."""
MiroFish Backend - Flask应用工厂
"""
import os import os
import warnings import warnings
# 抑制 multiprocessing resource_tracker 的警告（来自第三方库如 transformers） # Silence multiprocessing.resource_tracker warnings emitted by some third-party
# 需要在所有其他导入之前设置 # libraries (e.g. transformers); must run before those modules are imported.
warnings.filterwarnings("ignore", message=".*resource_tracker.*") warnings.filterwarnings("ignore", message=".*resource_tracker.*")
from flask import Flask, request from flask import Flask, request
@ -18,19 +16,21 @@ from .utils.locale import t
def create_app(config_class=Config): def create_app(config_class=Config):
"""Flask应用工厂函数""" """Flask application factory."""
app = Flask(__name__) app = Flask(__name__)
app.config.from_object(config_class) app.config.from_object(config_class)
# 设置JSON编码：确保中文直接显示（而不是 \uXXXX 格式） # Configure JSON encoding so non-ASCII characters render literally
# Flask >= 2.3 使用 app.json.ensure_ascii，旧版本使用 JSON_AS_ASCII 配置 # rather than as \uXXXX escape sequences. Flask >= 2.3 exposes
# ``app.json.ensure_ascii``; older versions use ``JSON_AS_ASCII``.
if hasattr(app, 'json') and hasattr(app.json, 'ensure_ascii'): if hasattr(app, 'json') and hasattr(app.json, 'ensure_ascii'):
app.json.ensure_ascii = False app.json.ensure_ascii = False
# 设置日志 # Configure logging.
logger = setup_logger('mirofish') logger = setup_logger('mirofish')
# 只在 reloader 子进程中打印启动信息（避免 debug 模式下打印两次） # Only print startup banners in the reloader child process to avoid
# double-printing in debug mode.
is_reloader_process = os.environ.get('WERKZEUG_RUN_MAIN') == 'true' is_reloader_process = os.environ.get('WERKZEUG_RUN_MAIN') == 'true'
debug_mode = app.config.get('DEBUG', False) debug_mode = app.config.get('DEBUG', False)
should_log_startup = not debug_mode or is_reloader_process should_log_startup = not debug_mode or is_reloader_process
@ -40,16 +40,17 @@ def create_app(config_class=Config):
logger.info(t("log.bootstrap.m001")) logger.info(t("log.bootstrap.m001"))
logger.info("=" * 50) logger.info("=" * 50)
# 启用CORS # Enable CORS.
CORS(app, resources={r"/api/*": {"origins": "*"}}) CORS(app, resources={r"/api/*": {"origins": "*"}})
# 注册模拟进程清理函数（确保服务器关闭时终止所有模拟进程） # Register simulation-process cleanup so all child processes are torn down
# when the Flask server shuts down.
from .services.simulation_runner import SimulationRunner from .services.simulation_runner import SimulationRunner
SimulationRunner.register_cleanup() SimulationRunner.register_cleanup()
if should_log_startup: if should_log_startup:
logger.info(t("log.bootstrap.m002")) logger.info(t("log.bootstrap.m002"))
# 请求日志中间件 # Request-logging middleware.
@app.before_request @app.before_request
def log_request(): def log_request():
logger = get_logger('mirofish.request') logger = get_logger('mirofish.request')
@ -63,13 +64,13 @@ def create_app(config_class=Config):
logger.debug(t("log.bootstrap.m005", response=response.status_code)) logger.debug(t("log.bootstrap.m005", response=response.status_code))
return response return response
# 注册蓝图 # Register API blueprints.
from .api import graph_bp, simulation_bp, report_bp from .api import graph_bp, simulation_bp, report_bp
app.register_blueprint(graph_bp, url_prefix='/api/graph') app.register_blueprint(graph_bp, url_prefix='/api/graph')
app.register_blueprint(simulation_bp, url_prefix='/api/simulation') app.register_blueprint(simulation_bp, url_prefix='/api/simulation')
app.register_blueprint(report_bp, url_prefix='/api/report') app.register_blueprint(report_bp, url_prefix='/api/report')
# 健康检查 # Health-check endpoint.
@app.route('/health') @app.route('/health')
def health(): def health():
return {'status': 'ok', 'service': 'MiroFish Backend'} return {'status': 'ok', 'service': 'MiroFish Backend'}

4
backend/app/api/__init__.py
View File

@ -1,6 +1,4 @@
""" """API blueprints package."""
API路由模块
"""
from flask import Blueprint from flask import Blueprint

151
backend/app/api/graph.py
View File

@ -1,6 +1,7 @@
""" """
图谱相关API路由 Graph-related API routes.
采用项目上下文机制，服务端持久化状态
Uses a project context mechanism with server-side state persistence.
""" """
import os import os
@ -26,25 +27,22 @@ _graph_data_cache: dict = {} # graph_id -> {"data": ..., "ts": float}
_graph_refresh_locks: dict = {} # graph_id -> threading.Lock (one refresh at a time) _graph_refresh_locks: dict = {} # graph_id -> threading.Lock (one refresh at a time)
_GRAPH_CACHE_TTL = 300 # seconds before triggering a background refresh _GRAPH_CACHE_TTL = 300 # seconds before triggering a background refresh
# 获取日志器
logger = get_logger('mirofish.api') logger = get_logger('mirofish.api')
def allowed_file(filename: str) -> bool: def allowed_file(filename: str) -> bool:
"""检查文件扩展名是否允许""" """Return True if the file extension is in the allowed list."""
if not filename or '.' not in filename: if not filename or '.' not in filename:
return False return False
ext = os.path.splitext(filename)[1].lower().lstrip('.') ext = os.path.splitext(filename)[1].lower().lstrip('.')
return ext in Config.ALLOWED_EXTENSIONS return ext in Config.ALLOWED_EXTENSIONS
# ============== 项目管理接口 ============== # ============== Project management endpoints ==============
@graph_bp.route('/project/<project_id>', methods=['GET']) @graph_bp.route('/project/<project_id>', methods=['GET'])
def get_project(project_id: str): def get_project(project_id: str):
""" """Get project details."""
获取项目详情
"""
project = ProjectManager.get_project(project_id) project = ProjectManager.get_project(project_id)
if not project: if not project:
@ -61,9 +59,7 @@ def get_project(project_id: str):
@graph_bp.route('/project/list', methods=['GET']) @graph_bp.route('/project/list', methods=['GET'])
def list_projects(): def list_projects():
""" """List all projects."""
列出所有项目
"""
limit = request.args.get('limit', 50, type=int) limit = request.args.get('limit', 50, type=int)
projects = ProjectManager.list_projects(limit=limit) projects = ProjectManager.list_projects(limit=limit)
@ -76,9 +72,7 @@ def list_projects():
@graph_bp.route('/project/<project_id>', methods=['DELETE']) @graph_bp.route('/project/<project_id>', methods=['DELETE'])
def delete_project(project_id: str): def delete_project(project_id: str):
""" """Delete a project."""
删除项目
"""
success = ProjectManager.delete_project(project_id) success = ProjectManager.delete_project(project_id)
if not success: if not success:
@ -95,9 +89,7 @@ def delete_project(project_id: str):
@graph_bp.route('/project/<project_id>/reset', methods=['POST']) @graph_bp.route('/project/<project_id>/reset', methods=['POST'])
def reset_project(project_id: str): def reset_project(project_id: str):
""" """Reset project state (used to rebuild the graph from scratch)."""
重置项目状态（用于重新构建图谱）
"""
project = ProjectManager.get_project(project_id) project = ProjectManager.get_project(project_id)
if not project: if not project:
@ -106,7 +98,8 @@ def reset_project(project_id: str):
"error": t("api.error.graph.m004", project_id=project_id) "error": t("api.error.graph.m004", project_id=project_id)
}), 404 }), 404
# 重置到本体已生成状态 # Roll back to the "ontology generated" state so the next build can resume
# from the existing ontology rather than re-running ontology generation.
if project.ontology: if project.ontology:
project.status = ProjectStatus.ONTOLOGY_GENERATED project.status = ProjectStatus.ONTOLOGY_GENERATED
else: else:
@ -124,22 +117,21 @@ def reset_project(project_id: str):
}) })
# ============== 接口1：上传文件并生成本体 ============== # ============== Endpoint 1: upload files and generate ontology ==============
@graph_bp.route('/ontology/generate', methods=['POST']) @graph_bp.route('/ontology/generate', methods=['POST'])
def generate_ontology(): def generate_ontology():
""" """Endpoint 1: upload files, analyze them, and generate an ontology definition.
接口1：上传文件，分析生成本体定义
请求方式：multipart/form-data Request format: multipart/form-data.
参数： Args:
files: 上传的文件（PDF/MD/TXT），可多个 files: Uploaded files (PDF/MD/TXT); one or more.
simulation_requirement: 模拟需求描述（必填） simulation_requirement: Description of the simulation requirement (required).
project_name: 项目名称（可选） project_name: Project name (optional).
additional_context: 额外说明（可选） additional_context: Additional context (optional).
返回： Returns:
{ {
"success": true, "success": true,
"data": { "data": {
@ -157,7 +149,6 @@ def generate_ontology():
try: try:
logger.info(t("log.graph_api.m006")) logger.info(t("log.graph_api.m006"))
# 获取参数
simulation_requirement = request.form.get('simulation_requirement', '') simulation_requirement = request.form.get('simulation_requirement', '')
project_name = request.form.get('project_name', 'Unnamed Project') project_name = request.form.get('project_name', 'Unnamed Project')
additional_context = request.form.get('additional_context', '') additional_context = request.form.get('additional_context', '')
@ -171,7 +162,6 @@ def generate_ontology():
"error": t("api.error.graph.m009") "error": t("api.error.graph.m009")
}), 400 }), 400
# 获取上传的文件
uploaded_files = request.files.getlist('files') uploaded_files = request.files.getlist('files')
if not uploaded_files or all(not f.filename for f in uploaded_files): if not uploaded_files or all(not f.filename for f in uploaded_files):
return jsonify({ return jsonify({
@ -179,18 +169,17 @@ def generate_ontology():
"error": t("api.error.graph.m010") "error": t("api.error.graph.m010")
}), 400 }), 400
# 创建项目
project = ProjectManager.create_project(name=project_name) project = ProjectManager.create_project(name=project_name)
project.simulation_requirement = simulation_requirement project.simulation_requirement = simulation_requirement
logger.info(t("log.graph_api.m011", project=project.project_id)) logger.info(t("log.graph_api.m011", project=project.project_id))
# 保存文件并提取文本 # Persist each uploaded file under the project's directory and pull its
# text out so the ontology generator has plain text to work with.
document_texts = [] document_texts = []
all_text = "" all_text = ""
for file in uploaded_files: for file in uploaded_files:
if file and file.filename and allowed_file(file.filename): if file and file.filename and allowed_file(file.filename):
# 保存文件到项目目录
file_info = ProjectManager.save_file_to_project( file_info = ProjectManager.save_file_to_project(
project.project_id, project.project_id,
file, file,
@ -201,7 +190,6 @@ def generate_ontology():
"size": file_info["size"] "size": file_info["size"]
}) })
# 提取文本
text = FileParser.extract_text(file_info["path"]) text = FileParser.extract_text(file_info["path"])
text = TextProcessor.preprocess_text(text) text = TextProcessor.preprocess_text(text)
document_texts.append(text) document_texts.append(text)
@ -214,12 +202,10 @@ def generate_ontology():
"error": t("api.error.graph.m012") "error": t("api.error.graph.m012")
}), 400 }), 400
# 保存提取的文本
project.total_text_length = len(all_text) project.total_text_length = len(all_text)
ProjectManager.save_extracted_text(project.project_id, all_text) ProjectManager.save_extracted_text(project.project_id, all_text)
logger.info(t("log.graph_api.m013", len=len(all_text))) logger.info(t("log.graph_api.m013", len=len(all_text)))
# 生成本体
logger.info(t("log.graph_api.m014")) logger.info(t("log.graph_api.m014"))
generator = OntologyGenerator() generator = OntologyGenerator()
ontology = generator.generate( ontology = generator.generate(
@ -228,7 +214,6 @@ def generate_ontology():
additional_context=additional_context if additional_context else None additional_context=additional_context if additional_context else None
) )
# 保存本体到项目
entity_count = len(ontology.get("entity_types", [])) entity_count = len(ontology.get("entity_types", []))
edge_count = len(ontology.get("edge_types", [])) edge_count = len(ontology.get("edge_types", []))
logger.info(t("log.graph_api.m015", entity_count=entity_count, edge_count=edge_count)) logger.info(t("log.graph_api.m015", entity_count=entity_count, edge_count=edge_count))
@ -262,35 +247,33 @@ def generate_ontology():
}), 500 }), 500
# ============== 接口2：构建图谱 ============== # ============== Endpoint 2: build graph ==============
@graph_bp.route('/build', methods=['POST']) @graph_bp.route('/build', methods=['POST'])
def build_graph(): def build_graph():
""" """Endpoint 2: build the graph for the given project_id.
接口2：根据project_id构建图谱
请求（JSON）： Request (JSON):
{ {
"project_id": "proj_xxxx", // 必填，来自接口1 "project_id": "proj_xxxx", // required, from endpoint 1
"graph_name": "图谱名称", // 可选 "graph_name": "Graph name", // optional
"chunk_size": 500, // 可选，默认500 "chunk_size": 500, // optional, default 500
"chunk_overlap": 50 // 可选，默认50 "chunk_overlap": 50 // optional, default 50
} }
返回： Returns:
{ {
"success": true, "success": true,
"data": { "data": {
"project_id": "proj_xxxx", "project_id": "proj_xxxx",
"task_id": "task_xxxx", "task_id": "task_xxxx",
"message": "图谱构建任务已启动" "message": "Graph build task started"
} }
} }
""" """
try: try:
logger.info(t("log.graph_api.m017")) logger.info(t("log.graph_api.m017"))
# 检查配置
errors = [] errors = []
if not Config.NEO4J_PASSWORD: if not Config.NEO4J_PASSWORD:
errors.append("NEO4J未配置") errors.append("NEO4J未配置")
@ -301,7 +284,6 @@ def build_graph():
"error": "配置错误: " + "; ".join(errors) "error": "配置错误: " + "; ".join(errors)
}), 500 }), 500
# 解析请求
data = request.get_json() or {} data = request.get_json() or {}
project_id = data.get('project_id') project_id = data.get('project_id')
logger.debug(t("log.graph_api.m019", project_id=project_id)) logger.debug(t("log.graph_api.m019", project_id=project_id))
@ -312,7 +294,6 @@ def build_graph():
"error": t("api.error.graph.m020") "error": t("api.error.graph.m020")
}), 400 }), 400
# 获取项目
project = ProjectManager.get_project(project_id) project = ProjectManager.get_project(project_id)
if not project: if not project:
return jsonify({ return jsonify({
@ -320,8 +301,8 @@ def build_graph():
"error": t("api.error.graph.m021", project_id=project_id) "error": t("api.error.graph.m021", project_id=project_id)
}), 404 }), 404
# 检查项目状态 # If True, abandon any existing build progress and rebuild from scratch.
force = data.get('force', False) # 强制重新构建 force = data.get('force', False)
if project.status == ProjectStatus.CREATED: if project.status == ProjectStatus.CREATED:
return jsonify({ return jsonify({
@ -336,23 +317,20 @@ def build_graph():
"task_id": project.graph_build_task_id "task_id": project.graph_build_task_id
}), 400 }), 400
# 如果强制重建，重置状态 # On a forced rebuild, drop any prior build artifacts so we restart cleanly.
if force and project.status in [ProjectStatus.GRAPH_BUILDING, ProjectStatus.FAILED, ProjectStatus.GRAPH_COMPLETED]: if force and project.status in [ProjectStatus.GRAPH_BUILDING, ProjectStatus.FAILED, ProjectStatus.GRAPH_COMPLETED]:
project.status = ProjectStatus.ONTOLOGY_GENERATED project.status = ProjectStatus.ONTOLOGY_GENERATED
project.graph_id = None project.graph_id = None
project.graph_build_task_id = None project.graph_build_task_id = None
project.error = None project.error = None
# 获取配置
graph_name = data.get('graph_name', project.name or 'MiroFish Graph') graph_name = data.get('graph_name', project.name or 'MiroFish Graph')
chunk_size = data.get('chunk_size', project.chunk_size or Config.DEFAULT_CHUNK_SIZE) chunk_size = data.get('chunk_size', project.chunk_size or Config.DEFAULT_CHUNK_SIZE)
chunk_overlap = data.get('chunk_overlap', project.chunk_overlap or Config.DEFAULT_CHUNK_OVERLAP) chunk_overlap = data.get('chunk_overlap', project.chunk_overlap or Config.DEFAULT_CHUNK_OVERLAP)
# 更新项目配置
project.chunk_size = chunk_size project.chunk_size = chunk_size
project.chunk_overlap = chunk_overlap project.chunk_overlap = chunk_overlap
# 获取提取的文本
text = ProjectManager.get_extracted_text(project_id) text = ProjectManager.get_extracted_text(project_id)
if not text: if not text:
return jsonify({ return jsonify({
@ -360,7 +338,6 @@ def build_graph():
"error": t("api.error.graph.m024") "error": t("api.error.graph.m024")
}), 400 }), 400
# 获取本体
ontology = project.ontology ontology = project.ontology
if not ontology: if not ontology:
return jsonify({ return jsonify({
@ -368,31 +345,26 @@ def build_graph():
"error": t("api.error.graph.m025") "error": t("api.error.graph.m025")
}), 400 }), 400
# 创建异步任务
task_manager = TaskManager() task_manager = TaskManager()
task_id = task_manager.create_task(f"构建图谱: {graph_name}") task_id = task_manager.create_task(f"构建图谱: {graph_name}")
logger.info(t("log.graph_api.m026", task_id=task_id, project_id=project_id)) logger.info(t("log.graph_api.m026", task_id=task_id, project_id=project_id))
# 更新项目状态
project.status = ProjectStatus.GRAPH_BUILDING project.status = ProjectStatus.GRAPH_BUILDING
project.graph_build_task_id = task_id project.graph_build_task_id = task_id
ProjectManager.save_project(project) ProjectManager.save_project(project)
# 启动后台任务
def build_task(): def build_task():
build_logger = get_logger('mirofish.build') build_logger = get_logger('mirofish.build')
try: try:
build_logger.info(f"[{task_id}] 开始构建图谱...") build_logger.info(t("log.graph_api.m027", task_id=task_id))
task_manager.update_task( task_manager.update_task(
task_id, task_id,
status=TaskStatus.PROCESSING, status=TaskStatus.PROCESSING,
message="初始化图谱构建服务..." message="初始化图谱构建服务..."
) )
# 创建图谱构建服务
builder = GraphBuilderService() builder = GraphBuilderService()
# 分块
task_manager.update_task( task_manager.update_task(
task_id, task_id,
message="文本分块中...", message="文本分块中...",
@ -405,7 +377,6 @@ def build_graph():
) )
total_chunks = len(chunks) total_chunks = len(chunks)
# 创建图谱
task_manager.update_task( task_manager.update_task(
task_id, task_id,
message="创建Zep图谱...", message="创建Zep图谱...",
@ -413,11 +384,9 @@ def build_graph():
) )
graph_id = builder.create_graph(name=graph_name) graph_id = builder.create_graph(name=graph_name)
# 更新项目的graph_id
project.graph_id = graph_id project.graph_id = graph_id
ProjectManager.save_project(project) ProjectManager.save_project(project)
# 设置本体
task_manager.update_task( task_manager.update_task(
task_id, task_id,
message="设置本体定义...", message="设置本体定义...",
@ -425,9 +394,9 @@ def build_graph():
) )
builder.set_ontology(graph_id, ontology) builder.set_ontology(graph_id, ontology)
# 添加文本（progress_callback 签名是 (msg, progress_ratio)） # Add text. The progress_callback signature is (msg, progress_ratio).
def add_progress_callback(msg, progress_ratio): def add_progress_callback(msg, progress_ratio):
progress = 15 + int(progress_ratio * 40) # 15% - 55% progress = 15 + int(progress_ratio * 40) # maps ratio onto 15%-55%
task_manager.update_task( task_manager.update_task(
task_id, task_id,
message=msg, message=msg,
@ -460,7 +429,7 @@ def build_graph():
skip_chunks=skip_chunks, skip_chunks=skip_chunks,
) )
# 等待Zep处理完成（查询每个episode的processed状态） # Wait for Zep to finish processing (poll each episode's processed flag).
task_manager.update_task( task_manager.update_task(
task_id, task_id,
message="等待Zep处理数据...", message="等待Zep处理数据...",
@ -468,7 +437,7 @@ def build_graph():
) )
def wait_progress_callback(msg, progress_ratio): def wait_progress_callback(msg, progress_ratio):
progress = 55 + int(progress_ratio * 35) # 55% - 90% progress = 55 + int(progress_ratio * 35) # maps ratio onto 55%-90%
task_manager.update_task( task_manager.update_task(
task_id, task_id,
message=msg, message=msg,
@ -477,7 +446,6 @@ def build_graph():
builder._wait_for_episodes(episode_uuids, wait_progress_callback) builder._wait_for_episodes(episode_uuids, wait_progress_callback)
# 获取图谱数据
task_manager.update_task( task_manager.update_task(
task_id, task_id,
message="获取图谱数据...", message="获取图谱数据...",
@ -485,15 +453,19 @@ def build_graph():
) )
graph_data = builder.get_graph_data(graph_id) graph_data = builder.get_graph_data(graph_id)
# 更新项目状态
project.status = ProjectStatus.GRAPH_COMPLETED project.status = ProjectStatus.GRAPH_COMPLETED
ProjectManager.save_project(project) ProjectManager.save_project(project)
node_count = graph_data.get("node_count", 0) node_count = graph_data.get("node_count", 0)
edge_count = graph_data.get("edge_count", 0) edge_count = graph_data.get("edge_count", 0)
build_logger.info(f"[{task_id}] 图谱构建完成: graph_id={graph_id}, 节点={node_count}, 边={edge_count}") build_logger.info(t(
"log.graph_api.m028",
task_id=task_id,
graph_id=graph_id,
node_count=node_count,
edge_count=edge_count,
))
# 完成
task_manager.update_task( task_manager.update_task(
task_id, task_id,
status=TaskStatus.COMPLETED, status=TaskStatus.COMPLETED,
@ -509,8 +481,8 @@ def build_graph():
) )
except Exception as e: except Exception as e:
# 更新项目状态为失败 # Mark the project as FAILED so the UI can surface the error.
build_logger.error(f"[{task_id}] 图谱构建失败: {str(e)}") build_logger.error(t("log.graph_api.m029", task_id=task_id, e=str(e)))
build_logger.debug(traceback.format_exc()) build_logger.debug(traceback.format_exc())
project.status = ProjectStatus.FAILED project.status = ProjectStatus.FAILED
@ -524,7 +496,6 @@ def build_graph():
error=traceback.format_exc() error=traceback.format_exc()
) )
# 启动后台线程
thread = threading.Thread(target=build_task, daemon=True) thread = threading.Thread(target=build_task, daemon=True)
thread.start() thread.start()
@ -545,13 +516,11 @@ def build_graph():
}), 500 }), 500
# ============== 任务查询接口 ============== # ============== Task query endpoints ==============
@graph_bp.route('/task/<task_id>', methods=['GET']) @graph_bp.route('/task/<task_id>', methods=['GET'])
def get_task(task_id: str): def get_task(task_id: str):
""" """Query the status of a task."""
查询任务状态
"""
task = TaskManager().get_task(task_id) task = TaskManager().get_task(task_id)
if not task: if not task:
@ -568,9 +537,7 @@ def get_task(task_id: str):
@graph_bp.route('/tasks', methods=['GET']) @graph_bp.route('/tasks', methods=['GET'])
def list_tasks(): def list_tasks():
""" """List all tasks."""
列出所有任务
"""
tasks = TaskManager().list_tasks() tasks = TaskManager().list_tasks()
return jsonify({ return jsonify({
@ -580,7 +547,7 @@ def list_tasks():
}) })
# ============== 图谱数据接口 ============== # ============== Graph data endpoints ==============
def _refresh_graph_cache(graph_id: str): def _refresh_graph_cache(graph_id: str):
"""Background thread: fetch graph data from Neo4j and update cache.""" """Background thread: fetch graph data from Neo4j and update cache."""
@ -607,11 +574,11 @@ def _refresh_graph_cache(graph_id: str):
@graph_bp.route('/data/<graph_id>', methods=['GET']) @graph_bp.route('/data/<graph_id>', methods=['GET'])
def get_graph_data(graph_id: str): def get_graph_data(graph_id: str):
""" """Return graph data (nodes and edges).
获取图谱数据（节点和边）。
- 有缓存且未过期：直接返回缓存，不调用 Zep - Fresh cache: serve from cache without hitting Zep.
- 有缓存但已过期：立即返回旧缓存，后台异步刷新 - Stale cache: return the old cache immediately and refresh in the background.
- 无缓存：后台线程拉取，返回 202 让前端稍后重试 - No cache: kick off a background fetch and return 202 so the frontend retries.
""" """
if not Config.NEO4J_PASSWORD: if not Config.NEO4J_PASSWORD:
return jsonify({"success": False, "error": t("api.error.graph.m028")}), 500 return jsonify({"success": False, "error": t("api.error.graph.m028")}), 500
@ -639,9 +606,7 @@ def get_graph_data(graph_id: str):
@graph_bp.route('/delete/<graph_id>', methods=['DELETE']) @graph_bp.route('/delete/<graph_id>', methods=['DELETE'])
def delete_graph(graph_id: str): def delete_graph(graph_id: str):
""" """Delete a Zep graph."""
删除Zep图谱
"""
try: try:
if not Config.NEO4J_PASSWORD: if not Config.NEO4J_PASSWORD:
return jsonify({ return jsonify({

204
backend/app/api/report.py
View File

@ -1,6 +1,7 @@
""" """
Report API路由 Report API routes.
提供模拟报告生成、获取、对话等接口
Provides endpoints for generating, retrieving, and chatting about simulation reports.
""" """
import os import os
@ -20,30 +21,30 @@ from ..utils.locale import t, get_locale, set_locale
logger = get_logger('mirofish.api.report') logger = get_logger('mirofish.api.report')
# ============== 报告生成接口 ============== # ============== Report generation endpoints ==============
@report_bp.route('/generate', methods=['POST']) @report_bp.route('/generate', methods=['POST'])
def generate_report(): def generate_report():
""" """
生成模拟分析报告（异步任务） Generate a simulation analysis report (asynchronous task).
这是一个耗时操作，接口会立即返回task_id， This is a long-running operation. The endpoint returns a task_id immediately;
使用 GET /api/report/generate/status 查询进度 use GET /api/report/generate/status to poll progress.
请求（JSON）： Request (JSON):
{ {
"simulation_id": "sim_xxxx", // 必填，模拟ID "simulation_id": "sim_xxxx", // required, simulation ID
"force_regenerate": false // 可选，强制重新生成 "force_regenerate": false // optional, force regeneration
} }
返回： Returns:
{ {
"success": true, "success": true,
"data": { "data": {
"simulation_id": "sim_xxxx", "simulation_id": "sim_xxxx",
"task_id": "task_xxxx", "task_id": "task_xxxx",
"status": "generating", "status": "generating",
"message": "报告生成任务已启动" "message": "Report generation task started"
} }
} }
""" """
@ -59,7 +60,6 @@ def generate_report():
force_regenerate = data.get('force_regenerate', False) force_regenerate = data.get('force_regenerate', False)
# 获取模拟信息
manager = SimulationManager() manager = SimulationManager()
state = manager.get_simulation(simulation_id) state = manager.get_simulation(simulation_id)
@ -69,7 +69,7 @@ def generate_report():
"error": t('api.simulationNotFound', id=simulation_id) "error": t('api.simulationNotFound', id=simulation_id)
}), 404 }), 404
# 检查是否已有报告 # Skip regeneration if a completed report already exists for this simulation.
if not force_regenerate: if not force_regenerate:
existing_report = ReportManager.get_report_by_simulation(simulation_id) existing_report = ReportManager.get_report_by_simulation(simulation_id)
if existing_report and existing_report.status == ReportStatus.COMPLETED: if existing_report and existing_report.status == ReportStatus.COMPLETED:
@ -84,7 +84,6 @@ def generate_report():
} }
}) })
# 获取项目信息
project = ProjectManager.get_project(state.project_id) project = ProjectManager.get_project(state.project_id)
if not project: if not project:
return jsonify({ return jsonify({
@ -106,11 +105,11 @@ def generate_report():
"error": t('api.missingSimRequirement') "error": t('api.missingSimRequirement')
}), 400 }), 400
# 提前生成 report_id，以便立即返回给前端 # Generate report_id eagerly so the frontend can use it immediately
# (before the background task has actually persisted anything).
import uuid import uuid
report_id = f"report_{uuid.uuid4().hex[:12]}" report_id = f"report_{uuid.uuid4().hex[:12]}"
# 创建异步任务
task_manager = TaskManager() task_manager = TaskManager()
task_id = task_manager.create_task( task_id = task_manager.create_task(
task_type="report_generate", task_type="report_generate",
@ -124,7 +123,6 @@ def generate_report():
# Capture locale before spawning background thread # Capture locale before spawning background thread
current_locale = get_locale() current_locale = get_locale()
# 定义后台任务
def run_generate(): def run_generate():
set_locale(current_locale) set_locale(current_locale)
try: try:
@ -135,14 +133,12 @@ def generate_report():
message=t('api.initReportAgent') message=t('api.initReportAgent')
) )
# 创建Report Agent
agent = ReportAgent( agent = ReportAgent(
graph_id=graph_id, graph_id=graph_id,
simulation_id=simulation_id, simulation_id=simulation_id,
simulation_requirement=simulation_requirement simulation_requirement=simulation_requirement
) )
# 进度回调
def progress_callback(stage, progress, message): def progress_callback(stage, progress, message):
task_manager.update_task( task_manager.update_task(
task_id, task_id,
@ -150,13 +146,13 @@ def generate_report():
message=f"[{stage}] {message}" message=f"[{stage}] {message}"
) )
# 生成报告（传入预先生成的 report_id） # Pass in the pre-generated report_id so the persisted report matches
# the id we already returned to the frontend.
report = agent.generate_report( report = agent.generate_report(
progress_callback=progress_callback, progress_callback=progress_callback,
report_id=report_id report_id=report_id
) )
# 保存报告
ReportManager.save_report(report) ReportManager.save_report(report)
if report.status == ReportStatus.COMPLETED: if report.status == ReportStatus.COMPLETED:
@ -175,7 +171,6 @@ def generate_report():
logger.error(t("log.report_api.m001", str=str(e))) logger.error(t("log.report_api.m001", str=str(e)))
task_manager.fail_task(task_id, str(e)) task_manager.fail_task(task_id, str(e))
# 启动后台线程
thread = threading.Thread(target=run_generate, daemon=True) thread = threading.Thread(target=run_generate, daemon=True)
thread.start() thread.start()
@ -203,15 +198,15 @@ def generate_report():
@report_bp.route('/generate/status', methods=['POST']) @report_bp.route('/generate/status', methods=['POST'])
def get_generate_status(): def get_generate_status():
""" """
查询报告生成任务进度 Query the progress of a report generation task.
请求（JSON）： Request (JSON):
{ {
"task_id": "task_xxxx", // 可选，generate返回的task_id "task_id": "task_xxxx", // optional, task_id returned by generate
"simulation_id": "sim_xxxx" // 可选，模拟ID "simulation_id": "sim_xxxx" // optional, simulation ID
} }
返回： Returns:
{ {
"success": true, "success": true,
"data": { "data": {
@ -228,7 +223,8 @@ def get_generate_status():
task_id = data.get('task_id') task_id = data.get('task_id')
simulation_id = data.get('simulation_id') simulation_id = data.get('simulation_id')
# 如果提供了simulation_id，先检查是否已有完成的报告 # If simulation_id is provided, short-circuit when a completed report already exists
# so callers don't have to track a stale task_id after a successful run.
if simulation_id: if simulation_id:
existing_report = ReportManager.get_report_by_simulation(simulation_id) existing_report = ReportManager.get_report_by_simulation(simulation_id)
if existing_report and existing_report.status == ReportStatus.COMPLETED: if existing_report and existing_report.status == ReportStatus.COMPLETED:
@ -272,14 +268,14 @@ def get_generate_status():
}), 500 }), 500
# ============== 报告获取接口 ============== # ============== Report retrieval endpoints ==============
@report_bp.route('/<report_id>', methods=['GET']) @report_bp.route('/<report_id>', methods=['GET'])
def get_report(report_id: str): def get_report(report_id: str):
""" """
获取报告详情 Get report details.
返回： Returns:
{ {
"success": true, "success": true,
"data": { "data": {
@ -319,9 +315,9 @@ def get_report(report_id: str):
@report_bp.route('/by-simulation/<simulation_id>', methods=['GET']) @report_bp.route('/by-simulation/<simulation_id>', methods=['GET'])
def get_report_by_simulation(simulation_id: str): def get_report_by_simulation(simulation_id: str):
""" """
根据模拟ID获取报告 Get the report for a given simulation ID.
返回： Returns:
{ {
"success": true, "success": true,
"data": { "data": {
@ -358,13 +354,13 @@ def get_report_by_simulation(simulation_id: str):
@report_bp.route('/list', methods=['GET']) @report_bp.route('/list', methods=['GET'])
def list_reports(): def list_reports():
""" """
列出所有报告 List all reports.
Query参数： Query parameters:
simulation_id: 按模拟ID过滤（可选） simulation_id: optional filter by simulation ID.
limit: 返回数量限制（默认50） limit: maximum number of reports to return (default 50).
返回： Returns:
{ {
"success": true, "success": true,
"data": [...], "data": [...],
@ -398,9 +394,9 @@ def list_reports():
@report_bp.route('/<report_id>/download', methods=['GET']) @report_bp.route('/<report_id>/download', methods=['GET'])
def download_report(report_id: str): def download_report(report_id: str):
""" """
下载报告（Markdown格式） Download a report as a Markdown file.
返回Markdown文件 Returns the Markdown file as an attachment.
""" """
try: try:
report = ReportManager.get_report(report_id) report = ReportManager.get_report(report_id)
@ -414,7 +410,8 @@ def download_report(report_id: str):
md_path = ReportManager._get_report_markdown_path(report_id) md_path = ReportManager._get_report_markdown_path(report_id)
if not os.path.exists(md_path): if not os.path.exists(md_path):
# 如果MD文件不存在，生成一个临时文件 # MD file is missing on disk; materialize a temp file from the in-memory content
# so the download still succeeds for older reports that were never persisted.
import tempfile import tempfile
with tempfile.NamedTemporaryFile(mode='w', suffix='.md', delete=False) as f: with tempfile.NamedTemporaryFile(mode='w', suffix='.md', delete=False) as f:
f.write(report.markdown_content) f.write(report.markdown_content)
@ -443,7 +440,7 @@ def download_report(report_id: str):
@report_bp.route('/<report_id>', methods=['DELETE']) @report_bp.route('/<report_id>', methods=['DELETE'])
def delete_report(report_id: str): def delete_report(report_id: str):
"""删除报告""" """Delete a report."""
try: try:
success = ReportManager.delete_report(report_id) success = ReportManager.delete_report(report_id)
@ -467,32 +464,33 @@ def delete_report(report_id: str):
}), 500 }), 500
# ============== Report Agent对话接口 ============== # ============== Report Agent chat endpoints ==============
@report_bp.route('/chat', methods=['POST']) @report_bp.route('/chat', methods=['POST'])
def chat_with_report_agent(): def chat_with_report_agent():
""" """
与Report Agent对话 Chat with the Report Agent.
Report Agent可以在对话中自主调用检索工具来回答问题 The Report Agent can autonomously invoke retrieval tools during the conversation
to answer the user's question.
请求（JSON）： Request (JSON):
{ {
"simulation_id": "sim_xxxx", // 必填，模拟ID "simulation_id": "sim_xxxx", // required, simulation ID
"message": "请解释一下舆情走向", // 必填，用户消息 "message": "Explain the sentiment trend", // required, user message
"chat_history": [ // 可选，对话历史 "chat_history": [ // optional, prior turns
{"role": "user", "content": "..."}, {"role": "user", "content": "..."},
{"role": "assistant", "content": "..."} {"role": "assistant", "content": "..."}
] ]
} }
返回： Returns:
{ {
"success": true, "success": true,
"data": { "data": {
"response": "Agent回复...", "response": "Agent reply...",
"tool_calls": [调用的工具列表], "tool_calls": [list of tools invoked],
"sources": [信息来源] "sources": [information sources]
} }
} }
""" """
@ -515,7 +513,6 @@ def chat_with_report_agent():
"error": t('api.requireMessage') "error": t('api.requireMessage')
}), 400 }), 400
# 获取模拟和项目信息
manager = SimulationManager() manager = SimulationManager()
state = manager.get_simulation(simulation_id) state = manager.get_simulation(simulation_id)
@ -541,7 +538,6 @@ def chat_with_report_agent():
simulation_requirement = project.simulation_requirement or "" simulation_requirement = project.simulation_requirement or ""
# 创建Agent并进行对话
agent = ReportAgent( agent = ReportAgent(
graph_id=graph_id, graph_id=graph_id,
simulation_id=simulation_id, simulation_id=simulation_id,
@ -564,22 +560,22 @@ def chat_with_report_agent():
}), 500 }), 500
# ============== 报告进度与分章节接口 ============== # ============== Report progress and section endpoints ==============
@report_bp.route('/<report_id>/progress', methods=['GET']) @report_bp.route('/<report_id>/progress', methods=['GET'])
def get_report_progress(report_id: str): def get_report_progress(report_id: str):
""" """
获取报告生成进度（实时） Get real-time report generation progress.
返回： Returns:
{ {
"success": true, "success": true,
"data": { "data": {
"status": "generating", "status": "generating",
"progress": 45, "progress": 45,
"message": "正在生成章节: 关键发现", "message": "Generating section: Key Findings",
"current_section": "关键发现", "current_section": "Key Findings",
"completed_sections": ["执行摘要", "模拟背景"], "completed_sections": ["Executive Summary", "Simulation Background"],
"updated_at": "2025-12-09T..." "updated_at": "2025-12-09T..."
} }
} }
@ -610,11 +606,12 @@ def get_report_progress(report_id: str):
@report_bp.route('/<report_id>/sections', methods=['GET']) @report_bp.route('/<report_id>/sections', methods=['GET'])
def get_report_sections(report_id: str): def get_report_sections(report_id: str):
""" """
获取已生成的章节列表（分章节输出） Get the list of sections generated so far (per-section streaming output).
前端可以轮询此接口获取已生成的章节内容，无需等待整个报告完成 The frontend can poll this endpoint to render sections incrementally,
without waiting for the entire report to finish.
返回： Returns:
{ {
"success": true, "success": true,
"data": { "data": {
@ -623,7 +620,7 @@ def get_report_sections(report_id: str):
{ {
"filename": "section_01.md", "filename": "section_01.md",
"section_index": 1, "section_index": 1,
"content": "## 执行摘要\\n\\n..." "content": "## Executive Summary\\n\\n..."
}, },
... ...
], ],
@ -635,7 +632,6 @@ def get_report_sections(report_id: str):
try: try:
sections = ReportManager.get_generated_sections(report_id) sections = ReportManager.get_generated_sections(report_id)
# 获取报告状态
report = ReportManager.get_report(report_id) report = ReportManager.get_report(report_id)
is_complete = report is not None and report.status == ReportStatus.COMPLETED is_complete = report is not None and report.status == ReportStatus.COMPLETED
@ -661,14 +657,14 @@ def get_report_sections(report_id: str):
@report_bp.route('/<report_id>/section/<int:section_index>', methods=['GET']) @report_bp.route('/<report_id>/section/<int:section_index>', methods=['GET'])
def get_single_section(report_id: str, section_index: int): def get_single_section(report_id: str, section_index: int):
""" """
获取单个章节内容 Get the content of a single section.
返回： Returns:
{ {
"success": true, "success": true,
"data": { "data": {
"filename": "section_01.md", "filename": "section_01.md",
"content": "## 执行摘要\\n\\n..." "content": "## Executive Summary\\n\\n..."
} }
} }
""" """
@ -702,16 +698,16 @@ def get_single_section(report_id: str, section_index: int):
}), 500 }), 500
# ============== 报告状态检查接口 ============== # ============== Report status check endpoints ==============
@report_bp.route('/check/<simulation_id>', methods=['GET']) @report_bp.route('/check/<simulation_id>', methods=['GET'])
def check_report_status(simulation_id: str): def check_report_status(simulation_id: str):
""" """
检查模拟是否有报告，以及报告状态 Check whether a simulation has a report, and report its status.
用于前端判断是否解锁Interview功能 Used by the frontend to decide whether to unlock the Interview feature.
返回： Returns:
{ {
"success": true, "success": true,
"data": { "data": {
@ -730,7 +726,7 @@ def check_report_status(simulation_id: str):
report_status = report.status.value if report else None report_status = report.status.value if report else None
report_id = report.report_id if report else None report_id = report.report_id if report else None
# 只有报告完成后才解锁interview # Interview feature is only unlocked once a report has finished generating.
interview_unlocked = has_report and report.status == ReportStatus.COMPLETED interview_unlocked = has_report and report.status == ReportStatus.COMPLETED
return jsonify({ return jsonify({
@ -753,22 +749,22 @@ def check_report_status(simulation_id: str):
}), 500 }), 500
# ============== Agent 日志接口 ============== # ============== Agent log endpoints ==============
@report_bp.route('/<report_id>/agent-log', methods=['GET']) @report_bp.route('/<report_id>/agent-log', methods=['GET'])
def get_agent_log(report_id: str): def get_agent_log(report_id: str):
""" """
获取 Report Agent 的详细执行日志 Get the detailed execution log of the Report Agent.
实时获取报告生成过程中的每一步动作，包括： Streams every step the agent took while generating the report, including:
- 报告开始、规划开始/完成 - Report start, planning start/complete.
- 每个章节的开始、工具调用、LLM响应、完成 - Per-section start, tool calls, LLM responses, and completion.
- 报告完成或失败 - Final report completion or failure.
Query参数： Query parameters:
from_line: 从第几行开始读取（可选，默认0，用于增量获取） from_line: line offset to start reading from (optional, default 0, for incremental polling).
返回： Returns:
{ {
"success": true, "success": true,
"data": { "data": {
@ -779,7 +775,7 @@ def get_agent_log(report_id: str):
"report_id": "report_xxxx", "report_id": "report_xxxx",
"action": "tool_call", "action": "tool_call",
"stage": "generating", "stage": "generating",
"section_title": "执行摘要", "section_title": "Executive Summary",
"section_index": 1, "section_index": 1,
"details": { "details": {
"tool_name": "insight_forge", "tool_name": "insight_forge",
@ -817,9 +813,9 @@ def get_agent_log(report_id: str):
@report_bp.route('/<report_id>/agent-log/stream', methods=['GET']) @report_bp.route('/<report_id>/agent-log/stream', methods=['GET'])
def stream_agent_log(report_id: str): def stream_agent_log(report_id: str):
""" """
获取完整的 Agent 日志（一次性获取全部） Get the full Agent log in one shot (no pagination).
返回： Returns:
{ {
"success": true, "success": true,
"data": { "data": {
@ -848,27 +844,27 @@ def stream_agent_log(report_id: str):
}), 500 }), 500
# ============== 控制台日志接口 ============== # ============== Console log endpoints ==============
@report_bp.route('/<report_id>/console-log', methods=['GET']) @report_bp.route('/<report_id>/console-log', methods=['GET'])
def get_console_log(report_id: str): def get_console_log(report_id: str):
""" """
获取 Report Agent 的控制台输出日志 Get the Report Agent's console output log.
实时获取报告生成过程中的控制台输出（INFO、WARNING等）， Streams the console output produced during report generation (INFO, WARNING, etc.).
这与 agent-log 接口返回的结构化 JSON 日志不同， Unlike the structured JSON returned by the agent-log endpoint, this is plain-text
是纯文本格式的控制台风格日志。 console-style output.
Query参数： Query parameters:
from_line: 从第几行开始读取（可选，默认0，用于增量获取） from_line: line offset to start reading from (optional, default 0, for incremental polling).
返回： Returns:
{ {
"success": true, "success": true,
"data": { "data": {
"logs": [ "logs": [
"[19:46:14] INFO: 搜索完成: 找到 15 条相关事实", "[19:46:14] INFO: Search complete: found 15 relevant facts",
"[19:46:14] INFO: 图谱搜索: graph_id=xxx, query=...", "[19:46:14] INFO: Graph search: graph_id=xxx, query=...",
... ...
], ],
"total_lines": 100, "total_lines": 100,
@ -899,9 +895,9 @@ def get_console_log(report_id: str):
@report_bp.route('/<report_id>/console-log/stream', methods=['GET']) @report_bp.route('/<report_id>/console-log/stream', methods=['GET'])
def stream_console_log(report_id: str): def stream_console_log(report_id: str):
""" """
获取完整的控制台日志（一次性获取全部） Get the full console log in one shot (no pagination).
返回： Returns:
{ {
"success": true, "success": true,
"data": { "data": {
@ -930,17 +926,17 @@ def stream_console_log(report_id: str):
}), 500 }), 500
# ============== 工具调用接口（供调试使用）============== # ============== Tool invocation endpoints (for debugging) ==============
@report_bp.route('/tools/search', methods=['POST']) @report_bp.route('/tools/search', methods=['POST'])
def search_graph_tool(): def search_graph_tool():
""" """
图谱搜索工具接口（供调试使用） Graph search tool endpoint (for debugging).
请求（JSON）： Request (JSON):
{ {
"graph_id": "mirofish_xxxx", "graph_id": "mirofish_xxxx",
"query": "搜索查询", "query": "search query",
"limit": 10 "limit": 10
} }
""" """
@ -983,9 +979,9 @@ def search_graph_tool():
@report_bp.route('/tools/statistics', methods=['POST']) @report_bp.route('/tools/statistics', methods=['POST'])
def get_graph_statistics_tool(): def get_graph_statistics_tool():
""" """
图谱统计工具接口（供调试使用） Graph statistics tool endpoint (for debugging).
请求（JSON）： Request (JSON):
{ {
"graph_id": "mirofish_xxxx" "graph_id": "mirofish_xxxx"
} }

715
backend/app/api/simulation.py
View File

File diff suppressed because it is too large Load Diff

42
backend/app/config.py
View File

@ -1,38 +1,40 @@
""" """Configuration management.
配置管理
统一从项目根目录的 .env 文件加载配置 Loads configuration values from the project-root ``.env`` file.
""" """
import os import os
from dotenv import load_dotenv from dotenv import load_dotenv
# 加载项目根目录的 .env 文件 # Load the project-root .env file.
# 路径: MiroFish/.env (相对于 backend/app/config.py) # Path: MiroFish/.env (relative to backend/app/config.py).
project_root_env = os.path.join(os.path.dirname(__file__), '../../.env') project_root_env = os.path.join(os.path.dirname(__file__), '../../.env')
if os.path.exists(project_root_env): if os.path.exists(project_root_env):
load_dotenv(project_root_env, override=True) load_dotenv(project_root_env, override=True)
else: else:
# 如果根目录没有 .env，尝试加载环境变量（用于生产环境） # If the project root has no .env, fall back to the process environment
# (used in production deployments).
load_dotenv(override=True) load_dotenv(override=True)
class Config: class Config:
"""Flask配置类""" """Flask configuration class."""
# Flask配置 # Flask settings.
SECRET_KEY = os.environ.get('SECRET_KEY', 'mirofish-secret-key') SECRET_KEY = os.environ.get('SECRET_KEY', 'mirofish-secret-key')
DEBUG = os.environ.get('FLASK_DEBUG', 'True').lower() == 'true' DEBUG = os.environ.get('FLASK_DEBUG', 'True').lower() == 'true'
# JSON配置 - 禁用ASCII转义，让中文直接显示（而不是 \uXXXX 格式） # JSON settings: disable ASCII escaping so non-ASCII output renders literally
# rather than as \uXXXX escape sequences.
JSON_AS_ASCII = False JSON_AS_ASCII = False
# LLM配置（统一使用OpenAI格式） # LLM settings (called via the OpenAI-compatible API surface).
LLM_API_KEY = os.environ.get('LLM_API_KEY') LLM_API_KEY = os.environ.get('LLM_API_KEY')
LLM_BASE_URL = os.environ.get('LLM_BASE_URL', 'https://api.openai.com/v1') LLM_BASE_URL = os.environ.get('LLM_BASE_URL', 'https://api.openai.com/v1')
LLM_MODEL_NAME = os.environ.get('LLM_MODEL_NAME', 'gpt-4o-mini') LLM_MODEL_NAME = os.environ.get('LLM_MODEL_NAME', 'gpt-4o-mini')
# Neo4j + Graphiti配置（替代 Zep Cloud） # Neo4j + Graphiti settings (replacement for Zep Cloud).
NEO4J_URI = os.environ.get('NEO4J_URI', 'bolt://localhost:7687') NEO4J_URI = os.environ.get('NEO4J_URI', 'bolt://localhost:7687')
NEO4J_USER = os.environ.get('NEO4J_USER', 'neo4j') NEO4J_USER = os.environ.get('NEO4J_USER', 'neo4j')
NEO4J_PASSWORD = os.environ.get('NEO4J_PASSWORD', 'mirofish123') NEO4J_PASSWORD = os.environ.get('NEO4J_PASSWORD', 'mirofish123')
@ -50,23 +52,23 @@ class Config:
EMBEDDING_API_KEY = os.environ.get('EMBEDDING_API_KEY') EMBEDDING_API_KEY = os.environ.get('EMBEDDING_API_KEY')
EMBEDDING_BASE_URL = os.environ.get('EMBEDDING_BASE_URL') EMBEDDING_BASE_URL = os.environ.get('EMBEDDING_BASE_URL')
# Zep配置（保留兼容性，已废弃） # Zep settings (kept for backwards compatibility; deprecated).
ZEP_API_KEY = os.environ.get('ZEP_API_KEY', '') ZEP_API_KEY = os.environ.get('ZEP_API_KEY', '')
# 文件上传配置 # File upload settings.
MAX_CONTENT_LENGTH = 50 * 1024 * 1024 # 50MB MAX_CONTENT_LENGTH = 50 * 1024 * 1024 # 50MB
UPLOAD_FOLDER = os.path.join(os.path.dirname(__file__), '../uploads') UPLOAD_FOLDER = os.path.join(os.path.dirname(__file__), '../uploads')
ALLOWED_EXTENSIONS = {'pdf', 'md', 'txt', 'markdown'} ALLOWED_EXTENSIONS = {'pdf', 'md', 'txt', 'markdown'}
# 文本处理配置 # Text processing settings.
DEFAULT_CHUNK_SIZE = 500 # 默认切块大小 DEFAULT_CHUNK_SIZE = 500 # default chunk size in characters
DEFAULT_CHUNK_OVERLAP = 50 # 默认重叠大小 DEFAULT_CHUNK_OVERLAP = 50 # default overlap in characters
# OASIS模拟配置 # OASIS simulation settings.
OASIS_DEFAULT_MAX_ROUNDS = int(os.environ.get('OASIS_DEFAULT_MAX_ROUNDS', '10')) OASIS_DEFAULT_MAX_ROUNDS = int(os.environ.get('OASIS_DEFAULT_MAX_ROUNDS', '10'))
OASIS_SIMULATION_DATA_DIR = os.path.join(os.path.dirname(__file__), '../uploads/simulations') OASIS_SIMULATION_DATA_DIR = os.path.join(os.path.dirname(__file__), '../uploads/simulations')
# OASIS平台可用动作配置 # OASIS per-platform allowed action lists.
OASIS_TWITTER_ACTIONS = [ OASIS_TWITTER_ACTIONS = [
'CREATE_POST', 'LIKE_POST', 'REPOST', 'FOLLOW', 'DO_NOTHING', 'QUOTE_POST' 'CREATE_POST', 'LIKE_POST', 'REPOST', 'FOLLOW', 'DO_NOTHING', 'QUOTE_POST'
] ]
@ -76,14 +78,14 @@ class Config:
'TREND', 'REFRESH', 'DO_NOTHING', 'FOLLOW', 'MUTE' 'TREND', 'REFRESH', 'DO_NOTHING', 'FOLLOW', 'MUTE'
] ]
# Report Agent配置 # Report agent settings.
REPORT_AGENT_MAX_TOOL_CALLS = int(os.environ.get('REPORT_AGENT_MAX_TOOL_CALLS', '5')) REPORT_AGENT_MAX_TOOL_CALLS = int(os.environ.get('REPORT_AGENT_MAX_TOOL_CALLS', '5'))
REPORT_AGENT_MAX_REFLECTION_ROUNDS = int(os.environ.get('REPORT_AGENT_MAX_REFLECTION_ROUNDS', '2')) REPORT_AGENT_MAX_REFLECTION_ROUNDS = int(os.environ.get('REPORT_AGENT_MAX_REFLECTION_ROUNDS', '2'))
REPORT_AGENT_TEMPERATURE = float(os.environ.get('REPORT_AGENT_TEMPERATURE', '0.5')) REPORT_AGENT_TEMPERATURE = float(os.environ.get('REPORT_AGENT_TEMPERATURE', '0.5'))
@classmethod @classmethod
def validate(cls): def validate(cls):
"""验证必要配置""" """Validate that required configuration values are present."""
errors = [] errors = []
if not cls.LLM_API_KEY: if not cls.LLM_API_KEY:
errors.append("LLM_API_KEY 未配置") errors.append("LLM_API_KEY 未配置")

4
backend/app/models/__init__.py
View File

@ -1,6 +1,4 @@
""" """Data model package."""
数据模型模块
"""
from .task import TaskManager, TaskStatus from .task import TaskManager, TaskStatus
from .project import Project, ProjectStatus, ProjectManager from .project import Project, ProjectStatus, ProjectManager

105
backend/app/models/project.py
View File

@ -1,6 +1,7 @@
""" """Project context management.
项目上下文管理
用于在服务端持久化项目状态，避免前端在接口间传递大量数据 Persists project state on the server so the frontend does not have to round-trip
large blobs of context between API calls.
""" """
import os import os
@ -15,45 +16,45 @@ from ..config import Config
class ProjectStatus(str, Enum): class ProjectStatus(str, Enum):
"""项目状态""" """Project lifecycle status."""
CREATED = "created" # 刚创建，文件已上传 CREATED = "created" # just created, files uploaded
ONTOLOGY_GENERATED = "ontology_generated" # 本体已生成 ONTOLOGY_GENERATED = "ontology_generated" # ontology has been generated
GRAPH_BUILDING = "graph_building" # 图谱构建中 GRAPH_BUILDING = "graph_building" # graph build in progress
GRAPH_COMPLETED = "graph_completed" # 图谱构建完成 GRAPH_COMPLETED = "graph_completed" # graph build finished
FAILED = "failed" # 失败 FAILED = "failed" # build failed
@dataclass @dataclass
class Project: class Project:
"""项目数据模型""" """Project data model."""
project_id: str project_id: str
name: str name: str
status: ProjectStatus status: ProjectStatus
created_at: str created_at: str
updated_at: str updated_at: str
# 文件信息 # File information
files: List[Dict[str, str]] = field(default_factory=list) # [{filename, path, size}] files: List[Dict[str, str]] = field(default_factory=list) # [{filename, path, size}]
total_text_length: int = 0 total_text_length: int = 0
# 本体信息（接口1生成后填充） # Ontology information (filled in after step 1 generates it)
ontology: Optional[Dict[str, Any]] = None ontology: Optional[Dict[str, Any]] = None
analysis_summary: Optional[str] = None analysis_summary: Optional[str] = None
# 图谱信息（接口2完成后填充） # Graph information (filled in after step 2 finishes)
graph_id: Optional[str] = None graph_id: Optional[str] = None
graph_build_task_id: Optional[str] = None graph_build_task_id: Optional[str] = None
# 配置 # Configuration
simulation_requirement: Optional[str] = None simulation_requirement: Optional[str] = None
chunk_size: int = 500 chunk_size: int = 500
chunk_overlap: int = 50 chunk_overlap: int = 50
# 错误信息 # Error message when status == FAILED
error: Optional[str] = None error: Optional[str] = None
def to_dict(self) -> Dict[str, Any]: def to_dict(self) -> Dict[str, Any]:
"""转换为字典""" """Serialize the project to a JSON-friendly dict."""
return { return {
"project_id": self.project_id, "project_id": self.project_id,
"name": self.name, "name": self.name,
@ -74,7 +75,7 @@ class Project:
@classmethod @classmethod
def from_dict(cls, data: Dict[str, Any]) -> 'Project': def from_dict(cls, data: Dict[str, Any]) -> 'Project':
"""从字典创建""" """Reconstruct a project from its serialized dict."""
status = data.get('status', 'created') status = data.get('status', 'created')
if isinstance(status, str): if isinstance(status, str):
status = ProjectStatus(status) status = ProjectStatus(status)
@ -99,46 +100,45 @@ class Project:
class ProjectManager: class ProjectManager:
"""项目管理器 - 负责项目的持久化存储和检索""" """Project manager: handles persistence and retrieval of projects on disk."""
# 项目存储根目录 # Root directory for project storage
PROJECTS_DIR = os.path.join(Config.UPLOAD_FOLDER, 'projects') PROJECTS_DIR = os.path.join(Config.UPLOAD_FOLDER, 'projects')
@classmethod @classmethod
def _ensure_projects_dir(cls): def _ensure_projects_dir(cls):
"""确保项目目录存在""" """Ensure the projects root directory exists."""
os.makedirs(cls.PROJECTS_DIR, exist_ok=True) os.makedirs(cls.PROJECTS_DIR, exist_ok=True)
@classmethod @classmethod
def _get_project_dir(cls, project_id: str) -> str: def _get_project_dir(cls, project_id: str) -> str:
"""获取项目目录路径""" """Return the on-disk directory for a project."""
return os.path.join(cls.PROJECTS_DIR, project_id) return os.path.join(cls.PROJECTS_DIR, project_id)
@classmethod @classmethod
def _get_project_meta_path(cls, project_id: str) -> str: def _get_project_meta_path(cls, project_id: str) -> str:
"""获取项目元数据文件路径""" """Return the path to a project's metadata JSON file."""
return os.path.join(cls._get_project_dir(project_id), 'project.json') return os.path.join(cls._get_project_dir(project_id), 'project.json')
@classmethod @classmethod
def _get_project_files_dir(cls, project_id: str) -> str: def _get_project_files_dir(cls, project_id: str) -> str:
"""获取项目文件存储目录""" """Return the directory where project source files are stored."""
return os.path.join(cls._get_project_dir(project_id), 'files') return os.path.join(cls._get_project_dir(project_id), 'files')
@classmethod @classmethod
def _get_project_text_path(cls, project_id: str) -> str: def _get_project_text_path(cls, project_id: str) -> str:
"""获取项目提取文本存储路径""" """Return the path to a project's extracted text file."""
return os.path.join(cls._get_project_dir(project_id), 'extracted_text.txt') return os.path.join(cls._get_project_dir(project_id), 'extracted_text.txt')
@classmethod @classmethod
def create_project(cls, name: str = "Unnamed Project") -> Project: def create_project(cls, name: str = "Unnamed Project") -> Project:
""" """Create a new project.
创建新项目
Args: Args:
name: 项目名称 name: Display name for the project.
Returns: Returns:
新创建的Project对象 The newly created ``Project`` instance.
""" """
cls._ensure_projects_dir() cls._ensure_projects_dir()
@ -153,20 +153,20 @@ class ProjectManager:
updated_at=now updated_at=now
) )
# 创建项目目录结构 # Create the on-disk project directory layout
project_dir = cls._get_project_dir(project_id) project_dir = cls._get_project_dir(project_id)
files_dir = cls._get_project_files_dir(project_id) files_dir = cls._get_project_files_dir(project_id)
os.makedirs(project_dir, exist_ok=True) os.makedirs(project_dir, exist_ok=True)
os.makedirs(files_dir, exist_ok=True) os.makedirs(files_dir, exist_ok=True)
# 保存项目元数据 # Persist project metadata
cls.save_project(project) cls.save_project(project)
return project return project
@classmethod @classmethod
def save_project(cls, project: Project) -> None: def save_project(cls, project: Project) -> None:
"""保存项目元数据""" """Persist project metadata to disk."""
project.updated_at = datetime.now().isoformat() project.updated_at = datetime.now().isoformat()
meta_path = cls._get_project_meta_path(project.project_id) meta_path = cls._get_project_meta_path(project.project_id)
@ -175,14 +175,13 @@ class ProjectManager:
@classmethod @classmethod
def get_project(cls, project_id: str) -> Optional[Project]: def get_project(cls, project_id: str) -> Optional[Project]:
""" """Load a project by id.
获取项目
Args: Args:
project_id: 项目ID project_id: Project identifier.
Returns: Returns:
Project对象，如果不存在返回None The ``Project`` if it exists, otherwise ``None``.
""" """
meta_path = cls._get_project_meta_path(project_id) meta_path = cls._get_project_meta_path(project_id)
@ -196,14 +195,13 @@ class ProjectManager:
@classmethod @classmethod
def list_projects(cls, limit: int = 50) -> List[Project]: def list_projects(cls, limit: int = 50) -> List[Project]:
""" """List existing projects, newest first.
列出所有项目
Args: Args:
limit: 返回数量限制 limit: Maximum number of projects to return.
Returns: Returns:
项目列表，按创建时间倒序 Projects ordered by ``created_at`` descending.
""" """
cls._ensure_projects_dir() cls._ensure_projects_dir()
@ -213,21 +211,19 @@ class ProjectManager:
if project: if project:
projects.append(project) projects.append(project)
# 按创建时间倒序排序
projects.sort(key=lambda p: p.created_at, reverse=True) projects.sort(key=lambda p: p.created_at, reverse=True)
return projects[:limit] return projects[:limit]
@classmethod @classmethod
def delete_project(cls, project_id: str) -> bool: def delete_project(cls, project_id: str) -> bool:
""" """Delete a project and all of its files.
删除项目及其所有文件
Args: Args:
project_id: 项目ID project_id: Project identifier.
Returns: Returns:
是否删除成功 ``True`` if the project existed and was removed, ``False`` otherwise.
""" """
project_dir = cls._get_project_dir(project_id) project_dir = cls._get_project_dir(project_id)
@ -239,29 +235,26 @@ class ProjectManager:
@classmethod @classmethod
def save_file_to_project(cls, project_id: str, file_storage, original_filename: str) -> Dict[str, str]: def save_file_to_project(cls, project_id: str, file_storage, original_filename: str) -> Dict[str, str]:
""" """Save an uploaded file under the project's files directory.
保存上传的文件到项目目录
Args: Args:
project_id: 项目ID project_id: Project identifier.
file_storage: Flask的FileStorage对象 file_storage: Flask ``FileStorage`` object from the request.
original_filename: 原始文件名 original_filename: The user-supplied filename.
Returns: Returns:
文件信息字典 {filename, path, size} Dict describing the saved file: ``{original_filename, saved_filename, path, size}``.
""" """
files_dir = cls._get_project_files_dir(project_id) files_dir = cls._get_project_files_dir(project_id)
os.makedirs(files_dir, exist_ok=True) os.makedirs(files_dir, exist_ok=True)
# 生成安全的文件名 # Generate a safe randomized filename to avoid collisions
ext = os.path.splitext(original_filename)[1].lower() ext = os.path.splitext(original_filename)[1].lower()
safe_filename = f"{uuid.uuid4().hex[:8]}{ext}" safe_filename = f"{uuid.uuid4().hex[:8]}{ext}"
file_path = os.path.join(files_dir, safe_filename) file_path = os.path.join(files_dir, safe_filename)
# 保存文件
file_storage.save(file_path) file_storage.save(file_path)
# 获取文件大小
file_size = os.path.getsize(file_path) file_size = os.path.getsize(file_path)
return { return {
@ -273,14 +266,14 @@ class ProjectManager:
@classmethod @classmethod
def save_extracted_text(cls, project_id: str, text: str) -> None: def save_extracted_text(cls, project_id: str, text: str) -> None:
"""保存提取的文本""" """Persist the project's extracted full text to disk."""
text_path = cls._get_project_text_path(project_id) text_path = cls._get_project_text_path(project_id)
with open(text_path, 'w', encoding='utf-8') as f: with open(text_path, 'w', encoding='utf-8') as f:
f.write(text) f.write(text)
@classmethod @classmethod
def get_extracted_text(cls, project_id: str) -> Optional[str]: def get_extracted_text(cls, project_id: str) -> Optional[str]:
"""获取提取的文本""" """Read back the project's extracted full text, or ``None`` if absent."""
text_path = cls._get_project_text_path(project_id) text_path = cls._get_project_text_path(project_id)
if not os.path.exists(text_path): if not os.path.exists(text_path):
@ -291,7 +284,7 @@ class ProjectManager:
@classmethod @classmethod
def get_project_files(cls, project_id: str) -> List[str]: def get_project_files(cls, project_id: str) -> List[str]:
"""获取项目的所有文件路径""" """Return the on-disk paths of all files in the project."""
files_dir = cls._get_project_files_dir(project_id) files_dir = cls._get_project_files_dir(project_id)
if not os.path.exists(files_dir): if not os.path.exists(files_dir):

74
backend/app/models/task.py
View File

@ -1,6 +1,6 @@
""" """Task state management.
任务状态管理
用于跟踪长时间运行的任务（如图谱构建） Tracks long-running tasks (e.g. graph build) so callers can poll progress.
""" """
import uuid import uuid
@ -14,30 +14,30 @@ from ..utils.locale import t
class TaskStatus(str, Enum): class TaskStatus(str, Enum):
"""任务状态枚举""" """Task status enum."""
PENDING = "pending" # 等待中 PENDING = "pending" # waiting
PROCESSING = "processing" # 处理中 PROCESSING = "processing" # in progress
COMPLETED = "completed" # 已完成 COMPLETED = "completed" # finished successfully
FAILED = "failed" # 失败 FAILED = "failed" # finished with error
@dataclass @dataclass
class Task: class Task:
"""任务数据类""" """Task data class."""
task_id: str task_id: str
task_type: str task_type: str
status: TaskStatus status: TaskStatus
created_at: datetime created_at: datetime
updated_at: datetime updated_at: datetime
progress: int = 0 # 总进度百分比 0-100 progress: int = 0 # overall progress percentage 0-100
message: str = "" # 状态消息 message: str = "" # human-readable status message
result: Optional[Dict] = None # 任务结果 result: Optional[Dict] = None # task result payload
error: Optional[str] = None # 错误信息 error: Optional[str] = None # error message when failed
metadata: Dict = field(default_factory=dict) # 额外元数据 metadata: Dict = field(default_factory=dict) # arbitrary caller metadata
progress_detail: Dict = field(default_factory=dict) # 详细进度信息 progress_detail: Dict = field(default_factory=dict) # fine-grained progress info
def to_dict(self) -> Dict[str, Any]: def to_dict(self) -> Dict[str, Any]:
"""转换为字典""" """Serialize the task to a JSON-friendly dict."""
return { return {
"task_id": self.task_id, "task_id": self.task_id,
"task_type": self.task_type, "task_type": self.task_type,
@ -54,16 +54,12 @@ class Task:
class TaskManager: class TaskManager:
""" """Thread-safe singleton task registry."""
任务管理器
线程安全的任务状态管理
"""
_instance = None _instance = None
_lock = threading.Lock() _lock = threading.Lock()
def __new__(cls): def __new__(cls):
"""单例模式"""
if cls._instance is None: if cls._instance is None:
with cls._lock: with cls._lock:
if cls._instance is None: if cls._instance is None:
@ -73,15 +69,14 @@ class TaskManager:
return cls._instance return cls._instance
def create_task(self, task_type: str, metadata: Optional[Dict] = None) -> str: def create_task(self, task_type: str, metadata: Optional[Dict] = None) -> str:
""" """Create a new task.
创建新任务
Args: Args:
task_type: 任务类型 task_type: Task type identifier.
metadata: 额外元数据 metadata: Optional caller-supplied metadata.
Returns: Returns:
任务ID The newly created task id.
""" """
task_id = str(uuid.uuid4()) task_id = str(uuid.uuid4())
now = datetime.now() now = datetime.now()
@ -101,7 +96,7 @@ class TaskManager:
return task_id return task_id
def get_task(self, task_id: str) -> Optional[Task]: def get_task(self, task_id: str) -> Optional[Task]:
"""获取任务""" """Return the task for ``task_id`` or ``None`` if unknown."""
with self._task_lock: with self._task_lock:
return self._tasks.get(task_id) return self._tasks.get(task_id)
@ -115,17 +110,16 @@ class TaskManager:
error: Optional[str] = None, error: Optional[str] = None,
progress_detail: Optional[Dict] = None progress_detail: Optional[Dict] = None
): ):
""" """Update mutable fields on an existing task.
更新任务状态
Args: Args:
task_id: 任务ID task_id: Task id to update.
status: 新状态 status: New status, if changing.
progress: 进度 progress: New overall progress (0-100), if changing.
message: 消息 message: New status message, if changing.
result: 结果 result: New result payload, if changing.
error: 错误信息 error: New error message, if changing.
progress_detail: 详细进度信息 progress_detail: New fine-grained progress info, if changing.
""" """
with self._task_lock: with self._task_lock:
task = self._tasks.get(task_id) task = self._tasks.get(task_id)
@ -145,7 +139,7 @@ class TaskManager:
task.progress_detail = progress_detail task.progress_detail = progress_detail
def complete_task(self, task_id: str, result: Dict): def complete_task(self, task_id: str, result: Dict):
"""标记任务完成""" """Mark a task as completed and attach the result."""
self.update_task( self.update_task(
task_id, task_id,
status=TaskStatus.COMPLETED, status=TaskStatus.COMPLETED,
@ -155,7 +149,7 @@ class TaskManager:
) )
def fail_task(self, task_id: str, error: str): def fail_task(self, task_id: str, error: str):
"""标记任务失败""" """Mark a task as failed and attach the error message."""
self.update_task( self.update_task(
task_id, task_id,
status=TaskStatus.FAILED, status=TaskStatus.FAILED,
@ -164,7 +158,7 @@ class TaskManager:
) )
def list_tasks(self, task_type: Optional[str] = None) -> list: def list_tasks(self, task_type: Optional[str] = None) -> list:
"""列出任务""" """List tasks, optionally filtered by ``task_type``, newest first."""
with self._task_lock: with self._task_lock:
tasks = list(self._tasks.values()) tasks = list(self._tasks.values())
if task_type: if task_type:
@ -172,7 +166,7 @@ class TaskManager:
return [t.to_dict() for t in sorted(tasks, key=lambda x: x.created_at, reverse=True)] return [t.to_dict() for t in sorted(tasks, key=lambda x: x.created_at, reverse=True)]
def cleanup_old_tasks(self, max_age_hours: int = 24): def cleanup_old_tasks(self, max_age_hours: int = 24):
"""清理旧任务""" """Drop completed/failed tasks older than ``max_age_hours``."""
from datetime import timedelta from datetime import timedelta
cutoff = datetime.now() - timedelta(hours=max_age_hours) cutoff = datetime.now() - timedelta(hours=max_age_hours)

4
backend/app/services/__init__.py
View File

@ -1,6 +1,4 @@
""" """Business services package."""
业务服务模块
"""
from .ontology_generator import OntologyGenerator from .ontology_generator import OntologyGenerator
from .graph_builder import GraphBuilderService from .graph_builder import GraphBuilderService

99
backend/app/services/graph_builder.py
View File

@ -1,6 +1,7 @@
""" """Graph build service.
图谱构建服务
接口2：使用Zep API构建Standalone Graph Pipeline step 2: build the project's standalone knowledge graph through the
Zep/Graphiti API.
""" """
import os import os
@ -69,7 +70,7 @@ def _classify_entity_type(name: str, summary: str, ontology: Optional[Dict]) ->
@dataclass @dataclass
class GraphInfo: class GraphInfo:
"""图谱信息""" """Summary information about a built graph."""
graph_id: str graph_id: str
node_count: int node_count: int
edge_count: int edge_count: int
@ -85,10 +86,7 @@ class GraphInfo:
class GraphBuilderService: class GraphBuilderService:
""" """Drives knowledge-graph construction via the Zep/Graphiti API."""
图谱构建服务
负责调用Zep API构建知识图谱
"""
def __init__(self, api_key: Optional[str] = None): def __init__(self, api_key: Optional[str] = None):
self.client = GraphitiAdapter() self.client = GraphitiAdapter()
@ -103,21 +101,20 @@ class GraphBuilderService:
chunk_overlap: int = 50, chunk_overlap: int = 50,
batch_size: int = 3 batch_size: int = 3
) -> str: ) -> str:
""" """Kick off a graph build asynchronously.
异步构建图谱
Args: Args:
text: 输入文本 text: Source text to ingest.
ontology: 本体定义（来自接口1的输出） ontology: Ontology definition (the output of pipeline step 1).
graph_name: 图谱名称 graph_name: Display name for the graph.
chunk_size: 文本块大小 chunk_size: Characters per text chunk.
chunk_overlap: 块重叠大小 chunk_overlap: Overlap (in characters) between consecutive chunks.
batch_size: 每批发送的块数量 batch_size: Number of chunks pushed to Zep per batch.
Returns: Returns:
任务ID The id of the task tracking the build.
""" """
# 创建任务 # Register a task to track build progress.
task_id = self.task_manager.create_task( task_id = self.task_manager.create_task(
task_type="graph_build", task_type="graph_build",
metadata={ metadata={
@ -130,7 +127,7 @@ class GraphBuilderService:
# Capture locale before spawning background thread # Capture locale before spawning background thread
current_locale = get_locale() current_locale = get_locale()
# 在后台线程中执行构建 # Run the build on a background thread so the request returns immediately.
thread = threading.Thread( thread = threading.Thread(
target=self._build_graph_worker, target=self._build_graph_worker,
args=(task_id, text, ontology, graph_name, chunk_size, chunk_overlap, batch_size, current_locale) args=(task_id, text, ontology, graph_name, chunk_size, chunk_overlap, batch_size, current_locale)
@ -151,7 +148,7 @@ class GraphBuilderService:
batch_size: int, batch_size: int,
locale: str = 'zh' locale: str = 'zh'
): ):
"""图谱构建工作线程""" """Background worker that performs the graph build."""
set_locale(locale) set_locale(locale)
try: try:
self.task_manager.update_task( self.task_manager.update_task(
@ -161,7 +158,7 @@ class GraphBuilderService:
message=t('progress.startBuildingGraph') message=t('progress.startBuildingGraph')
) )
# 1. 创建图谱 # 1. Create the graph.
graph_id = self.create_graph(graph_name) graph_id = self.create_graph(graph_name)
self.task_manager.update_task( self.task_manager.update_task(
task_id, task_id,
@ -169,7 +166,7 @@ class GraphBuilderService:
message=t('progress.graphCreated', graphId=graph_id) message=t('progress.graphCreated', graphId=graph_id)
) )
# 2. 设置本体 # 2. Set the ontology.
self.set_ontology(graph_id, ontology) self.set_ontology(graph_id, ontology)
self.task_manager.update_task( self.task_manager.update_task(
task_id, task_id,
@ -177,7 +174,7 @@ class GraphBuilderService:
message=t('progress.ontologySet') message=t('progress.ontologySet')
) )
# 3. 文本分块 # 3. Split source text into chunks.
chunks = TextProcessor.split_text(text, chunk_size, chunk_overlap) chunks = TextProcessor.split_text(text, chunk_size, chunk_overlap)
total_chunks = len(chunks) total_chunks = len(chunks)
self.task_manager.update_task( self.task_manager.update_task(
@ -186,7 +183,7 @@ class GraphBuilderService:
message=t('progress.textSplit', count=total_chunks) message=t('progress.textSplit', count=total_chunks)
) )
# 4. 分批发送数据 # 4. Push chunks to the graph in batches.
episode_uuids = self.add_text_batches( episode_uuids = self.add_text_batches(
graph_id, chunks, batch_size, graph_id, chunks, batch_size,
lambda msg, prog: self.task_manager.update_task( lambda msg, prog: self.task_manager.update_task(
@ -196,7 +193,7 @@ class GraphBuilderService:
) )
) )
# 5. 等待Zep处理完成 # 5. Wait for Zep to finish processing the episodes.
self.task_manager.update_task( self.task_manager.update_task(
task_id, task_id,
progress=60, progress=60,
@ -212,7 +209,7 @@ class GraphBuilderService:
) )
) )
# 6. 获取图谱信息 # 6. Fetch the final graph metadata.
self.task_manager.update_task( self.task_manager.update_task(
task_id, task_id,
progress=90, progress=90,
@ -221,7 +218,6 @@ class GraphBuilderService:
graph_info = self._get_graph_info(graph_id) graph_info = self._get_graph_info(graph_id)
# 完成
self.task_manager.complete_task(task_id, { self.task_manager.complete_task(task_id, {
"graph_id": graph_id, "graph_id": graph_id,
"graph_info": graph_info.to_dict(), "graph_info": graph_info.to_dict(),
@ -234,7 +230,7 @@ class GraphBuilderService:
self.task_manager.fail_task(task_id, error_msg) self.task_manager.fail_task(task_id, error_msg)
def create_graph(self, name: str) -> str: def create_graph(self, name: str) -> str:
"""创建Zep图谱（公开方法）""" """Create a new Zep graph and return its id (public API)."""
graph_id = f"mirofish_{uuid.uuid4().hex[:16]}" graph_id = f"mirofish_{uuid.uuid4().hex[:16]}"
self.client.graph.create( self.client.graph.create(
@ -246,7 +242,7 @@ class GraphBuilderService:
return graph_id return graph_id
def set_ontology(self, graph_id: str, ontology: Dict[str, Any]): def set_ontology(self, graph_id: str, ontology: Dict[str, Any]):
"""设置图谱本体提示（Graphiti自动提取实体，本体作为提示存储）""" """Register the ontology with the graph (Graphiti uses it as an extraction prompt)."""
self.client.graph.set_ontology( self.client.graph.set_ontology(
graph_ids=[graph_id], graph_ids=[graph_id],
entities=ontology.get("entity_types"), entities=ontology.get("entity_types"),
@ -261,8 +257,11 @@ class GraphBuilderService:
progress_callback: Optional[Callable] = None, progress_callback: Optional[Callable] = None,
skip_chunks: int = 0, skip_chunks: int = 0,
) -> List[str]: ) -> List[str]:
"""分批添加文本到图谱，返回所有 episode 的 uuid 列表。 """Push chunks to the graph in batches; returns the uuids of all episodes added.
skip_chunks: 跳过已处理的块数（用于断点续传）。"""
Args:
skip_chunks: Number of chunks to skip (used for resume-after-restart).
"""
episode_uuids = [] episode_uuids = []
total_chunks = len(chunks) total_chunks = len(chunks)
@ -279,27 +278,26 @@ class GraphBuilderService:
) )
# 构建episode数据 # Build the per-episode payload structures expected by the client.
episodes = [ episodes = [
type('Episode', (), {'data': chunk, 'type': 'text'})() type('Episode', (), {'data': chunk, 'type': 'text'})()
for chunk in batch_chunks for chunk in batch_chunks
] ]
# 发送到Zep
try: try:
batch_result = self.client.graph.add_batch( batch_result = self.client.graph.add_batch(
graph_id=graph_id, graph_id=graph_id,
episodes=episodes episodes=episodes
) )
# 收集返回的 episode uuid # Collect the uuids returned for each episode.
if batch_result and isinstance(batch_result, list): if batch_result and isinstance(batch_result, list):
for ep in batch_result: for ep in batch_result:
ep_uuid = getattr(ep, 'uuid_', None) or getattr(ep, 'uuid', None) ep_uuid = getattr(ep, 'uuid_', None) or getattr(ep, 'uuid', None)
if ep_uuid: if ep_uuid:
episode_uuids.append(ep_uuid) episode_uuids.append(ep_uuid)
# 避免请求过快 # Throttle to avoid overwhelming the upstream API.
time.sleep(1) time.sleep(1)
except Exception as e: except Exception as e:
@ -315,7 +313,7 @@ class GraphBuilderService:
progress_callback: Optional[Callable] = None, progress_callback: Optional[Callable] = None,
timeout: int = 600 timeout: int = 600
): ):
"""等待所有 episode 处理完成（通过查询每个 episode 的 processed 状态）""" """Poll each episode until Zep marks it processed, or the timeout expires."""
if not episode_uuids: if not episode_uuids:
if progress_callback: if progress_callback:
progress_callback(t('progress.noEpisodesWait'), 1.0) progress_callback(t('progress.noEpisodesWait'), 1.0)
@ -338,7 +336,7 @@ class GraphBuilderService:
) )
break break
# 检查每个 episode 的处理状态 # Check the processing state of each pending episode.
for ep_uuid in list(pending_episodes): for ep_uuid in list(pending_episodes):
try: try:
episode = self.client.graph.episode.get(uuid_=ep_uuid) episode = self.client.graph.episode.get(uuid_=ep_uuid)
@ -349,7 +347,7 @@ class GraphBuilderService:
completed_count += 1 completed_count += 1
except Exception as e: except Exception as e:
# 忽略单个查询错误，继续 # Tolerate a single failed query; the next loop iteration retries.
pass pass
elapsed = int(time.time() - start_time) elapsed = int(time.time() - start_time)
@ -360,20 +358,17 @@ class GraphBuilderService:
) )
if pending_episodes: if pending_episodes:
time.sleep(3) # 每3秒检查一次 time.sleep(3) # poll every 3 seconds
if progress_callback: if progress_callback:
progress_callback(t('progress.processingComplete', completed=completed_count, total=total_episodes), 1.0) progress_callback(t('progress.processingComplete', completed=completed_count, total=total_episodes), 1.0)
def _get_graph_info(self, graph_id: str) -> GraphInfo: def _get_graph_info(self, graph_id: str) -> GraphInfo:
"""获取图谱信息""" """Fetch summary info (counts and entity types) for a graph."""
# 获取节点（分页）
nodes = fetch_all_nodes(self.client, graph_id) nodes = fetch_all_nodes(self.client, graph_id)
# 获取边（分页）
edges = fetch_all_edges(self.client, graph_id) edges = fetch_all_edges(self.client, graph_id)
# 统计实体类型 # Tally distinct entity types across all nodes.
entity_types = set() entity_types = set()
for node in nodes: for node in nodes:
if node.labels: if node.labels:
@ -389,26 +384,24 @@ class GraphBuilderService:
) )
def get_graph_data(self, graph_id: str, ontology: Optional[Dict] = None) -> Dict[str, Any]: def get_graph_data(self, graph_id: str, ontology: Optional[Dict] = None) -> Dict[str, Any]:
""" """Return the full graph payload including timestamps, attributes, and edges.
获取完整图谱数据（包含详细信息）
Args: Args:
graph_id: 图谱ID graph_id: Graph identifier.
Returns: Returns:
包含nodes和edges的字典，包括时间信息、属性等详细数据 Dict with ``nodes``, ``edges``, and aggregate counts.
""" """
nodes = fetch_all_nodes(self.client, graph_id) nodes = fetch_all_nodes(self.client, graph_id)
edges = fetch_all_edges(self.client, graph_id) edges = fetch_all_edges(self.client, graph_id)
# 创建节点映射用于获取节点名称 # Build a uuid->name map so edge endpoints can be labeled.
node_map = {} node_map = {}
for node in nodes: for node in nodes:
node_map[node.uuid_] = node.name or "" node_map[node.uuid_] = node.name or ""
nodes_data = [] nodes_data = []
for node in nodes: for node in nodes:
# 获取创建时间
created_at = getattr(node, 'created_at', None) created_at = getattr(node, 'created_at', None)
if created_at: if created_at:
created_at = str(created_at) created_at = str(created_at)
@ -429,20 +422,18 @@ class GraphBuilderService:
edges_data = [] edges_data = []
for edge in edges: for edge in edges:
# 获取时间信息
created_at = getattr(edge, 'created_at', None) created_at = getattr(edge, 'created_at', None)
valid_at = getattr(edge, 'valid_at', None) valid_at = getattr(edge, 'valid_at', None)
invalid_at = getattr(edge, 'invalid_at', None) invalid_at = getattr(edge, 'invalid_at', None)
expired_at = getattr(edge, 'expired_at', None) expired_at = getattr(edge, 'expired_at', None)
# 获取 episodes # Normalize the episode list (the field may be missing or a single id).
episodes = getattr(edge, 'episodes', None) or getattr(edge, 'episode_ids', None) episodes = getattr(edge, 'episodes', None) or getattr(edge, 'episode_ids', None)
if episodes and not isinstance(episodes, list): if episodes and not isinstance(episodes, list):
episodes = [str(episodes)] episodes = [str(episodes)]
elif episodes: elif episodes:
episodes = [str(e) for e in episodes] episodes = [str(e) for e in episodes]
# 获取 fact_type
fact_type = getattr(edge, 'fact_type', None) or edge.name or "" fact_type = getattr(edge, 'fact_type', None) or edge.name or ""
edges_data.append({ edges_data.append({
@ -471,6 +462,6 @@ class GraphBuilderService:
} }
def delete_graph(self, graph_id: str): def delete_graph(self, graph_id: str):
"""删除图谱""" """Delete a graph by id."""
self.client.graph.delete(graph_id=graph_id) self.client.graph.delete(graph_id=graph_id)

506
backend/app/services/oasis_profile_generator.py
View File

File diff suppressed because it is too large Load Diff

103
backend/app/services/ontology_generator.py
View File

@ -1,6 +1,7 @@
""" """Ontology generation service.
本体生成服务
接口1：分析文本内容，生成适合社会模拟的实体和关系类型定义 Pipeline step 1: analyze the source text and propose entity and relationship
types that fit a social-media opinion simulation.
""" """
import json import json
@ -14,19 +15,19 @@ logger = logging.getLogger(__name__)
def _to_pascal_case(name: str) -> str: def _to_pascal_case(name: str) -> str:
"""将任意格式的名称转换为 PascalCase（如 'works_for' -> 'WorksFor', 'person' -> 'Person'）""" """Convert an arbitrary identifier to PascalCase (e.g. ``works_for`` -> ``WorksFor``)."""
# 按非字母数字字符分割 # Split on non-alphanumeric separators first.
parts = re.split(r'[^a-zA-Z0-9]+', name) parts = re.split(r'[^a-zA-Z0-9]+', name)
# 再按 camelCase 边界分割（如 'camelCase' -> ['camel', 'Case']） # Then split on camelCase boundaries (e.g. ``camelCase`` -> ``['camel', 'Case']``).
words = [] words = []
for part in parts: for part in parts:
words.extend(re.sub(r'([a-z])([A-Z])', r'\1_\2', part).split('_')) words.extend(re.sub(r'([a-z])([A-Z])', r'\1_\2', part).split('_'))
# 每个词首字母大写，过滤空串 # Title-case each non-empty word and concatenate.
result = ''.join(word.capitalize() for word in words if word) result = ''.join(word.capitalize() for word in words if word)
return result if result else 'Unknown' return result if result else 'Unknown'
# 本体生成的系统提示词 # System prompt template for ontology generation.
ONTOLOGY_SYSTEM_PROMPT = """You are a professional knowledge-graph ontology designer. Your task is to analyze the supplied text and simulation requirement and design entity types and relationship types suitable for a **social-media public-opinion simulation**. ONTOLOGY_SYSTEM_PROMPT = """You are a professional knowledge-graph ontology designer. Your task is to analyze the supplied text and simulation requirement and design entity types and relationship types suitable for a **social-media public-opinion simulation**.
**Important: you must output valid JSON data and nothing else.** **Important: you must output valid JSON data and nothing else.**
@ -174,10 +175,7 @@ B. **Concrete types (8 entries, designed from the text content)**:
class OntologyGenerator: class OntologyGenerator:
""" """Generate an entity- and edge-type ontology from arbitrary input text."""
本体生成器
分析文本内容，生成实体和关系类型定义
"""
def __init__(self, llm_client: Optional[LLMClient] = None): def __init__(self, llm_client: Optional[LLMClient] = None):
self.llm_client = llm_client or LLMClient() self.llm_client = llm_client or LLMClient()
@ -188,18 +186,17 @@ class OntologyGenerator:
simulation_requirement: str, simulation_requirement: str,
additional_context: Optional[str] = None additional_context: Optional[str] = None
) -> Dict[str, Any]: ) -> Dict[str, Any]:
""" """Generate an ontology definition.
生成本体定义
Args: Args:
document_texts: 文档文本列表 document_texts: Source document text segments.
simulation_requirement: 模拟需求描述 simulation_requirement: Description of the simulation goal.
additional_context: 额外上下文 additional_context: Optional supplemental context.
Returns: Returns:
本体定义（entity_types, edge_types等） The ontology dict with ``entity_types``, ``edge_types``, and a summary.
""" """
# 构建用户消息 # Compose the user message that frames the LLM request.
user_message = self._build_user_message( user_message = self._build_user_message(
document_texts, document_texts,
simulation_requirement, simulation_requirement,
@ -213,19 +210,19 @@ class OntologyGenerator:
{"role": "user", "content": user_message} {"role": "user", "content": user_message}
] ]
# 调用LLM # Invoke the LLM.
result = self.llm_client.chat_json( result = self.llm_client.chat_json(
messages=messages, messages=messages,
temperature=0.3, temperature=0.3,
max_tokens=4096 max_tokens=4096
) )
# 验证和后处理 # Validate the LLM response and post-process it.
result = self._validate_and_process(result) result = self._validate_and_process(result)
return result return result
# 传给 LLM 的文本最大长度（5万字） # Maximum length of source text passed to the LLM (50k characters).
MAX_TEXT_LENGTH_FOR_LLM = 50000 MAX_TEXT_LENGTH_FOR_LLM = 50000
def _build_user_message( def _build_user_message(
@ -234,13 +231,14 @@ class OntologyGenerator:
simulation_requirement: str, simulation_requirement: str,
additional_context: Optional[str] additional_context: Optional[str]
) -> str: ) -> str:
"""构建用户消息""" """Build the user-message string for the ontology LLM call."""
# 合并文本 # Concatenate the source documents into a single string.
combined_text = "\n\n---\n\n".join(document_texts) combined_text = "\n\n---\n\n".join(document_texts)
original_length = len(combined_text) original_length = len(combined_text)
# 如果文本超过5万字，截断（仅影响传给LLM的内容，不影响图谱构建） # If the combined text exceeds the LLM input cap, truncate it for the
# LLM call only. The full text is still used for graph construction.
if len(combined_text) > self.MAX_TEXT_LENGTH_FOR_LLM: if len(combined_text) > self.MAX_TEXT_LENGTH_FOR_LLM:
combined_text = combined_text[:self.MAX_TEXT_LENGTH_FOR_LLM] combined_text = combined_text[:self.MAX_TEXT_LENGTH_FOR_LLM]
combined_text += f"\n\n...(original text is {original_length} characters; only the first {self.MAX_TEXT_LENGTH_FOR_LLM} characters were used for ontology analysis)..." combined_text += f"\n\n...(original text is {original_length} characters; only the first {self.MAX_TEXT_LENGTH_FOR_LLM} characters were used for ontology analysis)..."
@ -275,9 +273,9 @@ Based on the content above, design entity types and relationship types suitable
return message return message
def _validate_and_process(self, result: Dict[str, Any]) -> Dict[str, Any]: def _validate_and_process(self, result: Dict[str, Any]) -> Dict[str, Any]:
"""验证和后处理结果""" """Validate and post-process the LLM-generated ontology dict."""
# 确保必要字段存在 # Ensure required top-level fields exist.
if "entity_types" not in result: if "entity_types" not in result:
result["entity_types"] = [] result["entity_types"] = []
if "edge_types" not in result: if "edge_types" not in result:
@ -285,11 +283,12 @@ Based on the content above, design entity types and relationship types suitable
if "analysis_summary" not in result: if "analysis_summary" not in result:
result["analysis_summary"] = "" result["analysis_summary"] = ""
# 验证实体类型 # Validate entity types.
# 记录原始名称到 PascalCase 的映射，用于后续修正 edge 的 source_targets 引用 # Track original-name -> PascalCase mapping so edge source_targets
# references can be fixed up consistently below.
entity_name_map = {} entity_name_map = {}
for entity in result["entity_types"]: for entity in result["entity_types"]:
# 强制将 entity name 转为 PascalCase（Zep API 要求） # Force entity names to PascalCase (required by the Zep API).
if "name" in entity: if "name" in entity:
original_name = entity["name"] original_name = entity["name"]
entity["name"] = _to_pascal_case(original_name) entity["name"] = _to_pascal_case(original_name)
@ -300,19 +299,20 @@ Based on the content above, design entity types and relationship types suitable
entity["attributes"] = [] entity["attributes"] = []
if "examples" not in entity: if "examples" not in entity:
entity["examples"] = [] entity["examples"] = []
# 确保description不超过100字符 # Truncate descriptions longer than 100 characters.
if len(entity.get("description", "")) > 100: if len(entity.get("description", "")) > 100:
entity["description"] = entity["description"][:97] + "..." entity["description"] = entity["description"][:97] + "..."
# 验证关系类型 # Validate edge types.
for edge in result["edge_types"]: for edge in result["edge_types"]:
# 强制将 edge name 转为 SCREAMING_SNAKE_CASE（Zep API 要求） # Force edge names to SCREAMING_SNAKE_CASE (required by the Zep API).
if "name" in edge: if "name" in edge:
original_name = edge["name"] original_name = edge["name"]
edge["name"] = original_name.upper() edge["name"] = original_name.upper()
if edge["name"] != original_name: if edge["name"] != original_name:
logger.warning(f"Edge type name '{original_name}' auto-converted to '{edge['name']}'") logger.warning(f"Edge type name '{original_name}' auto-converted to '{edge['name']}'")
# 修正 source_targets 中的实体名称引用，与转换后的 PascalCase 保持一致 # Rewrite source_targets entity-name references to match the
# PascalCase-normalized entity names.
for st in edge.get("source_targets", []): for st in edge.get("source_targets", []):
if st.get("source") in entity_name_map: if st.get("source") in entity_name_map:
st["source"] = entity_name_map[st["source"]] st["source"] = entity_name_map[st["source"]]
@ -325,11 +325,11 @@ Based on the content above, design entity types and relationship types suitable
if len(edge.get("description", "")) > 100: if len(edge.get("description", "")) > 100:
edge["description"] = edge["description"][:97] + "..." edge["description"] = edge["description"][:97] + "..."
# Zep API 限制：最多 10 个自定义实体类型，最多 10 个自定义边类型 # Zep API caps: at most 10 custom entity types and 10 custom edge types.
MAX_ENTITY_TYPES = 10 MAX_ENTITY_TYPES = 10
MAX_EDGE_TYPES = 10 MAX_EDGE_TYPES = 10
# 去重：按 name 去重，保留首次出现的 # Deduplicate by name, keeping the first occurrence.
seen_names = set() seen_names = set()
deduped = [] deduped = []
for entity in result["entity_types"]: for entity in result["entity_types"]:
@ -341,7 +341,7 @@ Based on the content above, design entity types and relationship types suitable
logger.warning(f"Duplicate entity type '{name}' removed during validation") logger.warning(f"Duplicate entity type '{name}' removed during validation")
result["entity_types"] = deduped result["entity_types"] = deduped
# 兜底类型定义 # Fallback entity-type definitions used when the LLM omits them.
person_fallback = { person_fallback = {
"name": "Person", "name": "Person",
"description": "Any individual person not fitting other specific person types.", "description": "Any individual person not fitting other specific person types.",
@ -362,12 +362,12 @@ Based on the content above, design entity types and relationship types suitable
"examples": ["small business", "community group"] "examples": ["small business", "community group"]
} }
# 检查是否已有兜底类型 # Check whether the fallback types are already present.
entity_names = {e["name"] for e in result["entity_types"]} entity_names = {e["name"] for e in result["entity_types"]}
has_person = "Person" in entity_names has_person = "Person" in entity_names
has_organization = "Organization" in entity_names has_organization = "Organization" in entity_names
# 需要添加的兜底类型 # Collect missing fallback types to add below.
fallbacks_to_add = [] fallbacks_to_add = []
if not has_person: if not has_person:
fallbacks_to_add.append(person_fallback) fallbacks_to_add.append(person_fallback)
@ -378,17 +378,15 @@ Based on the content above, design entity types and relationship types suitable
current_count = len(result["entity_types"]) current_count = len(result["entity_types"])
needed_slots = len(fallbacks_to_add) needed_slots = len(fallbacks_to_add)
# 如果添加后会超过 10 个，需要移除一些现有类型 # If adding the fallbacks would exceed the cap, drop some existing types.
if current_count + needed_slots > MAX_ENTITY_TYPES: if current_count + needed_slots > MAX_ENTITY_TYPES:
# 计算需要移除多少个
to_remove = current_count + needed_slots - MAX_ENTITY_TYPES to_remove = current_count + needed_slots - MAX_ENTITY_TYPES
# 从末尾移除（保留前面更重要的具体类型） # Drop trailing types first; the more specific types come earlier.
result["entity_types"] = result["entity_types"][:-to_remove] result["entity_types"] = result["entity_types"][:-to_remove]
# 添加兜底类型
result["entity_types"].extend(fallbacks_to_add) result["entity_types"].extend(fallbacks_to_add)
# 最终确保不超过限制（防御性编程） # Defensive cap enforcement: hard-trim if anything slipped through.
if len(result["entity_types"]) > MAX_ENTITY_TYPES: if len(result["entity_types"]) > MAX_ENTITY_TYPES:
result["entity_types"] = result["entity_types"][:MAX_ENTITY_TYPES] result["entity_types"] = result["entity_types"][:MAX_ENTITY_TYPES]
@ -398,14 +396,13 @@ Based on the content above, design entity types and relationship types suitable
return result return result
def generate_python_code(self, ontology: Dict[str, Any]) -> str: def generate_python_code(self, ontology: Dict[str, Any]) -> str:
""" """Render the ontology definition as Python source code.
将本体定义转换为Python代码（类似ontology.py）
Args: Args:
ontology: 本体定义 ontology: Ontology definition dict.
Returns: Returns:
Python代码字符串 Python source code as a single string.
""" """
code_lines = [ code_lines = [
'"""', '"""',
@ -421,7 +418,7 @@ Based on the content above, design entity types and relationship types suitable
'', '',
] ]
# 生成实体类型 # Emit each entity type as a Python class.
for entity in ontology.get("entity_types", []): for entity in ontology.get("entity_types", []):
name = entity["name"] name = entity["name"]
desc = entity.get("description", f"A {name} entity.") desc = entity.get("description", f"A {name} entity.")
@ -447,10 +444,10 @@ Based on the content above, design entity types and relationship types suitable
code_lines.append('# ============== 关系类型定义 ==============') code_lines.append('# ============== 关系类型定义 ==============')
code_lines.append('') code_lines.append('')
# 生成关系类型 # Emit each edge type as a Python class.
for edge in ontology.get("edge_types", []): for edge in ontology.get("edge_types", []):
name = edge["name"] name = edge["name"]
# 转换为PascalCase类名 # Convert SCREAMING_SNAKE_CASE -> PascalCase for the class name.
class_name = ''.join(word.capitalize() for word in name.split('_')) class_name = ''.join(word.capitalize() for word in name.split('_'))
desc = edge.get("description", f"A {name} relationship.") desc = edge.get("description", f"A {name} relationship.")
@ -472,7 +469,7 @@ Based on the content above, design entity types and relationship types suitable
code_lines.append('') code_lines.append('')
code_lines.append('') code_lines.append('')
# 生成类型字典 # Emit the type registries.
code_lines.append('# ============== 类型配置 ==============') code_lines.append('# ============== 类型配置 ==============')
code_lines.append('') code_lines.append('')
code_lines.append('ENTITY_TYPES = {') code_lines.append('ENTITY_TYPES = {')
@ -489,7 +486,7 @@ Based on the content above, design entity types and relationship types suitable
code_lines.append('}') code_lines.append('}')
code_lines.append('') code_lines.append('')
# 生成边的source_targets映射 # Emit the edge source_targets map.
code_lines.append('EDGE_SOURCE_TARGETS = {') code_lines.append('EDGE_SOURCE_TARGETS = {')
for edge in ontology.get("edge_types", []): for edge in ontology.get("edge_types", []):
name = edge["name"] name = edge["name"]

713
backend/app/services/report_agent.py
View File

File diff suppressed because it is too large Load Diff

331
backend/app/services/simulation_config_generator.py
View File

@ -1,13 +1,16 @@
""" """
模拟配置智能生成器 Intelligent simulation-configuration generator.
使用LLM根据模拟需求、文档内容、图谱信息自动生成细致的模拟参数
实现全程自动化，无需人工设置参数
采用分步生成策略，避免一次性生成过长内容导致失败： Uses an LLM to derive detailed simulation parameters from the simulation
1. 生成时间配置 requirement, document content, and knowledge-graph information, fully
2. 生成事件配置 automating parameter setup without manual intervention.
3. 分批生成Agent配置
4. 生成平台配置 Employs a step-wise generation strategy to avoid failures caused by
producing too much content in a single call:
1. Generate time configuration
2. Generate event configuration
3. Generate agent configurations in batches
4. Generate platform configuration
""" """
import json import json
@ -25,156 +28,156 @@ from .zep_entity_reader import EntityNode, ZepEntityReader
logger = get_logger('mirofish.simulation_config') logger = get_logger('mirofish.simulation_config')
# 中国作息时间配置（北京时间） # Daily-rhythm config for China (Beijing time, UTC+8).
CHINA_TIMEZONE_CONFIG = { CHINA_TIMEZONE_CONFIG = {
# 深夜时段（几乎无人活动） # Late-night hours: almost no activity.
"dead_hours": [0, 1, 2, 3, 4, 5], "dead_hours": [0, 1, 2, 3, 4, 5],
# 早间时段（逐渐醒来） # Morning hours: gradually waking up.
"morning_hours": [6, 7, 8], "morning_hours": [6, 7, 8],
# 工作时段 # Working hours.
"work_hours": [9, 10, 11, 12, 13, 14, 15, 16, 17, 18], "work_hours": [9, 10, 11, 12, 13, 14, 15, 16, 17, 18],
# 晚间高峰（最活跃） # Evening peak: most active.
"peak_hours": [19, 20, 21, 22], "peak_hours": [19, 20, 21, 22],
# 夜间时段（活跃度下降） # Late-evening hours: activity declining.
"night_hours": [23], "night_hours": [23],
# 活跃度系数 # Activity multipliers.
"activity_multipliers": { "activity_multipliers": {
"dead": 0.05, # 凌晨几乎无人 "dead": 0.05, # Overnight: almost no one online.
"morning": 0.4, # 早间逐渐活跃 "morning": 0.4, # Morning ramp-up.
"work": 0.7, # 工作时段中等 "work": 0.7, # Working hours: moderate activity.
"peak": 1.5, # 晚间高峰 "peak": 1.5, # Evening peak.
"night": 0.5 # 深夜下降 "night": 0.5 # Late-night decline.
} }
} }
@dataclass @dataclass
class AgentActivityConfig: class AgentActivityConfig:
"""单个Agent的活动配置""" """Activity configuration for a single agent."""
agent_id: int agent_id: int
entity_uuid: str entity_uuid: str
entity_name: str entity_name: str
entity_type: str entity_type: str
# 活跃度配置 (0.0-1.0) # Activity configuration (0.0-1.0).
activity_level: float = 0.5 # 整体活跃度 activity_level: float = 0.5 # Overall activity level.
# 发言频率（每小时预期发言次数） # Posting frequency (expected posts per hour).
posts_per_hour: float = 1.0 posts_per_hour: float = 1.0
comments_per_hour: float = 2.0 comments_per_hour: float = 2.0
# 活跃时间段（24小时制，0-23） # Active hours (24-hour clock, 0-23).
active_hours: List[int] = field(default_factory=lambda: list(range(8, 23))) active_hours: List[int] = field(default_factory=lambda: list(range(8, 23)))
# 响应速度（对热点事件的反应延迟，单位：模拟分钟） # Response speed: latency to react to hot events, in simulated minutes.
response_delay_min: int = 5 response_delay_min: int = 5
response_delay_max: int = 60 response_delay_max: int = 60
# 情感倾向 (-1.0到1.0，负面到正面) # Sentiment bias (-1.0 to 1.0, negative to positive).
sentiment_bias: float = 0.0 sentiment_bias: float = 0.0
# 立场（对特定话题的态度） # Stance: attitude toward a given topic.
stance: str = "neutral" # supportive, opposing, neutral, observer stance: str = "neutral" # supportive, opposing, neutral, observer
# 影响力权重（决定其发言被其他Agent看到的概率） # Influence weight: probability of an agent's post being seen by others.
influence_weight: float = 1.0 influence_weight: float = 1.0
@dataclass @dataclass
class TimeSimulationConfig: class TimeSimulationConfig:
"""时间模拟配置（基于中国人作息习惯）""" """Time-simulation configuration (modelled on a Chinese daily rhythm)."""
# 模拟总时长（模拟小时数） # Total simulated duration (simulated hours).
total_simulation_hours: int = 72 # 默认模拟72小时（3天） total_simulation_hours: int = 72 # Default: 72 simulated hours (3 days).
# 每轮代表的时间（模拟分钟）- 默认60分钟（1小时），加快时间流速 # Time represented by each round (simulated minutes); default 60 (1 hour) to speed up the simulated clock.
minutes_per_round: int = 60 minutes_per_round: int = 60
# 每小时激活的Agent数量范围 # Range of agents activated per hour.
agents_per_hour_min: int = 5 agents_per_hour_min: int = 5
agents_per_hour_max: int = 20 agents_per_hour_max: int = 20
# 高峰时段（晚间19-22点，中国人最活跃的时间） # Peak hours (evenings 19:00-22:00, most active for the modelled audience).
peak_hours: List[int] = field(default_factory=lambda: [19, 20, 21, 22]) peak_hours: List[int] = field(default_factory=lambda: [19, 20, 21, 22])
peak_activity_multiplier: float = 1.5 peak_activity_multiplier: float = 1.5
# 低谷时段（凌晨0-5点，几乎无人活动） # Off-peak hours (00:00-05:00, almost no activity).
off_peak_hours: List[int] = field(default_factory=lambda: [0, 1, 2, 3, 4, 5]) off_peak_hours: List[int] = field(default_factory=lambda: [0, 1, 2, 3, 4, 5])
off_peak_activity_multiplier: float = 0.05 # 凌晨活跃度极低 off_peak_activity_multiplier: float = 0.05 # Overnight activity is very low.
# 早间时段 # Morning hours.
morning_hours: List[int] = field(default_factory=lambda: [6, 7, 8]) morning_hours: List[int] = field(default_factory=lambda: [6, 7, 8])
morning_activity_multiplier: float = 0.4 morning_activity_multiplier: float = 0.4
# 工作时段 # Working hours.
work_hours: List[int] = field(default_factory=lambda: [9, 10, 11, 12, 13, 14, 15, 16, 17, 18]) work_hours: List[int] = field(default_factory=lambda: [9, 10, 11, 12, 13, 14, 15, 16, 17, 18])
work_activity_multiplier: float = 0.7 work_activity_multiplier: float = 0.7
@dataclass @dataclass
class EventConfig: class EventConfig:
"""事件配置""" """Event configuration."""
# 初始事件（模拟开始时的触发事件） # Initial events: triggers fired when the simulation begins.
initial_posts: List[Dict[str, Any]] = field(default_factory=list) initial_posts: List[Dict[str, Any]] = field(default_factory=list)
# 定时事件（在特定时间触发的事件） # Scheduled events: events fired at specific times.
scheduled_events: List[Dict[str, Any]] = field(default_factory=list) scheduled_events: List[Dict[str, Any]] = field(default_factory=list)
# 热点话题关键词 # Hot-topic keywords.
hot_topics: List[str] = field(default_factory=list) hot_topics: List[str] = field(default_factory=list)
# 舆论引导方向 # Narrative direction for public-opinion guidance.
narrative_direction: str = "" narrative_direction: str = ""
@dataclass @dataclass
class PlatformConfig: class PlatformConfig:
"""平台特定配置""" """Platform-specific configuration."""
platform: str # twitter or reddit platform: str # twitter or reddit
# 推荐算法权重 # Recommendation-algorithm weights.
recency_weight: float = 0.4 # 时间新鲜度 recency_weight: float = 0.4 # Recency.
popularity_weight: float = 0.3 # 热度 popularity_weight: float = 0.3 # Popularity.
relevance_weight: float = 0.3 # 相关性 relevance_weight: float = 0.3 # Relevance.
# 病毒传播阈值（达到多少互动后触发扩散） # Viral-spread threshold: number of interactions required to trigger spreading.
viral_threshold: int = 10 viral_threshold: int = 10
# 回声室效应强度（相似观点聚集程度） # Echo-chamber strength: how strongly similar viewpoints cluster together.
echo_chamber_strength: float = 0.5 echo_chamber_strength: float = 0.5
@dataclass @dataclass
class SimulationParameters: class SimulationParameters:
"""完整的模拟参数配置""" """Complete simulation-parameter configuration."""
# 基础信息 # Basic identifiers.
simulation_id: str simulation_id: str
project_id: str project_id: str
graph_id: str graph_id: str
simulation_requirement: str simulation_requirement: str
# 时间配置 # Time configuration.
time_config: TimeSimulationConfig = field(default_factory=TimeSimulationConfig) time_config: TimeSimulationConfig = field(default_factory=TimeSimulationConfig)
# Agent配置列表 # Agent configuration list.
agent_configs: List[AgentActivityConfig] = field(default_factory=list) agent_configs: List[AgentActivityConfig] = field(default_factory=list)
# 事件配置 # Event configuration.
event_config: EventConfig = field(default_factory=EventConfig) event_config: EventConfig = field(default_factory=EventConfig)
# 平台配置 # Platform configurations.
twitter_config: Optional[PlatformConfig] = None twitter_config: Optional[PlatformConfig] = None
reddit_config: Optional[PlatformConfig] = None reddit_config: Optional[PlatformConfig] = None
# LLM配置 # LLM configuration.
llm_model: str = "" llm_model: str = ""
llm_base_url: str = "" llm_base_url: str = ""
# 生成元数据 # Generation metadata.
generated_at: str = field(default_factory=lambda: datetime.now().isoformat()) generated_at: str = field(default_factory=lambda: datetime.now().isoformat())
generation_reasoning: str = "" # LLM的推理说明 generation_reasoning: str = "" # LLM-provided rationale.
def to_dict(self) -> Dict[str, Any]: def to_dict(self) -> Dict[str, Any]:
"""转换为字典""" """Return the parameters as a dictionary."""
time_dict = asdict(self.time_config) time_dict = asdict(self.time_config)
return { return {
"simulation_id": self.simulation_id, "simulation_id": self.simulation_id,
@ -193,34 +196,35 @@ class SimulationParameters:
} }
def to_json(self, indent: int = 2) -> str: def to_json(self, indent: int = 2) -> str:
"""转换为JSON字符串""" """Return the parameters as a JSON string."""
return json.dumps(self.to_dict(), ensure_ascii=False, indent=indent) return json.dumps(self.to_dict(), ensure_ascii=False, indent=indent)
class SimulationConfigGenerator: class SimulationConfigGenerator:
""" """
模拟配置智能生成器 Intelligent simulation-configuration generator.
使用LLM分析模拟需求、文档内容、图谱实体信息， Uses an LLM to analyse the simulation requirement, document content,
自动生成最佳的模拟参数配置 and graph entity information to automatically derive the best
simulation parameter configuration.
采用分步生成策略： Step-wise generation strategy:
1. 生成时间配置和事件配置（轻量级） 1. Generate time and event configurations (lightweight).
2. 分批生成Agent配置（每批10-20个） 2. Generate agent configurations in batches (10-20 per batch).
3. 生成平台配置 3. Generate platform configuration.
""" """
# 上下文最大字符数 # Maximum context length (characters).
MAX_CONTEXT_LENGTH = 50000 MAX_CONTEXT_LENGTH = 50000
# 每批生成的Agent数量 # Number of agents generated per batch.
AGENTS_PER_BATCH = 15 AGENTS_PER_BATCH = 15
# 各步骤的上下文截断长度（字符数） # Per-step context truncation lengths (characters).
TIME_CONFIG_CONTEXT_LENGTH = 10000 # 时间配置 TIME_CONFIG_CONTEXT_LENGTH = 10000 # Time configuration.
EVENT_CONFIG_CONTEXT_LENGTH = 8000 # 事件配置 EVENT_CONFIG_CONTEXT_LENGTH = 8000 # Event configuration.
ENTITY_SUMMARY_LENGTH = 300 # 实体摘要 ENTITY_SUMMARY_LENGTH = 300 # Entity summary.
AGENT_SUMMARY_LENGTH = 300 # Agent配置中的实体摘要 AGENT_SUMMARY_LENGTH = 300 # Entity summary used in agent configs.
ENTITIES_PER_TYPE_DISPLAY = 20 # 每类实体显示数量 ENTITIES_PER_TYPE_DISPLAY = 20 # Number of entities displayed per type.
def __init__( def __init__(
self, self,
@ -252,28 +256,27 @@ class SimulationConfigGenerator:
enable_reddit: bool = True, enable_reddit: bool = True,
progress_callback: Optional[Callable[[int, int, str], None]] = None, progress_callback: Optional[Callable[[int, int, str], None]] = None,
) -> SimulationParameters: ) -> SimulationParameters:
""" """Intelligently generate a complete simulation configuration (step-wise).
智能生成完整的模拟配置（分步生成）
Args: Args:
simulation_id: 模拟ID simulation_id: Simulation ID.
project_id: 项目ID project_id: Project ID.
graph_id: 图谱ID graph_id: Graph ID.
simulation_requirement: 模拟需求描述 simulation_requirement: Description of the simulation requirement.
document_text: 原始文档内容 document_text: Original document content.
entities: 过滤后的实体列表 entities: Filtered list of entities.
enable_twitter: 是否启用Twitter enable_twitter: Whether to enable Twitter.
enable_reddit: 是否启用Reddit enable_reddit: Whether to enable Reddit.
progress_callback: 进度回调函数(current_step, total_steps, message) progress_callback: Progress callback (current_step, total_steps, message).
Returns: Returns:
SimulationParameters: 完整的模拟参数 SimulationParameters: The complete simulation parameters.
""" """
logger.info(t("log.simulation_config.m001", simulation_id=simulation_id, len=len(entities))) logger.info(t("log.simulation_config.m001", simulation_id=simulation_id, len=len(entities)))
# 计算总步骤数 # Compute total step count.
num_batches = math.ceil(len(entities) / self.AGENTS_PER_BATCH) num_batches = math.ceil(len(entities) / self.AGENTS_PER_BATCH)
total_steps = 3 + num_batches # 时间配置 + 事件配置 + N批Agent + 平台配置 total_steps = 3 + num_batches # Time config + event config + N agent batches + platform config.
current_step = 0 current_step = 0
def report_progress(step: int, message: str): def report_progress(step: int, message: str):
@ -283,7 +286,7 @@ class SimulationConfigGenerator:
progress_callback(step, total_steps, message) progress_callback(step, total_steps, message)
logger.info(f"[{step}/{total_steps}] {message}") logger.info(f"[{step}/{total_steps}] {message}")
# 1. 构建基础上下文信息 # 1. Build base context information.
context = self._build_context( context = self._build_context(
simulation_requirement=simulation_requirement, simulation_requirement=simulation_requirement,
document_text=document_text, document_text=document_text,
@ -292,20 +295,20 @@ class SimulationConfigGenerator:
reasoning_parts = [] reasoning_parts = []
# ========== 步骤1: 生成时间配置 ========== # ========== Step 1: generate time configuration ==========
report_progress(1, t('progress.generatingTimeConfig')) report_progress(1, t('progress.generatingTimeConfig'))
num_entities = len(entities) num_entities = len(entities)
time_config_result = self._generate_time_config(context, num_entities) time_config_result = self._generate_time_config(context, num_entities)
time_config = self._parse_time_config(time_config_result, num_entities) time_config = self._parse_time_config(time_config_result, num_entities)
reasoning_parts.append(f"{t('progress.timeConfigLabel')}: {time_config_result.get('reasoning', t('common.success'))}") reasoning_parts.append(f"{t('progress.timeConfigLabel')}: {time_config_result.get('reasoning', t('common.success'))}")
# ========== 步骤2: 生成事件配置 ========== # ========== Step 2: generate event configuration ==========
report_progress(2, t('progress.generatingEventConfig')) report_progress(2, t('progress.generatingEventConfig'))
event_config_result = self._generate_event_config(context, simulation_requirement, entities) event_config_result = self._generate_event_config(context, simulation_requirement, entities)
event_config = self._parse_event_config(event_config_result) event_config = self._parse_event_config(event_config_result)
reasoning_parts.append(f"{t('progress.eventConfigLabel')}: {event_config_result.get('reasoning', t('common.success'))}") reasoning_parts.append(f"{t('progress.eventConfigLabel')}: {event_config_result.get('reasoning', t('common.success'))}")
# ========== 步骤3-N: 分批生成Agent配置 ========== # ========== Steps 3-N: generate agent configurations in batches ==========
all_agent_configs = [] all_agent_configs = []
for batch_idx in range(num_batches): for batch_idx in range(num_batches):
start_idx = batch_idx * self.AGENTS_PER_BATCH start_idx = batch_idx * self.AGENTS_PER_BATCH
@ -327,13 +330,13 @@ class SimulationConfigGenerator:
reasoning_parts.append(t('progress.agentConfigResult', count=len(all_agent_configs))) reasoning_parts.append(t('progress.agentConfigResult', count=len(all_agent_configs)))
# ========== 为初始帖子分配发布者 Agent ========== # ========== Assign poster agents to initial posts ==========
logger.info(t("log.simulation_config.m002")) logger.info(t("log.simulation_config.m002"))
event_config = self._assign_initial_post_agents(event_config, all_agent_configs) event_config = self._assign_initial_post_agents(event_config, all_agent_configs)
assigned_count = len([p for p in event_config.initial_posts if p.get("poster_agent_id") is not None]) assigned_count = len([p for p in event_config.initial_posts if p.get("poster_agent_id") is not None])
reasoning_parts.append(t('progress.postAssignResult', count=assigned_count)) reasoning_parts.append(t('progress.postAssignResult', count=assigned_count))
# ========== 最后一步: 生成平台配置 ========== # ========== Final step: generate platform configuration ==========
report_progress(total_steps, t('progress.generatingPlatformConfig')) report_progress(total_steps, t('progress.generatingPlatformConfig'))
twitter_config = None twitter_config = None
reddit_config = None reddit_config = None
@ -358,7 +361,7 @@ class SimulationConfigGenerator:
echo_chamber_strength=0.6 echo_chamber_strength=0.6
) )
# 构建最终参数 # Build final parameters.
params = SimulationParameters( params = SimulationParameters(
simulation_id=simulation_id, simulation_id=simulation_id,
project_id=project_id, project_id=project_id,
@ -384,19 +387,19 @@ class SimulationConfigGenerator:
document_text: str, document_text: str,
entities: List[EntityNode] entities: List[EntityNode]
) -> str: ) -> str:
"""构建LLM上下文，截断到最大长度""" """Build the LLM context, truncated to the maximum length."""
# 实体摘要 # Entity summary.
entity_summary = self._summarize_entities(entities) entity_summary = self._summarize_entities(entities)
# 构建上下文 # Build the context.
context_parts = [ context_parts = [
f"## Simulation Requirement\n{simulation_requirement}", f"## Simulation Requirement\n{simulation_requirement}",
f"\n## Entities ({len(entities)})\n{entity_summary}", f"\n## Entities ({len(entities)})\n{entity_summary}",
] ]
current_length = sum(len(p) for p in context_parts) current_length = sum(len(p) for p in context_parts)
remaining_length = self.MAX_CONTEXT_LENGTH - current_length - 500 # 留500字符余量 remaining_length = self.MAX_CONTEXT_LENGTH - current_length - 500 # Reserve 500-char headroom.
if remaining_length > 0 and document_text: if remaining_length > 0 and document_text:
doc_text = document_text[:remaining_length] doc_text = document_text[:remaining_length]
@ -407,10 +410,10 @@ class SimulationConfigGenerator:
return "\n".join(context_parts) return "\n".join(context_parts)
def _summarize_entities(self, entities: List[EntityNode]) -> str: def _summarize_entities(self, entities: List[EntityNode]) -> str:
"""生成实体摘要""" """Generate an entity summary."""
lines = [] lines = []
# 按类型分组 # Group by type.
by_type: Dict[str, List[EntityNode]] = {} by_type: Dict[str, List[EntityNode]] = {}
for e in entities: for e in entities:
t = e.get_entity_type() or "Unknown" t = e.get_entity_type() or "Unknown"
@ -420,7 +423,7 @@ class SimulationConfigGenerator:
for entity_type, type_entities in by_type.items(): for entity_type, type_entities in by_type.items():
lines.append(f"\n### {entity_type} ({len(type_entities)})") lines.append(f"\n### {entity_type} ({len(type_entities)})")
# 使用配置的显示数量和摘要长度 # Use configured display count and summary length.
display_count = self.ENTITIES_PER_TYPE_DISPLAY display_count = self.ENTITIES_PER_TYPE_DISPLAY
summary_len = self.ENTITY_SUMMARY_LENGTH summary_len = self.ENTITY_SUMMARY_LENGTH
for e in type_entities[:display_count]: for e in type_entities[:display_count]:
@ -432,7 +435,7 @@ class SimulationConfigGenerator:
return "\n".join(lines) return "\n".join(lines)
def _call_llm_with_retry(self, prompt: str, system_prompt: str) -> Dict[str, Any]: def _call_llm_with_retry(self, prompt: str, system_prompt: str) -> Dict[str, Any]:
"""带重试的LLM调用，包含JSON修复逻辑""" """LLM call with retries, including JSON repair logic."""
import re import re
max_attempts = 3 max_attempts = 3
@ -447,25 +450,25 @@ class SimulationConfigGenerator:
{"role": "user", "content": prompt} {"role": "user", "content": prompt}
], ],
response_format={"type": "json_object"}, response_format={"type": "json_object"},
temperature=0.7 - (attempt * 0.1) # 每次重试降低温度 temperature=0.7 - (attempt * 0.1) # Lower temperature on each retry.
# 不设置max_tokens，让LLM自由发挥 # max_tokens is intentionally unset so the LLM can use its full budget.
) )
content = response.choices[0].message.content content = response.choices[0].message.content
finish_reason = response.choices[0].finish_reason finish_reason = response.choices[0].finish_reason
# 检查是否被截断 # Detect truncation.
if finish_reason == 'length': if finish_reason == 'length':
logger.warning(t("log.simulation_config.m004", attempt=attempt + 1)) logger.warning(t("log.simulation_config.m004", attempt=attempt + 1))
content = self._fix_truncated_json(content) content = self._fix_truncated_json(content)
# 尝试解析JSON # Attempt to parse JSON.
try: try:
return json.loads(content) return json.loads(content)
except json.JSONDecodeError as e: except json.JSONDecodeError as e:
logger.warning(t("log.simulation_config.m005", attempt=attempt + 1, str=str(e)[:80])) logger.warning(t("log.simulation_config.m005", attempt=attempt + 1, str=str(e)[:80]))
# 尝试修复JSON # Attempt to repair the JSON.
fixed = self._try_fix_config_json(content) fixed = self._try_fix_config_json(content)
if fixed: if fixed:
return fixed return fixed
@ -481,36 +484,36 @@ class SimulationConfigGenerator:
raise last_error or Exception("LLM调用失败") raise last_error or Exception("LLM调用失败")
def _fix_truncated_json(self, content: str) -> str: def _fix_truncated_json(self, content: str) -> str:
"""修复被截断的JSON""" """Repair truncated JSON."""
content = content.strip() content = content.strip()
# 计算未闭合的括号 # Count unclosed brackets.
open_braces = content.count('{') - content.count('}') open_braces = content.count('{') - content.count('}')
open_brackets = content.count('[') - content.count(']') open_brackets = content.count('[') - content.count(']')
# 检查是否有未闭合的字符串 # Check for an unclosed string.
if content and content[-1] not in '",}]': if content and content[-1] not in '",}]':
content += '"' content += '"'
# 闭合括号 # Close brackets.
content += ']' * open_brackets content += ']' * open_brackets
content += '}' * open_braces content += '}' * open_braces
return content return content
def _try_fix_config_json(self, content: str) -> Optional[Dict[str, Any]]: def _try_fix_config_json(self, content: str) -> Optional[Dict[str, Any]]:
"""尝试修复配置JSON""" """Attempt to repair a configuration JSON payload."""
import re import re
# 修复被截断的情况 # Repair truncation first.
content = self._fix_truncated_json(content) content = self._fix_truncated_json(content)
# 提取JSON部分 # Extract the JSON portion.
json_match = re.search(r'\{[\s\S]*\}', content) json_match = re.search(r'\{[\s\S]*\}', content)
if json_match: if json_match:
json_str = json_match.group() json_str = json_match.group()
# 移除字符串中的换行符 # Remove line breaks from inside strings.
def fix_string(match): def fix_string(match):
s = match.group(0) s = match.group(0)
s = s.replace('\n', ' ').replace('\r', ' ') s = s.replace('\n', ' ').replace('\r', ' ')
@ -522,7 +525,7 @@ class SimulationConfigGenerator:
try: try:
return json.loads(json_str) return json.loads(json_str)
except: except:
# 尝试移除所有控制字符 # Strip all control characters and try again.
json_str = re.sub(r'[\x00-\x1f\x7f-\x9f]', ' ', json_str) json_str = re.sub(r'[\x00-\x1f\x7f-\x9f]', ' ', json_str)
json_str = re.sub(r'\s+', ' ', json_str) json_str = re.sub(r'\s+', ' ', json_str)
try: try:
@ -533,11 +536,11 @@ class SimulationConfigGenerator:
return None return None
def _generate_time_config(self, context: str, num_entities: int) -> Dict[str, Any]: def _generate_time_config(self, context: str, num_entities: int) -> Dict[str, Any]:
"""生成时间配置""" """Generate the time configuration."""
# 使用配置的上下文截断长度 # Use the configured context truncation length.
context_truncated = context[:self.TIME_CONFIG_CONTEXT_LENGTH] context_truncated = context[:self.TIME_CONFIG_CONTEXT_LENGTH]
# 计算最大允许值（80%的agent数） # Compute the upper bound (90% of the agent count).
max_agents_allowed = max(1, int(num_entities * 0.9)) max_agents_allowed = max(1, int(num_entities * 0.9))
prompt = f"""Based on the simulation requirement below, generate a time-simulation configuration. prompt = f"""Based on the simulation requirement below, generate a time-simulation configuration.
@ -595,10 +598,10 @@ Field guide:
return self._get_default_time_config(num_entities) return self._get_default_time_config(num_entities)
def _get_default_time_config(self, num_entities: int) -> Dict[str, Any]: def _get_default_time_config(self, num_entities: int) -> Dict[str, Any]:
"""获取默认时间配置（中国人作息）""" """Return the default time configuration (Chinese daily rhythm)."""
return { return {
"total_simulation_hours": 72, "total_simulation_hours": 72,
"minutes_per_round": 60, # 每轮1小时，加快时间流速 "minutes_per_round": 60, # 1 hour per round to speed up the simulated clock.
"agents_per_hour_min": max(1, num_entities // 15), "agents_per_hour_min": max(1, num_entities // 15),
"agents_per_hour_max": max(5, num_entities // 5), "agents_per_hour_max": max(5, num_entities // 5),
"peak_hours": [19, 20, 21, 22], "peak_hours": [19, 20, 21, 22],
@ -609,12 +612,12 @@ Field guide:
} }
def _parse_time_config(self, result: Dict[str, Any], num_entities: int) -> TimeSimulationConfig: def _parse_time_config(self, result: Dict[str, Any], num_entities: int) -> TimeSimulationConfig:
"""解析时间配置结果，并验证agents_per_hour值不超过总agent数""" """Parse the time-configuration result and ensure agents_per_hour values do not exceed the total agent count."""
# 获取原始值 # Pull raw values.
agents_per_hour_min = result.get("agents_per_hour_min", max(1, num_entities // 15)) agents_per_hour_min = result.get("agents_per_hour_min", max(1, num_entities // 15))
agents_per_hour_max = result.get("agents_per_hour_max", max(5, num_entities // 5)) agents_per_hour_max = result.get("agents_per_hour_max", max(5, num_entities // 5))
# 验证并修正：确保不超过总agent数 # Validate and correct: ensure values do not exceed the total agent count.
if agents_per_hour_min > num_entities: if agents_per_hour_min > num_entities:
logger.warning(t("log.simulation_config.m008", agents_per_hour_min=agents_per_hour_min, num_entities=num_entities)) logger.warning(t("log.simulation_config.m008", agents_per_hour_min=agents_per_hour_min, num_entities=num_entities))
agents_per_hour_min = max(1, num_entities // 10) agents_per_hour_min = max(1, num_entities // 10)
@ -623,19 +626,19 @@ Field guide:
logger.warning(t("log.simulation_config.m009", agents_per_hour_max=agents_per_hour_max, num_entities=num_entities)) logger.warning(t("log.simulation_config.m009", agents_per_hour_max=agents_per_hour_max, num_entities=num_entities))
agents_per_hour_max = max(agents_per_hour_min + 1, num_entities // 2) agents_per_hour_max = max(agents_per_hour_min + 1, num_entities // 2)
# 确保 min < max # Ensure min < max.
if agents_per_hour_min >= agents_per_hour_max: if agents_per_hour_min >= agents_per_hour_max:
agents_per_hour_min = max(1, agents_per_hour_max // 2) agents_per_hour_min = max(1, agents_per_hour_max // 2)
logger.warning(t("log.simulation_config.m010", agents_per_hour_min=agents_per_hour_min)) logger.warning(t("log.simulation_config.m010", agents_per_hour_min=agents_per_hour_min))
return TimeSimulationConfig( return TimeSimulationConfig(
total_simulation_hours=result.get("total_simulation_hours", 72), total_simulation_hours=result.get("total_simulation_hours", 72),
minutes_per_round=result.get("minutes_per_round", 60), # 默认每轮1小时 minutes_per_round=result.get("minutes_per_round", 60), # Default: 1 simulated hour per round.
agents_per_hour_min=agents_per_hour_min, agents_per_hour_min=agents_per_hour_min,
agents_per_hour_max=agents_per_hour_max, agents_per_hour_max=agents_per_hour_max,
peak_hours=result.get("peak_hours", [19, 20, 21, 22]), peak_hours=result.get("peak_hours", [19, 20, 21, 22]),
off_peak_hours=result.get("off_peak_hours", [0, 1, 2, 3, 4, 5]), off_peak_hours=result.get("off_peak_hours", [0, 1, 2, 3, 4, 5]),
off_peak_activity_multiplier=0.05, # 凌晨几乎无人 off_peak_activity_multiplier=0.05, # Overnight: almost no one online.
morning_hours=result.get("morning_hours", [6, 7, 8]), morning_hours=result.get("morning_hours", [6, 7, 8]),
morning_activity_multiplier=0.4, morning_activity_multiplier=0.4,
work_hours=result.get("work_hours", list(range(9, 19))), work_hours=result.get("work_hours", list(range(9, 19))),
@ -649,14 +652,14 @@ Field guide:
simulation_requirement: str, simulation_requirement: str,
entities: List[EntityNode] entities: List[EntityNode]
) -> Dict[str, Any]: ) -> Dict[str, Any]:
"""生成事件配置""" """Generate the event configuration."""
# 获取可用的实体类型列表，供 LLM 参考 # Build the list of available entity types for the LLM to reference.
entity_types_available = list(set( entity_types_available = list(set(
e.get_entity_type() or "Unknown" for e in entities e.get_entity_type() or "Unknown" for e in entities
)) ))
# 为每种类型列出代表性实体名称 # Collect representative entity names per type.
type_examples = {} type_examples = {}
for e in entities: for e in entities:
etype = e.get_entity_type() or "Unknown" etype = e.get_entity_type() or "Unknown"
@ -670,7 +673,7 @@ Field guide:
for t, examples in type_examples.items() for t, examples in type_examples.items()
]) ])
# 使用配置的上下文截断长度 # Use the configured context truncation length.
context_truncated = context[:self.EVENT_CONFIG_CONTEXT_LENGTH] context_truncated = context[:self.EVENT_CONFIG_CONTEXT_LENGTH]
prompt = f"""Based on the simulation requirement below, generate an event configuration. prompt = f"""Based on the simulation requirement below, generate an event configuration.
@ -717,7 +720,7 @@ Return strict JSON (no markdown):
} }
def _parse_event_config(self, result: Dict[str, Any]) -> EventConfig: def _parse_event_config(self, result: Dict[str, Any]) -> EventConfig:
"""解析事件配置结果""" """Parse the event-configuration result."""
return EventConfig( return EventConfig(
initial_posts=result.get("initial_posts", []), initial_posts=result.get("initial_posts", []),
scheduled_events=[], scheduled_events=[],
@ -730,15 +733,15 @@ Return strict JSON (no markdown):
event_config: EventConfig, event_config: EventConfig,
agent_configs: List[AgentActivityConfig] agent_configs: List[AgentActivityConfig]
) -> EventConfig: ) -> EventConfig:
""" """Assign a suitable poster agent to each initial post.
为初始帖子分配合适的发布者 Agent
根据每个帖子的 poster_type 匹配最合适的 agent_id Matches the most appropriate agent_id for each post based on its
poster_type.
""" """
if not event_config.initial_posts: if not event_config.initial_posts:
return event_config return event_config
# 按实体类型建立 agent 索引 # Build an agent index keyed by entity type.
agents_by_type: Dict[str, List[AgentActivityConfig]] = {} agents_by_type: Dict[str, List[AgentActivityConfig]] = {}
for agent in agent_configs: for agent in agent_configs:
etype = agent.entity_type.lower() etype = agent.entity_type.lower()
@ -746,7 +749,7 @@ Return strict JSON (no markdown):
agents_by_type[etype] = [] agents_by_type[etype] = []
agents_by_type[etype].append(agent) agents_by_type[etype].append(agent)
# 类型映射表（处理 LLM 可能输出的不同格式） # Type alias map (handles the different formats the LLM might emit).
type_aliases = { type_aliases = {
"official": ["official", "university", "governmentagency", "government"], "official": ["official", "university", "governmentagency", "government"],
"university": ["university", "official"], "university": ["university", "official"],
@ -758,7 +761,7 @@ Return strict JSON (no markdown):
"person": ["person", "student", "alumni"], "person": ["person", "student", "alumni"],
} }
# 记录每种类型已使用的 agent 索引，避免重复使用同一个 agent # Track the next agent index used per type to avoid reusing the same agent twice.
used_indices: Dict[str, int] = {} used_indices: Dict[str, int] = {}
updated_posts = [] updated_posts = []
@ -766,17 +769,17 @@ Return strict JSON (no markdown):
poster_type = post.get("poster_type", "").lower() poster_type = post.get("poster_type", "").lower()
content = post.get("content", "") content = post.get("content", "")
# 尝试找到匹配的 agent # Try to find a matching agent.
matched_agent_id = None matched_agent_id = None
# 1. 直接匹配 # 1. Direct match.
if poster_type in agents_by_type: if poster_type in agents_by_type:
agents = agents_by_type[poster_type] agents = agents_by_type[poster_type]
idx = used_indices.get(poster_type, 0) % len(agents) idx = used_indices.get(poster_type, 0) % len(agents)
matched_agent_id = agents[idx].agent_id matched_agent_id = agents[idx].agent_id
used_indices[poster_type] = idx + 1 used_indices[poster_type] = idx + 1
else: else:
# 2. 使用别名匹配 # 2. Match via aliases.
for alias_key, aliases in type_aliases.items(): for alias_key, aliases in type_aliases.items():
if poster_type in aliases or alias_key == poster_type: if poster_type in aliases or alias_key == poster_type:
for alias in aliases: for alias in aliases:
@ -789,11 +792,11 @@ Return strict JSON (no markdown):
if matched_agent_id is not None: if matched_agent_id is not None:
break break
# 3. 如果仍未找到，使用影响力最高的 agent # 3. If still unresolved, fall back to the most influential agent.
if matched_agent_id is None: if matched_agent_id is None:
logger.warning(t("log.simulation_config.m012", poster_type=poster_type)) logger.warning(t("log.simulation_config.m012", poster_type=poster_type))
if agent_configs: if agent_configs:
# 按影响力排序，选择影响力最高的 # Sort by influence and pick the highest.
sorted_agents = sorted(agent_configs, key=lambda a: a.influence_weight, reverse=True) sorted_agents = sorted(agent_configs, key=lambda a: a.influence_weight, reverse=True)
matched_agent_id = sorted_agents[0].agent_id matched_agent_id = sorted_agents[0].agent_id
else: else:
@ -817,9 +820,9 @@ Return strict JSON (no markdown):
start_idx: int, start_idx: int,
simulation_requirement: str simulation_requirement: str
) -> List[AgentActivityConfig]: ) -> List[AgentActivityConfig]:
"""分批生成Agent配置""" """Generate agent configurations in batches."""
# 构建实体信息（使用配置的摘要长度） # Build entity information (using the configured summary length).
entity_list = [] entity_list = []
summary_len = self.AGENT_SUMMARY_LENGTH summary_len = self.AGENT_SUMMARY_LENGTH
for i, e in enumerate(entities): for i, e in enumerate(entities):
@ -876,13 +879,13 @@ Return strict JSON (no markdown):
logger.warning(t("log.simulation_config.m014", e=e)) logger.warning(t("log.simulation_config.m014", e=e))
llm_configs = {} llm_configs = {}
# 构建AgentActivityConfig对象 # Build AgentActivityConfig objects.
configs = [] configs = []
for i, entity in enumerate(entities): for i, entity in enumerate(entities):
agent_id = start_idx + i agent_id = start_idx + i
cfg = llm_configs.get(agent_id, {}) cfg = llm_configs.get(agent_id, {})
# 如果LLM没有生成，使用规则生成 # If the LLM did not produce a config, fall back to rule-based generation.
if not cfg: if not cfg:
cfg = self._generate_agent_config_by_rule(entity) cfg = self._generate_agent_config_by_rule(entity)
@ -906,16 +909,16 @@ Return strict JSON (no markdown):
return configs return configs
def _generate_agent_config_by_rule(self, entity: EntityNode) -> Dict[str, Any]: def _generate_agent_config_by_rule(self, entity: EntityNode) -> Dict[str, Any]:
"""基于规则生成单个Agent配置（中国人作息）""" """Rule-based generation for a single agent's configuration (Chinese daily rhythm)."""
entity_type = (entity.get_entity_type() or "Unknown").lower() entity_type = (entity.get_entity_type() or "Unknown").lower()
if entity_type in ["university", "governmentagency", "ngo"]: if entity_type in ["university", "governmentagency", "ngo"]:
# 官方机构：工作时间活动，低频率，高影响力 # Official institutions: active during working hours, low frequency, high influence.
return { return {
"activity_level": 0.2, "activity_level": 0.2,
"posts_per_hour": 0.1, "posts_per_hour": 0.1,
"comments_per_hour": 0.05, "comments_per_hour": 0.05,
"active_hours": list(range(9, 18)), # 9:00-17:59 "active_hours": list(range(9, 18)), # 09:00-17:59
"response_delay_min": 60, "response_delay_min": 60,
"response_delay_max": 240, "response_delay_max": 240,
"sentiment_bias": 0.0, "sentiment_bias": 0.0,
@ -923,12 +926,12 @@ Return strict JSON (no markdown):
"influence_weight": 3.0 "influence_weight": 3.0
} }
elif entity_type in ["mediaoutlet"]: elif entity_type in ["mediaoutlet"]:
# 媒体：全天活动，中等频率，高影响力 # Media: active throughout the day, medium frequency, high influence.
return { return {
"activity_level": 0.5, "activity_level": 0.5,
"posts_per_hour": 0.8, "posts_per_hour": 0.8,
"comments_per_hour": 0.3, "comments_per_hour": 0.3,
"active_hours": list(range(7, 24)), # 7:00-23:59 "active_hours": list(range(7, 24)), # 07:00-23:59
"response_delay_min": 5, "response_delay_min": 5,
"response_delay_max": 30, "response_delay_max": 30,
"sentiment_bias": 0.0, "sentiment_bias": 0.0,
@ -936,12 +939,12 @@ Return strict JSON (no markdown):
"influence_weight": 2.5 "influence_weight": 2.5
} }
elif entity_type in ["professor", "expert", "official"]: elif entity_type in ["professor", "expert", "official"]:
# 专家/教授：工作+晚间活动，中等频率 # Experts / professors: active during work and evening, medium frequency.
return { return {
"activity_level": 0.4, "activity_level": 0.4,
"posts_per_hour": 0.3, "posts_per_hour": 0.3,
"comments_per_hour": 0.5, "comments_per_hour": 0.5,
"active_hours": list(range(8, 22)), # 8:00-21:59 "active_hours": list(range(8, 22)), # 08:00-21:59
"response_delay_min": 15, "response_delay_min": 15,
"response_delay_max": 90, "response_delay_max": 90,
"sentiment_bias": 0.0, "sentiment_bias": 0.0,
@ -949,12 +952,12 @@ Return strict JSON (no markdown):
"influence_weight": 2.0 "influence_weight": 2.0
} }
elif entity_type in ["student"]: elif entity_type in ["student"]:
# 学生：晚间为主，高频率 # Students: mostly evening, high frequency.
return { return {
"activity_level": 0.8, "activity_level": 0.8,
"posts_per_hour": 0.6, "posts_per_hour": 0.6,
"comments_per_hour": 1.5, "comments_per_hour": 1.5,
"active_hours": [8, 9, 10, 11, 12, 13, 18, 19, 20, 21, 22, 23], # 上午+晚间 "active_hours": [8, 9, 10, 11, 12, 13, 18, 19, 20, 21, 22, 23], # Morning + evening.
"response_delay_min": 1, "response_delay_min": 1,
"response_delay_max": 15, "response_delay_max": 15,
"sentiment_bias": 0.0, "sentiment_bias": 0.0,
@ -962,12 +965,12 @@ Return strict JSON (no markdown):
"influence_weight": 0.8 "influence_weight": 0.8
} }
elif entity_type in ["alumni"]: elif entity_type in ["alumni"]:
# 校友：晚间为主 # Alumni: mostly evening.
return { return {
"activity_level": 0.6, "activity_level": 0.6,
"posts_per_hour": 0.4, "posts_per_hour": 0.4,
"comments_per_hour": 0.8, "comments_per_hour": 0.8,
"active_hours": [12, 13, 19, 20, 21, 22, 23], # 午休+晚间 "active_hours": [12, 13, 19, 20, 21, 22, 23], # Lunch break + evening.
"response_delay_min": 5, "response_delay_min": 5,
"response_delay_max": 30, "response_delay_max": 30,
"sentiment_bias": 0.0, "sentiment_bias": 0.0,
@ -975,12 +978,12 @@ Return strict JSON (no markdown):
"influence_weight": 1.0 "influence_weight": 1.0
} }
else: else:
# 普通人：晚间高峰 # General public: evening peak.
return { return {
"activity_level": 0.7, "activity_level": 0.7,
"posts_per_hour": 0.5, "posts_per_hour": 0.5,
"comments_per_hour": 1.2, "comments_per_hour": 1.2,
"active_hours": [9, 10, 11, 12, 13, 18, 19, 20, 21, 22, 23], # 白天+晚间 "active_hours": [9, 10, 11, 12, 13, 18, 19, 20, 21, 22, 23], # Daytime + evening.
"response_delay_min": 2, "response_delay_min": 2,
"response_delay_max": 20, "response_delay_max": 20,
"sentiment_bias": 0.0, "sentiment_bias": 0.0,

152
backend/app/services/simulation_ipc.py
View File

@ -1,11 +1,12 @@
""" """Simulation IPC module.
模拟IPC通信模块
用于Flask后端和模拟脚本之间的进程间通信
通过文件系统实现简单的命令/响应模式： Inter-process communication between the Flask backend and the simulation
1. Flask写入命令到 commands/ 目录 subprocess. Implements a simple file-system command/response pattern:
2. 模拟脚本轮询命令目录，执行命令并写入响应到 responses/ 目录
3. Flask轮询响应目录获取结果 1. Flask writes commands into ``commands/``.
2. The simulation script polls for commands, executes them, and writes
responses into ``responses/``.
3. Flask polls the responses directory for results.
""" """
import os import os
@ -24,14 +25,14 @@ logger = get_logger('mirofish.simulation_ipc')
class CommandType(str, Enum): class CommandType(str, Enum):
"""命令类型""" """IPC command types."""
INTERVIEW = "interview" # 单个Agent采访 INTERVIEW = "interview" # interview a single agent
BATCH_INTERVIEW = "batch_interview" # 批量采访 BATCH_INTERVIEW = "batch_interview" # interview multiple agents at once
CLOSE_ENV = "close_env" # 关闭环境 CLOSE_ENV = "close_env" # tear down the environment
class CommandStatus(str, Enum): class CommandStatus(str, Enum):
"""命令状态""" """IPC command status."""
PENDING = "pending" PENDING = "pending"
PROCESSING = "processing" PROCESSING = "processing"
COMPLETED = "completed" COMPLETED = "completed"
@ -40,7 +41,7 @@ class CommandStatus(str, Enum):
@dataclass @dataclass
class IPCCommand: class IPCCommand:
"""IPC命令""" """A command sent over the IPC channel."""
command_id: str command_id: str
command_type: CommandType command_type: CommandType
args: Dict[str, Any] args: Dict[str, Any]
@ -66,7 +67,7 @@ class IPCCommand:
@dataclass @dataclass
class IPCResponse: class IPCResponse:
"""IPC响应""" """A response returned over the IPC channel."""
command_id: str command_id: str
status: CommandStatus status: CommandStatus
result: Optional[Dict[str, Any]] = None result: Optional[Dict[str, Any]] = None
@ -94,24 +95,22 @@ class IPCResponse:
class SimulationIPCClient: class SimulationIPCClient:
""" """IPC client used by the Flask side.
模拟IPC客户端（Flask端使用）
用于向模拟进程发送命令并等待响应 Sends commands to the simulation process and waits for responses.
""" """
def __init__(self, simulation_dir: str): def __init__(self, simulation_dir: str):
""" """Initialize the IPC client.
初始化IPC客户端
Args: Args:
simulation_dir: 模拟数据目录 simulation_dir: Directory holding the simulation's IPC files.
""" """
self.simulation_dir = simulation_dir self.simulation_dir = simulation_dir
self.commands_dir = os.path.join(simulation_dir, "ipc_commands") self.commands_dir = os.path.join(simulation_dir, "ipc_commands")
self.responses_dir = os.path.join(simulation_dir, "ipc_responses") self.responses_dir = os.path.join(simulation_dir, "ipc_responses")
# 确保目录存在 # Ensure both directories exist before use.
os.makedirs(self.commands_dir, exist_ok=True) os.makedirs(self.commands_dir, exist_ok=True)
os.makedirs(self.responses_dir, exist_ok=True) os.makedirs(self.responses_dir, exist_ok=True)
@ -122,20 +121,19 @@ class SimulationIPCClient:
timeout: float = 60.0, timeout: float = 60.0,
poll_interval: float = 0.5 poll_interval: float = 0.5
) -> IPCResponse: ) -> IPCResponse:
""" """Send a command and wait for the response.
发送命令并等待响应
Args: Args:
command_type: 命令类型 command_type: Command type to send.
args: 命令参数 args: Command arguments.
timeout: 超时时间（秒） timeout: Timeout in seconds.
poll_interval: 轮询间隔（秒） poll_interval: Polling interval in seconds.
Returns: Returns:
IPCResponse The ``IPCResponse``.
Raises: Raises:
TimeoutError: 等待响应超时 TimeoutError: When no response arrives before ``timeout``.
""" """
command_id = str(uuid.uuid4()) command_id = str(uuid.uuid4())
command = IPCCommand( command = IPCCommand(
@ -144,14 +142,14 @@ class SimulationIPCClient:
args=args args=args
) )
# 写入命令文件 # Write the command file.
command_file = os.path.join(self.commands_dir, f"{command_id}.json") command_file = os.path.join(self.commands_dir, f"{command_id}.json")
with open(command_file, 'w', encoding='utf-8') as f: with open(command_file, 'w', encoding='utf-8') as f:
json.dump(command.to_dict(), f, ensure_ascii=False, indent=2) json.dump(command.to_dict(), f, ensure_ascii=False, indent=2)
logger.info(t("log.simulation_ipc.m001", command_type=command_type.value, command_id=command_id)) logger.info(t("log.simulation_ipc.m001", command_type=command_type.value, command_id=command_id))
# 等待响应 # Poll for the response file.
response_file = os.path.join(self.responses_dir, f"{command_id}.json") response_file = os.path.join(self.responses_dir, f"{command_id}.json")
start_time = time.time() start_time = time.time()
@ -162,7 +160,7 @@ class SimulationIPCClient:
response_data = json.load(f) response_data = json.load(f)
response = IPCResponse.from_dict(response_data) response = IPCResponse.from_dict(response_data)
# 清理命令和响应文件 # Clean up command and response files after successful read.
try: try:
os.remove(command_file) os.remove(command_file)
os.remove(response_file) os.remove(response_file)
@ -176,10 +174,10 @@ class SimulationIPCClient:
time.sleep(poll_interval) time.sleep(poll_interval)
# 超时 # Timed out waiting for the response.
logger.error(t("log.simulation_ipc.m004", command_id=command_id)) logger.error(t("log.simulation_ipc.m004", command_id=command_id))
# 清理命令文件 # Clean up the unanswered command file.
try: try:
os.remove(command_file) os.remove(command_file)
except OSError: except OSError:
@ -194,20 +192,19 @@ class SimulationIPCClient:
platform: str = None, platform: str = None,
timeout: float = 60.0 timeout: float = 60.0
) -> IPCResponse: ) -> IPCResponse:
""" """Send a single-agent interview command.
发送单个Agent采访命令
Args: Args:
agent_id: Agent ID agent_id: Agent id to interview.
prompt: 采访问题 prompt: Interview question.
platform: 指定平台（可选） platform: Optional platform selector.
- "twitter": 只采访Twitter平台 - ``"twitter"``: interview only on Twitter.
- "reddit": 只采访Reddit平台 - ``"reddit"``: interview only on Reddit.
- None: 双平台模拟时同时采访两个平台，单平台模拟时采访该平台 - ``None``: dual-platform if applicable, else the single active platform.
timeout: 超时时间 timeout: Timeout in seconds.
Returns: Returns:
IPCResponse，result字段包含采访结果 ``IPCResponse`` whose ``result`` carries the interview response.
""" """
args = { args = {
"agent_id": agent_id, "agent_id": agent_id,
@ -228,19 +225,18 @@ class SimulationIPCClient:
platform: str = None, platform: str = None,
timeout: float = 120.0 timeout: float = 120.0
) -> IPCResponse: ) -> IPCResponse:
""" """Send a batched interview command.
发送批量采访命令
Args: Args:
interviews: 采访列表，每个元素包含 {"agent_id": int, "prompt": str, "platform": str(可选)} interviews: List of items shaped ``{"agent_id": int, "prompt": str, "platform": str?}``.
platform: 默认平台（可选，会被每个采访项的platform覆盖） platform: Default platform; per-item ``platform`` overrides this.
- "twitter": 默认只采访Twitter平台 - ``"twitter"``: default to Twitter.
- "reddit": 默认只采访Reddit平台 - ``"reddit"``: default to Reddit.
- None: 双平台模拟时每个Agent同时采访两个平台 - ``None``: dual-platform interview when applicable.
timeout: 超时时间 timeout: Timeout in seconds.
Returns: Returns:
IPCResponse，result字段包含所有采访结果 ``IPCResponse`` whose ``result`` carries every interview response.
""" """
args = {"interviews": interviews} args = {"interviews": interviews}
if platform: if platform:
@ -253,14 +249,13 @@ class SimulationIPCClient:
) )
def send_close_env(self, timeout: float = 30.0) -> IPCResponse: def send_close_env(self, timeout: float = 30.0) -> IPCResponse:
""" """Send a tear-down-environment command.
发送关闭环境命令
Args: Args:
timeout: 超时时间 timeout: Timeout in seconds.
Returns: Returns:
IPCResponse ``IPCResponse``.
""" """
return self.send_command( return self.send_command(
command_type=CommandType.CLOSE_ENV, command_type=CommandType.CLOSE_ENV,
@ -269,10 +264,9 @@ class SimulationIPCClient:
) )
def check_env_alive(self) -> bool: def check_env_alive(self) -> bool:
""" """Return ``True`` if the simulation environment reports as alive.
检查模拟环境是否存活
通过检查 env_status.json 文件来判断 Reads ``env_status.json`` written by the IPC server side.
""" """
status_file = os.path.join(self.simulation_dir, "env_status.json") status_file = os.path.join(self.simulation_dir, "env_status.json")
if not os.path.exists(status_file): if not os.path.exists(status_file):
@ -287,42 +281,40 @@ class SimulationIPCClient:
class SimulationIPCServer: class SimulationIPCServer:
""" """IPC server used by the simulation script.
模拟IPC服务器（模拟脚本端使用）
轮询命令目录，执行命令并返回响应 Polls the commands directory, executes commands, and writes responses.
""" """
def __init__(self, simulation_dir: str): def __init__(self, simulation_dir: str):
""" """Initialize the IPC server.
初始化IPC服务器
Args: Args:
simulation_dir: 模拟数据目录 simulation_dir: Directory holding the simulation's IPC files.
""" """
self.simulation_dir = simulation_dir self.simulation_dir = simulation_dir
self.commands_dir = os.path.join(simulation_dir, "ipc_commands") self.commands_dir = os.path.join(simulation_dir, "ipc_commands")
self.responses_dir = os.path.join(simulation_dir, "ipc_responses") self.responses_dir = os.path.join(simulation_dir, "ipc_responses")
# 确保目录存在 # Ensure both directories exist before use.
os.makedirs(self.commands_dir, exist_ok=True) os.makedirs(self.commands_dir, exist_ok=True)
os.makedirs(self.responses_dir, exist_ok=True) os.makedirs(self.responses_dir, exist_ok=True)
# 环境状态 # Server-running flag.
self._running = False self._running = False
def start(self): def start(self):
"""标记服务器为运行状态""" """Mark the server as alive and persist the state."""
self._running = True self._running = True
self._update_env_status("alive") self._update_env_status("alive")
def stop(self): def stop(self):
"""标记服务器为停止状态""" """Mark the server as stopped and persist the state."""
self._running = False self._running = False
self._update_env_status("stopped") self._update_env_status("stopped")
def _update_env_status(self, status: str): def _update_env_status(self, status: str):
"""更新环境状态文件""" """Update the persistent environment-status file."""
status_file = os.path.join(self.simulation_dir, "env_status.json") status_file = os.path.join(self.simulation_dir, "env_status.json")
with open(status_file, 'w', encoding='utf-8') as f: with open(status_file, 'w', encoding='utf-8') as f:
json.dump({ json.dump({
@ -331,16 +323,15 @@ class SimulationIPCServer:
}, f, ensure_ascii=False, indent=2) }, f, ensure_ascii=False, indent=2)
def poll_commands(self) -> Optional[IPCCommand]: def poll_commands(self) -> Optional[IPCCommand]:
""" """Poll the commands directory and return the next pending command.
轮询命令目录，返回第一个待处理的命令
Returns: Returns:
IPCCommand 或 None ``IPCCommand`` or ``None`` if no pending commands remain.
""" """
if not os.path.exists(self.commands_dir): if not os.path.exists(self.commands_dir):
return None return None
# 按时间排序获取命令文件 # Sort by mtime so we process commands in arrival order.
command_files = [] command_files = []
for filename in os.listdir(self.commands_dir): for filename in os.listdir(self.commands_dir):
if filename.endswith('.json'): if filename.endswith('.json'):
@ -361,17 +352,16 @@ class SimulationIPCServer:
return None return None
def send_response(self, response: IPCResponse): def send_response(self, response: IPCResponse):
""" """Write a response file.
发送响应
Args: Args:
response: IPC响应 response: The response to send.
""" """
response_file = os.path.join(self.responses_dir, f"{response.command_id}.json") response_file = os.path.join(self.responses_dir, f"{response.command_id}.json")
with open(response_file, 'w', encoding='utf-8') as f: with open(response_file, 'w', encoding='utf-8') as f:
json.dump(response.to_dict(), f, ensure_ascii=False, indent=2) json.dump(response.to_dict(), f, ensure_ascii=False, indent=2)
# 删除命令文件 # Delete the matching command file.
command_file = os.path.join(self.commands_dir, f"{response.command_id}.json") command_file = os.path.join(self.commands_dir, f"{response.command_id}.json")
try: try:
os.remove(command_file) os.remove(command_file)
@ -379,7 +369,7 @@ class SimulationIPCServer:
pass pass
def send_success(self, command_id: str, result: Dict[str, Any]): def send_success(self, command_id: str, result: Dict[str, Any]):
"""发送成功响应""" """Send a success response."""
self.send_response(IPCResponse( self.send_response(IPCResponse(
command_id=command_id, command_id=command_id,
status=CommandStatus.COMPLETED, status=CommandStatus.COMPLETED,
@ -387,7 +377,7 @@ class SimulationIPCServer:
)) ))
def send_error(self, command_id: str, error: str): def send_error(self, command_id: str, error: str):
"""发送错误响应""" """Send a failure response."""
self.send_response(IPCResponse( self.send_response(IPCResponse(
command_id=command_id, command_id=command_id,
status=CommandStatus.FAILED, status=CommandStatus.FAILED,

148
backend/app/services/simulation_manager.py
View File

@ -1,7 +1,7 @@
""" """OASIS simulation manager.
OASIS模拟管理器
管理Twitter和Reddit双平台并行模拟 Drives parallel Twitter + Reddit simulations using preset scripts plus
使用预设脚本 + LLM智能生成配置参数 LLM-generated configuration parameters.
""" """
import os import os
@ -23,60 +23,60 @@ logger = get_logger('mirofish.simulation')
class SimulationStatus(str, Enum): class SimulationStatus(str, Enum):
"""模拟状态""" """Simulation lifecycle status."""
CREATED = "created" CREATED = "created"
PREPARING = "preparing" PREPARING = "preparing"
READY = "ready" READY = "ready"
RUNNING = "running" RUNNING = "running"
PAUSED = "paused" PAUSED = "paused"
STOPPED = "stopped" # 模拟被手动停止 STOPPED = "stopped" # manually stopped
COMPLETED = "completed" # 模拟自然完成 COMPLETED = "completed" # finished naturally
FAILED = "failed" FAILED = "failed"
class PlatformType(str, Enum): class PlatformType(str, Enum):
"""平台类型""" """Simulated platform types."""
TWITTER = "twitter" TWITTER = "twitter"
REDDIT = "reddit" REDDIT = "reddit"
@dataclass @dataclass
class SimulationState: class SimulationState:
"""模拟状态""" """In-memory + persisted state for a single simulation."""
simulation_id: str simulation_id: str
project_id: str project_id: str
graph_id: str graph_id: str
# 平台启用状态 # Per-platform enable flags.
enable_twitter: bool = True enable_twitter: bool = True
enable_reddit: bool = True enable_reddit: bool = True
# 状态 # Lifecycle status.
status: SimulationStatus = SimulationStatus.CREATED status: SimulationStatus = SimulationStatus.CREATED
# 准备阶段数据 # Counters captured during the prepare phase.
entities_count: int = 0 entities_count: int = 0
profiles_count: int = 0 profiles_count: int = 0
entity_types: List[str] = field(default_factory=list) entity_types: List[str] = field(default_factory=list)
# 配置生成信息 # Information about the auto-generated config.
config_generated: bool = False config_generated: bool = False
config_reasoning: str = "" config_reasoning: str = ""
# 运行时数据 # Runtime data.
current_round: int = 0 current_round: int = 0
twitter_status: str = "not_started" twitter_status: str = "not_started"
reddit_status: str = "not_started" reddit_status: str = "not_started"
# 时间戳 # Timestamps.
created_at: str = field(default_factory=lambda: datetime.now().isoformat()) created_at: str = field(default_factory=lambda: datetime.now().isoformat())
updated_at: str = field(default_factory=lambda: datetime.now().isoformat()) updated_at: str = field(default_factory=lambda: datetime.now().isoformat())
# 错误信息 # Error message when status == FAILED.
error: Optional[str] = None error: Optional[str] = None
def to_dict(self) -> Dict[str, Any]: def to_dict(self) -> Dict[str, Any]:
"""完整状态字典（内部使用）""" """Full state dict (used for persistence and internal callers)."""
return { return {
"simulation_id": self.simulation_id, "simulation_id": self.simulation_id,
"project_id": self.project_id, "project_id": self.project_id,
@ -98,7 +98,7 @@ class SimulationState:
} }
def to_simple_dict(self) -> Dict[str, Any]: def to_simple_dict(self) -> Dict[str, Any]:
"""简化状态字典（API返回使用）""" """Simplified state dict (used for API responses)."""
return { return {
"simulation_id": self.simulation_id, "simulation_id": self.simulation_id,
"project_id": self.project_id, "project_id": self.project_id,
@ -113,37 +113,36 @@ class SimulationState:
class SimulationManager: class SimulationManager:
""" """Simulation manager.
模拟管理器
核心功能： Core responsibilities:
1. 从Zep图谱读取实体并过滤 1. Read entities from the Zep graph and filter to the configured types.
2. 生成OASIS Agent Profile 2. Generate OASIS agent profiles per entity.
3. 使用LLM智能生成模拟配置参数 3. Use the LLM to generate simulation configuration parameters.
4. 准备预设脚本所需的所有文件 4. Materialize the files the preset scripts expect.
""" """
# 模拟数据存储目录 # Root directory for persisted simulation data.
SIMULATION_DATA_DIR = os.path.join( SIMULATION_DATA_DIR = os.path.join(
os.path.dirname(__file__), os.path.dirname(__file__),
'../../uploads/simulations' '../../uploads/simulations'
) )
def __init__(self): def __init__(self):
# 确保目录存在 # Ensure the simulation data directory exists.
os.makedirs(self.SIMULATION_DATA_DIR, exist_ok=True) os.makedirs(self.SIMULATION_DATA_DIR, exist_ok=True)
# 内存中的模拟状态缓存 # In-memory cache of simulation state objects.
self._simulations: Dict[str, SimulationState] = {} self._simulations: Dict[str, SimulationState] = {}
def _get_simulation_dir(self, simulation_id: str) -> str: def _get_simulation_dir(self, simulation_id: str) -> str:
"""获取模拟数据目录""" """Return the on-disk directory for a simulation, creating if missing."""
sim_dir = os.path.join(self.SIMULATION_DATA_DIR, simulation_id) sim_dir = os.path.join(self.SIMULATION_DATA_DIR, simulation_id)
os.makedirs(sim_dir, exist_ok=True) os.makedirs(sim_dir, exist_ok=True)
return sim_dir return sim_dir
def _save_simulation_state(self, state: SimulationState): def _save_simulation_state(self, state: SimulationState):
"""保存模拟状态到文件""" """Persist a simulation state to disk and update the cache."""
sim_dir = self._get_simulation_dir(state.simulation_id) sim_dir = self._get_simulation_dir(state.simulation_id)
state_file = os.path.join(sim_dir, "state.json") state_file = os.path.join(sim_dir, "state.json")
@ -155,7 +154,7 @@ class SimulationManager:
self._simulations[state.simulation_id] = state self._simulations[state.simulation_id] = state
def _load_simulation_state(self, simulation_id: str) -> Optional[SimulationState]: def _load_simulation_state(self, simulation_id: str) -> Optional[SimulationState]:
"""从文件加载模拟状态""" """Load a simulation state from disk (or cache) by id."""
if simulation_id in self._simulations: if simulation_id in self._simulations:
return self._simulations[simulation_id] return self._simulations[simulation_id]
@ -198,17 +197,16 @@ class SimulationManager:
enable_twitter: bool = True, enable_twitter: bool = True,
enable_reddit: bool = True, enable_reddit: bool = True,
) -> SimulationState: ) -> SimulationState:
""" """Create a new simulation in the ``CREATED`` state.
创建新的模拟
Args: Args:
project_id: 项目ID project_id: Owning project id.
graph_id: Zep图谱ID graph_id: Source Zep graph id.
enable_twitter: 是否启用Twitter模拟 enable_twitter: When ``True``, the Twitter simulation runs.
enable_reddit: 是否启用Reddit模拟 enable_reddit: When ``True``, the Reddit simulation runs.
Returns: Returns:
SimulationState The created ``SimulationState``.
""" """
import uuid import uuid
simulation_id = f"sim_{uuid.uuid4().hex[:12]}" simulation_id = f"sim_{uuid.uuid4().hex[:12]}"
@ -237,27 +235,26 @@ class SimulationManager:
progress_callback: Optional[callable] = None, progress_callback: Optional[callable] = None,
parallel_profile_count: int = 3 parallel_profile_count: int = 3
) -> SimulationState: ) -> SimulationState:
""" """Prepare the simulation environment end-to-end.
准备模拟环境（全程自动化）
步骤： Steps:
1. 从Zep图谱读取并过滤实体 1. Read and filter entities from the graph.
2. 为每个实体生成OASIS Agent Profile（可选LLM增强，支持并行） 2. Generate OASIS agent profiles (optional LLM enrichment, parallel-capable).
3. 使用LLM智能生成模拟配置参数（时间、活跃度、发言频率等） 3. Use the LLM to produce simulation parameters (timing, activity, posting frequency).
4. 保存配置文件和Profile文件 4. Save the configuration and profile files.
5. 复制预设脚本到模拟目录 5. Copy preset scripts into the simulation directory.
Args: Args:
simulation_id: 模拟ID simulation_id: Simulation id.
simulation_requirement: 模拟需求描述（用于LLM生成配置） simulation_requirement: Free-text description of the simulation goal.
document_text: 原始文档内容（用于LLM理解背景） document_text: Raw source document text passed to the LLM for context.
defined_entity_types: 预定义的实体类型（可选） defined_entity_types: Optional list of allowed entity types.
use_llm_for_profiles: 是否使用LLM生成详细人设 use_llm_for_profiles: When ``True``, enrich profiles via the LLM.
progress_callback: 进度回调函数 (stage, progress, message) progress_callback: Optional callback ``(stage, progress, message, **extras)``.
parallel_profile_count: 并行生成人设的数量，默认3 parallel_profile_count: Number of profile generations to run in parallel.
Returns: Returns:
SimulationState The updated ``SimulationState``.
""" """
state = self._load_simulation_state(simulation_id) state = self._load_simulation_state(simulation_id)
if not state: if not state:
@ -269,7 +266,7 @@ class SimulationManager:
sim_dir = self._get_simulation_dir(simulation_id) sim_dir = self._get_simulation_dir(simulation_id)
# ========== 阶段1: 读取并过滤实体 ========== # ========== Stage 1: read and filter entities ==========
if progress_callback: if progress_callback:
progress_callback("reading", 0, t('progress.connectingZepGraph')) progress_callback("reading", 0, t('progress.connectingZepGraph'))
@ -301,7 +298,7 @@ class SimulationManager:
self._save_simulation_state(state) self._save_simulation_state(state)
return state return state
# ========== 阶段2: 生成Agent Profile ========== # ========== Stage 2: generate agent profiles ==========
total_entities = len(filtered.entities) total_entities = len(filtered.entities)
if progress_callback: if progress_callback:
@ -312,7 +309,7 @@ class SimulationManager:
total=total_entities total=total_entities
) )
# 传入graph_id以启用Zep检索功能，获取更丰富的上下文 # Pass the graph_id so the generator can use Zep retrieval for richer context.
generator = OasisProfileGenerator(graph_id=state.graph_id) generator = OasisProfileGenerator(graph_id=state.graph_id)
def profile_progress(current, total, msg): def profile_progress(current, total, msg):
@ -326,7 +323,7 @@ class SimulationManager:
item_name=msg item_name=msg
) )
# 设置实时保存的文件路径（优先使用 Reddit JSON 格式） # Configure the realtime save target (prefer Reddit JSON if Reddit is enabled).
realtime_output_path = None realtime_output_path = None
realtime_platform = "reddit" realtime_platform = "reddit"
if state.enable_reddit: if state.enable_reddit:
@ -340,16 +337,16 @@ class SimulationManager:
entities=filtered.entities, entities=filtered.entities,
use_llm=use_llm_for_profiles, use_llm=use_llm_for_profiles,
progress_callback=profile_progress, progress_callback=profile_progress,
graph_id=state.graph_id, # 传入graph_id用于Zep检索 graph_id=state.graph_id, # used for Zep retrieval enrichment
parallel_count=parallel_profile_count, # 并行生成数量 parallel_count=parallel_profile_count,
realtime_output_path=realtime_output_path, # 实时保存路径 realtime_output_path=realtime_output_path,
output_platform=realtime_platform # 输出格式 output_platform=realtime_platform
) )
state.profiles_count = len(profiles) state.profiles_count = len(profiles)
# 保存Profile文件（注意：Twitter使用CSV格式，Reddit使用JSON格式） # Save profile files. Reddit also writes JSON during generation; this is
# Reddit 已经在生成过程中实时保存了，这里再保存一次确保完整性 # a final consistency write. Twitter requires CSV per OASIS conventions.
if progress_callback: if progress_callback:
progress_callback( progress_callback(
"generating_profiles", 95, "generating_profiles", 95,
@ -366,7 +363,7 @@ class SimulationManager:
) )
if state.enable_twitter: if state.enable_twitter:
# Twitter使用CSV格式！这是OASIS的要求 # Twitter uses CSV format — required by OASIS.
generator.save_profiles( generator.save_profiles(
profiles=profiles, profiles=profiles,
file_path=os.path.join(sim_dir, "twitter_profiles.csv"), file_path=os.path.join(sim_dir, "twitter_profiles.csv"),
@ -381,7 +378,7 @@ class SimulationManager:
total=len(profiles) total=len(profiles)
) )
# ========== 阶段3: LLM智能生成模拟配置 ========== # ========== Stage 3: LLM-driven simulation config ==========
if progress_callback: if progress_callback:
progress_callback( progress_callback(
"generating_config", 0, "generating_config", 0,
@ -419,7 +416,7 @@ class SimulationManager:
total=3 total=3
) )
# 保存配置文件 # Save the configuration file.
config_path = os.path.join(sim_dir, "simulation_config.json") config_path = os.path.join(sim_dir, "simulation_config.json")
with open(config_path, 'w', encoding='utf-8') as f: with open(config_path, 'w', encoding='utf-8') as f:
f.write(sim_params.to_json()) f.write(sim_params.to_json())
@ -435,10 +432,9 @@ class SimulationManager:
total=3 total=3
) )
# 注意：运行脚本保留在 backend/scripts/ 目录，不再复制到模拟目录 # The runtime scripts now live under backend/scripts/; we no longer copy
# 启动模拟时，simulation_runner 会从 scripts/ 目录运行脚本 # them per-simulation. simulation_runner invokes them in place.
# 更新状态
state.status = SimulationStatus.READY state.status = SimulationStatus.READY
self._save_simulation_state(state) self._save_simulation_state(state)
@ -456,16 +452,16 @@ class SimulationManager:
raise raise
def get_simulation(self, simulation_id: str) -> Optional[SimulationState]: def get_simulation(self, simulation_id: str) -> Optional[SimulationState]:
"""获取模拟状态""" """Return the simulation's state, or ``None`` if unknown."""
return self._load_simulation_state(simulation_id) return self._load_simulation_state(simulation_id)
def list_simulations(self, project_id: Optional[str] = None) -> List[SimulationState]: def list_simulations(self, project_id: Optional[str] = None) -> List[SimulationState]:
"""列出所有模拟""" """List all simulations, optionally filtered by ``project_id``."""
simulations = [] simulations = []
if os.path.exists(self.SIMULATION_DATA_DIR): if os.path.exists(self.SIMULATION_DATA_DIR):
for sim_id in os.listdir(self.SIMULATION_DATA_DIR): for sim_id in os.listdir(self.SIMULATION_DATA_DIR):
# 跳过隐藏文件（如 .DS_Store）和非目录文件 # Skip dotfiles (e.g. .DS_Store) and non-directories.
sim_path = os.path.join(self.SIMULATION_DATA_DIR, sim_id) sim_path = os.path.join(self.SIMULATION_DATA_DIR, sim_id)
if sim_id.startswith('.') or not os.path.isdir(sim_path): if sim_id.startswith('.') or not os.path.isdir(sim_path):
continue continue
@ -478,7 +474,7 @@ class SimulationManager:
return simulations return simulations
def get_profiles(self, simulation_id: str, platform: str = "reddit") -> List[Dict[str, Any]]: def get_profiles(self, simulation_id: str, platform: str = "reddit") -> List[Dict[str, Any]]:
"""获取模拟的Agent Profile""" """Return the persisted agent profiles for a platform."""
state = self._load_simulation_state(simulation_id) state = self._load_simulation_state(simulation_id)
if not state: if not state:
raise ValueError(f"模拟不存在: {simulation_id}") raise ValueError(f"模拟不存在: {simulation_id}")
@ -493,7 +489,7 @@ class SimulationManager:
return json.load(f) return json.load(f)
def get_simulation_config(self, simulation_id: str) -> Optional[Dict[str, Any]]: def get_simulation_config(self, simulation_id: str) -> Optional[Dict[str, Any]]:
"""获取模拟配置""" """Return the persisted simulation config dict, or ``None`` if absent."""
sim_dir = self._get_simulation_dir(simulation_id) sim_dir = self._get_simulation_dir(simulation_id)
config_path = os.path.join(sim_dir, "simulation_config.json") config_path = os.path.join(sim_dir, "simulation_config.json")
@ -504,7 +500,7 @@ class SimulationManager:
return json.load(f) return json.load(f)
def get_run_instructions(self, simulation_id: str) -> Dict[str, str]: def get_run_instructions(self, simulation_id: str) -> Dict[str, str]:
"""获取运行说明""" """Return shell commands and instructions to launch the simulation manually."""
sim_dir = self._get_simulation_dir(simulation_id) sim_dir = self._get_simulation_dir(simulation_id)
config_path = os.path.join(sim_dir, "simulation_config.json") config_path = os.path.join(sim_dir, "simulation_config.json")
scripts_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '../../scripts')) scripts_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '../../scripts'))

549
backend/app/services/simulation_runner.py
View File

File diff suppressed because it is too large Load Diff

38
backend/app/services/text_processor.py
View File

@ -1,17 +1,15 @@
""" """Text processing service."""
文本处理服务
"""
from typing import List, Optional from typing import List, Optional
from ..utils.file_parser import FileParser, split_text_into_chunks from ..utils.file_parser import FileParser, split_text_into_chunks
class TextProcessor: class TextProcessor:
"""文本处理器""" """Facade for the text-extraction and chunking pipeline."""
@staticmethod @staticmethod
def extract_from_files(file_paths: List[str]) -> str: def extract_from_files(file_paths: List[str]) -> str:
"""从多个文件提取文本""" """Extract and concatenate text from multiple files."""
return FileParser.extract_from_multiple(file_paths) return FileParser.extract_from_multiple(file_paths)
@staticmethod @staticmethod
@ -20,41 +18,39 @@ class TextProcessor:
chunk_size: int = 500, chunk_size: int = 500,
overlap: int = 50 overlap: int = 50
) -> List[str]: ) -> List[str]:
""" """Split text into chunks.
分割文本
Args: Args:
text: 原始文本 text: The source text.
chunk_size: 块大小 chunk_size: Target characters per chunk.
overlap: 重叠大小 overlap: Overlap between consecutive chunks.
Returns: Returns:
文本块列表 A list of chunk strings.
""" """
return split_text_into_chunks(text, chunk_size, overlap) return split_text_into_chunks(text, chunk_size, overlap)
@staticmethod @staticmethod
def preprocess_text(text: str) -> str: def preprocess_text(text: str) -> str:
""" """Pre-process text by normalizing whitespace and line endings.
预处理文本
- 移除多余空白 - Collapse runs of blank lines to at most two newlines.
- 标准化换行 - Normalize line endings to ``\\n``.
- Strip leading/trailing whitespace from each line.
Args: Args:
text: 原始文本 text: The source text.
Returns: Returns:
处理后的文本 The cleaned text.
""" """
import re import re
# 标准化换行
text = text.replace('\r\n', '\n').replace('\r', '\n') text = text.replace('\r\n', '\n').replace('\r', '\n')
# 移除连续空行（保留最多两个换行） # Collapse 3+ consecutive newlines down to a blank-line separator.
text = re.sub(r'\n{3,}', '\n\n', text) text = re.sub(r'\n{3,}', '\n\n', text)
# 移除行首行尾空白
lines = [line.strip() for line in text.split('\n')] lines = [line.strip() for line in text.split('\n')]
text = '\n'.join(lines) text = '\n'.join(lines)
@ -62,7 +58,7 @@ class TextProcessor:
@staticmethod @staticmethod
def get_text_stats(text: str) -> dict: def get_text_stats(text: str) -> dict:
"""获取文本统计信息""" """Return basic text statistics: total chars, lines, and words."""
return { return {
"total_chars": len(text), "total_chars": len(text),
"total_lines": text.count('\n') + 1, "total_lines": text.count('\n') + 1,

134
backend/app/services/zep_entity_reader.py
View File

@ -1,6 +1,7 @@
""" """Zep entity reader and filter service.
Zep实体读取与过滤服务
从Zep图谱中读取节点，筛选出符合预定义实体类型的节点 Reads nodes from a Zep graph and filters down to those that match a
predefined ontology of entity types.
""" """
import time import time
@ -16,21 +17,21 @@ from ..utils.locale import t
logger = get_logger('mirofish.zep_entity_reader') logger = get_logger('mirofish.zep_entity_reader')
# 用于泛型返回类型 # Generic return-type variable.
T = TypeVar('T') T = TypeVar('T')
@dataclass @dataclass
class EntityNode: class EntityNode:
"""实体节点数据结构""" """In-memory representation of an entity node from the graph."""
uuid: str uuid: str
name: str name: str
labels: List[str] labels: List[str]
summary: str summary: str
attributes: Dict[str, Any] attributes: Dict[str, Any]
# 相关的边信息 # Edges connected to this entity.
related_edges: List[Dict[str, Any]] = field(default_factory=list) related_edges: List[Dict[str, Any]] = field(default_factory=list)
# 相关的其他节点信息 # Other nodes connected through related edges.
related_nodes: List[Dict[str, Any]] = field(default_factory=list) related_nodes: List[Dict[str, Any]] = field(default_factory=list)
def to_dict(self) -> Dict[str, Any]: def to_dict(self) -> Dict[str, Any]:
@ -45,7 +46,7 @@ class EntityNode:
} }
def get_entity_type(self) -> Optional[str]: def get_entity_type(self) -> Optional[str]:
"""获取实体类型（排除默认的Entity标签）""" """Return the first non-default label, or ``None`` if only defaults are present."""
for label in self.labels: for label in self.labels:
if label not in ["Entity", "Node"]: if label not in ["Entity", "Node"]:
return label return label
@ -54,7 +55,7 @@ class EntityNode:
@dataclass @dataclass
class FilteredEntities: class FilteredEntities:
"""过滤后的实体集合""" """Result of a filter pass over the graph: matching entities + counts."""
entities: List[EntityNode] entities: List[EntityNode]
entity_types: Set[str] entity_types: Set[str]
total_count: int total_count: int
@ -70,13 +71,12 @@ class FilteredEntities:
class ZepEntityReader: class ZepEntityReader:
""" """Read entities from a Zep graph and filter to ontology-defined types.
Zep实体读取与过滤服务
主要功能： Capabilities:
1. 从Zep图谱读取所有节点 1. Read all nodes from the graph.
2. 筛选出符合预定义实体类型的节点（Labels不只是Entity的节点） 2. Keep nodes whose labels include something other than the default ``Entity``.
3. 获取每个实体的相关边和关联节点信息 3. Optionally enrich each entity with its connected edges and neighboring nodes.
""" """
def __init__(self, api_key: Optional[str] = None): def __init__(self, api_key: Optional[str] = None):
@ -89,17 +89,16 @@ class ZepEntityReader:
max_retries: int = 3, max_retries: int = 3,
initial_delay: float = 2.0 initial_delay: float = 2.0
) -> T: ) -> T:
""" """Call a Zep API function with retry on failure.
带重试机制的Zep API调用
Args: Args:
func: 要执行的函数（无参数的lambda或callable） func: A zero-argument callable performing the request.
operation_name: 操作名称，用于日志 operation_name: Operation label used in log output.
max_retries: 最大重试次数（默认3次，即最多尝试3次） max_retries: Maximum number of attempts (default 3 — i.e. up to 3 tries total).
initial_delay: 初始延迟秒数 initial_delay: Initial delay between retries in seconds.
Returns: Returns:
API调用结果 The return value of ``func``.
""" """
last_exception = None last_exception = None
delay = initial_delay delay = initial_delay
@ -114,21 +113,20 @@ class ZepEntityReader:
t("log.zep_entity_reader.m001", operation_name=operation_name, attempt=attempt + 1, str=str(e)[:100], delay=delay) t("log.zep_entity_reader.m001", operation_name=operation_name, attempt=attempt + 1, str=str(e)[:100], delay=delay)
) )
time.sleep(delay) time.sleep(delay)
delay *= 2 # 指数退避 delay *= 2 # exponential backoff
else: else:
logger.error(t("log.zep_entity_reader.m002", operation_name=operation_name, max_retries=max_retries, str=str(e))) logger.error(t("log.zep_entity_reader.m002", operation_name=operation_name, max_retries=max_retries, str=str(e)))
raise last_exception raise last_exception
def get_all_nodes(self, graph_id: str) -> List[Dict[str, Any]]: def get_all_nodes(self, graph_id: str) -> List[Dict[str, Any]]:
""" """Return every node in the graph (paginated under the hood).
获取图谱的所有节点（分页获取）
Args: Args:
graph_id: 图谱ID graph_id: Graph identifier.
Returns: Returns:
节点列表 A list of node dicts.
""" """
logger.info(t("log.zep_entity_reader.m003", graph_id=graph_id)) logger.info(t("log.zep_entity_reader.m003", graph_id=graph_id))
@ -148,14 +146,13 @@ class ZepEntityReader:
return nodes_data return nodes_data
def get_all_edges(self, graph_id: str) -> List[Dict[str, Any]]: def get_all_edges(self, graph_id: str) -> List[Dict[str, Any]]:
""" """Return every edge in the graph (paginated under the hood).
获取图谱的所有边（分页获取）
Args: Args:
graph_id: 图谱ID graph_id: Graph identifier.
Returns: Returns:
边列表 A list of edge dicts.
""" """
logger.info(t("log.zep_entity_reader.m005", graph_id=graph_id)) logger.info(t("log.zep_entity_reader.m005", graph_id=graph_id))
@ -176,17 +173,16 @@ class ZepEntityReader:
return edges_data return edges_data
def get_node_edges(self, node_uuid: str) -> List[Dict[str, Any]]: def get_node_edges(self, node_uuid: str) -> List[Dict[str, Any]]:
""" """Return every edge connected to the given node (with retry).
获取指定节点的所有相关边（带重试机制）
Args: Args:
node_uuid: 节点UUID node_uuid: Node UUID.
Returns: Returns:
边列表 A list of edge dicts.
""" """
try: try:
# 使用重试机制调用Zep API # Wrap the API call in retry logic.
edges = self._call_with_retry( edges = self._call_with_retry(
func=lambda: self.client.graph.node.get_entity_edges(node_uuid=node_uuid), func=lambda: self.client.graph.node.get_entity_edges(node_uuid=node_uuid),
operation_name=f"获取节点边(node={node_uuid[:8]}...)" operation_name=f"获取节点边(node={node_uuid[:8]}...)"
@ -214,20 +210,19 @@ class ZepEntityReader:
defined_entity_types: Optional[List[str]] = None, defined_entity_types: Optional[List[str]] = None,
enrich_with_edges: bool = True enrich_with_edges: bool = True
) -> FilteredEntities: ) -> FilteredEntities:
""" """Filter nodes down to entities matching the predefined ontology types.
筛选出符合预定义实体类型的节点
筛选逻辑： Filtering rules:
- 如果节点的Labels只有一个"Entity"，说明这个实体不符合我们预定义的类型，跳过 - Skip nodes whose only label is ``Entity`` (uncategorized).
- 如果节点的Labels包含除"Entity"和"Node"之外的标签，说明符合预定义类型，保留 - Keep nodes whose labels include anything other than ``Entity`` and ``Node``.
Args: Args:
graph_id: 图谱ID graph_id: Graph identifier.
defined_entity_types: 预定义的实体类型列表（可选，如果提供则只保留这些类型） defined_entity_types: Optional allow-list; when provided, only matching types are kept.
enrich_with_edges: 是否获取每个实体的相关边信息 enrich_with_edges: When ``True``, populate related_edges and related_nodes.
Returns: Returns:
FilteredEntities: 过滤后的实体集合 A ``FilteredEntities`` summary.
""" """
logger.info(t("log.zep_entity_reader.m008", graph_id=graph_id)) logger.info(t("log.zep_entity_reader.m008", graph_id=graph_id))
@ -243,7 +238,7 @@ class ZepEntityReader:
except Exception: except Exception:
pass pass
# 获取所有节点 # Read every node from the graph.
all_nodes = self.get_all_nodes(graph_id) all_nodes = self.get_all_nodes(graph_id)
total_count = len(all_nodes) total_count = len(all_nodes)
@ -259,27 +254,27 @@ class ZepEntityReader:
if entity_type != "Entity": if entity_type != "Entity":
node["labels"] = [entity_type] + labels node["labels"] = [entity_type] + labels
# 获取所有边（用于后续关联查找） # Read every edge so we can enrich entities later.
all_edges = self.get_all_edges(graph_id) if enrich_with_edges else [] all_edges = self.get_all_edges(graph_id) if enrich_with_edges else []
# 构建节点UUID到节点数据的映射 # uuid -> node-data map for fast lookup.
node_map = {n["uuid"]: n for n in all_nodes} node_map = {n["uuid"]: n for n in all_nodes}
# 筛选符合条件的实体 # Filter to entities that match the criteria.
filtered_entities = [] filtered_entities = []
entity_types_found = set() entity_types_found = set()
for node in all_nodes: for node in all_nodes:
labels = node.get("labels", []) labels = node.get("labels", [])
# 筛选逻辑：Labels必须包含除"Entity"和"Node"之外的标签 # Filtering rule: labels must contain something other than the defaults.
custom_labels = [l for l in labels if l not in ["Entity", "Node"]] custom_labels = [l for l in labels if l not in ["Entity", "Node"]]
if not custom_labels: if not custom_labels:
# 只有默认标签，跳过 # Only default labels — skip.
continue continue
# 如果指定了预定义类型，检查是否匹配 # When a predefined-type list is supplied, require a match against it.
if defined_entity_types: if defined_entity_types:
matching_labels = [l for l in custom_labels if l in defined_entity_types] matching_labels = [l for l in custom_labels if l in defined_entity_types]
if not matching_labels: if not matching_labels:
@ -290,7 +285,6 @@ class ZepEntityReader:
entity_types_found.add(entity_type) entity_types_found.add(entity_type)
# 创建实体节点对象
entity = EntityNode( entity = EntityNode(
uuid=node["uuid"], uuid=node["uuid"],
name=node["name"], name=node["name"],
@ -299,7 +293,7 @@ class ZepEntityReader:
attributes=node["attributes"], attributes=node["attributes"],
) )
# 获取相关边和节点 # Enrich with related edges and neighboring nodes.
if enrich_with_edges: if enrich_with_edges:
related_edges = [] related_edges = []
related_node_uuids = set() related_node_uuids = set()
@ -324,7 +318,7 @@ class ZepEntityReader:
entity.related_edges = related_edges entity.related_edges = related_edges
# 获取关联节点的基本信息 # Populate basic info for each neighboring node.
related_nodes = [] related_nodes = []
for related_uuid in related_node_uuids: for related_uuid in related_node_uuids:
if related_uuid in node_map: if related_uuid in node_map:
@ -354,18 +348,17 @@ class ZepEntityReader:
graph_id: str, graph_id: str,
entity_uuid: str entity_uuid: str
) -> Optional[EntityNode]: ) -> Optional[EntityNode]:
""" """Fetch a single entity with its full context (edges + neighbors), with retry.
获取单个实体及其完整上下文（边和关联节点，带重试机制）
Args: Args:
graph_id: 图谱ID graph_id: Graph identifier.
entity_uuid: 实体UUID entity_uuid: Entity UUID.
Returns: Returns:
EntityNode或None ``EntityNode`` or ``None`` if not found.
""" """
try: try:
# 使用重试机制获取节点 # Fetch the node with retry.
node = self._call_with_retry( node = self._call_with_retry(
func=lambda: self.client.graph.node.get(uuid_=entity_uuid), func=lambda: self.client.graph.node.get(uuid_=entity_uuid),
operation_name=f"获取节点详情(uuid={entity_uuid[:8]}...)" operation_name=f"获取节点详情(uuid={entity_uuid[:8]}...)"
@ -374,14 +367,14 @@ class ZepEntityReader:
if not node: if not node:
return None return None
# 获取节点的边 # Edges connected to this node.
edges = self.get_node_edges(entity_uuid) edges = self.get_node_edges(entity_uuid)
# 获取所有节点用于关联查找 # All graph nodes, used for neighbor lookup.
all_nodes = self.get_all_nodes(graph_id) all_nodes = self.get_all_nodes(graph_id)
node_map = {n["uuid"]: n for n in all_nodes} node_map = {n["uuid"]: n for n in all_nodes}
# 处理相关边和节点 # Collect related edges and neighboring uuids.
related_edges = [] related_edges = []
related_node_uuids = set() related_node_uuids = set()
@ -403,7 +396,7 @@ class ZepEntityReader:
}) })
related_node_uuids.add(edge["source_node_uuid"]) related_node_uuids.add(edge["source_node_uuid"])
# 获取关联节点信息 # Populate basic info for each neighboring node.
related_nodes = [] related_nodes = []
for related_uuid in related_node_uuids: for related_uuid in related_node_uuids:
if related_uuid in node_map: if related_uuid in node_map:
@ -435,16 +428,15 @@ class ZepEntityReader:
entity_type: str, entity_type: str,
enrich_with_edges: bool = True enrich_with_edges: bool = True
) -> List[EntityNode]: ) -> List[EntityNode]:
""" """Return every entity matching the given type.
获取指定类型的所有实体
Args: Args:
graph_id: 图谱ID graph_id: Graph identifier.
entity_type: 实体类型（如 "Student", "PublicFigure" 等） entity_type: Entity type label (e.g. ``Student``, ``PublicFigure``).
enrich_with_edges: 是否获取相关边信息 enrich_with_edges: When ``True``, populate related edges/nodes.
Returns: Returns:
实体列表 A list of matching ``EntityNode`` instances.
""" """
result = self.filter_defined_entities( result = self.filter_defined_entities(
graph_id=graph_id, graph_id=graph_id,

214
backend/app/services/zep_graph_memory_updater.py
View File

@ -1,6 +1,7 @@
""" """
Zep图谱记忆更新服务 Zep graph memory update service.
将模拟中的Agent活动动态更新到Zep图谱中
Streams agent activity from running simulations into the Zep knowledge graph.
""" """
import os import os
@ -23,7 +24,7 @@ logger = get_logger('mirofish.zep_graph_memory_updater')
@dataclass @dataclass
class AgentActivity: class AgentActivity:
"""Agent活动记录""" """Record of a single agent activity."""
platform: str # twitter / reddit platform: str # twitter / reddit
agent_id: int agent_id: int
agent_name: str agent_name: str
@ -33,13 +34,12 @@ class AgentActivity:
timestamp: str timestamp: str
def to_episode_text(self) -> str: def to_episode_text(self) -> str:
""" """Render the activity as a natural-language episode for Zep.
将活动转换为可以发送给Zep的文本描述
采用自然语言描述格式，让Zep能够从中提取实体和关系 The text uses plain narrative phrasing so Zep can extract entities and
不添加模拟相关的前缀，避免误导图谱更新 relationships from it. No simulation-specific prefix is prepended, so
the graph update is not biased by framing words.
""" """
# 根据不同的动作类型生成不同的描述
action_descriptions = { action_descriptions = {
"CREATE_POST": self._describe_create_post, "CREATE_POST": self._describe_create_post,
"LIKE_POST": self._describe_like_post, "LIKE_POST": self._describe_like_post,
@ -58,7 +58,7 @@ class AgentActivity:
describe_func = action_descriptions.get(self.action_type, self._describe_generic) describe_func = action_descriptions.get(self.action_type, self._describe_generic)
description = describe_func() description = describe_func()
# 直接返回 "agent名称: 活动描述" 格式，不添加模拟前缀 # Return "<agent name>: <activity>" with no simulation prefix.
return f"{self.agent_name}: {description}" return f"{self.agent_name}: {description}"
def _describe_create_post(self) -> str: def _describe_create_post(self) -> str:
@ -68,7 +68,7 @@ class AgentActivity:
return "发布了一条帖子" return "发布了一条帖子"
def _describe_like_post(self) -> str: def _describe_like_post(self) -> str:
"""点赞帖子 - 包含帖子原文和作者信息""" """Like a post — includes the post text and author when available."""
post_content = self.action_args.get("post_content", "") post_content = self.action_args.get("post_content", "")
post_author = self.action_args.get("post_author_name", "") post_author = self.action_args.get("post_author_name", "")
@ -81,7 +81,7 @@ class AgentActivity:
return "点赞了一条帖子" return "点赞了一条帖子"
def _describe_dislike_post(self) -> str: def _describe_dislike_post(self) -> str:
"""踩帖子 - 包含帖子原文和作者信息""" """Dislike a post — includes the post text and author when available."""
post_content = self.action_args.get("post_content", "") post_content = self.action_args.get("post_content", "")
post_author = self.action_args.get("post_author_name", "") post_author = self.action_args.get("post_author_name", "")
@ -94,7 +94,7 @@ class AgentActivity:
return "踩了一条帖子" return "踩了一条帖子"
def _describe_repost(self) -> str: def _describe_repost(self) -> str:
"""转发帖子 - 包含原帖内容和作者信息""" """Repost — includes the original post text and author when available."""
original_content = self.action_args.get("original_content", "") original_content = self.action_args.get("original_content", "")
original_author = self.action_args.get("original_author_name", "") original_author = self.action_args.get("original_author_name", "")
@ -107,7 +107,7 @@ class AgentActivity:
return "转发了一条帖子" return "转发了一条帖子"
def _describe_quote_post(self) -> str: def _describe_quote_post(self) -> str:
"""引用帖子 - 包含原帖内容、作者信息和引用评论""" """Quote-post — includes the original post, author, and the quote comment."""
original_content = self.action_args.get("original_content", "") original_content = self.action_args.get("original_content", "")
original_author = self.action_args.get("original_author_name", "") original_author = self.action_args.get("original_author_name", "")
quote_content = self.action_args.get("quote_content", "") or self.action_args.get("content", "") quote_content = self.action_args.get("quote_content", "") or self.action_args.get("content", "")
@ -127,7 +127,7 @@ class AgentActivity:
return base return base
def _describe_follow(self) -> str: def _describe_follow(self) -> str:
"""关注用户 - 包含被关注用户的名称""" """Follow a user — includes the followed user's name."""
target_user_name = self.action_args.get("target_user_name", "") target_user_name = self.action_args.get("target_user_name", "")
if target_user_name: if target_user_name:
@ -135,7 +135,7 @@ class AgentActivity:
return "关注了一个用户" return "关注了一个用户"
def _describe_create_comment(self) -> str: def _describe_create_comment(self) -> str:
"""发表评论 - 包含评论内容和所评论的帖子信息""" """Create a comment — includes the comment text and the parent post."""
content = self.action_args.get("content", "") content = self.action_args.get("content", "")
post_content = self.action_args.get("post_content", "") post_content = self.action_args.get("post_content", "")
post_author = self.action_args.get("post_author_name", "") post_author = self.action_args.get("post_author_name", "")
@ -151,7 +151,7 @@ class AgentActivity:
return "发表了评论" return "发表了评论"
def _describe_like_comment(self) -> str: def _describe_like_comment(self) -> str:
"""点赞评论 - 包含评论内容和作者信息""" """Like a comment — includes the comment text and author when available."""
comment_content = self.action_args.get("comment_content", "") comment_content = self.action_args.get("comment_content", "")
comment_author = self.action_args.get("comment_author_name", "") comment_author = self.action_args.get("comment_author_name", "")
@ -164,7 +164,7 @@ class AgentActivity:
return "点赞了一条评论" return "点赞了一条评论"
def _describe_dislike_comment(self) -> str: def _describe_dislike_comment(self) -> str:
"""踩评论 - 包含评论内容和作者信息""" """Dislike a comment — includes the comment text and author when available."""
comment_content = self.action_args.get("comment_content", "") comment_content = self.action_args.get("comment_content", "")
comment_author = self.action_args.get("comment_author_name", "") comment_author = self.action_args.get("comment_author_name", "")
@ -177,17 +177,17 @@ class AgentActivity:
return "踩了一条评论" return "踩了一条评论"
def _describe_search(self) -> str: def _describe_search(self) -> str:
"""搜索帖子 - 包含搜索关键词""" """Search posts — includes the search query."""
query = self.action_args.get("query", "") or self.action_args.get("keyword", "") query = self.action_args.get("query", "") or self.action_args.get("keyword", "")
return f"搜索了「{query}」" if query else "进行了搜索" return f"搜索了「{query}」" if query else "进行了搜索"
def _describe_search_user(self) -> str: def _describe_search_user(self) -> str:
"""搜索用户 - 包含搜索关键词""" """Search users — includes the search query."""
query = self.action_args.get("query", "") or self.action_args.get("username", "") query = self.action_args.get("query", "") or self.action_args.get("username", "")
return f"搜索了用户「{query}」" if query else "搜索了用户" return f"搜索了用户「{query}」" if query else "搜索了用户"
def _describe_mute(self) -> str: def _describe_mute(self) -> str:
"""屏蔽用户 - 包含被屏蔽用户的名称""" """Mute a user — includes the muted user's name."""
target_user_name = self.action_args.get("target_user_name", "") target_user_name = self.action_args.get("target_user_name", "")
if target_user_name: if target_user_name:
@ -195,80 +195,79 @@ class AgentActivity:
return "屏蔽了一个用户" return "屏蔽了一个用户"
def _describe_generic(self) -> str: def _describe_generic(self) -> str:
# 对于未知的动作类型，生成通用描述 # Fallback narration for action types not handled explicitly above.
return f"执行了{self.action_type}操作" return f"执行了{self.action_type}操作"
class ZepGraphMemoryUpdater: class ZepGraphMemoryUpdater:
""" """Zep graph memory updater.
Zep图谱记忆更新器
监控模拟的actions日志文件，将新的agent活动实时更新到Zep图谱中。 Watches a simulation's actions log file and streams new agent activity
按平台分组，每累积BATCH_SIZE条活动后批量发送到Zep。 into the Zep knowledge graph in near real time. Activities are grouped
by platform; each platform sends a batch once it has accumulated
``BATCH_SIZE`` items.
所有有意义的行为都会被更新到Zep，action_args中会包含完整的上下文信息： Every meaningful action is forwarded to Zep, with full context preserved
- 点赞/踩的帖子原文 in ``action_args``:
- 转发/引用的帖子原文
- 关注/屏蔽的用户名 - Original text of liked / disliked posts
- 点赞/踩的评论原文 - Original text of reposted / quoted posts
- Names of followed / muted users
- Original text of liked / disliked comments
""" """
# 批量发送大小（每个平台累积多少条后发送） # Number of activities to accumulate per platform before sending a batch.
BATCH_SIZE = 5 BATCH_SIZE = 5
# 平台名称映射（用于控制台显示） # Platform display names used for console / log output.
PLATFORM_DISPLAY_NAMES = { PLATFORM_DISPLAY_NAMES = {
'twitter': '世界1', 'twitter': '世界1',
'reddit': '世界2', 'reddit': '世界2',
} }
# 发送间隔（秒），避免请求过快 # Pause between sends (seconds) to avoid hammering the Zep API.
SEND_INTERVAL = 0.5 SEND_INTERVAL = 0.5
# 重试配置
MAX_RETRIES = 3 MAX_RETRIES = 3
RETRY_DELAY = 2 # 秒 RETRY_DELAY = 2 # seconds
def __init__(self, graph_id: str, api_key: Optional[str] = None): def __init__(self, graph_id: str, api_key: Optional[str] = None):
""" """Initialize the updater.
初始化更新器
Args: Args:
graph_id: Zep图谱ID graph_id: Zep graph ID.
api_key: Zep API Key（可选，默认从配置读取） api_key: Optional Zep API key; defaults to the value from config.
""" """
self.graph_id = graph_id self.graph_id = graph_id
self.client = GraphitiAdapter() self.client = GraphitiAdapter()
# 活动队列
self._activity_queue: Queue = Queue() self._activity_queue: Queue = Queue()
# 按平台分组的活动缓冲区（每个平台各自累积到BATCH_SIZE后批量发送） # Per-platform buffer; each platform flushes once it reaches BATCH_SIZE.
self._platform_buffers: Dict[str, List[AgentActivity]] = { self._platform_buffers: Dict[str, List[AgentActivity]] = {
'twitter': [], 'twitter': [],
'reddit': [], 'reddit': [],
} }
self._buffer_lock = threading.Lock() self._buffer_lock = threading.Lock()
# 控制标志
self._running = False self._running = False
self._worker_thread: Optional[threading.Thread] = None self._worker_thread: Optional[threading.Thread] = None
# 统计 # Counters
self._total_activities = 0 # 实际添加到队列的活动数 self._total_activities = 0 # activities accepted into the queue
self._total_sent = 0 # 成功发送到Zep的批次数 self._total_sent = 0 # batches successfully sent to Zep
self._total_items_sent = 0 # 成功发送到Zep的活动条数 self._total_items_sent = 0 # individual activities successfully sent to Zep
self._failed_count = 0 # 发送失败的批次数 self._failed_count = 0 # batches that failed to send
self._skipped_count = 0 # 被过滤跳过的活动数（DO_NOTHING） self._skipped_count = 0 # activities filtered out (e.g. DO_NOTHING)
logger.info(t("log.zep_graph_memory_updater.m001", graph_id=graph_id, self=self.BATCH_SIZE)) logger.info(t("log.zep_graph_memory_updater.m001", graph_id=graph_id, self=self.BATCH_SIZE))
def _get_platform_display_name(self, platform: str) -> str: def _get_platform_display_name(self, platform: str) -> str:
"""获取平台的显示名称""" """Return the human-friendly display name for a platform."""
return self.PLATFORM_DISPLAY_NAMES.get(platform.lower(), platform) return self.PLATFORM_DISPLAY_NAMES.get(platform.lower(), platform)
def start(self): def start(self):
"""启动后台工作线程""" """Start the background worker thread."""
if self._running: if self._running:
return return
@ -286,10 +285,9 @@ class ZepGraphMemoryUpdater:
logger.info(t("log.zep_graph_memory_updater.m002", self=self.graph_id)) logger.info(t("log.zep_graph_memory_updater.m002", self=self.graph_id))
def stop(self): def stop(self):
"""停止后台工作线程""" """Stop the background worker thread and flush pending activity."""
self._running = False self._running = False
# 发送剩余的活动
self._flush_remaining() self._flush_remaining()
if self._worker_thread and self._worker_thread.is_alive(): if self._worker_thread and self._worker_thread.is_alive():
@ -298,27 +296,28 @@ class ZepGraphMemoryUpdater:
logger.info(t("log.zep_graph_memory_updater.m003", self=self.graph_id, self_2=self._total_activities, self_3=self._total_sent, self_4=self._total_items_sent, self_5=self._failed_count, self_6=self._skipped_count)) logger.info(t("log.zep_graph_memory_updater.m003", self=self.graph_id, self_2=self._total_activities, self_3=self._total_sent, self_4=self._total_items_sent, self_5=self._failed_count, self_6=self._skipped_count))
def add_activity(self, activity: AgentActivity): def add_activity(self, activity: AgentActivity):
""" """Enqueue a single agent activity for delivery to Zep.
添加一个agent活动到队列
所有有意义的行为都会被添加到队列，包括： Every meaningful action is queued, including:
- CREATE_POST（发帖）
- CREATE_COMMENT（评论）
- QUOTE_POST（引用帖子）
- SEARCH_POSTS（搜索帖子）
- SEARCH_USER（搜索用户）
- LIKE_POST/DISLIKE_POST（点赞/踩帖子）
- REPOST（转发）
- FOLLOW（关注）
- MUTE（屏蔽）
- LIKE_COMMENT/DISLIKE_COMMENT（点赞/踩评论）
action_args中会包含完整的上下文信息（如帖子原文、用户名等）。 - CREATE_POST (post)
- CREATE_COMMENT (comment)
- QUOTE_POST (quote a post)
- SEARCH_POSTS (search posts)
- SEARCH_USER (search users)
- LIKE_POST / DISLIKE_POST (like / dislike a post)
- REPOST (repost)
- FOLLOW (follow)
- MUTE (mute)
- LIKE_COMMENT / DISLIKE_COMMENT (like / dislike a comment)
``action_args`` carries the full context (e.g. original post text,
user names) so the graph episode is self-contained.
Args: Args:
activity: Agent活动记录 activity: The agent activity record to enqueue.
""" """
# 跳过DO_NOTHING类型的活动 # DO_NOTHING actions carry no information worth indexing.
if activity.action_type == "DO_NOTHING": if activity.action_type == "DO_NOTHING":
self._skipped_count += 1 self._skipped_count += 1
return return
@ -328,14 +327,13 @@ class ZepGraphMemoryUpdater:
logger.debug(t("log.zep_graph_memory_updater.m004", activity=activity.agent_name, activity_2=activity.action_type)) logger.debug(t("log.zep_graph_memory_updater.m004", activity=activity.agent_name, activity_2=activity.action_type))
def add_activity_from_dict(self, data: Dict[str, Any], platform: str): def add_activity_from_dict(self, data: Dict[str, Any], platform: str):
""" """Build an ``AgentActivity`` from a parsed JSON record and enqueue it.
从字典数据添加活动
Args: Args:
data: 从actions.jsonl解析的字典数据 data: A dict parsed from a single ``actions.jsonl`` line.
platform: 平台名称 (twitter/reddit) platform: Source platform name (``twitter`` or ``reddit``).
""" """
# 跳过事件类型的条目 # Event-type rows describe simulation lifecycle, not agent activity.
if "event_type" in data: if "event_type" in data:
return return
@ -352,28 +350,26 @@ class ZepGraphMemoryUpdater:
self.add_activity(activity) self.add_activity(activity)
def _worker_loop(self, locale: str = 'zh'): def _worker_loop(self, locale: str = 'zh'):
"""后台工作循环 - 按平台批量发送活动到Zep""" """Background loop that drains the queue and flushes per-platform batches."""
set_locale(locale) set_locale(locale)
while self._running or not self._activity_queue.empty(): while self._running or not self._activity_queue.empty():
try: try:
# 尝试从队列获取活动（超时1秒） # Block briefly so the loop can also notice shutdown requests.
try: try:
activity = self._activity_queue.get(timeout=1) activity = self._activity_queue.get(timeout=1)
# 将活动添加到对应平台的缓冲区
platform = activity.platform.lower() platform = activity.platform.lower()
with self._buffer_lock: with self._buffer_lock:
if platform not in self._platform_buffers: if platform not in self._platform_buffers:
self._platform_buffers[platform] = [] self._platform_buffers[platform] = []
self._platform_buffers[platform].append(activity) self._platform_buffers[platform].append(activity)
# 检查该平台是否达到批量大小
if len(self._platform_buffers[platform]) >= self.BATCH_SIZE: if len(self._platform_buffers[platform]) >= self.BATCH_SIZE:
batch = self._platform_buffers[platform][:self.BATCH_SIZE] batch = self._platform_buffers[platform][:self.BATCH_SIZE]
self._platform_buffers[platform] = self._platform_buffers[platform][self.BATCH_SIZE:] self._platform_buffers[platform] = self._platform_buffers[platform][self.BATCH_SIZE:]
# 释放锁后再发送 # Release the lock before issuing the network call.
self._send_batch_activities(batch, platform) self._send_batch_activities(batch, platform)
# 发送间隔，避免请求过快 # Throttle so we don't hammer the Zep API.
time.sleep(self.SEND_INTERVAL) time.sleep(self.SEND_INTERVAL)
except Empty: except Empty:
@ -384,21 +380,20 @@ class ZepGraphMemoryUpdater:
time.sleep(1) time.sleep(1)
def _send_batch_activities(self, activities: List[AgentActivity], platform: str): def _send_batch_activities(self, activities: List[AgentActivity], platform: str):
""" """Send a batch of activities to the Zep graph as one combined episode.
批量发送活动到Zep图谱（合并为一条文本）
Args: Args:
activities: Agent活动列表 activities: Agent activity records to send.
platform: 平台名称 platform: Source platform name.
""" """
if not activities: if not activities:
return return
# 将多条活动合并为一条文本，用换行分隔 # Concatenate the per-activity narrations into a single newline-separated episode.
episode_texts = [activity.to_episode_text() for activity in activities] episode_texts = [activity.to_episode_text() for activity in activities]
combined_text = "\n".join(episode_texts) combined_text = "\n".join(episode_texts)
# 带重试的发送 # Retry on failure with linear backoff.
for attempt in range(self.MAX_RETRIES): for attempt in range(self.MAX_RETRIES):
try: try:
self.client.graph.add( self.client.graph.add(
@ -423,8 +418,8 @@ class ZepGraphMemoryUpdater:
self._failed_count += 1 self._failed_count += 1
def _flush_remaining(self): def _flush_remaining(self):
"""发送队列和缓冲区中剩余的活动""" """Drain the queue and flush every platform buffer, even partial ones."""
# 首先处理队列中剩余的活动，添加到缓冲区 # Move anything still in the queue into the per-platform buffers.
while not self._activity_queue.empty(): while not self._activity_queue.empty():
try: try:
activity = self._activity_queue.get_nowait() activity = self._activity_queue.get_nowait()
@ -436,60 +431,54 @@ class ZepGraphMemoryUpdater:
except Empty: except Empty:
break break
# 然后发送各平台缓冲区中剩余的活动（即使不足BATCH_SIZE条） # Flush each platform buffer regardless of whether it reached BATCH_SIZE.
with self._buffer_lock: with self._buffer_lock:
for platform, buffer in self._platform_buffers.items(): for platform, buffer in self._platform_buffers.items():
if buffer: if buffer:
display_name = self._get_platform_display_name(platform) display_name = self._get_platform_display_name(platform)
logger.info(t("log.zep_graph_memory_updater.m010", display_name=display_name, len=len(buffer))) logger.info(t("log.zep_graph_memory_updater.m010", display_name=display_name, len=len(buffer)))
self._send_batch_activities(buffer, platform) self._send_batch_activities(buffer, platform)
# 清空所有缓冲区
for platform in self._platform_buffers: for platform in self._platform_buffers:
self._platform_buffers[platform] = [] self._platform_buffers[platform] = []
def get_stats(self) -> Dict[str, Any]: def get_stats(self) -> Dict[str, Any]:
"""获取统计信息""" """Return a snapshot of updater statistics."""
with self._buffer_lock: with self._buffer_lock:
buffer_sizes = {p: len(b) for p, b in self._platform_buffers.items()} buffer_sizes = {p: len(b) for p, b in self._platform_buffers.items()}
return { return {
"graph_id": self.graph_id, "graph_id": self.graph_id,
"batch_size": self.BATCH_SIZE, "batch_size": self.BATCH_SIZE,
"total_activities": self._total_activities, # 添加到队列的活动总数 "total_activities": self._total_activities, # activities accepted into the queue
"batches_sent": self._total_sent, # 成功发送的批次数 "batches_sent": self._total_sent, # batches successfully sent
"items_sent": self._total_items_sent, # 成功发送的活动条数 "items_sent": self._total_items_sent, # activities successfully sent
"failed_count": self._failed_count, # 发送失败的批次数 "failed_count": self._failed_count, # batches that failed to send
"skipped_count": self._skipped_count, # 被过滤跳过的活动数（DO_NOTHING） "skipped_count": self._skipped_count, # activities filtered out (e.g. DO_NOTHING)
"queue_size": self._activity_queue.qsize(), "queue_size": self._activity_queue.qsize(),
"buffer_sizes": buffer_sizes, # 各平台缓冲区大小 "buffer_sizes": buffer_sizes, # per-platform buffer depth
"running": self._running, "running": self._running,
} }
class ZepGraphMemoryManager: class ZepGraphMemoryManager:
""" """Registry that owns one ``ZepGraphMemoryUpdater`` per active simulation."""
管理多个模拟的Zep图谱记忆更新器
每个模拟可以有自己的更新器实例
"""
_updaters: Dict[str, ZepGraphMemoryUpdater] = {} _updaters: Dict[str, ZepGraphMemoryUpdater] = {}
_lock = threading.Lock() _lock = threading.Lock()
@classmethod @classmethod
def create_updater(cls, simulation_id: str, graph_id: str) -> ZepGraphMemoryUpdater: def create_updater(cls, simulation_id: str, graph_id: str) -> ZepGraphMemoryUpdater:
""" """Create (and start) a graph-memory updater for a simulation.
为模拟创建图谱记忆更新器
Args: Args:
simulation_id: 模拟ID simulation_id: Simulation ID.
graph_id: Zep图谱ID graph_id: Zep graph ID.
Returns: Returns:
ZepGraphMemoryUpdater实例 The started ``ZepGraphMemoryUpdater`` instance.
""" """
with cls._lock: with cls._lock:
# 如果已存在，先停止旧的 # An updater already exists for this simulation — stop it first.
if simulation_id in cls._updaters: if simulation_id in cls._updaters:
cls._updaters[simulation_id].stop() cls._updaters[simulation_id].stop()
@ -502,25 +491,24 @@ class ZepGraphMemoryManager:
@classmethod @classmethod
def get_updater(cls, simulation_id: str) -> Optional[ZepGraphMemoryUpdater]: def get_updater(cls, simulation_id: str) -> Optional[ZepGraphMemoryUpdater]:
"""获取模拟的更新器""" """Return the updater for a simulation, or ``None`` if absent."""
return cls._updaters.get(simulation_id) return cls._updaters.get(simulation_id)
@classmethod @classmethod
def stop_updater(cls, simulation_id: str): def stop_updater(cls, simulation_id: str):
"""停止并移除模拟的更新器""" """Stop and deregister the updater belonging to a simulation."""
with cls._lock: with cls._lock:
if simulation_id in cls._updaters: if simulation_id in cls._updaters:
cls._updaters[simulation_id].stop() cls._updaters[simulation_id].stop()
del cls._updaters[simulation_id] del cls._updaters[simulation_id]
logger.info(t("log.zep_graph_memory_updater.m012", simulation_id=simulation_id)) logger.info(t("log.zep_graph_memory_updater.m012", simulation_id=simulation_id))
# 防止 stop_all 重复调用的标志 # Idempotency guard so ``stop_all`` only runs once per process lifetime.
_stop_all_done = False _stop_all_done = False
@classmethod @classmethod
def stop_all(cls): def stop_all(cls):
"""停止所有更新器""" """Stop every registered updater (idempotent)."""
# 防止重复调用
if cls._stop_all_done: if cls._stop_all_done:
return return
cls._stop_all_done = True cls._stop_all_done = True
@ -537,7 +525,7 @@ class ZepGraphMemoryManager:
@classmethod @classmethod
def get_all_stats(cls) -> Dict[str, Dict[str, Any]]: def get_all_stats(cls) -> Dict[str, Dict[str, Any]]:
"""获取所有更新器的统计信息""" """Return statistics for every registered updater."""
return { return {
sim_id: updater.get_stats() sim_id: updater.get_stats()
for sim_id, updater in cls._updaters.items() for sim_id, updater in cls._updaters.items()

576
backend/app/services/zep_tools.py
View File

File diff suppressed because it is too large Load Diff

4
backend/app/utils/__init__.py
View File

@ -1,6 +1,4 @@
""" """Backend utilities package."""
工具模块
"""
from .file_parser import FileParser from .file_parser import FileParser
from .llm_client import LLMClient from .llm_client import LLMClient

65
backend/app/utils/file_parser.py
View File

@ -1,6 +1,6 @@
""" """File parsing utilities.
文件解析工具
支持PDF、Markdown、TXT文件的文本提取 Supports text extraction from PDF, Markdown, and plain-text files.
""" """
import os import os
@ -9,30 +9,27 @@ from typing import List, Optional
def _read_text_with_fallback(file_path: str) -> str: def _read_text_with_fallback(file_path: str) -> str:
""" """Read a text file, falling back through encoding detectors when UTF-8 fails.
读取文本文件，UTF-8失败时自动探测编码。
采用多级回退策略： Multi-stage fallback strategy:
1. 首先尝试 UTF-8 解码 1. Try UTF-8 first.
2. 使用 charset_normalizer 检测编码 2. Use ``charset_normalizer`` to detect the encoding.
3. 回退到 chardet 检测编码 3. Fall back to ``chardet``.
4. 最终使用 UTF-8 + errors='replace' 兜底 4. Last resort: decode with UTF-8 + ``errors='replace'``.
Args: Args:
file_path: 文件路径 file_path: Path to the file to read.
Returns: Returns:
解码后的文本内容 The decoded text content.
""" """
data = Path(file_path).read_bytes() data = Path(file_path).read_bytes()
# 首先尝试 UTF-8
try: try:
return data.decode('utf-8') return data.decode('utf-8')
except UnicodeDecodeError: except UnicodeDecodeError:
pass pass
# 尝试使用 charset_normalizer 检测编码
encoding = None encoding = None
try: try:
from charset_normalizer import from_bytes from charset_normalizer import from_bytes
@ -42,7 +39,6 @@ def _read_text_with_fallback(file_path: str) -> str:
except Exception: except Exception:
pass pass
# 回退到 chardet
if not encoding: if not encoding:
try: try:
import chardet import chardet
@ -51,7 +47,6 @@ def _read_text_with_fallback(file_path: str) -> str:
except Exception: except Exception:
pass pass
# 最终兜底：使用 UTF-8 + replace
if not encoding: if not encoding:
encoding = 'utf-8' encoding = 'utf-8'
@ -59,20 +54,19 @@ def _read_text_with_fallback(file_path: str) -> str:
class FileParser: class FileParser:
"""文件解析器""" """Parser for the supported document formats."""
SUPPORTED_EXTENSIONS = {'.pdf', '.md', '.markdown', '.txt'} SUPPORTED_EXTENSIONS = {'.pdf', '.md', '.markdown', '.txt'}
@classmethod @classmethod
def extract_text(cls, file_path: str) -> str: def extract_text(cls, file_path: str) -> str:
""" """Extract plain text from a single supported file.
从文件中提取文本
Args: Args:
file_path: 文件路径 file_path: Path to the file.
Returns: Returns:
提取的文本内容 The extracted text content.
""" """
path = Path(file_path) path = Path(file_path)
@ -95,7 +89,7 @@ class FileParser:
@staticmethod @staticmethod
def _extract_from_pdf(file_path: str) -> str: def _extract_from_pdf(file_path: str) -> str:
"""从PDF提取文本""" """Extract text from a PDF file using PyMuPDF."""
try: try:
import fitz # PyMuPDF import fitz # PyMuPDF
except ImportError: except ImportError:
@ -112,24 +106,23 @@ class FileParser:
@staticmethod @staticmethod
def _extract_from_md(file_path: str) -> str: def _extract_from_md(file_path: str) -> str:
"""从Markdown提取文本，支持自动编码检测""" """Extract text from a Markdown file with automatic encoding detection."""
return _read_text_with_fallback(file_path) return _read_text_with_fallback(file_path)
@staticmethod @staticmethod
def _extract_from_txt(file_path: str) -> str: def _extract_from_txt(file_path: str) -> str:
"""从TXT提取文本，支持自动编码检测""" """Extract text from a plain-text file with automatic encoding detection."""
return _read_text_with_fallback(file_path) return _read_text_with_fallback(file_path)
@classmethod @classmethod
def extract_from_multiple(cls, file_paths: List[str]) -> str: def extract_from_multiple(cls, file_paths: List[str]) -> str:
""" """Extract and concatenate text from multiple files.
从多个文件提取文本并合并
Args: Args:
file_paths: 文件路径列表 file_paths: Paths of files to read.
Returns: Returns:
合并后的文本 The merged text, with per-file headers separating each section.
""" """
all_texts = [] all_texts = []
@ -149,16 +142,15 @@ def split_text_into_chunks(
chunk_size: int = 500, chunk_size: int = 500,
overlap: int = 50 overlap: int = 50
) -> List[str]: ) -> List[str]:
""" """Split text into overlapping chunks.
将文本分割成小块
Args: Args:
text: 原始文本 text: The source text to split.
chunk_size: 每块的字符数 chunk_size: Target characters per chunk.
overlap: 重叠字符数 overlap: Number of characters overlapping between consecutive chunks.
Returns: Returns:
文本块列表 A list of chunk strings.
""" """
if len(text) <= chunk_size: if len(text) <= chunk_size:
return [text] if text.strip() else [] return [text] if text.strip() else []
@ -169,9 +161,8 @@ def split_text_into_chunks(
while start < len(text): while start < len(text):
end = start + chunk_size end = start + chunk_size
# 尝试在句子边界处分割 # Prefer splitting on a sentence boundary near the chunk end
if end < len(text): if end < len(text):
# 查找最近的句子结束符
for sep in ['。', '！', '？', '.\n', '!\n', '?\n', '\n\n', '. ', '! ', '? ']: for sep in ['。', '！', '？', '.\n', '!\n', '?\n', '\n\n', '. ', '! ', '? ']:
last_sep = text[start:end].rfind(sep) last_sep = text[start:end].rfind(sep)
if last_sep != -1 and last_sep > chunk_size * 0.3: if last_sep != -1 and last_sep > chunk_size * 0.3:
@ -182,7 +173,7 @@ def split_text_into_chunks(
if chunk: if chunk:
chunks.append(chunk) chunks.append(chunk)
# 下一个块从重叠位置开始 # Next chunk starts at the overlap point
start = end - overlap if end < len(text) else len(text) start = end - overlap if end < len(text) else len(text)
return chunks return chunks

25
backend/app/utils/llm_client.py
View File

@ -1,6 +1,6 @@
""" """LLM client wrapper.
LLM客户端封装
统一使用OpenAI格式调用 All providers are called through the OpenAI-compatible API surface.
""" """
import json import json
@ -13,7 +13,7 @@ from ..config import Config
class LLMClient: class LLMClient:
"""LLM客户端""" """Thin wrapper around the OpenAI-compatible chat completions API."""
def __init__( def __init__(
self, self,
@ -37,17 +37,16 @@ class LLMClient:
max_tokens: int = 4096, max_tokens: int = 4096,
response_format: Optional[Dict] = None, response_format: Optional[Dict] = None,
) -> str: ) -> str:
""" """Send a chat completion request.
发送聊天请求
Args: Args:
messages: 消息列表 messages: Chat messages in OpenAI format.
temperature: 温度参数 temperature: Sampling temperature.
max_tokens: 最大token数 max_tokens: Maximum number of tokens to generate.
response_format: 响应格式（如JSON模式） response_format: Optional response format hint (e.g. JSON mode).
Returns: Returns:
模型响应文本 The assistant's response text.
""" """
kwargs = { kwargs = {
"model": self.model, "model": self.model,
@ -61,7 +60,7 @@ class LLMClient:
response = self.client.chat.completions.create(**kwargs) response = self.client.chat.completions.create(**kwargs)
content = response.choices[0].message.content content = response.choices[0].message.content
# 部分模型（如MiniMax M2.5）会在content中包含<think>思考内容，需要移除 # Some reasoning models (e.g. MiniMax M2.5) embed <think>...</think> blocks; strip them.
content = re.sub(r"<think>[\s\S]*?</think>", "", content).strip() content = re.sub(r"<think>[\s\S]*?</think>", "", content).strip()
return content return content
@ -79,7 +78,7 @@ class LLMClient:
messages=messages, temperature=temperature, max_tokens=max_tokens messages=messages, temperature=temperature, max_tokens=max_tokens
) )
# 清理markdown代码块标记 # Strip surrounding markdown code-fence markers if present.
cleaned_response = response.strip() cleaned_response = response.strip()
cleaned_response = re.sub( cleaned_response = re.sub(
r"^```(?:json)?\s*\n?", "", cleaned_response, flags=re.IGNORECASE r"^```(?:json)?\s*\n?", "", cleaned_response, flags=re.IGNORECASE

51
backend/app/utils/logger.py
View File

@ -1,6 +1,7 @@
""" """Logger configuration module.
日志配置模块
提供统一的日志管理，同时输出到控制台和文件 Provides unified logging that writes simultaneously to the console and a
rotating log file.
""" """
import os import os
@ -11,48 +12,44 @@ from logging.handlers import RotatingFileHandler
def _ensure_utf8_stdout(): def _ensure_utf8_stdout():
""" """Force stdout/stderr to UTF-8.
确保 stdout/stderr 使用 UTF-8 编码
解决 Windows 控制台中文乱码问题 Fixes garbled non-ASCII output on the Windows console.
""" """
if sys.platform == 'win32': if sys.platform == 'win32':
# Windows 下重新配置标准输出为 UTF-8 # On Windows, reconfigure the standard streams to UTF-8.
if hasattr(sys.stdout, 'reconfigure'): if hasattr(sys.stdout, 'reconfigure'):
sys.stdout.reconfigure(encoding='utf-8', errors='replace') sys.stdout.reconfigure(encoding='utf-8', errors='replace')
if hasattr(sys.stderr, 'reconfigure'): if hasattr(sys.stderr, 'reconfigure'):
sys.stderr.reconfigure(encoding='utf-8', errors='replace') sys.stderr.reconfigure(encoding='utf-8', errors='replace')
# 日志目录 # Directory that holds rotated log files.
LOG_DIR = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'logs') LOG_DIR = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'logs')
def setup_logger(name: str = 'mirofish', level: int = logging.DEBUG) -> logging.Logger: def setup_logger(name: str = 'mirofish', level: int = logging.DEBUG) -> logging.Logger:
""" """Configure and return a logger.
设置日志器
Args: Args:
name: 日志器名称 name: Logger name.
level: 日志级别 level: Minimum log level for the logger.
Returns: Returns:
配置好的日志器 The configured logger.
""" """
# 确保日志目录存在
os.makedirs(LOG_DIR, exist_ok=True) os.makedirs(LOG_DIR, exist_ok=True)
# 创建日志器
logger = logging.getLogger(name) logger = logging.getLogger(name)
logger.setLevel(level) logger.setLevel(level)
# 阻止日志向上传播到根 logger，避免重复输出 # Prevent propagation to the root logger to avoid duplicate output.
logger.propagate = False logger.propagate = False
# 如果已经有处理器，不重复添加 # If handlers are already attached, do not re-add them.
if logger.handlers: if logger.handlers:
return logger return logger
# 日志格式
detailed_formatter = logging.Formatter( detailed_formatter = logging.Formatter(
'[%(asctime)s] %(levelname)s [%(name)s.%(funcName)s:%(lineno)d] %(message)s', '[%(asctime)s] %(levelname)s [%(name)s.%(funcName)s:%(lineno)d] %(message)s',
datefmt='%Y-%m-%d %H:%M:%S' datefmt='%Y-%m-%d %H:%M:%S'
@ -63,7 +60,7 @@ def setup_logger(name: str = 'mirofish', level: int = logging.DEBUG) -> logging.
datefmt='%H:%M:%S' datefmt='%H:%M:%S'
) )
# 1. 文件处理器 - 详细日志（按日期命名，带轮转） # 1. File handler — detailed log, named by date and rotated by size.
log_filename = datetime.now().strftime('%Y-%m-%d') + '.log' log_filename = datetime.now().strftime('%Y-%m-%d') + '.log'
file_handler = RotatingFileHandler( file_handler = RotatingFileHandler(
os.path.join(LOG_DIR, log_filename), os.path.join(LOG_DIR, log_filename),
@ -74,14 +71,13 @@ def setup_logger(name: str = 'mirofish', level: int = logging.DEBUG) -> logging.
file_handler.setLevel(logging.DEBUG) file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(detailed_formatter) file_handler.setFormatter(detailed_formatter)
# 2. 控制台处理器 - 简洁日志（INFO及以上） # 2. Console handler — concise log, INFO and above.
# 确保 Windows 下使用 UTF-8 编码，避免中文乱码 # Ensure UTF-8 on Windows so non-ASCII characters render correctly.
_ensure_utf8_stdout() _ensure_utf8_stdout()
console_handler = logging.StreamHandler(sys.stdout) console_handler = logging.StreamHandler(sys.stdout)
console_handler.setLevel(logging.INFO) console_handler.setLevel(logging.INFO)
console_handler.setFormatter(simple_formatter) console_handler.setFormatter(simple_formatter)
# 添加处理器
logger.addHandler(file_handler) logger.addHandler(file_handler)
logger.addHandler(console_handler) logger.addHandler(console_handler)
@ -89,14 +85,13 @@ def setup_logger(name: str = 'mirofish', level: int = logging.DEBUG) -> logging.
def get_logger(name: str = 'mirofish') -> logging.Logger: def get_logger(name: str = 'mirofish') -> logging.Logger:
""" """Return an existing logger by name, creating it lazily if needed.
获取日志器（如果不存在则创建）
Args: Args:
name: 日志器名称 name: Logger name.
Returns: Returns:
日志器实例 The logger instance.
""" """
logger = logging.getLogger(name) logger = logging.getLogger(name)
if not logger.handlers: if not logger.handlers:
@ -104,11 +99,11 @@ def get_logger(name: str = 'mirofish') -> logging.Logger:
return logger return logger
# 创建默认日志器 # Default module-level logger.
logger = setup_logger() logger = setup_logger()
# 便捷方法 # Convenience module-level helpers.
def debug(msg, *args, **kwargs): def debug(msg, *args, **kwargs):
logger.debug(msg, *args, **kwargs) logger.debug(msg, *args, **kwargs)

84
backend/app/utils/retry.py
View File

@ -1,6 +1,7 @@
""" """API call retry primitives.
API调用重试机制
用于处理LLM等外部API调用的重试逻辑 Helpers for retrying calls to external APIs (LLMs, etc.) with exponential
backoff and jitter.
""" """
import time import time
@ -8,6 +9,7 @@ import random
import functools import functools
from typing import Callable, Any, Optional, Type, Tuple from typing import Callable, Any, Optional, Type, Tuple
from ..utils.logger import get_logger from ..utils.logger import get_logger
from .locale import t
logger = get_logger('mirofish.retry') logger = get_logger('mirofish.retry')
@ -21,17 +23,16 @@ def retry_with_backoff(
exceptions: Tuple[Type[Exception], ...] = (Exception,), exceptions: Tuple[Type[Exception], ...] = (Exception,),
on_retry: Optional[Callable[[Exception, int], None]] = None on_retry: Optional[Callable[[Exception, int], None]] = None
): ):
""" """Decorator that retries a callable with exponential backoff.
带指数退避的重试装饰器
Args: Args:
max_retries: 最大重试次数 max_retries: Maximum number of retries before giving up.
initial_delay: 初始延迟（秒） initial_delay: Initial delay in seconds before the first retry.
max_delay: 最大延迟（秒） max_delay: Cap on the delay between retries (seconds).
backoff_factor: 退避因子 backoff_factor: Multiplicative factor applied to the delay each retry.
jitter: 是否添加随机抖动 jitter: When ``True``, randomize the delay to avoid thundering herd.
exceptions: 需要重试的异常类型 exceptions: Exception types that should trigger a retry.
on_retry: 重试时的回调函数 (exception, retry_count) on_retry: Optional callback invoked on each retry as ``(exception, retry_count)``.
Usage: Usage:
@retry_with_backoff(max_retries=3) @retry_with_backoff(max_retries=3)
@ -52,10 +53,15 @@ def retry_with_backoff(
last_exception = e last_exception = e
if attempt == max_retries: if attempt == max_retries:
logger.error(f"函数 {func.__name__} 在 {max_retries} 次重试后仍失败: {str(e)}") logger.error(t(
"log.retry.m001",
func_name=func.__name__,
max_retries=max_retries,
e=str(e),
))
raise raise
# 计算延迟 # Compute the next delay, capped at ``max_delay``.
current_delay = min(delay, max_delay) current_delay = min(delay, max_delay)
if jitter: if jitter:
current_delay = current_delay * (0.5 + random.random()) current_delay = current_delay * (0.5 + random.random())
@ -86,9 +92,7 @@ def retry_with_backoff_async(
exceptions: Tuple[Type[Exception], ...] = (Exception,), exceptions: Tuple[Type[Exception], ...] = (Exception,),
on_retry: Optional[Callable[[Exception, int], None]] = None on_retry: Optional[Callable[[Exception, int], None]] = None
): ):
""" """Async variant of :func:`retry_with_backoff`."""
异步版本的重试装饰器
"""
import asyncio import asyncio
def decorator(func: Callable) -> Callable: def decorator(func: Callable) -> Callable:
@ -105,7 +109,12 @@ def retry_with_backoff_async(
last_exception = e last_exception = e
if attempt == max_retries: if attempt == max_retries:
logger.error(f"异步函数 {func.__name__} 在 {max_retries} 次重试后仍失败: {str(e)}") logger.error(t(
"log.retry.m002",
func_name=func.__name__,
max_retries=max_retries,
e=str(e),
))
raise raise
current_delay = min(delay, max_delay) current_delay = min(delay, max_delay)
@ -130,9 +139,7 @@ def retry_with_backoff_async(
class RetryableAPIClient: class RetryableAPIClient:
""" """Class-based wrapper around the retry helpers."""
可重试的API客户端封装
"""
def __init__( def __init__(
self, self,
@ -153,17 +160,16 @@ class RetryableAPIClient:
exceptions: Tuple[Type[Exception], ...] = (Exception,), exceptions: Tuple[Type[Exception], ...] = (Exception,),
**kwargs **kwargs
) -> Any: ) -> Any:
""" """Invoke ``func`` with retry on failure.
执行函数调用并在失败时重试
Args: Args:
func: 要调用的函数 func: Callable to invoke.
*args: 函数参数 *args: Positional arguments forwarded to ``func``.
exceptions: 需要重试的异常类型 exceptions: Exception types that should trigger a retry.
**kwargs: 函数关键字参数 **kwargs: Keyword arguments forwarded to ``func``.
Returns: Returns:
函数返回值 The value returned by ``func``.
""" """
last_exception = None last_exception = None
delay = self.initial_delay delay = self.initial_delay
@ -176,7 +182,11 @@ class RetryableAPIClient:
last_exception = e last_exception = e
if attempt == self.max_retries: if attempt == self.max_retries:
logger.error(f"API调用在 {self.max_retries} 次重试后仍失败: {str(e)}") logger.error(t(
"log.retry.m003",
max_retries=self.max_retries,
e=str(e),
))
raise raise
current_delay = min(delay, self.max_delay) current_delay = min(delay, self.max_delay)
@ -199,17 +209,17 @@ class RetryableAPIClient:
exceptions: Tuple[Type[Exception], ...] = (Exception,), exceptions: Tuple[Type[Exception], ...] = (Exception,),
continue_on_failure: bool = True continue_on_failure: bool = True
) -> Tuple[list, list]: ) -> Tuple[list, list]:
""" """Process ``items`` in sequence, retrying each independently on failure.
批量调用并对每个失败项单独重试
Args: Args:
items: 要处理的项目列表 items: Items to process.
process_func: 处理函数，接收单个item作为参数 process_func: Callable invoked once per item.
exceptions: 需要重试的异常类型 exceptions: Exception types that should trigger a retry.
continue_on_failure: 单项失败后是否继续处理其他项 continue_on_failure: When ``True``, keep processing remaining items after a failure.
Returns: Returns:
(成功结果列表, 失败项列表) ``(successes, failures)`` — a list of successful results and a list
of failure descriptors ``{"index", "item", "error"}``.
""" """
results = [] results = []
failures = [] failures = []
@ -224,7 +234,7 @@ class RetryableAPIClient:
results.append(result) results.append(result)
except Exception as e: except Exception as e:
logger.error(f"处理第 {idx + 1} 项失败: {str(e)}") logger.error(t("log.retry.m004", index=idx + 1, e=str(e)))
failures.append({ failures.append({
"index": idx, "index": idx,
"item": item, "item": item,

15
backend/app/utils/zep_paging.py
View File

@ -1,7 +1,8 @@
"""Zep Graph 分页读取工具。 """Zep Graph paging helpers.
Zep 的 node/edge 列表接口使用 UUID cursor 分页， Zep's node/edge list APIs paginate with a UUID cursor. This module wraps the
本模块封装自动翻页逻辑（含单页重试），对调用方透明地返回完整列表。 auto-paging loop (including per-page retry) so callers see the full list
transparently.
""" """
from __future__ import annotations from __future__ import annotations
@ -30,7 +31,7 @@ def _fetch_page_with_retry(
page_description: str = "page", page_description: str = "page",
**kwargs: Any, **kwargs: Any,
) -> list[Any]: ) -> list[Any]:
"""单页请求，失败时指数退避重试。自动处理429限速。""" """Fetch one page, retrying with exponential backoff. Handles 429 rate limits."""
if max_retries < 1: if max_retries < 1:
raise ValueError("max_retries must be >= 1") raise ValueError("max_retries must be >= 1")
@ -43,7 +44,7 @@ def _fetch_page_with_retry(
except Exception as e: except Exception as e:
last_exception = e last_exception = e
if attempt < max_retries - 1: if attempt < max_retries - 1:
# 检测429限速，使用retry-after头部指定的等待时间 # If a 429 rate limit is detected, prefer the retry-after header for the wait.
wait = delay wait = delay
logger.warning( logger.warning(
f"Zep {page_description} attempt {attempt + 1} failed: {str(e)[:100]}, retrying in {wait:.1f}s..." f"Zep {page_description} attempt {attempt + 1} failed: {str(e)[:100]}, retrying in {wait:.1f}s..."
@ -65,7 +66,7 @@ def fetch_all_nodes(
max_retries: int = _DEFAULT_MAX_RETRIES, max_retries: int = _DEFAULT_MAX_RETRIES,
retry_delay: float = _DEFAULT_RETRY_DELAY, retry_delay: float = _DEFAULT_RETRY_DELAY,
) -> list[Any]: ) -> list[Any]:
"""分页获取图谱节点，最多返回 max_items 条（默认 2000）。每页请求自带重试。""" """Page through graph nodes; return at most ``max_items`` (default 2000). Each page is retried internally."""
all_nodes: list[Any] = [] all_nodes: list[Any] = []
cursor: str | None = None cursor: str | None = None
page_num = 0 page_num = 0
@ -110,7 +111,7 @@ def fetch_all_edges(
max_retries: int = _DEFAULT_MAX_RETRIES, max_retries: int = _DEFAULT_MAX_RETRIES,
retry_delay: float = _DEFAULT_RETRY_DELAY, retry_delay: float = _DEFAULT_RETRY_DELAY,
) -> list[Any]: ) -> list[Any]:
"""分页获取图谱所有边，返回完整列表。每页请求自带重试。""" """Page through every graph edge and return the full list. Each page is retried internally."""
all_edges: list[Any] = [] all_edges: list[Any] = []
cursor: str | None = None cursor: str | None = None
page_num = 0 page_num = 0

2
backend/pyproject.toml
View File

@ -1,7 +1,7 @@
[project] [project]
name = "mirofish-backend" name = "mirofish-backend"
version = "0.1.0" version = "0.1.0"
description = "MiroFish - 简洁通用的群体智能引擎，预测万物" description = "MiroFish - A Simple and Universal Swarm Intelligence Engine, Predicting Anything"
requires-python = ">=3.11" requires-python = ">=3.11"
license = { text = "AGPL-3.0" } license = { text = "AGPL-3.0" }
authors = [ authors = [

21
backend/run.py
View File

@ -1,21 +1,20 @@
""" """MiroFish backend entry point."""
MiroFish Backend 启动入口
"""
import os import os
import sys import sys
# 解决 Windows 控制台中文乱码问题：在所有导入之前设置 UTF-8 编码 # Force UTF-8 on Windows console before importing anything that might write to
# stdout/stderr; otherwise non-ASCII characters render as mojibake.
if sys.platform == 'win32': if sys.platform == 'win32':
# 设置环境变量确保 Python 使用 UTF-8 # Make sure Python itself uses UTF-8.
os.environ.setdefault('PYTHONIOENCODING', 'utf-8') os.environ.setdefault('PYTHONIOENCODING', 'utf-8')
# 重新配置标准输出流为 UTF-8 # Reconfigure the standard streams to UTF-8.
if hasattr(sys.stdout, 'reconfigure'): if hasattr(sys.stdout, 'reconfigure'):
sys.stdout.reconfigure(encoding='utf-8', errors='replace') sys.stdout.reconfigure(encoding='utf-8', errors='replace')
if hasattr(sys.stderr, 'reconfigure'): if hasattr(sys.stderr, 'reconfigure'):
sys.stderr.reconfigure(encoding='utf-8', errors='replace') sys.stderr.reconfigure(encoding='utf-8', errors='replace')
# 添加项目根目录到路径 # Add the project root to sys.path so the ``app`` package resolves.
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from app import create_app from app import create_app
@ -23,8 +22,7 @@ from app.config import Config
def main(): def main():
"""主函数""" """Validate configuration and start the Flask development server."""
# 验证配置
errors = Config.validate() errors = Config.validate()
if errors: if errors:
print("配置错误:") print("配置错误:")
@ -33,18 +31,15 @@ def main():
print("\n请检查 .env 文件中的配置") print("\n请检查 .env 文件中的配置")
sys.exit(1) sys.exit(1)
# 创建应用
app = create_app() app = create_app()
# 获取运行配置 # Resolve runtime host/port from the environment.
host = os.environ.get('FLASK_HOST', '0.0.0.0') host = os.environ.get('FLASK_HOST', '0.0.0.0')
port = int(os.environ.get('FLASK_PORT', 5001)) port = int(os.environ.get('FLASK_PORT', 5001))
debug = Config.DEBUG debug = Config.DEBUG
# 启动服务
app.run(host=host, port=port, debug=debug, threaded=True) app.run(host=host, port=port, debug=debug, threaded=True)
if __name__ == '__main__': if __name__ == '__main__':
main() main()

Some files were not shown because too many files have changed in this diff Show More