RomanNum3ral
/
MicroFish
mirror of https://github.com/666ghj/MiroFish.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
293 Commits 1 Branch 3 Tags 74 MiB
Commit Graph

7 Commits

Author SHA1 Message Date
liyizhouAI a6a12d7775 Enhance Foresight demo replay and evidence citations 2026-06-08 19:09:16 +08:00
liyizhouAI 8b77a7b9ba fix: report 429 retry + theme toggle on all pages 
- LLMClient: increase retries 5→8, backoff 1s→2s, max 30s→60s for GLM rate limit recovery
- Report agent: add 3s delay between sections to avoid sustained 429
- Add theme-toggle-anchor to SimulationRun/Report/Interaction/Replay views
- All pages now have dark/light theme toggle button
 2026-04-17 08:36:58 +08:00
liyizhouAI b7df4880c8 fix: v0.3.4 - production stability fixes for simulation pipeline 
- LLM JSON truncation: add _repair_truncated_json() to handle max_tokens cutoff
- Ontology generation: increase max_tokens 4096→8192 to prevent truncation
- Config generation: parallel batch processing (3 threads × 30 agents/batch) for ~3x speedup
- Rate limit: reduce semaphore 30→8 to avoid GLM API 429 storms
- HuggingFace offline: set HF_HUB_OFFLINE=1 to skip unreachable hf-mirror.com
- Simulation recovery: auto-reset stuck "preparing" states on Flask restart
- Prepare endpoint: handle concurrent prepare requests gracefully
- Frontend: handle page-refresh-during-prepare edge case
- API: return 429 instead of 500 for LLM rate limit errors
 2026-04-16 22:44:40 +08:00
liyizhouAI f1f2ea4ea5 fix: v0.3.1 - 7 bug fixes unblock end-to-end production flow 
First successful E2E test uncovered 7 blocking bugs. All fixed and deployed
to production server. User confirmed full pipeline now runs through.

## Bug Fixes

### #1 LLMClient exponential backoff retry
- recognize RateLimitError / 429 / 5xx / 1302 / timeout / connection errors
- 5 retries with 1→2→4→8→16s backoff + random jitter
- file: backend/app/utils/llm_client.py

### #2 GLM → Qwen 32B dual-LLM fallback (ontology generation)
- primary LLM (智谱 GLM-4-Flash) retries exhausted → single attempt fallback to
  SiliconFlow Qwen 2.5-32B via GRAPHITI_LLM_* config
- solves GLM low RPM quota intermittent throttling
- fallback does NOT retry (avoid cascade)
- file: backend/app/services/ontology_generator.py

### #3 Qwen 32K context overflow
- MAX_TEXT_LENGTH_FOR_LLM 50000 → 28000 chars
- ensures prompt+response stays under 32768 tokens
- truncated docs get marker line "[文档已截断以适应 LLM 上下文窗口]"
- file: backend/app/services/ontology_generator.py

### #4 Neo4j entity summary flatten via monkey-patch
- Qwen occasionally returns {summary: {value, type, title, description}}
  which Neo4j rejects (properties must be primitives)
- monkey-patch Neo4jEntityNodeOperations.save / save_bulk
- recursive _flatten_entity_property extracts .value from nested dicts,
  json.dumps as string fallback
- file: backend/app/services/graphiti_client.py

### #5 Frontend stuck at /process/new with no pending state
- pendingUpload is in-memory reactive, lost on refresh / direct URL
- MainView.handleNewProject early-returned on empty state without navigating,
  leaving UI permanently in "waiting for ontology" limbo
- fix: detect empty state, log redirect msg, router.replace to Home after 800ms
- file: frontend/src/views/MainView.vue

### #6 Semaphore 100 → 30 (OASIS simulation)
- high concurrency triggered GLM rate limit cascade during profile/action LLM calls
- drop to 30 eliminates 1302 retries, total runtime impact < 10%
- file: backend/scripts/run_parallel_simulation.py

### #7 SimulationReplayView redesigned to Manus cinematic style
- previous 3-column analyst panel felt like a dashboard, not a replay
- new single-column immersive layout matching Manus's "observation window":
  - top breadcrumb: "Foresight is running Reddit simulation · Round 8/15"
  - main stage: browser chrome + native-style platform post card
    (Reddit subreddit header / Twitter tweet header)
  - action type badges, agent avatars with gradient palettes
  - stage meta bar: per-round action counts by type
- bottom scrubber row: timestamp chip + Jump to live button + live indicator
  (pulsing green dot when sim is running) + step/play/step buttons + 5 speed levels
- bottom task bar: current pipeline step icon/label/progress
- dark theme (#0A0A0B base + #FF5722 accent)
- analyst mode toggle (◫/▦) preserves original 3-column view
- auto-polling every 10s while sim is running, auto-tracks live position
- file: frontend/src/views/SimulationReplayView.vue

## Docs
- PRD.md bumped to v0.3.1 with full hotfix changelog + decision log entries
 2026-04-15 13:25:05 +08:00
liyizhouAI 1fef01d979 feat: v0.3 - Manus replay + token tracking + SIGTERM fix + accelerate button 
This is the v0.3 milestone commit before the v0.4 big version push.
Major themes: process replay, runtime stability, cost observability.

## New Features

- **Manus-style process replay** (frontend + backend)
  - `GET /api/simulation/<id>/replay` returns full workflow + agents + rounds + aggregate
  - `frontend/src/views/SimulationReplayView.vue` 3-column layout (workflow / actions / stats)
  - bottom scrubber with play/pause/step + 5 speed levels (0.5x-10x)
  - filters out stale actions from previous runs via latest simulation_start timestamp

- **Token usage tracking** (`backend/app/utils/token_tracker.py`)
  - process-wide stage→model→tokens counter
  - LLMClient auto-records prompt/completion tokens after each call
  - stages tagged at API entry: step1_ontology, step2_graph_build, step3_prepare, step5_report
  - `GET /api/usage/summary` for live stats + CNY cost estimate
  - `GET /api/usage/estimate-simulation` for OASIS subprocess estimation
  - pricing table for GLM/SiliconFlow/MiniMax/OpenAI/Anthropic models
  - documented as internal-use, removed from customer-facing builds

- **Step 2 "Skip & Continue" button**
  - lets user stop profile generation early and proceed with what's already generated
  - `simulation_manager.request_accelerate()` + cancel_check in oasis_profile_generator
  - new endpoint `POST /api/simulation/prepare/accelerate`

## Critical Bug Fix

- **SIGTERM no longer kills running simulation subprocess**
  - root cause: `SimulationRunner.register_cleanup()` registered SIGTERM/SIGINT/SIGHUP handlers
    that called `os.killpg` on every tracked sim child, even though spawn already used
    `start_new_session=True` to give children isolated sessions
  - fix: neutered `register_cleanup` to a no-op; `cleanup_all_simulations` itself preserved
    for explicit stop_simulation paths
  - validated: killed Flask backend twice, simulation subprocess kept running
  - impact: hot-reload backend code without interrupting in-flight simulations

## Performance & Tuning

- semaphore 30 → 100 (twitter + reddit) for higher LLM concurrency
- discovered 200 agents as memory/cost/statistical sweet spot for 8G server
  (503 agents OOMs both platforms; 200 agents fits cleanly with 95% confidence margin)

## Documentation

- **PRD.md** rewritten as v0.3 baseline (10 chapters + 2 appendices, 639 lines)
  - product positioning across 3 usage modes (one-shot / model-reuse / SaaS)
  - v0.4 roadmap: domestic platforms (douyin/wechat/xiaohongshu/weibo), fork sim, multi-tenant
  - operational lessons: HF mirror, Tencent PyPI mirror, GLM-4-Flash choice, agent count
  - decision log with dates
  - per-stage token/cost breakdown for typical 200-agent run
- **README.md / README-ZH.md** updated with replay step + Graphiti+Neo4j

## Files Touched

19 files changed, 1105 insertions(+), 246 deletions(-)
 2026-04-15 09:37:22 +08:00
666ghj 985f89f49a fix: resolve 500 error caused by <think> tags and markdown code fences in content field from reasoning models like MiniMax/GLM 2026-03-06 00:30:31 +08:00
666ghj 08f417f3b7 Introduce Project ID for context management, finalizing the stateful API pipeline from file submission to graph construction. 2025-11-28 17:21:08 +08:00