liyizhouAI
|
3bdaeaf5e7
|
docs: v0.3.3 - PRD rewrite + Graphiti deprecation record + deploy guide
PRD.md:
- Version bumped to v0.3.3
- Architecture diagram: Graphiti → CustomGraphBuilder
- v0.3.3 changelog: CustomGraphBuilder, deploy.sh, CDN SPA fix, COS deploy
- Infrastructure decisions table updated
- Decision log: 4 new entries for 2026-04-16
- Appendix C: Full Graphiti deprecation record (10 patch attempts, why self-built)
- Appendix D: Complete deployment workflow (backend/frontend/CDN/COS)
README.md / README-ZH.md:
- Step 1 description updated: Graphiti → custom builder with 10x concurrency
|
2026-04-16 13:00:13 +08:00 |
|
liyizhouAI
|
f1f2ea4ea5
|
fix: v0.3.1 - 7 bug fixes unblock end-to-end production flow
First successful E2E test uncovered 7 blocking bugs. All fixed and deployed
to production server. User confirmed full pipeline now runs through.
## Bug Fixes
### #1 LLMClient exponential backoff retry
- recognize RateLimitError / 429 / 5xx / 1302 / timeout / connection errors
- 5 retries with 1→2→4→8→16s backoff + random jitter
- file: backend/app/utils/llm_client.py
### #2 GLM → Qwen 32B dual-LLM fallback (ontology generation)
- primary LLM (智谱 GLM-4-Flash) retries exhausted → single attempt fallback to
SiliconFlow Qwen 2.5-32B via GRAPHITI_LLM_* config
- solves GLM low RPM quota intermittent throttling
- fallback does NOT retry (avoid cascade)
- file: backend/app/services/ontology_generator.py
### #3 Qwen 32K context overflow
- MAX_TEXT_LENGTH_FOR_LLM 50000 → 28000 chars
- ensures prompt+response stays under 32768 tokens
- truncated docs get marker line "[文档已截断以适应 LLM 上下文窗口]"
- file: backend/app/services/ontology_generator.py
### #4 Neo4j entity summary flatten via monkey-patch
- Qwen occasionally returns {summary: {value, type, title, description}}
which Neo4j rejects (properties must be primitives)
- monkey-patch Neo4jEntityNodeOperations.save / save_bulk
- recursive _flatten_entity_property extracts .value from nested dicts,
json.dumps as string fallback
- file: backend/app/services/graphiti_client.py
### #5 Frontend stuck at /process/new with no pending state
- pendingUpload is in-memory reactive, lost on refresh / direct URL
- MainView.handleNewProject early-returned on empty state without navigating,
leaving UI permanently in "waiting for ontology" limbo
- fix: detect empty state, log redirect msg, router.replace to Home after 800ms
- file: frontend/src/views/MainView.vue
### #6 Semaphore 100 → 30 (OASIS simulation)
- high concurrency triggered GLM rate limit cascade during profile/action LLM calls
- drop to 30 eliminates 1302 retries, total runtime impact < 10%
- file: backend/scripts/run_parallel_simulation.py
### #7 SimulationReplayView redesigned to Manus cinematic style
- previous 3-column analyst panel felt like a dashboard, not a replay
- new single-column immersive layout matching Manus's "observation window":
- top breadcrumb: "Foresight is running Reddit simulation · Round 8/15"
- main stage: browser chrome + native-style platform post card
(Reddit subreddit header / Twitter tweet header)
- action type badges, agent avatars with gradient palettes
- stage meta bar: per-round action counts by type
- bottom scrubber row: timestamp chip + Jump to live button + live indicator
(pulsing green dot when sim is running) + step/play/step buttons + 5 speed levels
- bottom task bar: current pipeline step icon/label/progress
- dark theme (#0A0A0B base + #FF5722 accent)
- analyst mode toggle (◫/▦) preserves original 3-column view
- auto-polling every 10s while sim is running, auto-tracks live position
- file: frontend/src/views/SimulationReplayView.vue
## Docs
- PRD.md bumped to v0.3.1 with full hotfix changelog + decision log entries
|
2026-04-15 13:25:05 +08:00 |
|
liyizhouAI
|
1fef01d979
|
feat: v0.3 - Manus replay + token tracking + SIGTERM fix + accelerate button
This is the v0.3 milestone commit before the v0.4 big version push.
Major themes: process replay, runtime stability, cost observability.
## New Features
- **Manus-style process replay** (frontend + backend)
- `GET /api/simulation/<id>/replay` returns full workflow + agents + rounds + aggregate
- `frontend/src/views/SimulationReplayView.vue` 3-column layout (workflow / actions / stats)
- bottom scrubber with play/pause/step + 5 speed levels (0.5x-10x)
- filters out stale actions from previous runs via latest simulation_start timestamp
- **Token usage tracking** (`backend/app/utils/token_tracker.py`)
- process-wide stage→model→tokens counter
- LLMClient auto-records prompt/completion tokens after each call
- stages tagged at API entry: step1_ontology, step2_graph_build, step3_prepare, step5_report
- `GET /api/usage/summary` for live stats + CNY cost estimate
- `GET /api/usage/estimate-simulation` for OASIS subprocess estimation
- pricing table for GLM/SiliconFlow/MiniMax/OpenAI/Anthropic models
- documented as internal-use, removed from customer-facing builds
- **Step 2 "Skip & Continue" button**
- lets user stop profile generation early and proceed with what's already generated
- `simulation_manager.request_accelerate()` + cancel_check in oasis_profile_generator
- new endpoint `POST /api/simulation/prepare/accelerate`
## Critical Bug Fix
- **SIGTERM no longer kills running simulation subprocess**
- root cause: `SimulationRunner.register_cleanup()` registered SIGTERM/SIGINT/SIGHUP handlers
that called `os.killpg` on every tracked sim child, even though spawn already used
`start_new_session=True` to give children isolated sessions
- fix: neutered `register_cleanup` to a no-op; `cleanup_all_simulations` itself preserved
for explicit stop_simulation paths
- validated: killed Flask backend twice, simulation subprocess kept running
- impact: hot-reload backend code without interrupting in-flight simulations
## Performance & Tuning
- semaphore 30 → 100 (twitter + reddit) for higher LLM concurrency
- discovered 200 agents as memory/cost/statistical sweet spot for 8G server
(503 agents OOMs both platforms; 200 agents fits cleanly with 95% confidence margin)
## Documentation
- **PRD.md** rewritten as v0.3 baseline (10 chapters + 2 appendices, 639 lines)
- product positioning across 3 usage modes (one-shot / model-reuse / SaaS)
- v0.4 roadmap: domestic platforms (douyin/wechat/xiaohongshu/weibo), fork sim, multi-tenant
- operational lessons: HF mirror, Tencent PyPI mirror, GLM-4-Flash choice, agent count
- decision log with dates
- per-stage token/cost breakdown for typical 200-agent run
- **README.md / README-ZH.md** updated with replay step + Graphiti+Neo4j
## Files Touched
19 files changed, 1105 insertions(+), 246 deletions(-)
|
2026-04-15 09:37:22 +08:00 |