Commit Graph

10 Commits

Author SHA1 Message Date
liyizhouAI b7df4880c8 fix: v0.3.4 - production stability fixes for simulation pipeline
- LLM JSON truncation: add _repair_truncated_json() to handle max_tokens cutoff
- Ontology generation: increase max_tokens 4096→8192 to prevent truncation
- Config generation: parallel batch processing (3 threads × 30 agents/batch) for ~3x speedup
- Rate limit: reduce semaphore 30→8 to avoid GLM API 429 storms
- HuggingFace offline: set HF_HUB_OFFLINE=1 to skip unreachable hf-mirror.com
- Simulation recovery: auto-reset stuck "preparing" states on Flask restart
- Prepare endpoint: handle concurrent prepare requests gracefully
- Frontend: handle page-refresh-during-prepare edge case
- API: return 429 instead of 500 for LLM rate limit errors
2026-04-16 22:44:40 +08:00
liyizhouAI 1fef01d979 feat: v0.3 - Manus replay + token tracking + SIGTERM fix + accelerate button
This is the v0.3 milestone commit before the v0.4 big version push.
Major themes: process replay, runtime stability, cost observability.

## New Features

- **Manus-style process replay** (frontend + backend)
  - `GET /api/simulation/<id>/replay` returns full workflow + agents + rounds + aggregate
  - `frontend/src/views/SimulationReplayView.vue` 3-column layout (workflow / actions / stats)
  - bottom scrubber with play/pause/step + 5 speed levels (0.5x-10x)
  - filters out stale actions from previous runs via latest simulation_start timestamp

- **Token usage tracking** (`backend/app/utils/token_tracker.py`)
  - process-wide stage→model→tokens counter
  - LLMClient auto-records prompt/completion tokens after each call
  - stages tagged at API entry: step1_ontology, step2_graph_build, step3_prepare, step5_report
  - `GET /api/usage/summary` for live stats + CNY cost estimate
  - `GET /api/usage/estimate-simulation` for OASIS subprocess estimation
  - pricing table for GLM/SiliconFlow/MiniMax/OpenAI/Anthropic models
  - documented as internal-use, removed from customer-facing builds

- **Step 2 "Skip & Continue" button**
  - lets user stop profile generation early and proceed with what's already generated
  - `simulation_manager.request_accelerate()` + cancel_check in oasis_profile_generator
  - new endpoint `POST /api/simulation/prepare/accelerate`

## Critical Bug Fix

- **SIGTERM no longer kills running simulation subprocess**
  - root cause: `SimulationRunner.register_cleanup()` registered SIGTERM/SIGINT/SIGHUP handlers
    that called `os.killpg` on every tracked sim child, even though spawn already used
    `start_new_session=True` to give children isolated sessions
  - fix: neutered `register_cleanup` to a no-op; `cleanup_all_simulations` itself preserved
    for explicit stop_simulation paths
  - validated: killed Flask backend twice, simulation subprocess kept running
  - impact: hot-reload backend code without interrupting in-flight simulations

## Performance & Tuning

- semaphore 30 → 100 (twitter + reddit) for higher LLM concurrency
- discovered 200 agents as memory/cost/statistical sweet spot for 8G server
  (503 agents OOMs both platforms; 200 agents fits cleanly with 95% confidence margin)

## Documentation

- **PRD.md** rewritten as v0.3 baseline (10 chapters + 2 appendices, 639 lines)
  - product positioning across 3 usage modes (one-shot / model-reuse / SaaS)
  - v0.4 roadmap: domestic platforms (douyin/wechat/xiaohongshu/weibo), fork sim, multi-tenant
  - operational lessons: HF mirror, Tencent PyPI mirror, GLM-4-Flash choice, agent count
  - decision log with dates
  - per-stage token/cost breakdown for typical 200-agent run
- **README.md / README-ZH.md** updated with replay step + Graphiti+Neo4j

## Files Touched

19 files changed, 1105 insertions(+), 246 deletions(-)
2026-04-15 09:37:22 +08:00
liyizhouAI ded715feb2 feat: merge upstream i18n + rebrand MiroFish → Foresight 先见之明
- Merge 41 upstream commits (i18n for 7 languages, security fixes, new features)
- Rebrand all MiroFish references to Foresight/先见之明 across 37 files
- Re-apply dark theme CSS overhaul (pure black/gray, no blue tints)
- Re-apply Teleport-based theme toggle (inline with brand, 20px gap)
- Restore Foresight logo and favicon
- Update GitHub links to liyizhouAI/foresight
- Update locale files, README, package.json

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
2026-04-11 16:40:13 +08:00
666ghj 5ece3f670b Implement Report Agent for automated report generation and interaction
- Introduced the Report Agent module to facilitate the automatic generation of simulation analysis reports using LangChain and Zep, following the ReACT model.
- Added functionality for report outline planning, segmented content generation, and user interaction through a dialogue interface.
- Implemented new API endpoints for report generation, status checking, and retrieval, enhancing the overall reporting capabilities.
- Updated README.md to include detailed instructions on the new report generation features and API usage.
- Enhanced the project structure to accommodate the new report management functionalities, including report storage and retrieval mechanisms.
2025-12-09 15:10:55 +08:00
666ghj 91eb73ae44 Enhance signal handling and suppress warnings in simulation scripts
- Added signal handling to gracefully manage shutdown events across simulation scripts, ensuring proper cleanup of resources.
- Introduced a global shutdown event to allow simulations to respond to termination signals, improving robustness.
- Suppressed warnings related to multiprocessing resource tracking to avoid unnecessary log clutter during execution.
- Updated cleanup logic in `SimulationRunner` and `ZepGraphMemoryManager` to prevent redundant calls and ensure efficient resource management.
- Enhanced logging to provide clearer feedback during shutdown processes, improving traceability.
2025-12-09 00:37:12 +08:00
666ghj 5b4f02f421 Enhance simulation configuration and management features
- Added support for a `max_rounds` parameter in simulation API, allowing users to limit the number of simulation rounds, improving control over simulation duration.
- Updated README.md to reflect the new `max_rounds` parameter and its usage in simulation requests.
- Enhanced error handling for `max_rounds` input validation to ensure it is a positive integer.
- Modified simulation runner and related scripts to incorporate `max_rounds` functionality, ensuring consistent application across Twitter and Reddit simulations.
- Improved logging to indicate when the number of rounds is truncated due to the `max_rounds` setting, enhancing traceability during simulation execution.
2025-12-05 15:50:54 +08:00
666ghj d4fac63eb4 Enhance simulation management and logging features
- Registered a cleanup function for simulation processes to ensure proper termination on server shutdown.
- Improved logging during application startup to confirm the registration of the cleanup function.
- Updated simulation preparation checks to clarify the conditions for considering a simulation ready, enhancing error handling and user feedback.
- Added detailed logging for simulation status changes, improving traceability during the simulation lifecycle.
- Introduced new files for simulation configuration and profile data, supporting enhanced testing and visualization capabilities.
2025-12-02 17:11:47 +08:00
666ghj 5f159f6d88 Enhance backend functionality with OASIS simulation features
- Updated README.md to include new simulation scripts and configuration details for OASIS, including API retry mechanisms and environment variable settings.
- Added simulation management and configuration generation services to streamline the simulation process across Twitter and Reddit platforms.
- Introduced new API routes for simulation-related operations, including entity retrieval and simulation status management.
- Implemented a robust retry mechanism for external API calls to improve system stability.
- Enhanced task management model to include detailed progress tracking.
- Added logging capabilities for action tracking during simulations.
- Included new scripts for running parallel simulations and testing profile formats.
2025-12-01 15:03:44 +08:00
666ghj e98da6b53e Enhance backend startup logging and API endpoint display
- Updated `run.py` to conditionally print startup information only in the reloader process to avoid duplicate logs in debug mode.
- Modified `__init__.py` to log startup and completion messages based on the reloader process condition.
- Added warnings suppression in `graph_builder.py` for Pydantic v2 regarding Field usage.
- Revised `ontology_generator.py` to enforce strict design guidelines for entity types and relationships, ensuring compliance with new requirements.
- Improved logging behavior in `logger.py` to prevent log propagation to the root logger, avoiding duplicate outputs.
2025-11-28 18:59:36 +08:00
666ghj 08f417f3b7 Introduce Project ID for context management, finalizing the stateful API pipeline from file submission to graph construction. 2025-11-28 17:21:08 +08:00