RomanNum3ral
/
MicroFish
mirror of https://github.com/666ghj/MiroFish.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
268 Commits 1 Branch 3 Tags 74 MiB
Commit Graph

5 Commits

Author SHA1 Message Date
Vivian Prabaswara 7f4ce46579 feat: Add performance optimizations for MiroFish simulation 
Implements 5 major optimizations to handle 96+ rounds without memory issues:

1. Fix Memory Leak
   - Added max_rounds_kept limit (20 rounds in memory)
   - Auto-cleanup old rounds via add_round_summary()
   - Prevents OOM for long simulations

2. Checkpoint System
   - Save progress every 10 rounds (configurable)
   - Resume from last checkpoint if crash
   - Compressed storage with gzip
   - Keep max 5 checkpoints per platform

3. Decision Caching
   - Cache LLM decisions based on agent_id + context hash
   - 24-hour TTL by default
   - Reduces API calls by 20-40%
   - Thread-safe singleton pattern

4. Streaming Progress
   - Real-time progress via Server-Sent Events (SSE)
   - Broadcast round_complete, simulation_complete events
   - Better UX - no more 'stuck' perception
   - Easy client integration with EventSource

5. Batch Processing
   - Process agents in batches (10 per batch)
   - Memory cleanup between batches
   - Per-batch error handling
   - Cancellation support

Expected improvements:
- Memory usage: 60-75% reduction (1-2GB -> 300-500MB)
- API calls: 20-40% reduction with caching
- Resume capability: Checkpoint every 10 rounds
- Real-time progress: SSE streaming

New files:
- backend/app/services/checkpoint_manager.py
- backend/app/services/decision_cache.py
- backend/app/services/batch_processor.py
- backend/app/api/streaming.py
- backend/scripts/run_optimized_twitter_simulation.py
- OPTIMIZATION_GUIDE.md

Modified files:
- backend/app/services/simulation_runner.py
- backend/app/services/__init__.py
 2026-07-09 01:20:39 +00:00
666ghj 1042d50306 Implement Interview feature for agent interactions in simulations 
- Added a new Interview module to facilitate interactions with agents post-simulation, allowing for single and batch interviews.
- Introduced IPC communication mechanism for command and response handling between the Flask backend and simulation scripts.
- Updated README.md to include detailed instructions on the new Interview functionality, including API endpoints and usage examples.
- Enhanced simulation scripts to support waiting for commands after completion, improving user control over the simulation environment.
- Implemented error handling and logging for interview processes, ensuring robust operation and traceability.
 2025-12-08 15:55:39 +08:00
666ghj e3768e2707 Implement dynamic graph memory update feature for simulations 
- Added a new optional parameter `enable_graph_memory_update` to the simulation API, allowing real-time updates of agent activities to the Zep knowledge graph.
- Introduced `ZepGraphMemoryUpdater` and `ZepGraphMemoryManager` classes to handle the background processing of activity updates, ensuring efficient API calls and data management.
- Updated the README.md to include detailed instructions on the new graph memory update functionality and its configuration.
- Enhanced the simulation runner to manage the lifecycle of the graph memory updater, including starting and stopping the updater based on user configuration.
- Improved logging to track the status of graph memory updates, providing better insights into the simulation process.
 2025-12-05 17:53:45 +08:00
666ghj 5f159f6d88 Enhance backend functionality with OASIS simulation features 
- Updated README.md to include new simulation scripts and configuration details for OASIS, including API retry mechanisms and environment variable settings.
- Added simulation management and configuration generation services to streamline the simulation process across Twitter and Reddit platforms.
- Introduced new API routes for simulation-related operations, including entity retrieval and simulation status management.
- Implemented a robust retry mechanism for external API calls to improve system stability.
- Enhanced task management model to include detailed progress tracking.
- Added logging capabilities for action tracking during simulations.
- Included new scripts for running parallel simulations and testing profile formats.
 2025-12-01 15:03:44 +08:00
666ghj 08f417f3b7 Introduce Project ID for context management, finalizing the stateful API pipeline from file submission to graph construction. 2025-11-28 17:21:08 +08:00