1. Report prompt now extracts user's core questions from simulation_requirement
and structures the report to directly answer them (e.g. "will it go well?",
"where are the problems?") instead of generic future-prediction questions.
2. Step3Simulation auto-triggers report generation 1.5s after simulation completes.
3. ReportView auto-redirects to latest successful report when current report
has failed, via /api/report/check/<simulation_id> endpoint.
Simulation completion detection now automatically triggers report
generation after a 1.5s delay, eliminating the need for users to
manually click the "generate report" button.
Report generation now supports REPORT_LLM_* env vars to route report
LLM calls through a separate provider (SiliconFlow Qwen-32B) instead
of sharing the GLM rate limit with simulation. Also passes the report
LLM client to ZepToolsService so tool calls (PanoramaSearch, QuickSearch,
InsightForge, InterviewAgents) use the same provider.
- Filter out non-social entity types (API, SDK, Database, etc.) from agent configs
reduces agent count ~20%, fewer LLM calls per round
- Make semaphore configurable via SIMULATION_SEMAPHORE env var (default 50, up from 30)
allows higher concurrent LLM requests for faster round execution
- Reddit semaphore also reads from env var
- Estimated speedup: ~2x (25min → ~12min for 40 rounds)
scripts/deploy.sh unifies backend/frontend deployment:
- --backend (default): rsync app+scripts → ubuntu@124.223.92.72:/opt/foresight/backend
+ kill/restart Flask + health check
- --frontend: vite build + coscmd upload to COS bucket foresight-1317962478
+ tccli CDN purge
- --full: both
- --no-restart: skip Flask restart
- --dry-run: preview only
Safety:
- Python syntax check before rsync (aborts on error)
- health check loop after restart (aborts if /health fails 5x)
- rsync --delete excludes __pycache__ / *.pyc / .pytest_cache
- rsync --rsync-path="sudo rsync" for remote permissions
- set -euo pipefail throughout
- "$@" not eval (handles spaces in project path)
Solves:
- Manual deploys of backend bits scattered across sessions missed v0.3
accelerate methods on server until this was caught in production
- No consistent way to push frontend + purge CDN in one step
End of a long debugging rabbit hole. After ~10 attempts patching Graphiti's
compatibility with non-OpenAI LLMs (GLM, Qwen via SiliconFlow), every fix
uncovered another bug. Made the strategic call to abandon Graphiti entirely
for the graph-build path and write a minimal custom builder.
## Why Graphiti had to go
- Qwen 32B 32K context overflow on cumulative episode retrieval (60K-82K tokens)
- SiliconFlow Qwen 72B also 32K (not 128K as docs implied)
- GLM 4 Flash gives 128K context but triggers "20015 parameter invalid" on
Graphiti's structured output calls (logprobs in reranker, empty input in
embedder, nested dict in Neo4j writes)
- Each patch target was 1-2 levels deep in Graphiti internals
- 4 separate monkey-patches (EntityNode.save, bulk_utils, driver layer,
reranker, embedder) still couldn't cover the extract_nodes path
## New architecture
backend/app/services/custom_graph_builder.py (NEW, 348 lines):
- Uses Foresight's own LLMClient (retry + GLM→Qwen fallback + token tracker)
- 1 LLM call per chunk (Graphiti needed 4-5)
- Single prompt extracts entities + relationships as JSON
- Writes directly to Neo4j via cypher MERGE (no Graphiti dependency)
- Schema identical to what Graphiti produced — downstream get_all_nodes /
get_all_edges / zep_entity_reader all work unchanged
- ThreadPoolExecutor (10 workers) for concurrent chunk extraction
- Sequential Neo4j writes via shared session (avoids name conflicts in dedup)
- DDL wrapped in try/except (tolerates pre-existing Graphiti indexes)
- Entity name dedup via in-memory map → single uuid per canonical name
- Regex-sanitized label / relation type to prevent cypher injection
backend/app/api/graph.py:
- /api/graph/build endpoint now calls CustomGraphBuilder instead of
builder.add_text_batches() → client.add_episodes_batch() (the old Graphiti path)
backend/app/services/graph_builder.py:
- _build_graph_worker also updated to use CustomGraphBuilder (dead code path
but kept in sync)
backend/app/models/project.py:
- Default chunk_size 500 → 250 (to keep individual LLM prompts small)
backend/app/services/graphiti_client.py:
- Kept all monkey-patches for backward compat — they now only affect legacy
Graphiti code paths that CustomGraphBuilder bypasses entirely:
* _patch_neo4j_driver (AsyncSession.run nested-dict sanitize)
* _patch_reranker_for_non_openai (GLM logprobs workaround)
* _patch_embedder_empty_input (SiliconFlow empty-input guard)
* _patch_entity_node_ops (EntityNode.save sanitize)
* _patch_add_nodes_and_edges_bulk_tx (bulk-episode sanitize)
- These are kept for safety; Graphiti code path is no longer invoked in the
production build flow but methods like add_episode still exist on the class
## Performance
- Sequential: ~194 chunks × 2-5s = 10-15 min
- Concurrent (10 workers): ~194 / 10 × 3-5s = **1-2 min** (5-8x speedup)
- Rate limiting handled by LLMClient retry/backoff, not raw thread contention
## Downstream compatibility
Verified:
- get_all_nodes: MATCH (n:Entity) WHERE n.group_id = $gid RETURN n, labels(n)
- get_all_edges: MATCH (a)-[r]->(b) WHERE r.group_id = $gid RETURN r, type(r)
- get_node / get_node_edges: MATCH by uuid
CustomGraphBuilder writes:
- (n:Entity [optional second label]) with uuid, name, summary, group_id, created_at
- [r:RELATION_TYPE] with uuid, name, fact, group_id, created_at
Schema matches exactly.
## Pipeline wiring
Step 1 ontology generation → Step 2 graph build (CustomGraphBuilder) →
Step 3 profile generation (ZepEntityReader queries Neo4j) → Step 4 simulation
No changes needed downstream of Step 2.
Graphiti doesn't assign custom labels like Zep does - all nodes get
generic 'Entity' label. Updated filter to accept all named Entity
nodes instead of requiring custom labels like 'Person', 'Organization'.
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
- Increase chunk size 500→2000 (reduces chunks from ~125 to ~32 for 60K docs)
- Use Graphiti add_episode_bulk for parallel processing (5 episodes per batch)
- Fallback to sequential if bulk fails
- Target: 60K doc graph build in ~10-15 min instead of 45 min
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
- Remove CORS headers from nginx (let Flask-CORS handle it alone)
- Fix Neo4j edge query to match all relationship types (RELATES_TO + MENTIONS)
- Convert Neo4j DateTime objects to strings for JSON serialization
- Add .serena/, .vercel/, .venv_direct/ to gitignore
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
- Convert Neo4j DateTime objects to strings via _safe_str() helper
- Query all edge types (RELATES_TO + MENTIONS), not just RELATES_TO
- Fix get_node_edges to match any relationship type
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
- Add retry/home buttons when errors occur in graph building
- Improve error messages: Network Error, timeout, missing files
- Clear guidance for users when page refresh loses upload state
- API endpoint switched from HTTP to HTTPS (api.foresight.yizhou.chat)
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
- Upgrade from Qwen2.5-7B to Qwen2.5-32B-Instruct (faster, better quality)
- Add real-time progress messages: "正在处理第 X/Y 个文本块"
- Log per-episode processing time
- Expand progress range from 15-55% to 15-90% (no more episode polling step)
- Add GRAPHITI_LLM_* separate config for graph building LLM
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
- MiniMaxEmbedder: custom EmbedderClient for MiniMax's non-OpenAI-compatible
embedding API (uses 'texts' field instead of 'input')
- Pass LLM config to OpenAIRerankerClient to avoid OPENAI_API_KEY requirement
- Both embedder and reranker now use MiniMax API credentials
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
Override inner container max-width constraints (.main-content, .dashboard-section,
.content-area, .main-content-area, .panel-wrapper) so dark mode content fills
widescreen displays like light mode does natively.
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
- Merge 41 upstream commits (i18n for 7 languages, security fixes, new features)
- Rebrand all MiroFish references to Foresight/先见之明 across 37 files
- Re-apply dark theme CSS overhaul (pure black/gray, no blue tints)
- Re-apply Teleport-based theme toggle (inline with brand, 20px gap)
- Restore Foresight logo and favicon
- Update GitHub links to liyizhouAI/foresight
- Update locale files, README, package.json
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
- Add sans-serif font for English left-pane (status, workflow sections)
- Shorten English workflow step descriptions
- Reduce English report title font-size from 36px to 28px
- Use sans-serif font for English titles, descriptions and navbar
- Shorten English hero text to avoid overflow
- Fix :global() scoped CSS issue that was setting root font-size to 3.5rem
- Use separate unscoped style block for html[lang] selectors
liyizhouAI
666ghj
ghostubborn