Commit Graph

132 Commits

Author SHA1 Message Date
Ubuntu cf63bccc39 feat(api): add download endpoints for source doc, report and simulation log
Adds GET /api/graph/project/<id>/download/source, GET /api/simulation/<id>/download/report,
and GET /api/simulation/<id>/download/log with corresponding tests (5/5 passing).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 22:18:23 +00:00
Ubuntu 123be0eea9 fix(project): expose uploaded files in project _to_dict response
Add _get_project_files classmethod that queries ProjectFileModel for
file_type=="upload" records, and wire it into _to_dict replacing the
hardcoded empty list.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 22:15:13 +00:00
Ubuntu 0d5a2cf993 feat(api): add PATCH /api/graph/project/<id> to update project name
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 22:14:13 +00:00
Ubuntu ca38800a53 feat(project): trigger async LLM name generation on project creation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 22:12:43 +00:00
Ubuntu f301eff373 feat(project): add LLM-based project name generator service
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 22:11:13 +00:00
Ubuntu df6c77255a fix(simulation): fix SQLite lock on Azure Files and Neo4j clone query
- Add nobrl,cache=strict,nosharesock,actimeo=30 mount options to the
  Azure Files volume so SQLite can use byte-range locking correctly over
  SMB (without nobrl, Reddit DB creation fails with 'database is locked')
- Fix _execute_neo4j_query to pass parameters as params= keyword arg
  (was parameters_= which the installed neo4j driver does not accept),
  restoring per-simulation graph isolation via clone_graph

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 22:49:38 +00:00
Ubuntu e7641e9831 feat(history): enable 'Analysis Report' button in simulation history
- ProjectManager._to_dict now scans simulation state files to find
  last_simulation_id and last_report_id for each project
- HistoryDatabase: report button enabled when last_report_id or
  last_simulation_id available; goToReport looks up report by
  simulation ID if no direct report_id; report status icon reflects
  availability
- Add getReportBySimulation() to frontend report API

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-04 23:27:10 +00:00
Ubuntu 2aa44760d1 feat(f2a-b): step02/03 UX overhaul, Azure OpenAI fix, offline report interviews
Step 02/03 UX:
- Remove duplicate simulation params (total_hours, minutes_per_round) from Fase B form
- Add following_probability + recsys_type to step 03 config panel instead
- Step 03 now read-only by default with Edit / Save toolbar toggle
- Add common.edit and log.configSaved i18n keys to all 4 locales

Backend - Azure OpenAI fix (run_parallel_simulation.py):
- create_model() now detects Azure URLs (cognitiveservices.azure.com / openai.azure.com)
  and uses ModelPlatformType.AZURE with AzureOpenAIModel, passing api_version and
  azure_deployment_name explicitly — fixes 404 "Resource not found" from camel-ai

Backend - offline report interviews (simulation_runner.py):
- interview_agents_batch() falls back to _offline_batch_interview() when OASIS env is
  closed (simulation finished), reading posts from SQLite DB instead of failing with
  "No successful interviews / check OASIS environment status"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-04 23:20:41 +00:00
Ubuntu 5ec98a3e0e fix(graph): use get_running_loop() in _execute_neo4j_query, unify delete_graph to single async-backed method 2026-05-03 22:10:14 +00:00
Ubuntu 0343cad632 feat(simulation): clone graph on start for per-simulation isolation, use graph_id_simulation in report
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 22:06:27 +00:00
Ubuntu 6ea83f31a4 feat(graph): add clone_graph and delete_graph to GraphitiBackend via APOC
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 22:05:19 +00:00
Ubuntu 594144e195 fix(simulation): move regenerate_agent validation to request handler (404/400 before task creation) 2026-05-03 22:02:56 +00:00
Ubuntu 5d679024c3 feat(simulation): add POST /agent create and /agent/{id}/regenerate endpoints
Implements Tasks 8 and 9 of F2-A+B: async agent creation from entity UUID
and async per-agent profile regeneration, both with atomic profile file writes.
Refactors generate_profile_from_entity to accept extra_instructions instead
of requiring user_id (renamed internal logic to _generate_single_profile).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 21:59:25 +00:00
Ubuntu 306414a555 fix(simulation): add int coercion and 400 validation for max_agents in prepare endpoint 2026-05-03 21:56:03 +00:00
Ubuntu ef46ba1743 feat(simulation): add max_agents selector via top-N connectivity in prepare
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 21:54:34 +00:00
Ubuntu a74c0975fe fix(simulation): fix clone 400→404 for not found, remove redundant makedirs, hoist uuid import
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 21:52:35 +00:00
Ubuntu a6b4ccca3c feat(simulation): add POST /<simulation_id>/clone endpoint (Task 6)
Implements SimulationManager.clone_simulation() and the corresponding
Flask route. The clone copies reddit_profiles.json, twitter_profiles.csv
and agent_profiles.json from the source simulation; does not copy
simulation_config.json; sets status=PROFILES_READY and records
parent_simulation_id. All 3 tests in test_simulation_clone.py pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 21:48:57 +00:00
Ubuntu ed71afe442 fix(simulation): add FileNotFoundError check in delete_agent_profile and endpoint 2026-05-03 21:44:55 +00:00
Ubuntu 83cf890c83 feat(simulation): add DELETE agent, POST generate-config, PATCH config endpoints
- DELETE /<sim_id>/agent/<user_id>: removes agent from reddit_profiles.json (atomic write, guards against running/completed status)
- POST /<sim_id>/generate-config: transitions profiles_ready→configuring→ready, runs LLM config generation in background thread, returns task_id
- PATCH /<sim_id>/config: merges time/platform config fields into simulation_config.json (atomic write)
- Corresponding SimulationManager methods: delete_agent_profile(), patch_simulation_config()
- 7 tests all passing (3 original + 4 new)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 21:43:02 +00:00
Ubuntu c0356e706a fix(simulation): catch FileNotFoundError in patch_agent endpoint (return 404 not 500) 2026-05-03 21:40:16 +00:00
Ubuntu f7dd353a31 feat(simulation): add PATCH /simulation/{id}/agent/{user_id} endpoint
Implements atomic agent profile editing with manually_edited flag.
Blocks edits when simulation is running or completed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 21:37:19 +00:00
Ubuntu 8a0854ceba feat(simulation): add profiles_ready/configuring states and parent_simulation_id/graph_id_simulation fields 2026-05-03 21:33:14 +00:00
Ubuntu 83b69dfd87 fix(graph-api): fix build_graph endpoint for dict-based ProjectManager
Convert all project.attribute accesses to project["key"]/project.get("key"),
use save_project({...}) dict calls, and wire save_graph_record/complete_graph_record
helpers for proper graph persistence.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 01:24:56 +00:00
Ubuntu 5d6d4d787a fix(graph-api): cleanup orphan project on ontology generation failure
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 01:23:15 +00:00
Ubuntu 284f962e17 fix(graph-api): rewrite generate_ontology/import_ontology to use dict ProjectManager
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 01:20:32 +00:00
Ubuntu ea9ac6a1cd fix(graph-api): fix project/task dict access after ProjectManager refactor
Replace .to_dict() calls and attribute access with direct dict usage,
add get_storage() import for delete_project, and rewrite reset_project
to use save_project() with a partial dict instead of mutating an ORM object.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 01:17:21 +00:00
Ubuntu 8a6d00b26e refactor(project): clean up _to_dict delegation and remove unused import
- Remove unused `import io`
- Avoid double-encoding in save_extracted_text by caching encoded bytes
- Delegate _to_dict queries to get_ontology/get_latest_graph_external_id
- Add comment to save_ontology noting full versioning is planned for F2-3

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 01:16:03 +00:00
Ubuntu 949921344b feat(project): add ontology/graph persistence helpers to ProjectManager
- Add `from ..config import Config` import
- Convert `_to_dict` from @staticmethod to @classmethod; it now opens a
  fresh session to populate `ontology` and `graph_id` from OntologyModel
  and GraphModel respectively
- Use `db.expunge(proj)` before closing sessions in create_project,
  get_project and list_projects so `_to_dict` can open its own session
  without nesting conflicts (SQLite StaticPool safe)
- Add five new classmethods: save_ontology, get_ontology,
  save_graph_record, get_latest_graph_external_id, complete_graph_record

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 01:13:26 +00:00
Ubuntu 9ed2412745 test(conftest): add in_memory_db and task manager singleton reset fixtures; fix broken tests
- Replace conftest.py with two autouse fixtures: reset_graph_factory_singleton
  and reset_task_manager_singleton, plus in_memory_db fixture for tests
  requiring a real DB
- Rewrite test_project_task_recovery.py to use ProjectManager + SQLite in-memory
  DB instead of the removed Project dataclass (same guarantee: active_task_id
  persists and defaults to None)
- Fix db.py init_db() to import db_models before Base.metadata.create_all() so
  all ORM tables are registered and created on first startup

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 00:15:42 +00:00
Ubuntu 0b760eed0b fix(task): rename loop var t→task_row to avoid shadowing locale.t import 2026-05-03 00:13:12 +00:00
Ubuntu 2478beb9ea feat(project): refactor ProjectManager to persist via SQLAlchemy + StorageService
Replace file-based JSON persistence with SQLAlchemy DB (ProjectModel/ProjectFileModel)
and StorageService for file content; remove Project dataclass; add 7 passing tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 00:11:12 +00:00
Ubuntu 1f43d35d59 feat(task): refactor TaskManager to persist tasks in SQLAlchemy DB
Replace in-memory dict-based TaskManager with a SQLAlchemy-backed implementation
using TaskModel. Tasks now survive process restarts. 6 new tests added and passing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 00:10:16 +00:00
Ubuntu 479ae0b712 feat(app): inject SQLAlchemy DB and StorageService into Flask app factory
Initializes the database (via init_db) and registers the StorageService
instance in app.extensions['storage'] inside create_app(). Adds get_storage()
helper for convenient access from any Flask context.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 00:08:13 +00:00
Ubuntu 868ce39577 fix(storage,db): path traversal fix, delete_prefix validation, remove dead import, factory uses Config
- local.py: use relative_to() for path traversal guard (fixes prefix-collision false negative)
- local.py: validate delete_prefix rejects empty/root prefix to prevent full-storage wipe
- local.py: remove unused `import os`
- db_models.py: remove dead UniqueConstraint import
- db_models.py: replace deprecated datetime.utcnow() with datetime.now(timezone.utc)
- factory.py: read STORAGE_TYPE and related settings from Config instead of os.environ directly; remove `import os`

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 00:06:58 +00:00
Ubuntu 773cc250c9 feat(db): add SQLAlchemy Base, session factory, and all ORM models
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 23:54:05 +00:00
Ubuntu 9e33f823d9 feat(storage): add StorageService protocol, LocalFSStorage, AzureBlobStorage, factory 2026-05-02 23:53:19 +00:00
Ubuntu ca2f9e2a8f feat(config): add DATABASE_URL, STORAGE_TYPE, AZURE_STORAGE_*, JWT config vars 2026-05-02 23:53:16 +00:00
Ubuntu 49f4da51b1 fix(report): strip fabricated tool_result blocks to prevent LLM hallucination loop 2026-04-26 15:06:24 +00:00
Ubuntu 866ed421e2 fix(ontology): handle string attributes from LLM response to prevent TypeError crash
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-26 14:59:10 +00:00
Ubuntu 117eabf607 feat(recovery): persist active_task_id to project.json for browser-refresh reconnection
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-26 14:54:44 +00:00
Ubuntu 656a3d7d5c fix(graph): cap get_all_edges to 5000 edges to prevent unbounded RAM growth 2026-04-26 14:50:21 +00:00
Ubuntu d09ee23cd3 feat: ontology import, project recovery, graphiti improvements, and docs
- Add ontology import from file (Step 1)
- Persist active task_id to project.json for browser-refresh recovery
- Add project list endpoint and frontend recovery UI
- Improve Graphiti backend stability and oasis profile generation
- Add TechnicalDesign.md and enterprise roadmap spec
- Update .gitignore to exclude secrets and local-only docs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-26 14:28:14 +00:00
Ubuntu 7daf8566ed fix(report): eliminate temp file leak in MD fallback; simplify invalid-format test 2026-04-26 00:11:40 +00:00
Ubuntu 8efedb55e5 feat(report): add PDF download endpoint via PyMuPDF
Adds ?format=md|pdf query param to GET /api/report/<id>/download.
PDF is generated from Markdown via fitz.Story (PyMuPDF). Also fixes
create_app() to support dict config and skip JWT auth in TESTING mode.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-26 00:07:09 +00:00
Ubuntu a209330b0b debug: log GRAPH_BACKEND and key presence at graph build start
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-25 22:18:26 +00:00
Ubuntu 09935e53a0 refactor(graph): remove hardcoded ZEP checks, use get_graph_config_errors; add embed/small LLM env docs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-25 21:10:40 +00:00
Ubuntu c074136a87 fix(llm): centralise Azure URL normalisation in parse_azure_url utility
Extract parse_azure_url from graphiti_backend into utils/llm_client so all
services (oasis_profile_generator, simulation_config_generator, graphiti_backend,
LLMClient) strip the /chat/completions suffix and api-version before passing
base_url to the OpenAI SDK, fixing 404 errors when the full Azure endpoint URL
is set in LLM_BASE_URL.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-25 21:09:48 +00:00
Ubuntu e7ebcdff75 fix(graphiti): patch _extract_entity_attributes to flatten nested attr dicts
The previous fix applied _flatten_attributes too broadly, breaking graphiti's
internal ExtractedEntities validation (lists were converted to strings).

Instead, monkey-patch only _extract_entity_attributes — the exact function
that returns entity attribute dicts to node.attributes before Neo4j write.
Lists of primitives are preserved; only dict-valued attributes are flattened.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-25 20:24:08 +00:00
Ubuntu 6bce05dca2 fix(graphiti): flatten nested attribute dicts before Neo4j write; hide summary from properties panel
Graphiti's attribute extraction LLM sometimes returns values wrapped in
nested dicts ({"value": "CTTI"}) instead of plain strings. Neo4j rejects
these with TypeError: "Property values can only be of primitive types".

Fix: after _AzureGenericClient gets the LLM response, validate it through
the Pydantic response_model and call _flatten_attributes() so every value
reaching Neo4j is a scalar. Also add _flatten_attributes() helper.

UI: exclude 'summary' from the Properties section of the node detail panel
since it already appears in the dedicated Summary section below.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-25 20:08:39 +00:00
Ubuntu ba9c8d1f42 feat(ontology): make entity/edge type limits configurable via env vars
Revert default limits back to 10 edges and 12 entities (10 specific + 2
fallback), matching Zep Cloud's maximum. Expose ONTOLOGY_MAX_ENTITY_TYPES
and ONTOLOGY_MAX_EDGE_TYPES in Config so operators can override without
touching code. Prompts now inject the live values at request time.
Also replace Chinese comments in .env.example with English equivalents.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-25 19:56:44 +00:00