MicroFish/.kiro/specs/graph-build-empty-fix/gap-analysis.md

Gap Analysis: graph-build-empty-fix

Scope Snapshot

The fix is small in code surface (config defaults, embedder construction, docs) but research-heavy on root cause (why does Graphiti add_episode appear to succeed yet leave Neo4j empty?). The Ollama documentation, OpenAI-compatible embedder support, and loud add_batch failure already exist from spec graphiti-ollama-embedder (issue #18). What's still missing: flipping the active default to Ollama and confirming the dimension-mismatch hypothesis that drives the empty-graph symptom.

Current State

Relevant Assets

• backend/app/services/graphiti_adapter.py:92–139 — _build_llm_and_embedder constructs OpenAI or Gemini providers. The OpenAI branch reads Config.EMBEDDING_BASE_URL or Config.LLM_BASE_URL and Config.EMBEDDING_API_KEY or Config.LLM_API_KEY. This branch is already Ollama-compatible (Ollama exposes an OpenAI-shaped /v1/embeddings).
• backend/app/services/graphiti_adapter.py:466–486 — _GraphNamespace.add_batch re-raises on episode-ingestion failures (spec #18). No placeholder UUIDs. Logger is ERROR with traceback.
• backend/app/services/graph_builder.py:227–230 — _build_graph_worker catches Exception, captures traceback, calls TaskManager().fail_task(task_id, error_msg).
• backend/app/__init__.py:88–109 — _recover_stuck_projects gates promotion to GRAPH_COMPLETED on count(n:Entity {group_id}) > 0. Matches Req 4 AC5 already.
• backend/app/config.py:42, 52–53 — current defaults:
  • EMBEDDING_MODEL = 'text-embedding-3-small' (OpenAI, 1536-dim)
  • EMBEDDING_BASE_URL = None → falls back to LLM_BASE_URL
  • EMBEDDING_API_KEY = None → falls back to LLM_API_KEY
• README.md:163–183 — Ollama section present but commented out; OpenAI defaults are still the active path.
• CLAUDE.md:72–80 — already names mxbai-embed-large (1024-dim) and explicitly rules out 768-dim nomic-embed-text. Documentation framing already treats Ollama as a supported provider.
• docker-compose.yml:31–33 — already notes the host.docker.internal:11434 reach-through for Ollama.

Conventions in Play

• All Neo4j/Graphiti access goes through services/graphiti_adapter.py (per .kiro/steering/database.md).
• Configuration is centralized in backend/app/config.py — env-driven, single file.
• Background-task error handling: worker try/except → fail_task(task_id, str(e)) (per .kiro/steering/error-handling.md).
• Graph is multi-tenant by group_id; every read/write must be scoped.

Integration Surfaces Out of This Repo

• graphiti-core package — owns EMBEDDING_DIM = 1024, Neo4j vector index DDL, and add_episode LLM-extraction → embedding → write pipeline. Not vendored here; behavior must be inferred from runtime + their public API.
• Local Ollama daemon (operator-managed) — out of scope to bundle, in scope to assume runs at host:11434.

Requirement-to-Asset Map

Requirement	Asset / Touchpoint	Gap
R1 Root cause	_build_llm_and_embedder, add_batch, add_episode runtime behavior, Neo4j vector-index dim	Unknown — need a reproduction run on default main config to capture the exact failure surface (dimension mismatch vs. embedder 404 vs. silent LLM-extraction-returns-empty).
R2 Local-default	config.py:42, 52–53, .env.example (protected — operator will reload), README.md:163–183	Missing — defaults still point to OpenAI; need to flip to Ollama (mxbai-embed-large @ http://localhost:11434/v1) and demote OpenAI/Gemini to commented fallbacks.
R3 Dimension consistency	graphiti-core's EMBEDDING_DIM = 1024 (external constant); CLAUDE.md:72–80 documentation	Constraint — keep dim at 1024, don't expose a tunable. The Ollama default mxbai-embed-large is 1024-dim, so the defaults align.
R4 Loud failure on every silent path	add_batch (already loud), _build_llm_and_embedder (no pre-flight), graph_builder._wait_for_episodes (polls a no-op episode.get)	Constraint + possibly Missing — if R1 turns up an additional silent-failure call site (likely candidates: graphiti add_episode swallowing extraction failures, or a dim-mismatch returning soft errors), add a remediation there.
R5 Backwards compatibility	_build_llm_and_embedder OpenAI/Gemini branches	Constraint — no logic change required; only defaults change. Operator's explicit EMBEDDING_* settings continue to win.
R6 Documentation	README.md:163–183, CLAUDE.md:72–80, docker-compose.yml:31–33, .env.example (protected)	Missing — flip the README from "Ollama as commented option" to "Ollama as active default, OpenAI/Gemini commented fallback"; CLAUDE.md needs a small wording tweak; .env.example requires operator-coordinated edit (file is hook-protected).
R7 End-to-end smoke	Graph build → env setup (profile_generator) → report agent tools — not directly modified, just exercised	Constraint — requires a representative seed file. PR description documents whether the smoke test ran.

Implementation Approach Options

Option A — Defaults-Only Flip (extend existing)

Change backend/app/config.py defaults and .env.example + README/CLAUDE.md/docker-compose comments. No code-path changes to _build_llm_and_embedder (the "openai" branch already serves Ollama). Optionally add a one-line ERROR log in graphiti_adapter._build_llm_and_embedder when EMBEDDING_BASE_URL is unset, warning that the LLM base URL is being reused (which is a known dim/model mismatch trap with Dashscope/Qwen).

Trade-offs

• ✅ Tiny, reversible, matches conventions.
• ✅ Fully relies on existing loud-failure plumbing.
• ❌ If R1 turns up a silent path inside add_episode itself (graphiti-core), defaults-only does not fix it.

Option B — Defaults Flip + Pre-flight Embedder Probe (extend existing + small new helper)

Same as A, plus a one-shot embedder ping during _get_graphiti() initialization: synchronously call the configured embedder on a known string and assert the response length matches Graphiti's EMBEDDING_DIM. On mismatch or connectivity failure, raise so the first Project creation surfaces the error rather than the first graph-build worker.

Trade-offs

• ✅ Surfaces dimension/connectivity bugs before a long graph build runs.
• ✅ Avoids per-batch "is this even reachable?" guessing.
• ❌ Requirements explicitly call out "no startup-time embedder health probe that refuses to boot" (Boundary Context, out of scope). This option contradicts that boundary.

Option C — Defaults Flip + First-Batch Failure Surfacing (hybrid, recommended)

Option A, plus targeted hardening based on what R1 reveals. Likely candidates if root cause is a graphiti-core silent path:

• Wrap the first add_episode call with an explicit dimension-check on the produced embedding (compare against EMBEDDING_DIM) and raise a clear ValueError("embedding dim mismatch") from add_batch before the Neo4j write, so the worker fails the task with an actionable message.
• Tighten _get_graph_info such that complete_task is gated on a non-zero node count (Req 4 AC5), so a "succeeded but empty" graph never reaches GRAPH_COMPLETED.

Trade-offs

• ✅ Targets the actual failure mode identified by R1 instead of speculating.
• ✅ Stays within the boundary (no startup probe, no new env var, no new provider).
• ✅ Matches existing conventions: a small if not nodes: raise inside the worker, propagated to fail_task.
• ❌ Slightly larger PR than A; the dim-check helper is new code (10–20 lines).

Effort & Risk

• Effort: S (1–3 days) — code change is small; majority of the work is the root-cause repro + smoke-testing the end-to-end pipeline with a local Ollama instance.
• Risk: Medium — relies on a Graphiti-core/Neo4j interaction that we don't fully control. If the root cause is upstream and only fixable via a graphiti-core version bump, scope creeps. Mitigation: if upstream fix is required, capture it in the PR description and ship the defaults-flip + first-batch dim check now; the loud failure ensures operators see the real error rather than an empty graph.

Research Items for Design Phase

1. Confirm the exact silent-failure call site. Run a fresh build on main's default .env and trace where the entity-extraction-or-write disappears: graphiti-core LLM extraction stage, the embedder call, or the Neo4j vector-index write. Log/instrument as needed.
2. Verify the embedder-output dimension at runtime. With mxbai-embed-large via Ollama, confirm len(embedding) == 1024. With text-embedding-3-small, confirm 1536, and observe what Neo4j (or graphiti-core's vector-index check) does with the mismatch.
3. Decide whether _get_graph_info gating belongs in this PR. If R1 root cause is fully addressed by the defaults flip, the node_count > 0 gate in complete_task is belt-and-braces. If R1 reveals a residual silent path, the gate becomes essential.

Recommendations for Design Phase

• Preferred approach: Option C — flip defaults, instrument the first batch enough to capture R1 evidence, gate complete_task on a non-zero node count.
• Key decisions to lock in design:
  • Concrete default values for EMBEDDING_MODEL, EMBEDDING_BASE_URL, EMBEDDING_API_KEY in backend/app/config.py.
  • Whether _get_graph_info(graph_id) returning node_count == 0 should raise inside the worker (Req 4 AC5) or only when add_batch succeeded — the latter is the right semantics.
  • Wording for the README / CLAUDE.md flip: keep both providers documented; only change the active line.
• Carry forward to implementation:
  • .env.example is hook-protected; either coordinate with the developer to update it manually, or document the required diff in HANDOFF.md.
  • The end-to-end smoke test (graph build → profile generation → report query) needs a representative seed file; if unavailable, mark that explicitly in the PR.