From 43645e4bbcc5c0d9860f3413c727f85caeb1aa03 Mon Sep 17 00:00:00 2001 From: Chris Sherwood Date: Tue, 12 May 2026 12:47:34 -0700 Subject: [PATCH] fix(AI): rewrite RAG query on first follow-up (off-by-one in skip-rewrite threshold) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The short-conversation skip in `rewriteQueryWithContext` used `userMessages.length <= 2`, which short-circuits both the very first turn AND the first follow-up. The follow-up is the moment the rewriter matters most — it's where pronouns and shorthand ("the bars", "how long does it last?") need to be resolved against earlier turns before the embedding search runs. With the rewriter skipped, RAG queries against the raw last message, scores nothing above the 0.3 threshold, and no context gets injected for that turn. The visible symptom is the assistant treating the first follow-up in any chat as a brand-new question — e.g. "great - they threw up 2 of the bars it looks like" answered as if it were a recipe-bars question, with no carry-forward of the prior chocolate- poisoning context. Threshold lowered to `< 2`: skip only when there's exactly one user message (nothing to rewrite from). From the first follow-up onward the rewriter runs, as originally intended before commit 96e5027. Validated against `mistral-nemo:12b` on NOMAD3 by hot-patching the compiled controller and replaying the dog-chocolate scenario. Post-patch response correctly threads "3 Hershey's bars" from turn 1 into turn 2's answer; pre-patch (per reporter's screenshot) pivoted to peanut butter bar recipes. Co-Authored-By: Claude Opus 4.7 (1M context) --- admin/app/controllers/ollama_controller.ts | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/admin/app/controllers/ollama_controller.ts b/admin/app/controllers/ollama_controller.ts index 68b6b78..cac3994 100644 --- a/admin/app/controllers/ollama_controller.ts +++ b/admin/app/controllers/ollama_controller.ts @@ -383,10 +383,15 @@ export default class OllamaController { // Get recent conversation history (last 6 messages for 3 turns) const recentMessages = messages.slice(-6) - // Skip rewriting for short conversations. Rewriting adds latency with - // little RAG benefit until there is enough context to matter. + // Skip rewriting on the very first turn — with only one user message + // there is no prior context to fold in, so the rewrite would just echo + // the message back at the cost of an extra LLM round-trip. From the + // first follow-up onward we need the rewrite so the RAG query carries + // entities and topics from earlier turns ("the bars" → "Hershey's bars + // chocolate poisoning dog"); without it, embeddings match nothing and + // the assistant loses the thread. const userMessages = recentMessages.filter(msg => msg.role === 'user') - if (userMessages.length <= 2) { + if (userMessages.length < 2) { return lastUserMessage?.content || null }