fix(AI): rewrite RAG query on first follow-up (off-by-one in skip-rewrite threshold)
The short-conversation skip in `rewriteQueryWithContext` used `userMessages.length <= 2`,
which short-circuits both the very first turn AND the first follow-up. The follow-up is
the moment the rewriter matters most — it's where pronouns and shorthand ("the bars",
"how long does it last?") need to be resolved against earlier turns before the embedding
search runs. With the rewriter skipped, RAG queries against the raw last message, scores
nothing above the 0.3 threshold, and no context gets injected for that turn.
The visible symptom is the assistant treating the first follow-up in any chat as a
brand-new question — e.g. "great - they threw up 2 of the bars it looks like" answered
as if it were a recipe-bars question, with no carry-forward of the prior chocolate-
poisoning context.
Threshold lowered to `< 2`: skip only when there's exactly one user message (nothing to
rewrite from). From the first follow-up onward the rewriter runs, as originally intended
before commit 96e5027.
Validated against `mistral-nemo:12b` on NOMAD3 by hot-patching the compiled controller
and replaying the dog-chocolate scenario. Post-patch response correctly threads "3
Hershey's bars" from turn 1 into turn 2's answer; pre-patch (per reporter's screenshot)
pivoted to peanut butter bar recipes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
5694c91e3e
commit
43645e4bbc
|
|
@ -383,10 +383,15 @@ export default class OllamaController {
|
|||
// Get recent conversation history (last 6 messages for 3 turns)
|
||||
const recentMessages = messages.slice(-6)
|
||||
|
||||
// Skip rewriting for short conversations. Rewriting adds latency with
|
||||
// little RAG benefit until there is enough context to matter.
|
||||
// Skip rewriting on the very first turn — with only one user message
|
||||
// there is no prior context to fold in, so the rewrite would just echo
|
||||
// the message back at the cost of an extra LLM round-trip. From the
|
||||
// first follow-up onward we need the rewrite so the RAG query carries
|
||||
// entities and topics from earlier turns ("the bars" → "Hershey's bars
|
||||
// chocolate poisoning dog"); without it, embeddings match nothing and
|
||||
// the assistant loses the thread.
|
||||
const userMessages = recentMessages.filter(msg => msg.role === 'user')
|
||||
if (userMessages.length <= 2) {
|
||||
if (userMessages.length < 2) {
|
||||
return lastUserMessage?.content || null
|
||||
}
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue