RomanNum3ral
/
project-nomad
mirror of https://github.com/Crosstalk-Solutions/project-nomad.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

fix(KB): respect Manual ingest policy on post-download dispatch 

RunDownloadJob's onComplete handler was unconditionally firing
EmbedFileJob.dispatch after every ZIM download, gated only by "is
Ollama installed?". The rag.defaultIngestPolicy KV setting was never
consulted, so users who explicitly set Auto-index to Manual still got
every newly-downloaded ZIM auto-embedded.

RagService.scanAndSync already handles Manual correctly by recording
pending_decision rows instead of dispatching (rag_service.ts:1587-1638
via decideScanAction). The post-download path skipped that gate.

Mirror the same check at the dispatch site: read the policy KV; if
Manual, firstOrCreate a pending_decision row in kb_ingest_state so the
per-file Index affordance from PR #909 surfaces the file the same way
scan-time-discovered Manual files do. firstOrCreate (not create) so a
re-download doesn't demote an existing indexed/failed row — the user
can explicitly re-index from the KB panel if they want fresh content.

Verified on NOMAD3: with rag.defaultIngestPolicy='Manual', every ZIM
downloaded today via Content Explorer (agriculture-essential +
computing-essential, ~62 MB across 7 files) wrote kb_ingest_state
rows with state='indexed' instead of pending_decision. Real bug,
not a hot-patch artifact.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Sherwood 2026-05-19 12:46:03 -07:00 committed by Jake Turner
parent 059cf2afbe
commit a5fe52f66f
1 changed files with 34 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

41
admin/app/jobs/run_download_job.ts
View File

@ -147,13 +147,40 @@ export class RunDownloadJob {
// Only dispatch embedding job if AI Assistant (Ollama) is installed
const ollamaUrl = await dockerService.getServiceURL('nomad_ollama')
if (ollamaUrl) {
try {
await EmbedFileJob.dispatch({
fileName: url.split('/').pop() || '',
filePath: filepath,
})
} catch (error) {
console.error(`[RunDownloadJob] Error dispatching EmbedFileJob for URL ${url}:`, error)
// Respect the global ingest policy. Under Manual, record the file
// as pending_decision so the KB panel surfaces the per-file Index
// affordance (PR #909) instead of silently auto-embedding behind
// the user's back. Unset is treated as Always to preserve legacy
// behavior — mirrors rag_service.ts:1587-1588.
const { default: KVStore } = await import('#models/kv_store')
const { default: KbIngestState } = await import('#models/kb_ingest_state')
const policyRaw = await KVStore.getValue('rag.defaultIngestPolicy')
const policy: 'Always' | 'Manual' = policyRaw === 'Manual' ? 'Manual' : 'Always'
if (policy === 'Manual') {
try {
// firstOrCreate so a re-download doesn't demote an existing
// indexed/failed row — user keeps prior state and can re-index
// explicitly from the KB panel if they want fresh content.
await KbIngestState.firstOrCreate(
{ file_path: filepath },
{ file_path: filepath, state: 'pending_decision', chunks_embedded: 0 }
)
} catch (error) {
console.error(
`[RunDownloadJob] Error recording pending_decision state for ${filepath}:`,
error
)
}
} else {
try {
await EmbedFileJob.dispatch({
fileName: url.split('/').pop() || '',
filePath: filepath,
})
} catch (error) {
console.error(`[RunDownloadJob] Error dispatching EmbedFileJob for URL ${url}:`, error)
}
}
}
} else if (filetype === 'map') {