project-nomad/admin
John Cortright 68bb510351 fix(rag): reap orphaned qdrant sources during sync (#1170)
scanAndSyncStorage builds sourcesInQdrant and embeddableFiles in the
same pass but only ever checks one direction - is this on-disk file
already embedded? It never checks the reverse, so points left behind
by a deleted or replaced ZIM just sit in Qdrant forever and keep
showing up as search hits for content that doesn't exist anymore.

Added a reverse sweep that diffs the two sets and purges anything left
in Qdrant with no file backing it. Guarded so it's a no-op when the
disk scan comes back empty, so a filesystem hiccup can't get misread
as "delete everything."

Also has to exclude README.md/docs, since those get embedded by
discoverNomadDocs and live outside the kb_uploads/zim paths the
scanner actually walks. First pass without that exclusion would have
wiped the docs KB on every single sync - caught it by testing against
a live install where those sources were already indexed.

Verified live: found exactly one real orphan out of 29 indexed
sources (an old devdocs_en_bash zim superseded by a newer download),
purged it, and left the other 28 - including all the docs - alone.
2026-08-14 15:24:43 -07:00
..
app fix(rag): reap orphaned qdrant sources during sync (#1170) 2026-08-14 15:24:43 -07:00
bin feat: curated content system overhaul 2026-02-11 15:44:46 -08:00
commands feat(eval): standardized RAG quality evaluation harness (#1233) 2026-08-13 11:10:47 -07:00
config fix: lazy-connect + retry for ioredis to avoid blocked startup 2026-06-23 17:40:10 +00:00
constants feat(AI): make RAG retreival optional per chat (#1247) 2026-08-14 11:42:28 -07:00
database feat(benchmark): official multi-arch sysbench, resolved digest, platform metadata (#1158) 2026-08-04 03:39:53 +00:00
docs docs(release): finalize v1.34.0 release notes [skip ci] 2026-08-04 03:46:26 +00:00
inertia feat(AI): make RAG retreival optional per chat (#1247) 2026-08-14 11:42:28 -07:00
providers fix(kiwix): self-heal a missing or corrupt library XML on startup 2026-06-23 04:46:56 +00:00
public feat(creator-packs): add missing Modern Rogue banner asset (#1147) 2026-08-04 03:39:52 +00:00
resources chore: standardize brand name to Project NOMAD, retire backronym 2026-08-04 03:37:32 +00:00
scripts refactor(drug-reference): make collections JSON the single source for curated data (#1130) 2026-08-04 03:38:35 +00:00
start feat: auto-generating OpenAPI docs with Scalar UI (#1128) 2026-08-04 03:38:33 +00:00
tests fix(rag): reap orphaned qdrant sources during sync (#1170) 2026-08-14 15:24:43 -07:00
types feat(AI): make RAG retreival optional per chat (#1247) 2026-08-14 11:42:28 -07:00
util Add offline FDA drug reference (labels, interaction view, conditions, remedies) (#1040) 2026-08-04 03:38:33 +00:00
views chore: standardize brand name to Project NOMAD, retire backronym 2026-08-04 03:37:32 +00:00
.editorconfig feat: initial commit 2025-06-29 15:51:08 -07:00
.env.example feat(creator-packs): gated per-creator video packs, offline via Kiwix (#1106) 2026-08-04 03:38:27 +00:00
ace.js feat: initial commit 2025-06-29 15:51:08 -07:00
adonisrc.ts feat(GPU): auto-remediate nomad_ollama passthrough loss on admin boot (#755) 2026-05-20 10:16:00 -07:00
eslint.config.js feat: openwebui+ollama and zim management 2025-07-09 09:08:21 -07:00
package-lock.json chore(deps): bump axios and systeminformation in /admin 2026-08-04 03:38:40 +00:00
package.json feat(eval): standardized RAG quality evaluation harness (#1233) 2026-08-13 11:10:47 -07:00
tailwind.config.ts feat: initial commit 2025-06-29 15:51:08 -07:00
tsconfig.json Add offline FDA drug reference (labels, interaction view, conditions, remedies) (#1040) 2026-08-04 03:38:33 +00:00
vite.config.ts fix(Maps): ensure proper parsing of hostnames (#640) 2026-04-03 14:26:50 -07:00