project-nomad/admin
just-jbc 4f4cc5bc65
feat(rag): add subject/collection organization to knowledge base (#1063)
* feat(rag): add subject/collection organization to knowledge base

- Add nullable collection field to KbIngestState, propagated through the
  embed job, RAG service, and Qdrant point payloads (indexed for filtering)
- Add upload-time category selection and per-file collection reassignment
  in the Knowledge Base modal, with a filterable Stored Files table
- Add a 'Search in' collection filter to the chat interface, threaded
  through to searchSimilarDocuments as an optional Qdrant filter
- Fix .docx extraction: previously routed through raw-text extraction
  (garbage output for a ZIP-based XML format); adds a proper mammoth-based
  extractor and a dedicated 'docx' file-type case

* feat(rag): support dynamic KB collection creation, rename, and removal

Extends collection organization with a Manage Collections UI: collections
are created on the fly when a file is assigned to a new name, can be
renamed (bulk-updates every tagged file and Qdrant point), and can be
removed (reassigns tagged files back to Uncategorized rather than
deleting anything).

* fix(rag): use dynamic collections query in chat search filter

chat/index.tsx still imported the static KB_COLLECTIONS constant for its
'Search in' dropdown, inconsistent with KnowledgeBaseModal.tsx which already
uses the live getKnowledgeCollections() query. Renamed/added collections
via the new Manage Collections UI weren't reflected in the chat filter.

* feat(rag): broaden preset tags and add creatable collection combobox

Replaces the survival-specific preset list with general-purpose starter
tags (recipes, diy, health, technology, finance, travel, hobbies,
reference, survival, energy) so the Knowledge Base reads well for
home-lab/reference use, not just prepping.

Adds sanitizeCollectionName() (trim, lowercase, length cap) applied on
every write path server-side, and a dependency-free CollectionCombobox
component replacing the plain <select> + window.prompt pattern for
tagging — autocompletes against presets + tags already in use, with a
'+ Create' option for anything new.

* chore(rag): remove .docx fix from this branch, split into #1100

Per review feedback, the .docx extraction fix is unrelated to the
collections feature and can merge independently. Moved to a standalone
PR (Crosstalk-Solutions/project-nomad#1100) off dev.

* chore: remove unrelated diff noise (lockfile, comments, indentation)

---------

Co-authored-by: John Cortright <jcortright@zscaler.com>
2026-07-20 15:09:11 -07:00
..
app feat(rag): add subject/collection organization to knowledge base (#1063) 2026-07-20 15:09:11 -07:00
bin feat: curated content system overhaul 2026-02-11 15:44:46 -08:00
commands feat(content): opt-in automatic updates for installed ZIM & map content 2026-06-23 04:47:01 +00:00
config fix: lazy-connect + retry for ioredis to avoid blocked startup 2026-06-23 17:40:10 +00:00
constants feat(rag): add subject/collection organization to knowledge base (#1063) 2026-07-20 15:09:11 -07:00
database feat(rag): add subject/collection organization to knowledge base (#1063) 2026-07-20 15:09:11 -07:00
docs chore: standardize brand name to Project NOMAD, retire backronym 2026-07-18 09:56:54 -07:00
inertia feat(rag): add subject/collection organization to knowledge base (#1063) 2026-07-20 15:09:11 -07:00
providers fix(kiwix): self-heal a missing or corrupt library XML on startup 2026-06-23 04:46:56 +00:00
public feat: switch all PNG images to WEBP (#575) 2026-04-03 14:26:50 -07:00
resources chore: standardize brand name to Project NOMAD, retire backronym 2026-07-18 09:56:54 -07:00
scripts feat(supply-depot): add MeshCore Web with self-signed HTTPS 2026-06-23 04:47:06 +00:00
start feat(rag): add subject/collection organization to knowledge base (#1063) 2026-07-20 15:09:11 -07:00
tests feat(rag): add subject/collection organization to knowledge base (#1063) 2026-07-20 15:09:11 -07:00
types feat(rag): add subject/collection organization to knowledge base (#1063) 2026-07-20 15:09:11 -07:00
util feat: display model download progress 2026-02-06 16:22:23 -08:00
views chore: standardize brand name to Project NOMAD, retire backronym 2026-07-18 09:56:54 -07:00
.editorconfig feat: initial commit 2025-06-29 15:51:08 -07:00
.env.example fix(system): prevent false offline reports when Cloudflare endpoint is unreachable 2026-06-23 04:47:01 +00:00
ace.js feat: initial commit 2025-06-29 15:51:08 -07:00
adonisrc.ts feat(GPU): auto-remediate nomad_ollama passthrough loss on admin boot (#755) 2026-05-20 10:16:00 -07:00
eslint.config.js feat: openwebui+ollama and zim management 2025-07-09 09:08:21 -07:00
package-lock.json chore(deps): bump tar, vite, and dockerode in admin 2026-07-20 05:56:41 +00:00
package.json chore(deps): bump tar, vite, and dockerode in admin 2026-07-20 05:56:41 +00:00
tailwind.config.ts feat: initial commit 2025-06-29 15:51:08 -07:00
tsconfig.json feat: initial commit 2025-06-29 15:51:08 -07:00
vite.config.ts fix(Maps): ensure proper parsing of hostnames (#640) 2026-04-03 14:26:50 -07:00