project-nomad/admin/app/utils
just-jbc 84008663af
fix(rag): add proper .docx text extraction via mammoth (#1100)
* fix(rag): add proper .docx text extraction via mammoth

.docx files were classified as plain text and routed through raw-text
extraction (extractTXTText). Since .docx is a ZIP archive containing XML,
this produced garbage content in the Knowledge Base — XML tags and binary
noise instead of the actual document text.

Adds a dedicated 'docx' file type (split out of the generic 'text' bucket
in determineFileType) and a mammoth-based extractor that parses the
document XML properly.

* fix: clean up package-lock.json diff

* chore(deps): pin mammoth version

---------

Co-authored-by: John Cortright <jcortright@zscaler.com>
Co-authored-by: jakeaturner <jturner@cosmistack.com>
2026-07-20 15:17:50 -07:00
..
content_auto_update_backoff.ts feat(content): opt-in automatic updates for installed ZIM & map content 2026-06-23 04:47:01 +00:00
content_reindex_decision.ts feat(content): opt-in automatic updates for installed ZIM & map content 2026-06-23 04:47:01 +00:00
downloads.ts fix(downloads): send a descriptive User-Agent so Wikimedia mirrors don't 403 (#1114) 2026-07-19 22:37:57 -07:00
fs.ts fix(rag): add proper .docx text extraction via mammoth (#1100) 2026-07-20 15:17:50 -07:00
image_disk_preflight.ts feat(supply-depot): opt-in automatic updates for installed apps 2026-06-23 04:46:52 +00:00
kb_ingest_decision.ts feat(KB): Always/Manual ingest policy toggle (RFC #883 §1/§4) (#894) 2026-05-20 10:16:00 -07:00
kb_job_health.ts feat(KB): status pill + last-activity timestamp on Processing Queue (RFC #883 §5/§10) 2026-05-20 10:16:00 -07:00
kb_ratio_lookup.ts fix(KB): stop partial_stall warning firing on atypical ZIMs (link-out/PDF-heavy) 2026-06-23 04:46:54 +00:00
kb_warning_decision.ts refactor(KB): move FileWarning to shared types/rag following existing convention 2026-05-20 10:16:00 -07:00
misc.ts feat(AI): chat suggestions and assistant settings 2026-02-01 07:24:21 +00:00
superseded_resource.ts fix(content): remove superseded curated map/ZIM files when a new version installs 2026-06-23 04:47:01 +00:00
update_window.ts feat(supply-depot): opt-in automatic updates for installed apps 2026-06-23 04:46:52 +00:00
version.ts feat: support for updating services 2026-03-11 14:08:09 -07:00
zim_filename.ts fix(ZIM): preserve co-existing Wikipedia corpora on cleanup (#884) 2026-05-20 10:16:00 -07:00
zim_html.ts chore(KB): filter non-content sections + render tables in ZIM extraction (#1044) 2026-07-18 09:03:53 -07:00