project-nomad/admin/tests/unit
chriscrosstalk 8f56d76fe7
chore(KB): filter non-content sections + render tables in ZIM extraction (#1044)
Closes #902.

Two gaps in the structured ZIM extraction path:

1. NON_CONTENT_HEADING_PATTERNS was only used by the structure heuristic to
   count meaningful sections, never at section-emit time. Sections under
   "See also" / "References" / "External links" / etc. were still chunked and
   embedded. They're now flagged when the heading opens and dropped.

2. <table> elements were run through cheerio's `.text()`, concatenating every
   cell with no separators ("AgeDoseAdult500mg") into unsearchable word salad.
   New tableToText() joins cells with " | " and rows with newlines so
   row/column structure survives into the chunk.

Refactor: moved extractStructuredContent out of ZIMExtractionService into a
pure, cheerio-only util (app/utils/zim_html.ts) so it can be unit-tested
without the native @openzim/libzim binding. Service delegates to it; behavior
is otherwise unchanged. Adds tests/unit/zim_html.spec.ts (6 tests).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 09:03:53 -07:00
..
app_auto_update.spec.ts feat(supply-depot): opt-in automatic updates for installed apps 2026-06-23 04:46:52 +00:00
cloud_metadata_url.spec.ts fix(security): canonicalize hostnames to block IPv4-mapped IPv6 IMDS bypass 2026-05-20 10:16:00 -07:00
content_auto_update.spec.ts feat(content): opt-in automatic updates for installed ZIM & map content 2026-06-23 04:47:01 +00:00
content_auto_update_backoff.spec.ts feat(content): opt-in automatic updates for installed ZIM & map content 2026-06-23 04:47:01 +00:00
content_reindex_decision.spec.ts feat(content): opt-in automatic updates for installed ZIM & map content 2026-06-23 04:47:01 +00:00
custom_app_guard.spec.ts feat: supply depot 2026-06-23 04:46:45 +00:00
global_map_banner.spec.ts fix(UI): improve global map banner display logic (#702) 2026-05-20 10:16:00 -07:00
kb_file_grouping.spec.ts feat(KnowledgeBase): add document viewer, download, metadata, and sorting (#721) 2026-06-23 04:47:09 +00:00
kb_guardrail.spec.ts feat(KB): guardrail modal at 50GB / 10%-free thresholds (RFC #883 §7) 2026-05-20 10:16:00 -07:00
kb_ingest_decision.spec.ts feat(KB): Always/Manual ingest policy toggle (RFC #883 §1/§4) (#894) 2026-05-20 10:16:00 -07:00
kb_job_health.spec.ts feat(KB): status pill + last-activity timestamp on Processing Queue (RFC #883 §5/§10) 2026-05-20 10:16:00 -07:00
kb_ratio_lookup.spec.ts fix(KB): stop partial_stall warning firing on atypical ZIMs (link-out/PDF-heavy) 2026-06-23 04:46:54 +00:00
kb_warning_decision.spec.ts feat(KB): conditional warnings A + B on Stored Files (RFC #883 §6) 2026-05-20 10:16:00 -07:00
private_url.spec.ts fix(security): harden assertNotPrivateUrl with ipaddr.js + host normalization 2026-06-23 04:46:55 +00:00
superseded_resource.spec.ts fix(content): remove superseded curated map/ZIM files when a new version installs 2026-06-23 04:47:01 +00:00
zim_filename.spec.ts fix(ZIM): preserve co-existing Wikipedia corpora on cleanup (#884) 2026-05-20 10:16:00 -07:00
zim_html.spec.ts chore(KB): filter non-content sections + render tables in ZIM extraction (#1044) 2026-07-18 09:03:53 -07:00