project-nomad/admin/tests
chriscrosstalk 8f56d76fe7
chore(KB): filter non-content sections + render tables in ZIM extraction (#1044)
Closes #902.

Two gaps in the structured ZIM extraction path:

1. NON_CONTENT_HEADING_PATTERNS was only used by the structure heuristic to
   count meaningful sections, never at section-emit time. Sections under
   "See also" / "References" / "External links" / etc. were still chunked and
   embedded. They're now flagged when the heading opens and dropped.

2. <table> elements were run through cheerio's `.text()`, concatenating every
   cell with no separators ("AgeDoseAdult500mg") into unsearchable word salad.
   New tableToText() joins cells with " | " and rows with newlines so
   row/column structure survives into the chunk.

Refactor: moved extractStructuredContent out of ZIMExtractionService into a
pure, cheerio-only util (app/utils/zim_html.ts) so it can be unit-tested
without the native @openzim/libzim binding. Service delegates to it; behavior
is otherwise unchanged. Adds tests/unit/zim_html.spec.ts (6 tests).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 09:03:53 -07:00
..
unit chore(KB): filter non-content sections + render tables in ZIM extraction (#1044) 2026-07-18 09:03:53 -07:00
bootstrap.ts feat: initial commit 2025-06-29 15:51:08 -07:00