Commit Graph

12 Commits

Author SHA1 Message Date
Chris Sherwood 7ff6d5c37c
feat(KB): surface embedding-disk estimate in curated tier-change modal (RFC #883 §1)
When a user picks a tier in TierSelectionModal, show how much additional
disk space the AI Assistant will need if the new ZIMs are indexed, plus
a policy-aware footer explaining whether they'll auto-index (Always) or
wait for opt-in (Manual). Estimates consume #891's KbRatioRegistry via a
new POST /api/rag/estimate-batch endpoint.

Backend
- New POST /api/rag/estimate-batch route + RagController.estimateBatch
- VineJS schema accepting array of {filename, sizeBytes}, capped at 500
- KbRatioRegistry.estimateBatch aggregates via the existing prefix-match
  lookup, returns {totalChunks, totalBytes, hasUnknown}
- New BYTES_PER_CHUNK_ON_DISK constant (~8 KB: 3 KB vector + ~3 KB chunk
  text + ~2 KB payload/index overhead). Tunable; will be replaced by
  Phase 4 self-calibration once we have real measurements.
- Controller normalizes incoming filenames via path.basename so callers
  that send full paths or URLs still match registry prefixes correctly.

Frontend
- api.estimateEmbeddingBatch() client method
- TierSelectionModal: when localSelectedSlug is set, resolve the tier's
  resources (incl. inherited tiers), POST to /estimate-batch, and render
  a new info block with the +~X GB figure + ingest-policy copy. Also
  fetches rag.defaultIngestPolicy so the same block surfaces whether
  indexing will fire automatically or wait for the user.
- resourceFilename() helper extracts the basename from the resource URL
  so the registry lookup hits the right prefix regardless of mirror.

Tests
- 4 new cases in tests/unit/kb_ratio_lookup.spec.ts covering the
  estimateBatch aggregator: standard sum, unknown-flagging, video-only
  ZIM (0 chunks but known, hasUnknown stays false), empty input.

Stacks on feat/kb-ratio-registry (#891) — consumes the registry table
seeded by that PR. Once #891 merges to rc, this PR auto-rebases.

Out of scope for this PR (deferred to follow-ups):
- Per-batch opt-in checkbox (RFC §1's '☑ Also index these for AI') needs
  a per-batch policy override path and is a separate PR
- Guardrail modal at 50 GB / 10% free / 6 hr thresholds (RFC §7) is also
  separate; this PR is informational, not gating
- Time-to-embed estimate awaits a chunks-per-second metric per host
2026-05-20 17:14:42 +00:00
Jake Turner 7ba733a425
fix(KB): add re-embed and reset & rebuild opts to fix broken embeddings (#886) 2026-05-20 17:14:37 +00:00
Henry Estela d1dc8faf49
fix(RAG): add start button in kb modal and ensure restart policy exists (#700)
Adds a check to RAG health to make sure nomad_qdrant is online, if not
then the user will be blocked from clicking any buttons in the KB modal
until they click the start qdrant button and let the container start

There is a new file qdrant_restart_policy_provider.ts, which tries to
ensure that the restart policy always exists for the nomad_qdrant
container even though the policy should have been there when the
container is created.
2026-05-20 17:12:40 +00:00
Luís Miguel 806b2c1714 fix(security): SSRF validation for map downloads and error sanitization (CWE-918, CWE-209) (#552)
* fix(security): add SSRF validation to map download URLs from manifest
* fix(security): sanitize verbose error in rag controller scan endpoint
* fix(security): sanitize verbose errors in benchmark controller
* fix(security): sanitize verbose error in system controller version fetch
* fix(security): sanitize verbose errors in chats controller (6 instances)
* fix(security): sanitize verbose errors in docker service (6 instances)
* fix(security): sanitize verbose error in system update service
* fix(security): sanitize verbose errors in collection update service
---------
Co-authored-by: Jake Turner <52841588+jakeaturner@users.noreply.github.com>
2026-04-21 14:26:28 -07:00
Henry Estela 69c15b8b1e feat(AI): enable remote AI chat host 2026-04-03 14:26:50 -07:00
Jake Turner dfa896e86b feat(RAG): allow deletion of files from KB 2026-03-04 20:05:14 -08:00
Jake Turner 99b96c3df7 feat(RAG): display embedding queue and improve progress tracking 2026-03-04 20:05:14 -08:00
Jake Turner 4747863702 feat(AI Assistant): allow manual scan and resync KB 2026-02-09 15:16:18 -08:00
Jake Turner 8726700a0a feat: zim content embedding 2026-02-08 13:20:10 -08:00
Jake Turner d1f40663d3 feat(RAG): initial beta with preprocessing, embedding, semantic retrieval, and ctx passage 2026-02-01 23:59:21 +00:00
Jake Turner 243f749090 feat: [wip] native AI chat interface 2026-01-31 20:39:49 -08:00
Jake Turner 50174d2edb feat(RAG): [wip] RAG capabilities 2026-01-31 20:39:49 -08:00