fix(RAG): anchor continuation-batch initial progress to overall-file frame (#889)

Each continuation batch of a multi-batch ZIM embed runs as a fresh
BullMQ job, so handle() ran the hardcoded `safeUpdateProgress(job, 5)`
even when the file was already 100k articles into a 600k-article ZIM.
The UI gauge briefly dropped to 5% before the per-batch onProgress
callback caught up to the true overall percentage, reading as a
backward jump every time a new batch started.

Compute initialPercent from batchOffset / totalArticles when available,
falling back to 5 for single-batch files (uploaded PDFs, txts) where
totalArticles isn't set. Capped at 99 to leave headroom for the 100%
final-batch marker.

Follow-up to PR #880 (which fixed the 0-100% scaling during a batch
but still had the initial-frame regression).
This commit is contained in:
chriscrosstalk 2026-05-15 23:01:45 -07:00 committed by GitHub
parent 69cf66c1f3
commit 68c0a37cab
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 10 additions and 2 deletions

View File

@ -85,8 +85,16 @@ export class EmbedFileJob {
logger.info(`[EmbedFileJob] Services ready. Processing file: ${fileName}`)
// Update progress starting
await this.safeUpdateProgress(job, 5)
// Anchor initial progress to where we are in the overall file. For a
// continuation batch midway through a multi-batch ZIM (e.g. offset 100k of
// 600k), the hardcoded 5 used to make the gauge briefly flash 0→5→real,
// which read as a backward jump. Fall back to 5 for single-batch files
// where totalArticles isn't set.
const initialPercent =
totalArticles && totalArticles > 0
? Math.min(99, Math.round(((batchOffset || 0) / totalArticles) * 100))
: 5
await this.safeUpdateProgress(job, initialPercent)
await job.updateData({
...job.data,
status: 'processing',