Loop 4 response to Vineeth's CHANGES_REQUESTED on PR #573.
The pre-Loop-4 enqueue-time write of last_dream_document_count was serving
double duty: rate limiter AND stampede latch. By arming the 8h guard the
moment a dream entered the pipeline, it implicitly blocked a second dream
from being scheduled during the in-flight window. Loop 3 relocated the
last_dream_at write to completion without moving its sibling baseline,
splitting the semantic pair and exposing the latch role that had lived
only in Vineeth's head.
Invariant (now pinned to check_and_schedule_dream's docstring): from the
moment a dream is scheduled until it completes or fails, no second dream
may be enqueued for the same (workspace, observer, observed) — and the
baseline count advances only when consolidation actually happened.
Changes:
- enqueue_dream: remove the last_dream_document_count write entirely and
drop the document_count parameter. enqueue no longer touches dream
metadata; the implicit stampede latch is replaced by an explicit
queue-backed defense.
- process_dream: extend the existing row-locked RMW to write both guard
fields atomically. Current explicit-doc count is recomputed inside the
locked block (not carried on DreamPayload) so the pair reflects the
actual consolidation moment.
- check_and_schedule_dream: query QueueItem for pending dreams on this
collection's work_unit_keys (mirrors uq_queue_dream_pending_work_unit_key)
before arming a timer. Uses queue state as source of truth rather than
reflecting it into metadata.
- Tests: two new coherence tests under TestGuardPairCoherence —
test_pending_queue_item_blocks_second_schedule walks the stampede timeline,
test_silent_failure_allows_retry_on_same_corpus verifies failed dreams
don't consume the baseline. Existing tests updated to the new contract.
d24958dd7c