* fix(dreamer): threshold and time-guard semantics
Finding 2: filter count_stmt on documents.level == 'explicit' in
check_and_schedule_dream. Dreamer-created levels (deductive, inductive,
contradiction) are consolidation output, not input, and would otherwise
inflate the threshold count and create a feedback loop.
Finding 3 (code-level): relocate last_dream_at write from enqueue_dream
(enqueue.py) to process_dream (orchestrator.py), inside the
'if result is not None' block. Duplicate enqueues can no longer reset
the 8-hour time guard clock. Failed/never-run dreams don't advance it.
Success criteria: lenient (any non-null DreamResult counts). Pending
Vineeth confirmation — will adjust to strict/middle if requested.
Tests pending in follow-up commits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(dreamer): threshold filter + last_dream_at relocation regression tests
Tests for Finding 2 and Finding 3 (code-level):
- TestThresholdFilter (tests/dreamer/test_dream_scheduler.py):
* Mixed levels below explicit threshold: 30 explicit + 40 deductive
+ 10 inductive → no trigger (core regression, buggy count would trigger)
* Explicit-only at threshold: 60 explicit → triggers
* Contradiction excluded: 100 contradiction + 10 explicit → no trigger
(confirms positive == "explicit" filter excludes all dreamer output)
- TestEnqueueDreamMetadataShape (tests/deriver/test_enqueue_dream.py):
* AsyncMock-patched update_collection_internal_metadata verifies
enqueue writes last_dream_document_count but NOT last_dream_at
- TestLastDreamAtCompletionWrite (tests/dreamer/test_dreamer_integration.py):
* Happy path: run_dream returns DreamResult → last_dream_at written
* Failure path: run_dream returns None → last_dream_at absent
* Exception path: run_dream raises → last_dream_at absent,
process_dream swallows exception (queue-processed semantics preserved)
Docstring on check_and_schedule_dream tightened: "document threshold"
-> "explicit-observation threshold" to reflect filter semantics.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(dreamer): preserve last_dream_document_count in completion write
CodeRabbit caught this: update_collection_internal_metadata uses a
top-level JSONB `||` merge, so passing {"dream": {"last_dream_at": ...}}
replaces the entire "dream" subkey and drops last_dream_document_count
that was written by enqueue_dream.
Symptom: after every completed dream, the baseline drops to 0. Next
check_and_schedule_dream reads documents_since_last_dream as
current_count - 0 = current_count, so any collection with >= 50
explicit observations can re-trigger immediately once the 8h guard
expires, even with no new raw material.
Fix: read-modify-write. Fetch current collection, merge last_dream_at
into the existing "dream" dict, write the merged dict back. Preserves
sibling keys (current: last_dream_document_count; future-proof for
telemetry fields that might land in PR 4).
Regression test added to tests/dreamer/test_dreamer_integration.py:
pre-seeds {"dream": {"last_dream_document_count": 42}}, runs
process_dream, asserts both last_dream_at is written AND
last_dream_document_count == 42 is preserved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(dreamer): address CodeRabbit feedback on
|
||
|---|---|---|
| .. | ||
| README.md | ||
| __init__.py | ||
| conftest.py | ||
| test_deriver_processing.py | ||
| test_enqueue_dream.py | ||
| test_queue_operations.py | ||
| test_queue_processing.py | ||
| test_representation_crud.py | ||
| test_vector_reconciliation.py | ||
README.md
Deriver Testing
This directory contains tests for the deriver system, which handles background processing of messages to extract insights and update working representations.
Structure
conftest.py- Shared fixtures for deriver testingtest_queue_operations.py- Tests for basic queue operationstest_deriver_processing.py- Tests for deriver processing logictest_queue_processing.py- Tests for queue manager and work unit processing
Key Fixtures
Database Fixtures
sample_session_with_peers- Creates a session with multiple peers having different observation configurationssample_messages- Creates sample messages for testingsample_queue_items- Creates queue items with various payload types (representation, summary)
Queue Fixtures
create_queue_payload- Helper to create queue payloads for testingadd_queue_items- Helper to add queue items to the databasecreate_active_queue_session- Helper to create active queue sessions for work unit tracking
Mocking Fixtures
mock_critical_analysis_call- Mocks the critical analysis LLM callmock_queue_manager- Mocks the queue manager for testingmock_representation_manager- Mocks the representation manager operations
Testing Patterns
Creating Queue Items
# Create representation payloads
payload = create_queue_payload(
message=message,
task_type="representation",
observer=observer_peer.name,
observed=message.peer_name
)
# Add to queue
queue_items = await add_queue_items([payload], session.id)
Testing Work Units
# Create a work unit
work_unit = WorkUnit(
session_id=session.id,
task_type="representation",
observer=observer,
observed=observed
)
# Test string representation
assert str(work_unit) == f"({session.id}, {observed.name}, {observer.name}, representation)"