* fix(deriver): truncate oversize observations so one cannot drop the batch
simple_batch_embed raised ValueError when any input exceeded the per-input
token cap, which failed the entire deriver save when a single observation
was over-length. Add on_oversize="truncate": oversize inputs are embedded
from a token-capped prefix (re-encoded until it fits, with a warning),
preserving one vector per input. Default stays "raise" so existing callers
are unchanged. RepresentationManager opts into truncate.
Also add a live embedding test that fails on main (raise / missing kwarg)
and passes once a mixed short+oversize batch survives.
Refs #569
* fix(deriver): surface failure when all observer saves fail
When every observer's save_representation failed (e.g. embedding retries
exhausted under a sustained 429), the deriver logged the error and returned
normally, so the queue marked the work unit processed with zero documents
saved. Collect per-observer errors and, after telemetry is emitted, raise
RepresentationSaveError when no observer succeeded. Partial failures stay
processed (saved observers must not be discarded) and are recorded via an
additive failed_observer_count on RepresentationCompletedEvent.
Refs #728
* fix(embedding): guarantee truncation progress and truncate on re-embed
The retry slice in _truncate_to_token_limit always recomputed the same
keep count, so a slice whose re-encode grew past the cap could oscillate.
Decrement keep after each unsuccessful retry.
Document re-embed in the reconciler used the default on_oversize="raise",
so one oversize document failed every other document in the batch.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* chore: drop ticket ids and shrink comments to one sentence
Comments and docstrings describe current behavior, not the PR that
introduced them. Ticket numbers stay in the commit/PR.
* chore: annotate RepresentationSaveError and assert truncate on re-embed
* fix(embedding): truncate on conclusion create paths and document BPE loop
Storage callers in create_observations (API + agent tools) now pass
on_oversize="truncate" so a single oversize item cannot drop the batch.
Docstring on _truncate_to_token_limit notes why decode/re-encode is load-bearing.
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix: enforce explicit-document session purity in dedup/merge paths
Audit for DEV-2000 (Scopes RFC prerequisite): explicit-level documents must
stay session-pure so scope memory can be built by copying explicit documents
between collections. Two classes of violation were possible:
- Exact-content and semantic dedup in crud/document.py matched candidates
with no level or session scoping, so an explicit document could be
reinforced by — or soft-deleted in favor of — a same-content document from
a different session or a different level (silently merging cross-session
derivations into one row).
- The generic create_observations tool handler accepted level='explicit'
from agents with no message context (dreamer/dialectic), which would mint
session-less explicit documents.
Enforcement (refuse, never rewrite):
- create_documents refuses explicit documents with a null session_name
- exact dedup keys on (content, level, session-for-explicit); derived levels
keep cross-session consolidation
- is_rejected_duplicate scopes candidate search to the same level, and the
same session for explicit documents
- the create_observations tool rejects explicit-level input outside message
ingestion (deriver) context
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: add card_refresh dream type for event-driven peer-card updates
Adds a lightweight dream variant (DEV-2000, Scopes RFC prerequisite) that
runs ONLY the peer-card update — for event-driven refreshes such as scope
membership changes and cold starts:
- DreamType.CARD_REFRESH alongside OMNI; dispatched by process_dream to a
new run_card_refresh_dream orchestration
- CardRefreshSpecialist: restricted to get_recent_observations,
search_memory, and update_peer_card (no observation-mutating tools), with
a low tool-iteration cap of min(6, DREAM.MAX_TOOL_ITERATIONS)
- rebuild=True mode carried in the dream payload: the existing card is NOT
injected into the prompt and the specialist rebuilds it solely from
observations present in the collection (for use after removals)
- enqueue-able via the manual enqueue_dream path (bypasses volume gates);
the work-unit key already embeds the dream type so a card refresh never
collides with a pending omni dream. POST /v3/workspaces/{id}/schedule_dream
accepts dream_type=card_refresh plus the rebuild flag
- card refreshes never advance the omni dream guard pair
(last_dream_at / last_dream_document_count)
- shared PEER CARD prompt section extracted (verbatim) from
DeductionSpecialist for reuse; CallPurpose gains dream.card_refresh
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: reserve scope__ peer namespace with kind flag and guardrails
Introduce the scope peer namespace (scope__<name>) and the authoritative
{"kind": "scope"} configuration flag, plus the server-side guardrails:
- src/utils/scopes.py: single source of truth for the prefix, kind flag,
and name helpers (scope_peer_name / is_scope_peer_name /
scope_name_from_peer / validate_no_scope_peer_names)
- reject reserved-prefix names on peer get-or-create (422)
- reject scope peers as message authors in crud.create_messages (422)
- reject scope peers as chat/representation targets (422); a scope peer
as the path-level observer is deferred to Phase 2b
- reject scope peers on the generic session-peer add/set/remove routes
and the session-create peers mapping (422, directing to scopes routes)
- peers.list excludes scope peers by default; new PeerGet.kind option
("scope" | "all") switches the view via a configuration JSONB filter
- schemas: Scope / ScopeCreate / ScopeSessions(Add) and
SessionCreate.scopes (unprefixed scope names, validated)
Part of DEV-1997 (Scopes RFC DEV-1970).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: add scopes CRUD routes and session-create scopes wiring
New /v3/workspaces/{workspace_id}/scopes facade (workspace-level auth;
peer- and session-scoped keys are rejected):
POST "" create-or-get (201/200)
POST /list paginated scope list
GET /{scope_id} single scope
POST /{scope_id}/sessions add memberships
DELETE /{scope_id}/sessions/{session_id} remove membership
GET /{scope_id}/sessions list member session ids
- crud/scope.py: get_or_create_scopes stamps the backing peer with
{"kind": "scope", "observe_me": false} and refuses to adopt a
legacy peer occupying the reserved name without the flag (409)
- memberships are session_peers rows with observe_others=true /
observe_me=false — identical to a hand-built observer peer
- SessionCreate.scopes: create-or-get each scope peer and add the
membership at session creation (the no-backfill common path)
- crud/session.py: public upsert_session_peers wrapper so the facade
bypasses the route-level guardrails without reaching into privates
Backfill of pre-existing documents and reconciliation on removal land in
DEV-1999; membership only affects messages ingested after the change.
Part of DEV-1997 (Scopes RFC DEV-1970).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: cover scopes facade, guardrails, and observer semantics
- create-or-get idempotency, list/get, name validation, legacy-collision
rejection (409), auth scoping (workspace key ok, peer/session keys 401)
- reserved prefix rejected on peer create; peers.list kind filtering
- scope peers rejected as message authors, chat/representation targets,
and on the generic session-peer routes
- membership add/list/remove with observe_others=true / observe_me=false
row shape asserted via DB, and facade-less equivalence with a
hand-built observer peer
- end-to-end litmus: after adding a session to a scope, the deriver
enqueue fan-out includes the scope peer as an observer
- session creation with scopes: [a, b] creates both memberships
Part of DEV-1997 (Scopes RFC DEV-1970).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: scope backfill-by-copy and removal reconciliation jobs
Retroactive scope membership changes (DEV-1999):
- New queue task types scope_backfill / scope_removal with payloads,
work-unit keys ({task}:{workspace}:{scope_peer}:{session}), deduped
enqueue (mirrors enqueue_dream), and consumer dispatch.
- Backfill copies a session's explicit documents from each sender's
global (P, P) collection into the scope's (scope_peer, P) collection
with internal_metadata.copied_from as the idempotency marker;
soft-deleted copies from an earlier removal are restored, so
add -> remove -> re-add converges on exactly one live copy. Completion
enqueues one manual omni dream per touched collection.
- Removal soft-deletes the session's explicit documents in the scope's
collections and cascades (fail-closed, transitively) to derived
documents whose source_ids intersect anything removed, deletes the
vectors from the external store, then enqueues a card_refresh dream
with rebuild=True plus a manual omni dream per touched collection.
- Zero LLM re-derivation: explicit documents are session-pure (DEV-2000
invariant); the only external call is re-embedding rows whose
embedding column is NULL (external-store deployments).
- Per-session job status lives in the scope peer's internal_metadata
under backfill_status, written via single-statement JSONB merges
(concurrent-writer safe) and surfaced at
GET /v3/workspaces/{w}/scopes/{scope_id}/status.
- Enqueued from the scopes add-sessions route and SessionCreate.scopes
handling, only when the session already has messages.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: add scope backfill/removal test coverage, fix backfill status JSONB bug
Adds tests/deriver/test_scope_backfill.py covering the DEV-1999 scope
backfill-by-copy and removal reconciliation jobs implemented in a prior
commit: explicit-doc copying, copied_from idempotency (including
add->remove->re-add), multi-peer collection routing, removal cascade to
dependent derived docs, dream enqueues (manual omni on backfill;
card_refresh rebuild + omni on removal), the status route, and add-sessions
route wiring (backfill enqueued only when the session already has messages).
Fixes a real production bug surfaced by these tests: update_scope_backfill_status
passed json.dumps()'d strings through SQLAlchemy cast(..., JSONB), which
double-encodes (psycopg re-serializes the already-JSON string), producing a
JSONB string scalar instead of an object. Postgres's `||` between two
non-array jsonb scalars doesn't merge — it silently wraps both into a
2-element array, corrupting backfill_status into a list. This crashed
clear_scope_backfill_status's `#-` path delete (called on every removal)
with "path element is not an integer" once a session had ever completed a
backfill. Fixed by passing raw Python dicts to cast() instead, mirroring the
working pattern already used in update_collection_internal_metadata.
Also adds src.deriver.scope_backfill.tracked_db to conftest's tracked_db
patch list — the module was missing from that per-import-site allowlist, so
its DB work ran against the real configured database instead of the
isolated per-test database.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(crud): preserve cache invalidation across get_or_create retry
`get_or_create_peers` and `get_or_create_scopes` mutate existing rows, then
insert new ones inside `db.begin_nested()`. When a concurrent writer creates
one of those rows first, the insert raises IntegrityError and the function
retries.
`begin_nested()` autoflushes the pending UPDATEs *before* opening the
savepoint, so the rollback neither undoes them nor expires the now-clean ORM
state. The retry then compared already-updated values, found no change, and
dropped those peers from `changed_peers` — skipping the cache purge while the
row change committed anyway, leaving entries stale until the 300s TTL.
Carry the mutated names into the retry via `_pending_invalidation` so the
purge cannot be lost.
The scopes facade mirrors `get_or_create_peers`, so both copies carried this.
The peer path is pre-existing and runs on every message ingest.
Also add the missing /v3 prefix to `_SCOPES_ROUTE_GUIDANCE`, which pointed
callers at a 404.
Adds tests/crud/test_get_or_create_retry_invalidation.py, which drives a real
racing session and fails without this change.
* fix(scopes): make scope identity unforgeable, unblock non-pattern peer names
Three coupled changes to the scopes facade.
1. `PeerCreate` no longer gates internal lookups. It exists to validate a new,
user-supplied peer id at the API boundary, but crud used it as a DTO for
names that already exist, so any name outside RESOURCE_NAME_PATTERN raised a
raw pydantic ValidationError — which is not a HonchoException, so it fell
through to the catch-all handler as an HTTP 500. Adds `PeerSpec` (same
fields, no charset pattern) as `PeerCreate`'s base, widens
`get_or_create_peers` to accept it, and changes `get_peer` to take a plain
str. All 13 construction sites converted; the create route keeps full
validation.
This unbreaks the Dreamer: DreamScheduler passes `collection.observer`
straight into the specialist preflight, and scope peers have
`observe_others=true`, so every `(scope.x, peer)` dream died there — the
feature scopes exist to enable. It also fixes a pre-existing bug unrelated
to scopes: a peer named `alice.smith` (legal before d429de0e5338, which
validated names by length alone) 500s on message create, session peer add,
and peer update.
2. The `kind` flag moves from `configuration` to `internal_metadata`.
`configuration` is user-writable — `PeerCreate`/`PeerUpdate` accept a
free-form dict and `update_peer` replaces it wholesale — so a legitimate
`{"observe_me": true}` update silently dropped the flag, and a forged
`{"kind": "scope"}` injected an ordinary peer into `POST /scopes/list`.
`internal_metadata` appears in no API schema. `observe_me: false` stays in
`configuration`, where it belongs.
3. Scope identity requires prefix AND flag, via `is_scope_peer()` and
`scope_peer_clause()`. Neither half is forgeable: the prefix sits outside
RESOURCE_NAME_PATTERN, `internal_metadata` is unreachable. Usage-site guards
become flag-based so a legacy peer merely occupying the namespace keeps
working rather than 422-ing on its own traffic; peer create and update stay
name-based, since those must stop new names entering the namespace.
`update_peer` now returns 422 instead of 500.
Also swaps the reserved prefix from `scope__` to `scope.`: `_` is inside
RESOURCE_NAME_PATTERN, so any tenant could already own a `scope__x` peer.
No DB migration — `internal_metadata` already exists on `peers`, and no scope
peers exist in any deployment yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): validate peer names on create, close namespace squatting and upsert race
Addresses three review findings against 48047a6a.
1. `PeerSpec` let API callers create invalid and reserved-prefix peers.
Widening `get_or_create_peers` to accept a pattern-free schema fixed the
lookup 500s but also removed validation from the *insert* path, and
request-controlled names reach it via message authors, session peer maps, and
the chat observer path — none of which carry a charset pattern of their own.
Confirmed: `POST /sessions/{id}/messages` with `peer_id: "scope.x"` returned
201 and minted an unflagged squatter, after which `POST /scopes {id: x}` was
permanently 409-blocked — namespace denial of service by any caller able to
post a message. `peer_id: "not a valid name!@#"` was likewise created.
Fixed by validating only names about to be INSERTed
(`_validate_new_peer_names`), so already-existing names — legacy dotted
names, scope peers — still resolve without a spurious 422. That keeps the
Dreamer fix intact, since it reads through `get_peer`.
2. Existing reserved-prefix squatters could not be updated. The name-based guard
on `PUT /peers/{peer_id}` refused every `scope.` name, contradicting the
invariant that an unflagged squatter stays a normal peer. Now flag-based, so
behavior is three-way: a real scope is refused, an existing unflagged peer
updates, and a missing reserved-prefix name is refused by (1) rather than
minted.
3. Scope checks raced with get-or-create and the membership upsert. The
route-level guards run before peers are resolved, so a scope created
concurrently in that window would be attached by the generic path with a
default `SessionPeerConfig()`, clobbering its observer membership config.
Adds `_reject_resolved_scope_peers`, which runs on the resolved rows in the
same transaction as the upsert — no window, no extra query. The early checks
stay for better error messages.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): guard scope membership config, move checks to the mutation point
Addresses a second review pass against 10655792.
1. Scope membership configuration was directly user-mutable.
`PUT /sessions/{id}/peers/{peer_id}/config` had no scope guard at all, and
`crud.set_peer_config` resolved the peer only to discard the row. Confirmed:
posting `{"observe_others": false, "observe_me": true}` for a scope returned
204 and persisted, which silently stops all fan-out into the scope and makes
Honcho form a representation *of* a scope — neither of which is reachable
through the facade. Deterministic, no race required. Now checked on the row
`get_peer` already returns, so it costs nothing and cannot race.
2. Empty and over-long names were still 500s. Removing the charset pattern from
`PeerSpec` fixed one trap but left its length bounds, and request-bound peer
names carry no length limits of their own — so `peer_id: ""` or a 513-char
name reached `PeerSpec(...)` and raised a raw pydantic ValidationError that
the catch-all turned into a 500. `PeerSpec` now carries no constraints at all
(matching its documented purpose) and every rule for a new name lives in
`_validate_new_peer_names` on the insert path.
3. Resolved-row protection generalized. The previous pass applied it only to
membership upserts, leaving check-then-use windows elsewhere: peer update
could have a concurrently-created scope's configuration replaced wholesale
(create-path validation does not fire for a peer that now exists), the chat
observer get-or-create could resolve a fresh scope as its observer, and the
generic session-peer removal could silently detach a scope from its sessions.
Each now inspects the resolved peer immediately before acting; the redundant
name-level guard on the update route is dropped in favor of the race-free one.
`remove_peers_from_session` grows an internal `_allow_scope_peers` flag because
the scopes facade ends membership through that same path and must not be blocked
by its own guard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(scopes): enumerate every peer-touching route against a scope policy
Three review passes each found the same class of defect: a route nobody had
checked, rather than logic that was subtly wrong. One of them — `PUT
/sessions/{id}/peers/{id}/config`, which let any caller set a scope to
`observe_others=false` and silently stop all fan-out into it — predated this work
entirely, because the guardrail set was assembled guardrail-by-guardrail instead
of derived from the route list. Sampling review cannot close that kind of gap;
enumeration can.
Derives every route through which a peer name can reach the system and requires
each to be classified as GUARDED or EXEMPT-with-a-reason, so a newly added
peer-touching route fails the suite until someone classifies it. Detection is the
union of path shape and a walk of the dependant tree (including sub-dependency
`Form(...)` params and nested request-body models), because neither signal alone
suffices: parameter names miss `POST /sessions/{id}/peers`, whose peer names are
dict keys, and path shape misses `messages/upload`, whose `peer_id` arrives as a
form field behind a parser dependency.
Both invariants are then asserted behaviorally, by calling the routes rather than
inspecting annotations — the guards deliberately live in crud, which is what makes
`messages/upload` guarded for free via `crud.create_messages`:
- a real scope is refused on all 11 guarded routes, and the rejection must name
the scope, so an unrelated 422 (a malformed body) cannot pass the assertion;
- an *unflagged* peer merely occupying the reserved namespace is unaffected. That
half regressed once already when `update_peer` used a name-based check.
Mutation-tested all three failure modes: disabling the `set_peer_config` guard
fails the guarded test naming that route; regressing `update_peer` to name-based
fails the squatter test; adding an unclassified peer route fails the enumeration.
Covers the HTTP surface only. Peer names also reach the system through the
deriver, dreamer, and queue, which have no route table to enumerate — noted in
the module docstring rather than implied to be covered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): refuse a scope in every observed position; enumerate per position
Addresses a fourth review pass against 62681e4d. The headline finding is that the
previous commit's enumeration test had the wrong *model*, not a missing entry.
1. Manual conclusions could create knowledge about a scope. `POST /conclusions`
validated only that observer_id and observed_id exist, so a scope as
`observed_id` persisted a conclusion about a peer carrying observe_me=false and
created an (observer, scope) collection for it. Confirmed: 201, and it read
back. `POST /schedule_dream` had the same hole via `observed`.
The fix is positional, because the invariant is:
A scope may be an OBSERVER. A scope may never be OBSERVED.
A scope as `observer_id` is how scoped conclusions are stored and must keep
working (verified still 201); as `observed_id` it is now refused. Same split
applied to schedule_dream `observed`, the peer-card `target` (which also covers
a scope's self-card, since target-omitted collapses observed to peer_id), and
session-context `peer_target`.
2. Chat target and both representation roles kept check-to-use races. Only the
chat path-level observer was re-checked on its resolved row; the target was
checked by name and then resolved without inspecting scope identity. Both are
now checked at the dialectic preflight, where observer and observed are already
resolved — an absent name has already failed by then, and an existing squatter
cannot retroactively become a scope.
3. Generic membership removal was still racy. The adjacent SELECT narrowed the
window but could not close it under READ COMMITTED. The UPDATE now carries its
own correlated NOT EXISTS against scope_peer_clause(), so Postgres evaluates
the exclusion as part of the statement and a scope committed after the advisory
check still cannot be detached.
4. New-name validation ran after the name reached Postgres. A NUL byte passed the
request schemas and PeerSpec, then raised psycopg.DataError inside the lookup —
a 500. Values that cannot correspond to a stored row by construction (NUL
bytes, over-length names) are now refused before the query.
(Over-length names already returned 422; only the wasted query was real there.)
The enumeration test is rekeyed from (method, path) to (method, path, position).
A binary per-route verdict cannot express finding 1 at all: `POST /conclusions` is
one route with two positions and opposite verdicts. Detection widens to observer /
observed / target / peer_target / peer_perspective, which surfaced four routes the
previous version never saw — conclusions, schedule_dream, queue/status, and
session context.
Also registers `src.routers.workspaces.tracked_db` in the conftest patch list; the
new guard there would otherwise have run against the real configured database
instead of the per-test one.
Mutation-tested: disabling the conclusions observed-guard fails the positional
test naming that position; adding an unclassified `observed_id` param fails
enumeration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): refuse future scopes in observed positions, preserve scope membership
Addresses a fifth review pass against 14136e5b. All six findings reproduced
locally before fixing.
1. High — generic peer replacement removed scope memberships.
`set_peers_for_session` soft-deleted every active SessionPeer row, and the
request-level guard only inspected names *present* in the replacement map. A
caller detached a scope by simply omitting it, never naming it — so no
request-level guard could ever see it. Reproduced: scope sessions went
['<id>'] -> [] on a 200. The exclusion now lives in the UPDATE itself
(correlated NOT EXISTS against scope_peer_clause), so replacement means
"replace ordinary peers" regardless of request contents or concurrent creation.
2. High — peer cards could be pre-seeded for future scopes.
`set_peer_card` resolves only the observer and writes a JSONB key derived from
an unchecked observed name, and the route guard rejected only *existing*
flagged scopes. Reproduced: PUT card with target=scope.<missing> returned 200,
creating that scope then returned 201, and the card described the real scope.
3. High — dreams could be queued for future scopes.
The route checked `observed` in a read-only session that closed before
`enqueue_dream`, and a missing reserved name passes any is-it-a-scope check.
Reproduced: 204 with observed=scope.<missing>.
2 and 3 share a root cause, so they share a fix: a new `reject_scope_observed`
that is stricter than `reject_scope_peers` in exactly one case — a *missing*
reserved name is refused, because nothing on these paths creates the peer, so
nothing else would ever catch it. Existing unflagged squatters still pass.
Both guards moved to the mutation point: card validation into
`crud.set_peer_card` (same transaction as the JSONB write, so Dreamer and
agent-tool callers are covered), dream validation into `enqueue_dream` (same
transaction as the queue insert). The redundant route-level checks are dropped
rather than left as weaker duplicates.
4. Medium — prefixed NUL names still reached PostgreSQL.
`reject_scope_peers` filtered for the reserved prefix and sent matches to a
text comparison, so "scope.future\0name" raised psycopg.DataError — a 500.
Both guards now share `_reserved_name_candidates`, which materializes the input
once and rejects impossible values before any SQL. Materializing matters
independently: the message-author path passes a generator, and validation
iterates separately from the prefix filter, so a generator would be
half-consumed. `_reject_impossible_peer_names` now takes a Collection so the
type checker enforces that.
5. Medium — representation kept a check-to-use race.
The previous commit claimed both representation roles were rechecked after
resolution; that was wrong — only the dialectic preflight got that check, and
the representation route never goes through it. It now opens one short
read-only session *after* the embedding call, checks both positions, and passes
that same session to `get_working_representation`, so no connection is held
across external work and a scope committed later cannot have conclusions in the
collection being read.
6. Low — policy coverage was not exhaustive. `sender_id` reaches CRUD as
`observed` but was missing from the detected parameter set. ALLOW cases could
also not carry builders, so the suite never proved the other half of the
contract — that legitimate scope *observers* keep working, which a guard
rejecting scopes everywhere would satisfy. Both fixed; observer positions on
conclusions, dreams, cards, session context and queue status are now asserted
behaviorally.
Deliberately not implemented: the scope-creation backstop scanning for
pre-existing card keys and queue items naming a future backing peer. Reasoning is
recorded in `get_or_create_scopes` — no new such state can be created now, any
pre-existing row is coincidental since `scope.` was never a meaningful namespace,
the consequence is inert, and detecting card keys means a full table scan per
scope creation.
Mutation-tested each new guard: removing the replacement exclusion fails both
membership-preservation tests; weakening either observed guard to existing-only
fails the pre-seeding tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): exclude scope memberships from the session observer limit
Scope memberships carry observe_others=true, so every scope counted against
SESSION_OBSERVERS_LIMIT (default 10) — capping scopes-per-session at the limit
minus the session's real observers, and reporting the failure as
`400 Cannot create session <name> with 11 observers. ... Observers are peers
with 'observe_others' set to true.` on a membership call. Wrong on three counts:
the ceiling is undocumented and contradicts RFC §5.1 ("sessions belong to any
number of scopes"), the message describes session creation, and it leaks the
word "observer" through a facade whose entire job is hiding observers (RFC
goal 5). The limit exists to bound per-observer deriver fan-out for real peers;
a scope costs document rows, not LLM calls (RFC §5.2), so it does not belong in
that budget.
Excluded from both halves of the check in `_get_or_add_peers_to_session`: the
incoming names via a flag-based lookup, existing memberships via a correlated
NOT EXISTS on `scope_peer_clause()` — the same pattern the replacement and
removal paths already use, so the exclusion holds regardless of concurrent
scope creation. The early `count_observers_in_config(session.peer_names)` check
in `get_or_create_session` is left alone: `peer_names` cannot contain a scope,
and `scopes` is a separate field.
`reject_scope_peers` is split into a `scope_peer_names()` query helper plus a
two-line raiser so the observer count reuses the authoritative name-AND-flag
predicate instead of growing a third copy of it. Still costs nothing on the
common path — no reserved-prefix name in the input means no query at all.
Also caps `SessionCreate.scopes` at 100, matching `ScopeSessionsAdd.session_ids`.
This belongs in the same commit: the observer limit was the only thing bounding
that list, so removing it turns an unbounded `scopes` array into a peer row and
a membership row per element, committed — the single-request path to the
cardinality anti-pattern RFC §8 warns about. Partly answers OQ6: no per-session
cap, 100 per request.
Tests: a session joins SESSION_OBSERVERS_LIMIT + 2 scopes through both the
facade and session creation; real observers over the limit still 400, so the
carve-out cannot quietly disable the limit; 101 scopes is a 422.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): skip semantic retrieval when the embedding precompute failed
Addresses three open review comments.
1. Major — the representation read could embed inside its DB session. The
route's precompute is suppressed, and both
`RepresentationManager.get_working_representation` and
`crud.query_documents` fall back to embedding when a query arrives without
one, so a failed precompute meant an external call inside the read session
this branch opens for the scope re-check — the connection-holding rule the
route's own comment claimed to satisfy. The innermost fallback also only
catches ValueError, so a provider outage surfaced as a 500. The semantic
query is now passed only when an embedding exists, degrading to
derived+recent retrieval. (`crud.query_documents` embedding inside a caller's
session predates this branch and is left alone.)
2. Minor — `test_resolved_scope_peer_rejected_at_membership_upsert` described a
race it does not perform. It creates an already-flagged scope and calls crud
directly; the unflagged → flagged transition is not simulated. Docstring now
says what the test actually pins.
3. Minor — `test_empty_replacement_preserves_scope_membership` asserted only
half its docstring. It passed if the empty PUT left every ordinary
membership intact; now asserts the ordinary peer's left_at is set.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): close auth and observed-position gaps, paginate membership
Review response for #884.
Security:
- gate `SessionCreate.scopes` behind a workspace-level key; the session-create
route is self-authorizing, so a peer- or session-scoped token could mint scope
peers and join sessions to scopes it had no access to via `POST /scopes`
- refuse a reserved-but-nonexistent name in two observed positions that used the
permissive guard: chat `target` and session-context `peer_target`. Both let a
caller act on `scope.X` before it existed, then create the scope
Facade:
- exclude scope peers from `GET /sessions/{id}/peers` and refuse the membership
-config read for a real scope, matching its write side
- replace `GET /scopes/{id}/sessions` with `POST /scopes/{id}/sessions/list`
returning `Page[Session]`; the add route now returns 204. Membership was
unbounded on both, while every other list surface paginates
- rename `crud.get_scope` to `get_scope_or_raise`
Tests:
- add a missing-name axis to the route-policy table (`Case.refuse_missing`), which
is what surfaced the two guard gaps above
- delete 14 hand-written tests the table now enumerates; 52 -> 39 functions in
test_scopes.py with more cases covered
- tighten the squatter assertion from `!= 422` to `< 400`, which was passing on 5xx
- assert the FastAPI-internals traversal still derives positions, so a framework
upgrade can't silently empty the suite
Docs:
- drop internal ticket and RFC references from the published OpenAPI descriptions
and surrounding comments; state the behavior instead
- move implementation reasoning out of the `PUT /peers/{id}` docstring, which
FastAPI publishes, into a comment
* test(scopes): assert exact statuses for permissive missing-name cases
Follow-up review pass on #884.
- add `Case.missing_status` so a permissive missing-name position asserts the
status it should actually get (404, or 200 for the no-op removal) instead of
`!= 422`, which also passed on a 5xx — the same hole already closed in the
squatter assertion
- require it whenever `refuse_missing` is False, and require its absence when
True, so the policy table can't drift from the assertion
- repoint a stale allow-reason at POST /scopes/{scope_id}/sessions/list; the GET
it named was removed
- document the membership list's ordering under `reverse`
* chore: clean up stale docstring language
* fix: don't backfill a session that left the scope
scope_backfill and scope_removal carry different work-unit keys, so
nothing orders them: a removal enqueued right after the add — or one
that lands while the backfill is embedding — sweeps the scope before
the copies exist, leaving a departed session's documents live in the
scope forever.
_run_backfill now re-checks SessionPeer membership inside the write
transaction and returns None; process_scope_backfill then skips both
the dream enqueues and the status write, so a skipped backfill can't
resurrect the status entry removal just cleared.
Adds coverage for the skip, the NULL-embedding re-embed path, and the
failed-status write. Handler-driven tests now stand up the membership
row the guard requires.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix: apply session scoping to all working-representation query paths
session_name was only applied to the recent-documents query in
RepresentationManager; the semantic and most-derived paths ignored it,
so limit_to_session leaked cross-session conclusions into perspectives.
- Thread a session allowlist (session_names) uniformly through all
three query paths; pushed down to pgvector and external vector stores
- Accept a list so the upcoming session-allowlist API reuses this path
- Fail closed on an empty allowlist (downstream stores drop empty IN
clauses, which would silently widen scope)
Fixes DEV-1994
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: bare-list membership sugar in the filter DSL
{"session_id": ["s1", "s2"]} is now shorthand for
{"session_id": {"in": [...]}} on regular columns, generically
(peer_id, etc.). JSONB metadata columns are excluded — a bare list
there keeps JSONB containment semantics, unchanged.
Previously a bare list on a regular column compiled to a type-mismatched
equality that matched nothing, so this is strictly additive.
Also translates the same shape in the turbopuffer/lancedb filter
builders, and fixes lancedb dropping empty IN clauses (fail-open) —
an empty membership list now emits an always-false condition.
Groundwork for DEV-1995 (session allowlist via the existing filters
DSL, no new API params)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: session allowlist on dialectic and representation via filters
Adds a constrained 'filters' body to peer.chat and /representation —
the same DSL search and conclusions already accept, supporting only the
session_id key (a session id, a bare list, or {"in": [...]}).
Unsupported keys and shapes are rejected with 422, never silently
ignored. Composes with session_id (must be included in the allowlist
when both are given). Capped at 1,000 sessions per request.
Enforcement is uniform at every recall chokepoint, fail-closed:
- dialectic prefetch + search_memory: conclusion recall restricted to
the allowlist; dream docs (session_name IS NULL) excluded
- message tools (search/grep/date-range/temporal/context/history):
strict intersection of allowlist and observer session membership
- get_reasoning_chain: unavailable under an allowlist (chains traverse
provenance across sessions and cannot be scoped without leaking)
- empty allowlist short-circuits to empty results everywhere
Auth: workspace keys pass the allowlist as-given; peer-scoped JWTs must
be a member of every allowlisted session (403 otherwise), mirroring the
existing single-session check.
Fixes DEV-1995
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: reserve scope__ peer namespace with kind flag and guardrails
Introduce the scope peer namespace (scope__<name>) and the authoritative
{"kind": "scope"} configuration flag, plus the server-side guardrails:
- src/utils/scopes.py: single source of truth for the prefix, kind flag,
and name helpers (scope_peer_name / is_scope_peer_name /
scope_name_from_peer / validate_no_scope_peer_names)
- reject reserved-prefix names on peer get-or-create (422)
- reject scope peers as message authors in crud.create_messages (422)
- reject scope peers as chat/representation targets (422); a scope peer
as the path-level observer is deferred to Phase 2b
- reject scope peers on the generic session-peer add/set/remove routes
and the session-create peers mapping (422, directing to scopes routes)
- peers.list excludes scope peers by default; new PeerGet.kind option
("scope" | "all") switches the view via a configuration JSONB filter
- schemas: Scope / ScopeCreate / ScopeSessions(Add) and
SessionCreate.scopes (unprefixed scope names, validated)
Part of DEV-1997 (Scopes RFC DEV-1970).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: add scopes CRUD routes and session-create scopes wiring
New /v3/workspaces/{workspace_id}/scopes facade (workspace-level auth;
peer- and session-scoped keys are rejected):
POST "" create-or-get (201/200)
POST /list paginated scope list
GET /{scope_id} single scope
POST /{scope_id}/sessions add memberships
DELETE /{scope_id}/sessions/{session_id} remove membership
GET /{scope_id}/sessions list member session ids
- crud/scope.py: get_or_create_scopes stamps the backing peer with
{"kind": "scope", "observe_me": false} and refuses to adopt a
legacy peer occupying the reserved name without the flag (409)
- memberships are session_peers rows with observe_others=true /
observe_me=false — identical to a hand-built observer peer
- SessionCreate.scopes: create-or-get each scope peer and add the
membership at session creation (the no-backfill common path)
- crud/session.py: public upsert_session_peers wrapper so the facade
bypasses the route-level guardrails without reaching into privates
Backfill of pre-existing documents and reconciliation on removal land in
DEV-1999; membership only affects messages ingested after the change.
Part of DEV-1997 (Scopes RFC DEV-1970).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: cover scopes facade, guardrails, and observer semantics
- create-or-get idempotency, list/get, name validation, legacy-collision
rejection (409), auth scoping (workspace key ok, peer/session keys 401)
- reserved prefix rejected on peer create; peers.list kind filtering
- scope peers rejected as message authors, chat/representation targets,
and on the generic session-peer routes
- membership add/list/remove with observe_others=true / observe_me=false
row shape asserted via DB, and facade-less equivalence with a
hand-built observer peer
- end-to-end litmus: after adding a session to a scope, the deriver
enqueue fan-out includes the scope peer as an observer
- session creation with scopes: [a, b] creates both memberships
Part of DEV-1997 (Scopes RFC DEV-1970).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(scopes): scope resolution helper and read-route schemas
Add resolve_scope_peers (crud/scope.py) mapping unprefixed scope names to
their backing scope peers, 404 when missing and 422 when a non-scope peer
squats the reserved name. Add the `scope` option to DialecticOptions and
PeerRepresentationGet, and a WorkspaceMessageSearchOptions with `scope`.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(scopes): wire the `scope` read option into the routes
Chat and representation: a single scope swaps the observer to the scope
peer (recall confined to the scoped collection and the scope's sessions by
existing observer semantics); a scope list resolves to the union of member
sessions and rides the DEV-1995 dynamic allowlist arm with the path peer as
observer. `scope` is mutually exclusive with `filters`/`session_id` (422)
and requires a workspace/admin key (403 for peer-scoped JWTs).
Session context: `scope` swaps the perspective source for both the working
representation and the peer-card fetch. Workspace search: `scope` injects
the scope's session set into the message-search filter (empty scope -> no
results). Close the carry-over guardrail gap: scope peers are rejected as
peer_target/peer_perspective in session context and as the peer/target in
GET /peers/{id}/context.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(scopes): cover the `scope` read option end-to-end
Validation (404/422/403), single-scope observer swap vs. union allowlist,
scoped representation and session-context (scoped collection + scoped card),
workspace search restricted to a scope's sessions, and guardrail closure for
scope peers on the peer/session context surfaces. Patch the workspaces
tracked_db import site so scope resolution reads the per-test database.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(crud): preserve cache invalidation across get_or_create retry
`get_or_create_peers` and `get_or_create_scopes` mutate existing rows, then
insert new ones inside `db.begin_nested()`. When a concurrent writer creates
one of those rows first, the insert raises IntegrityError and the function
retries.
`begin_nested()` autoflushes the pending UPDATEs *before* opening the
savepoint, so the rollback neither undoes them nor expires the now-clean ORM
state. The retry then compared already-updated values, found no change, and
dropped those peers from `changed_peers` — skipping the cache purge while the
row change committed anyway, leaving entries stale until the 300s TTL.
Carry the mutated names into the retry via `_pending_invalidation` so the
purge cannot be lost.
The scopes facade mirrors `get_or_create_peers`, so both copies carried this.
The peer path is pre-existing and runs on every message ingest.
Also add the missing /v3 prefix to `_SCOPES_ROUTE_GUIDANCE`, which pointed
callers at a 404.
Adds tests/crud/test_get_or_create_retry_invalidation.py, which drives a real
racing session and fails without this change.
* fix(scopes): make scope identity unforgeable, unblock non-pattern peer names
Three coupled changes to the scopes facade.
1. `PeerCreate` no longer gates internal lookups. It exists to validate a new,
user-supplied peer id at the API boundary, but crud used it as a DTO for
names that already exist, so any name outside RESOURCE_NAME_PATTERN raised a
raw pydantic ValidationError — which is not a HonchoException, so it fell
through to the catch-all handler as an HTTP 500. Adds `PeerSpec` (same
fields, no charset pattern) as `PeerCreate`'s base, widens
`get_or_create_peers` to accept it, and changes `get_peer` to take a plain
str. All 13 construction sites converted; the create route keeps full
validation.
This unbreaks the Dreamer: DreamScheduler passes `collection.observer`
straight into the specialist preflight, and scope peers have
`observe_others=true`, so every `(scope.x, peer)` dream died there — the
feature scopes exist to enable. It also fixes a pre-existing bug unrelated
to scopes: a peer named `alice.smith` (legal before d429de0e5338, which
validated names by length alone) 500s on message create, session peer add,
and peer update.
2. The `kind` flag moves from `configuration` to `internal_metadata`.
`configuration` is user-writable — `PeerCreate`/`PeerUpdate` accept a
free-form dict and `update_peer` replaces it wholesale — so a legitimate
`{"observe_me": true}` update silently dropped the flag, and a forged
`{"kind": "scope"}` injected an ordinary peer into `POST /scopes/list`.
`internal_metadata` appears in no API schema. `observe_me: false` stays in
`configuration`, where it belongs.
3. Scope identity requires prefix AND flag, via `is_scope_peer()` and
`scope_peer_clause()`. Neither half is forgeable: the prefix sits outside
RESOURCE_NAME_PATTERN, `internal_metadata` is unreachable. Usage-site guards
become flag-based so a legacy peer merely occupying the namespace keeps
working rather than 422-ing on its own traffic; peer create and update stay
name-based, since those must stop new names entering the namespace.
`update_peer` now returns 422 instead of 500.
Also swaps the reserved prefix from `scope__` to `scope.`: `_` is inside
RESOURCE_NAME_PATTERN, so any tenant could already own a `scope__x` peer.
No DB migration — `internal_metadata` already exists on `peers`, and no scope
peers exist in any deployment yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): validate peer names on create, close namespace squatting and upsert race
Addresses three review findings against 48047a6a.
1. `PeerSpec` let API callers create invalid and reserved-prefix peers.
Widening `get_or_create_peers` to accept a pattern-free schema fixed the
lookup 500s but also removed validation from the *insert* path, and
request-controlled names reach it via message authors, session peer maps, and
the chat observer path — none of which carry a charset pattern of their own.
Confirmed: `POST /sessions/{id}/messages` with `peer_id: "scope.x"` returned
201 and minted an unflagged squatter, after which `POST /scopes {id: x}` was
permanently 409-blocked — namespace denial of service by any caller able to
post a message. `peer_id: "not a valid name!@#"` was likewise created.
Fixed by validating only names about to be INSERTed
(`_validate_new_peer_names`), so already-existing names — legacy dotted
names, scope peers — still resolve without a spurious 422. That keeps the
Dreamer fix intact, since it reads through `get_peer`.
2. Existing reserved-prefix squatters could not be updated. The name-based guard
on `PUT /peers/{peer_id}` refused every `scope.` name, contradicting the
invariant that an unflagged squatter stays a normal peer. Now flag-based, so
behavior is three-way: a real scope is refused, an existing unflagged peer
updates, and a missing reserved-prefix name is refused by (1) rather than
minted.
3. Scope checks raced with get-or-create and the membership upsert. The
route-level guards run before peers are resolved, so a scope created
concurrently in that window would be attached by the generic path with a
default `SessionPeerConfig()`, clobbering its observer membership config.
Adds `_reject_resolved_scope_peers`, which runs on the resolved rows in the
same transaction as the upsert — no window, no extra query. The early checks
stay for better error messages.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): guard scope membership config, move checks to the mutation point
Addresses a second review pass against 10655792.
1. Scope membership configuration was directly user-mutable.
`PUT /sessions/{id}/peers/{peer_id}/config` had no scope guard at all, and
`crud.set_peer_config` resolved the peer only to discard the row. Confirmed:
posting `{"observe_others": false, "observe_me": true}` for a scope returned
204 and persisted, which silently stops all fan-out into the scope and makes
Honcho form a representation *of* a scope — neither of which is reachable
through the facade. Deterministic, no race required. Now checked on the row
`get_peer` already returns, so it costs nothing and cannot race.
2. Empty and over-long names were still 500s. Removing the charset pattern from
`PeerSpec` fixed one trap but left its length bounds, and request-bound peer
names carry no length limits of their own — so `peer_id: ""` or a 513-char
name reached `PeerSpec(...)` and raised a raw pydantic ValidationError that
the catch-all turned into a 500. `PeerSpec` now carries no constraints at all
(matching its documented purpose) and every rule for a new name lives in
`_validate_new_peer_names` on the insert path.
3. Resolved-row protection generalized. The previous pass applied it only to
membership upserts, leaving check-then-use windows elsewhere: peer update
could have a concurrently-created scope's configuration replaced wholesale
(create-path validation does not fire for a peer that now exists), the chat
observer get-or-create could resolve a fresh scope as its observer, and the
generic session-peer removal could silently detach a scope from its sessions.
Each now inspects the resolved peer immediately before acting; the redundant
name-level guard on the update route is dropped in favor of the race-free one.
`remove_peers_from_session` grows an internal `_allow_scope_peers` flag because
the scopes facade ends membership through that same path and must not be blocked
by its own guard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(scopes): enumerate every peer-touching route against a scope policy
Three review passes each found the same class of defect: a route nobody had
checked, rather than logic that was subtly wrong. One of them — `PUT
/sessions/{id}/peers/{id}/config`, which let any caller set a scope to
`observe_others=false` and silently stop all fan-out into it — predated this work
entirely, because the guardrail set was assembled guardrail-by-guardrail instead
of derived from the route list. Sampling review cannot close that kind of gap;
enumeration can.
Derives every route through which a peer name can reach the system and requires
each to be classified as GUARDED or EXEMPT-with-a-reason, so a newly added
peer-touching route fails the suite until someone classifies it. Detection is the
union of path shape and a walk of the dependant tree (including sub-dependency
`Form(...)` params and nested request-body models), because neither signal alone
suffices: parameter names miss `POST /sessions/{id}/peers`, whose peer names are
dict keys, and path shape misses `messages/upload`, whose `peer_id` arrives as a
form field behind a parser dependency.
Both invariants are then asserted behaviorally, by calling the routes rather than
inspecting annotations — the guards deliberately live in crud, which is what makes
`messages/upload` guarded for free via `crud.create_messages`:
- a real scope is refused on all 11 guarded routes, and the rejection must name
the scope, so an unrelated 422 (a malformed body) cannot pass the assertion;
- an *unflagged* peer merely occupying the reserved namespace is unaffected. That
half regressed once already when `update_peer` used a name-based check.
Mutation-tested all three failure modes: disabling the `set_peer_config` guard
fails the guarded test naming that route; regressing `update_peer` to name-based
fails the squatter test; adding an unclassified peer route fails the enumeration.
Covers the HTTP surface only. Peer names also reach the system through the
deriver, dreamer, and queue, which have no route table to enumerate — noted in
the module docstring rather than implied to be covered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): refuse a scope in every observed position; enumerate per position
Addresses a fourth review pass against 62681e4d. The headline finding is that the
previous commit's enumeration test had the wrong *model*, not a missing entry.
1. Manual conclusions could create knowledge about a scope. `POST /conclusions`
validated only that observer_id and observed_id exist, so a scope as
`observed_id` persisted a conclusion about a peer carrying observe_me=false and
created an (observer, scope) collection for it. Confirmed: 201, and it read
back. `POST /schedule_dream` had the same hole via `observed`.
The fix is positional, because the invariant is:
A scope may be an OBSERVER. A scope may never be OBSERVED.
A scope as `observer_id` is how scoped conclusions are stored and must keep
working (verified still 201); as `observed_id` it is now refused. Same split
applied to schedule_dream `observed`, the peer-card `target` (which also covers
a scope's self-card, since target-omitted collapses observed to peer_id), and
session-context `peer_target`.
2. Chat target and both representation roles kept check-to-use races. Only the
chat path-level observer was re-checked on its resolved row; the target was
checked by name and then resolved without inspecting scope identity. Both are
now checked at the dialectic preflight, where observer and observed are already
resolved — an absent name has already failed by then, and an existing squatter
cannot retroactively become a scope.
3. Generic membership removal was still racy. The adjacent SELECT narrowed the
window but could not close it under READ COMMITTED. The UPDATE now carries its
own correlated NOT EXISTS against scope_peer_clause(), so Postgres evaluates
the exclusion as part of the statement and a scope committed after the advisory
check still cannot be detached.
4. New-name validation ran after the name reached Postgres. A NUL byte passed the
request schemas and PeerSpec, then raised psycopg.DataError inside the lookup —
a 500. Values that cannot correspond to a stored row by construction (NUL
bytes, over-length names) are now refused before the query.
(Over-length names already returned 422; only the wasted query was real there.)
The enumeration test is rekeyed from (method, path) to (method, path, position).
A binary per-route verdict cannot express finding 1 at all: `POST /conclusions` is
one route with two positions and opposite verdicts. Detection widens to observer /
observed / target / peer_target / peer_perspective, which surfaced four routes the
previous version never saw — conclusions, schedule_dream, queue/status, and
session context.
Also registers `src.routers.workspaces.tracked_db` in the conftest patch list; the
new guard there would otherwise have run against the real configured database
instead of the per-test one.
Mutation-tested: disabling the conclusions observed-guard fails the positional
test naming that position; adding an unclassified `observed_id` param fails
enumeration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): refuse future scopes in observed positions, preserve scope membership
Addresses a fifth review pass against 14136e5b. All six findings reproduced
locally before fixing.
1. High — generic peer replacement removed scope memberships.
`set_peers_for_session` soft-deleted every active SessionPeer row, and the
request-level guard only inspected names *present* in the replacement map. A
caller detached a scope by simply omitting it, never naming it — so no
request-level guard could ever see it. Reproduced: scope sessions went
['<id>'] -> [] on a 200. The exclusion now lives in the UPDATE itself
(correlated NOT EXISTS against scope_peer_clause), so replacement means
"replace ordinary peers" regardless of request contents or concurrent creation.
2. High — peer cards could be pre-seeded for future scopes.
`set_peer_card` resolves only the observer and writes a JSONB key derived from
an unchecked observed name, and the route guard rejected only *existing*
flagged scopes. Reproduced: PUT card with target=scope.<missing> returned 200,
creating that scope then returned 201, and the card described the real scope.
3. High — dreams could be queued for future scopes.
The route checked `observed` in a read-only session that closed before
`enqueue_dream`, and a missing reserved name passes any is-it-a-scope check.
Reproduced: 204 with observed=scope.<missing>.
2 and 3 share a root cause, so they share a fix: a new `reject_scope_observed`
that is stricter than `reject_scope_peers` in exactly one case — a *missing*
reserved name is refused, because nothing on these paths creates the peer, so
nothing else would ever catch it. Existing unflagged squatters still pass.
Both guards moved to the mutation point: card validation into
`crud.set_peer_card` (same transaction as the JSONB write, so Dreamer and
agent-tool callers are covered), dream validation into `enqueue_dream` (same
transaction as the queue insert). The redundant route-level checks are dropped
rather than left as weaker duplicates.
4. Medium — prefixed NUL names still reached PostgreSQL.
`reject_scope_peers` filtered for the reserved prefix and sent matches to a
text comparison, so "scope.future\0name" raised psycopg.DataError — a 500.
Both guards now share `_reserved_name_candidates`, which materializes the input
once and rejects impossible values before any SQL. Materializing matters
independently: the message-author path passes a generator, and validation
iterates separately from the prefix filter, so a generator would be
half-consumed. `_reject_impossible_peer_names` now takes a Collection so the
type checker enforces that.
5. Medium — representation kept a check-to-use race.
The previous commit claimed both representation roles were rechecked after
resolution; that was wrong — only the dialectic preflight got that check, and
the representation route never goes through it. It now opens one short
read-only session *after* the embedding call, checks both positions, and passes
that same session to `get_working_representation`, so no connection is held
across external work and a scope committed later cannot have conclusions in the
collection being read.
6. Low — policy coverage was not exhaustive. `sender_id` reaches CRUD as
`observed` but was missing from the detected parameter set. ALLOW cases could
also not carry builders, so the suite never proved the other half of the
contract — that legitimate scope *observers* keep working, which a guard
rejecting scopes everywhere would satisfy. Both fixed; observer positions on
conclusions, dreams, cards, session context and queue status are now asserted
behaviorally.
Deliberately not implemented: the scope-creation backstop scanning for
pre-existing card keys and queue items naming a future backing peer. Reasoning is
recorded in `get_or_create_scopes` — no new such state can be created now, any
pre-existing row is coincidental since `scope.` was never a meaningful namespace,
the consequence is inert, and detecting card keys means a full table scan per
scope creation.
Mutation-tested each new guard: removing the replacement exclusion fails both
membership-preservation tests; weakening either observed guard to existing-only
fails the pre-seeding tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): exclude scope memberships from the session observer limit
Scope memberships carry observe_others=true, so every scope counted against
SESSION_OBSERVERS_LIMIT (default 10) — capping scopes-per-session at the limit
minus the session's real observers, and reporting the failure as
`400 Cannot create session <name> with 11 observers. ... Observers are peers
with 'observe_others' set to true.` on a membership call. Wrong on three counts:
the ceiling is undocumented and contradicts RFC §5.1 ("sessions belong to any
number of scopes"), the message describes session creation, and it leaks the
word "observer" through a facade whose entire job is hiding observers (RFC
goal 5). The limit exists to bound per-observer deriver fan-out for real peers;
a scope costs document rows, not LLM calls (RFC §5.2), so it does not belong in
that budget.
Excluded from both halves of the check in `_get_or_add_peers_to_session`: the
incoming names via a flag-based lookup, existing memberships via a correlated
NOT EXISTS on `scope_peer_clause()` — the same pattern the replacement and
removal paths already use, so the exclusion holds regardless of concurrent
scope creation. The early `count_observers_in_config(session.peer_names)` check
in `get_or_create_session` is left alone: `peer_names` cannot contain a scope,
and `scopes` is a separate field.
`reject_scope_peers` is split into a `scope_peer_names()` query helper plus a
two-line raiser so the observer count reuses the authoritative name-AND-flag
predicate instead of growing a third copy of it. Still costs nothing on the
common path — no reserved-prefix name in the input means no query at all.
Also caps `SessionCreate.scopes` at 100, matching `ScopeSessionsAdd.session_ids`.
This belongs in the same commit: the observer limit was the only thing bounding
that list, so removing it turns an unbounded `scopes` array into a peer row and
a membership row per element, committed — the single-request path to the
cardinality anti-pattern RFC §8 warns about. Partly answers OQ6: no per-session
cap, 100 per request.
Tests: a session joins SESSION_OBSERVERS_LIMIT + 2 scopes through both the
facade and session creation; real observers over the limit still 400, so the
carve-out cannot quietly disable the limit; 101 scopes is a 422.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): skip semantic retrieval when the embedding precompute failed
Addresses three open review comments.
1. Major — the representation read could embed inside its DB session. The
route's precompute is suppressed, and both
`RepresentationManager.get_working_representation` and
`crud.query_documents` fall back to embedding when a query arrives without
one, so a failed precompute meant an external call inside the read session
this branch opens for the scope re-check — the connection-holding rule the
route's own comment claimed to satisfy. The innermost fallback also only
catches ValueError, so a provider outage surfaced as a 500. The semantic
query is now passed only when an embedding exists, degrading to
derived+recent retrieval. (`crud.query_documents` embedding inside a caller's
session predates this branch and is left alone.)
2. Minor — `test_resolved_scope_peer_rejected_at_membership_upsert` described a
race it does not perform. It creates an already-flagged scope and calls crud
directly; the unflagged → flagged transition is not simulated. Docstring now
says what the test actually pins.
3. Minor — `test_empty_replacement_preserves_scope_membership` asserted only
half its docstring. It passed if the empty PUT left every ordinary
membership intact; now asserts the ordinary peer's left_at is set.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): close auth and observed-position gaps, paginate membership
Review response for #884.
Security:
- gate `SessionCreate.scopes` behind a workspace-level key; the session-create
route is self-authorizing, so a peer- or session-scoped token could mint scope
peers and join sessions to scopes it had no access to via `POST /scopes`
- refuse a reserved-but-nonexistent name in two observed positions that used the
permissive guard: chat `target` and session-context `peer_target`. Both let a
caller act on `scope.X` before it existed, then create the scope
Facade:
- exclude scope peers from `GET /sessions/{id}/peers` and refuse the membership
-config read for a real scope, matching its write side
- replace `GET /scopes/{id}/sessions` with `POST /scopes/{id}/sessions/list`
returning `Page[Session]`; the add route now returns 204. Membership was
unbounded on both, while every other list surface paginates
- rename `crud.get_scope` to `get_scope_or_raise`
Tests:
- add a missing-name axis to the route-policy table (`Case.refuse_missing`), which
is what surfaced the two guard gaps above
- delete 14 hand-written tests the table now enumerates; 52 -> 39 functions in
test_scopes.py with more cases covered
- tighten the squatter assertion from `!= 422` to `< 400`, which was passing on 5xx
- assert the FastAPI-internals traversal still derives positions, so a framework
upgrade can't silently empty the suite
Docs:
- drop internal ticket and RFC references from the published OpenAPI descriptions
and surrounding comments; state the behavior instead
- move implementation reasoning out of the `PUT /peers/{id}` docstring, which
FastAPI publishes, into a comment
* test(scopes): assert exact statuses for permissive missing-name cases
Follow-up review pass on #884.
- add `Case.missing_status` so a permissive missing-name position asserts the
status it should actually get (404, or 200 for the no-op removal) instead of
`!= 422`, which also passed on a 5xx — the same hole already closed in the
squatter assertion
- require it whenever `refuse_missing` is False, and require its absence when
True, so the policy table can't drift from the assertion
- repoint a stale allow-reason at POST /scopes/{scope_id}/sessions/list; the GET
it named was removed
- document the membership list's ordering under `reverse`
* fix: Address Coderabbit Comments
* fix: Address more coderabbit comments
* chore: remove ai artifacting
* fix: remove unused fakeredis client in lieu of in-memory taskless cache for test suite
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Structured outputs for dialectic
* cleanup
* rename json_schema_to_pydantic to clarify it's not a general schema converter
* clean up schema DoS guards
* simplification and cleanup of schema conversion
* chore: ruff and pyproject toml
* chore: basedpyright cleanup in test
* fix: some needed unrelated test failures
* test(schema_conversion-and-anthropic-backend): expand test coverage
include table tests
* fix(llm): support combined tool calling and structured output across backends
- OpenAI: parse() 500s on non-strict function tools; route tool-carrying
structured requests through create() with an explicit json_schema
response_format (mirrors the streaming path)
- Anthropic: skip the '{' JSON prefill when tools are present so tool_use
blocks stay reachable; make the schema instruction conditional and rely
on parse + repair
- Gemini: native response_schema + function calling is rejected before
Gemini 3; with tools present, inject a schema instruction into the final
turn instead and rely on parse + repair
- All backends: tool-call turns carry no consumable content, so skip
structured-output parsing on them
Extracted from the dialectic structured-output branch (DEV-1652) so the
transport layer can land independently.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(live_llm): exercise combined tools + structured output per provider
Two-turn live flow per backend: a forced tool-call turn (structured
parsing must be skipped) followed by a replay turn that must return a
schema-conforming answer with tools still attached. Asserts the
provider-specific request shaping: no parse() for OpenAI (500s on
non-strict tools), no '{' prefill for Anthropic, no native
response_schema for Gemini.
Verified against live OpenAI (gpt-4.1, gpt-5, gpt-5.4, gpt-5.4-mini)
and Gemini (gemini-2.5-flash).
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(unified): dialectic chat with response_format schema under tool use
Adds response_format pass-through to the unified runner's chat query and
a test case that forces the dialectic tool loop (reasoning off + global
enumeration question) while requiring a schema-conforming JSON answer —
end-to-end coverage of the combined tools + structured output transport
path on whichever provider each level is configured with.
Verified locally against a full harness run (json_match assertions pass;
the llm_judge assertion additionally runs in CI where the Anthropic key
is available).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: some needed unrelated test failures
* ci: add label-triggered live LLM test workflow
Adding the run-live-llm label to a PR (or workflow_dispatch) runs
tests/live_llm/ against real provider APIs — the only place the
--live-llm suite runs in CI. Reuses the unified-tests environment and
its Secrets Manager staging-dotenv resolution for provider keys; runs
on ubuntu-latest (no Fly runner, no Docker — the suite only touches the
LLM backends). Pins LIVE_LLM_ANTHROPIC_45_PLUS_MODELS=claude-sonnet-4-5
since the Anthropic family has no default models and would otherwise
silently collect empty.
Opt-in by design: live model behavior is variable, so this is a signal,
not a required check.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci: run live LLM tests on main pushes touching the transport
Mirrors unified-tests' push trigger, scoped to paths that can affect
the live suite (src/llm/, config, the tests, deps, and the workflow
itself) so provider API calls aren't spent on unrelated changes.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci: disable auth in live LLM test environment
The staging dotenv sets AUTH_USE_AUTH=true without a usable JWT secret,
and src/config.py validates the pair at import time — the same reason
unified-tests overrides it. This suite never runs the API server.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(live_llm): fix gpt-5.4 reasoning_effort and gemini replay-turn flake
- test_live_openai: gpt-5.4 dropped 'minimal' from the reasoning_effort
vocabulary, so the gpt5 caching test 400'd — and the OpenAI backend's
BadRequestError terminal swallowed it into an empty CompletionResult.
Pick the effort per model generation.
- test_live_tools_structured_output: use tool_choice='auto' on the
replay turn, matching the production dialectic loop (which never
forces 'none') — NONE mode is what provoked gemini-2.5-flash's empty
candidates. Drop the temperature pin so retries actually resample,
and treat a repeat tool call as a retryable attempt.
Verified live: full suite green, gemini 4/4 consecutive passes.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci: fail live LLM run when no staging secret was loaded
If the latest-tag fetch fails and no second tag exists, the fallback
step is skipped rather than failed, and the job would proceed without
provider keys — every test then skips via require_provider_key and the
run goes green. Guard on both fetch outcomes so that path fails loudly.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(live-llm-tests-GHA): remove extra comments
* feat(structured-output): enable non-recursive schema references
* docs(structured-outputs): clean up new doc
* test(structured-output): fix caching refs memory leak, add tests
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* telemetry: use session and user IDs in langfuse
* test: update old span test
* fix: disable langfuse in unit tests
* fix: add post-loop synthesis span
* refactor: address PR review feedback on langfuse tracing
- Consolidate track_name onto LLMTelemetryContext as the sole home;
remove the honcho_llm_call kwarg and update 4 callers to set it on
telemetry directly. Sentry ai_track now reads telemetry.track_name.
- Decouple escaped-stream self-stamping from run-context exit ordering:
stream_final_response now resets _in_agent_run explicitly around drain.
- Narrow langfuse_agent_step wrap in the tool loop — between-turn
bookkeeping (iteration_callback, choice switch, increment) lifted
outside the span so it scopes only the LLM call + tools.
- Reword test conftest comment to behavior-only language.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: switch langfuse spans to imperative handles
Replaces the context-manager-based langfuse_agent_run/step with imperative
LangfuseAgentRun/Step handles so the run span can outlive the function that
opens it. Streaming responses now own the run handle from construction and
close it after drain, stamping the accumulated streamed text as trace output
(previously blank). Multi-turn generations always stamp provider/model and
step metadata, fixing the regression where only the first turn was annotated.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(llm): record effective prompt-only input on run span
The run-level Langfuse span recorded the raw messages parameter, which is
None for prompt-only calls. Mirror execute_tool_loop's handling and record
the synthesized user message so the trace input isn't blank.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(llm): drop StreamingResponseWithMetadata.__anext__ to prevent span leak
The standalone __anext__ delegated straight to the inner stream, bypassing
the token-folding and Langfuse run-handle close that live only in the
__aiter__ generator. Any caller driving the wrapper via anext() instead of
`async for` would leak the run span and lose final-stream token accounting.
Latent today (all callers use `async for`), removed to close the footgun.
Add tests covering the run-handle drain path: full drain stamps the
accumulated streamed text as the span output and closes once; an abandoned
stream still closes via the finally rather than leaking.
* chore(llm): document intentional empty-body propagate_attributes block
The `with propagate_attributes(...): pass` stamps the active @observe trace
root via the context manager's __enter__ side effect; the empty body reads
as deletable dead code. Add a comment so it isn't removed. Addresses PR review.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(llm): restore api.py types after __anext__ removal
Dropping StreamingResponseWithMetadata.__anext__ made it stop satisfying
the AsyncIterator protocol, breaking the result annotation and the
isinstance narrowing in honcho_llm_call. Widen the tool-less result
annotation to include StreamingResponseWithMetadata and narrow positively
to HonchoLLMCallResponse before reading .content.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* Peer- and session-scoped JWTs were effectively workspace-scoped: auth() walked the route's declared scope and fell through to a workspace match, so a {w: ws-a, p: alice} token could act on any peer in ws-a.
* feat: peer keys can read sessions they belong to; require workspace on scoped keys
* fix: authorize JWTs by narrowest scope and gate member reads
Follow-up hardening on the narrowest-claim auth fix:
- Scope get_peer_config member-read to the caller's own peer; a session
member could previously read a co-member's per-session config.
- Enforce session membership on POST /peers/{id}/chat: the session_id
arrives in the body (invisible to require_auth), so a peer key could
read any session's injected message history. Check is_peer_in_session
in the handler before the dialectic runs.
- Consolidate the workspace-match check in auth() to a single hoisted
guard so no branch can silently re-open cross-workspace access.
- Normalize empty-string scope claims to None in verify_jwt so a blank
workspace can't satisfy the peer/session token-shape invariant.
- Extract scope_requires_workspace(), shared by verify_jwt and the keys
API so the creation-time guard and verification invariant can't drift.
route requires auth) and CLAUDE.md auth-scoping guidance.
- docs: describe narrow-scope key semantics in the platform reference.
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* feat: defer embedding messages
* fix: rm gauges
* feat: embed messages immediately on create with reconciler fallback (#766)
Adds embed_messages_now background task so newly created messages are
searchable within seconds instead of waiting up to the reconciler
interval. Three-phase claim/lease → embed → persist never holds a DB
session across the embedding call; the reconciler remains the fallback
for failures and stragglers.
* fix: harden immediate-embed fast path and cover its error branches
Wrap embed_messages_now in a top-level try/except so a failure in the
claim or persist phase degrades to "reconciler will retry" instead of
escaping into the background-task runner; the rows stay pending+leased
and the reconciler heals them.
Add tests for the previously-uncovered branches: external-store-unavailable
persist path, the file-upload endpoint's embed scheduling, and direct unit
tests for the shared compute_chunk_positions / build_message_vector_record
helpers.
Document the semantic-search eventual-consistency window in search.mdx
(keyword matches are immediate; vector matches lag creation by seconds).
* fix: don't hold DB session across vector-store upserts
* fix: align semantic-search function to filter null rows
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* feat: implement read DB and fix queue stale cleanup
* fix: use read_db in internal methods
* fix: mention read db in the CLAUDE.md
* fix: make TRACING checkout hook autocommit-safe; sample cleanup-gate jitter once
The DB.TRACING checkout hook ran `SELECT set_config(...)` at pool checkout,
before the dialect applies the read engine's AUTOCOMMIT isolation level. That
statement autobegins a transaction, and psycopg then refuses to switch the
connection into AUTOCOMMIT ("can't change 'autocommit' now: connection in
transaction status INTRANS"), so every read_only session 500s under TRACING and
the INTRANS connection leaks back to poison later write checkouts. Run the hook
in autocommit and restore the prior mode so it never leaves an open transaction;
set_config(..., is_local=false) is session-scoped and survives the boundary.
Add a regression test (fails without the fix) covering read_only + TRACING.
Also sample the stale-cleanup gate's jittered interval once per attempt instead
of re-rolling it every poll, so the spacing is a fixed deadline per cycle rather
than a random walk (and is testable at non-zero jitter ratios).
* fix: reset request_context in TRACING checkout-hook test
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* feat: add generate_jwt.py script for creating scoped JWTs
Adds a CLI utility script for generating Honcho JWTs without needing
to call the /v1/keys API endpoint. Useful for local development and
bootstrapping admin tokens.
Features:
- --admin flag for full-access tokens
- --workspace / --peer / --session flags for scoped tokens
- --expires flag with human-friendly duration syntax (e.g. 5h, 30d, 1y)
- --print-only flag for scripting (outputs bare token)
Examples:
uv run python scripts/generate_jwt.py --admin
uv run python scripts/generate_jwt.py --admin --expires 24h
uv run python scripts/generate_jwt.py --workspace my-ws --expires 30d
uv run python scripts/generate_jwt.py --workspace my-ws --peer my-peer --expires 1y
* docs: document generate_jwt.py in README auth setup section
* fix: remove t='' override to preserve utc_now_iso default in JWTParams
Per CodeRabbit review: explicitly setting t="" bypasses JWTParams's
default utc_now_iso timestamp, causing tokens for the same scope to
become byte-identical. Omitting t lets the default apply, ensuring
each generated token is unique.
* fix: address JWT script review feedback
* fix: type, lint
---------
Co-authored-by: Rajat Ahuja <rahuja445@gmail.com>
* feat(embedding): add dimensions_mode for OpenAI dimensions= forwarding
Add EMBEDDING_MODEL_CONFIG__DIMENSIONS_MODE (auto|always|never) controlling
whether the dimensions= parameter is forwarded on OpenAI embeddings.create
calls. auto (default) sends it when the operator explicitly set
EMBEDDING_VECTOR_DIMENSIONS and the configured model is not on the
known-rejecting allowlist (currently text-embedding-ada-002).
The provenance check (was VECTOR_DIMENSIONS explicitly set?) lives as
EmbeddingSettings.resolve_send_dimensions() because it needs access to
model_fields_set, which the standalone resolver does not have. The
resolved boolean is passed into _EmbeddingClient at construction time;
the client never inspects mode or provenance.
Also pins cloudevents <2.0 — 2.0.0 reorganized the package and dropped
cloudevents.conversion and cloudevents.http, which src/telemetry/emitter.py
imports. The original `>=1.12.0` constraint allowed the broken 2.0 resolve.
With the pin, the imports resolve cleanly and the basedpyright warning
cascade (37+ warnings about unknown types) disappears.
Drive-by cleanups (all unnecessary cast/ignore comments flagged by
basedpyright after the cloudevents downgrade):
- vector_store/lancedb.py, tests/conftest.py, and
tests/deriver/test_vector_reconciliation.py — drop dead pyright ignores
- sdks/python/src/honcho/http/{async_,}client.py — drop unnecessary
cast(datetime, ...) (parsedate_to_datetime already returns datetime)
- vector_store/turbopuffer.py — cast(Any, rows) for the upsert_rows
TypedDict that the SDK exposes but our row builder doesn't satisfy
- tests/test_datetime_parsing.py — ignore reportArgumentType on the
test that deliberately passes wrong types to assert raises
* feat(models): honor EMBEDDING_VECTOR_DIMENSIONS in pgvector columns
* feat(startup): atomic swap dim-vs-MIGRATED guard for runtime schema validator
Add src/startup/embedding_validator.py that introspects the actual pgvector
column dim at boot and refuses to start if it does not match
EMBEDDING_VECTOR_DIMENSIONS. Runs after the DB pool is up and before the
embedding client is constructed, in both src/main.py (FastAPI lifespan) and
src/deriver/__main__.py.
Implementation details:
- Schema-qualified pg_attribute join through pg_class/pg_namespace respects
DB.SCHEMA rather than relying on search_path
- Bounded retry (3 attempts, 1s backoff) for transient introspection failure,
then fail-closed with "could not validate embedding schema" — uncertainty
is not a green light to serve traffic
- External-store sampler (turbopuffer, lancedb) enumerates workspaces from
the application DB and probes their lazy-created namespaces; current
per-namespace probe is a no-op stub since the SDKs do not expose
uniform dim introspection — full enumeration is left to
`configure_embeddings --report` in Phase 3
Atomic guard swap: deletes the old dim-vs-MIGRATED config validator (which
forbade non-1536 pgvector unless MIGRATED=True) in the same commit as the
new runtime validator. There is no release window where non-1536 pgvector
can start unprotected. The 9 dual-write branches that use VECTOR_STORE.MIGRATED
remain untouched and load-bearing for legacy-tenant backend swaps.
VECTOR_STORE_DIMENSIONS deprecation: drop the "must match" raise; in
propagate_namespace, check model_fields_set and emit logger.warning +
DeprecationWarning (DeprecationWarning alone is filtered by Python's default
config and would not reach operators). Always overwrite with
EMBEDDING.VECTOR_DIMENSIONS regardless.
Test changes:
- tests/test_models_vector_dim.py: Phase 1's VECTOR_STORE_TYPE=lancedb +
MIGRATED=true escape hatches removed; the test now passes on plain
EMBEDDING_VECTOR_DIMENSIONS=768
- tests/llm/test_model_config.py: the two tests asserting the old guards
replaced with tests for the new deprecation + acceptance behavior
- tests/startup/test_embedding_validator.py: 10 new tests — dim assertion
logic (pass/mismatch/missing/unbounded/non-public-schema), fail-closed
retry budget, real-test-DB pass, real-DB ALTER-then-validate, deprecation
warning capture, non-1536 + pgvector + MIGRATED=false at config time
* feat(scripts): add configure_embeddings bootstrap CLI
Adds scripts/configure_embeddings.py alongside the other one-off scripts
(provision_db, migrate_db, generate_jwt_secret, etc.). Invoked as
`uv run python scripts/configure_embeddings.py` — same convention as the
existing scripts in that directory, including the sys.path shim that
lets src.* imports resolve when run directly.
Bootstrap step for self-hosted installs at a non-default
EMBEDDING_VECTOR_DIMENSIONS — runs between `alembic upgrade head` and
starting the API/deriver.
pgvector ALTER safety (single transaction):
- LOCK TABLE {schema}.documents, {schema}.message_embeddings IN ACCESS
EXCLUSIVE MODE — closes the TOCTOU window between population check
and ALTER
- COUNT(*) WHERE embedding IS NOT NULL on both tables; refuse with a
non-zero exit if either is populated (ALTER ... USING NULL would
silently wipe those vectors)
- Snapshot HNSW index DDL from pg_indexes; drop, ALTER, recreate from
the captured DDL so operator-set HNSW params (m, ef_construction)
survive the round trip
External vector stores (turbopuffer, lancedb) are never created or
modified — namespaces are per-workspace and lazy-created on first write.
The --report mode enumerates workspaces and collections from the
application DB, derives the expected namespaces via
get_vector_namespace(), and prints a per-namespace status table.
CLI modes (mutually exclusive):
- (default) interactive: print plan, prompt to confirm
- --dry-run: print plan and exit 0 without touching the DB
- --yes: apply without prompt
- --report: print external-store namespace inventory and exit
Also updates src/startup/embedding_validator.py error-message paths and
docs/v3/contributing/configuration.mdx invocations to point at the new
script location.
Tests cover plan no-op, plan needs-alter, plan raises on missing column,
ALTER + HNSW round-trip, refuse-when-populated (monkeypatched count to
avoid wiring the full workspace/peer/collection/document FK chain just
to land one vector row), and idempotency.
* docs: add changing-embeddings operations page
Document the supported way to change EMBEDDING_VECTOR_DIMENSIONS or
EMBEDDING_MODEL_CONFIG__MODEL on a Honcho deployment: provision a new
deployment at the desired configuration, replay source data out of
band, cut over at the application layer.
The page explains the asymmetry:
- Dimension is machine-enforced as immutable. The startup validator
introspects pg_attribute and crashes the API/deriver on mismatch.
- Model is operator-owned. There is no persistent metadata recording
which model produced each vector, so a same-dim model swap is
silently undetectable — flagged with a Warning callout.
Also documents the truncation edge case (text-embedding-3-large truncated to 1536 with EMBEDDING_VECTOR_DIMENSIONS left at default)
and the DIMENSIONS_MODE=always mitigation, plus a pointer that
storage-backend swap (VECTOR_STORE_MIGRATED + reconciler) is a distinct operation unaffected by this work.
Registers the page in docs/docs.json under the Self-Hosting nav group
and cross-links from configuration.mdx.
* fix(embedding): correct turbopuffer regex + tighten DIMENSIONS_MODE docs
- Turbopuffer attribute type for a vector column is `[N]f32` / `[N]f16` /
`[N]i8`, not `f32_vector(N)` as the earlier probe assumed. The earlier
regex returned None for the real SDK format, so existing Turbopuffer
namespaces would have been reported as "missing" instead of validated
for mismatch. Regex switched to `\[(\d+)\]` which is the
vendor-stable shape. Test cases rewritten to lock the actual format.
- docs/v3/contributing/configuration.mdx had a contradictory pair of
bullets: 223 said explicit 1536 makes `auto` forward dimensions=, 224
said `auto` would skip the parameter because 1536 is the default.
Operators reading both would (rightly) conclude they need `always`
even when `auto` would work. Rewrote both bullets so:
- `auto` is provenance-driven (explicit-set, not non-default-value).
- `always` is positioned as defense-in-depth for config layers that
might strip explicit default-valued envs, not the only path for
same-as-default truncation.
* fix(embedding): address PR #678 review comments
CodeRabbit + Rajat review feedback. All actionable items addressed
except two false-positives (responded on PR).
Bug fixes:
- deriver telemetry leak: validator was called outside try/finally so
shutdown_telemetry() did not run on validation failure. Moved inside.
- _emit_report printed "no effect with pgvector" unconditionally,
including from implicit post-apply calls. Added is_report_mode flag;
only print on explicit --report.
- LanceDB and Turbopuffer probes returned None when the namespace
existed but its schema was malformed (no vector field / unparseable
type string), silently bucketing real corruption as "missing"
(lazy-create) and letting it pass the startup validator. Now raise
VectorStoreError with actionable diagnostics; None remains valid only
for "namespace does not exist."
- Startup validator only sampled message namespaces; added a parallel
Collection-row sample so document namespaces are probed too, with the
same dim assertion. Mirrors the --report path.
Hygiene:
- StartupValidationError now subclasses HonchoException so existing
exception handlers recognize it. ValidationException is @final and
has 422 request-validation semantics that would be misleading here.
- scripts/configure_embeddings.py main() no longer spins up two event
loops. engine.dispose() moved into a try/finally inside _async_main
so cleanup runs in the same loop as the pipeline.
- Replaced hand-rolled retry loop with tenacity.AsyncRetrying; same
fail-closed semantics, less code, before_sleep_log for visibility.
- Added _validate_identifier() defense-in-depth: DB.SCHEMA and HNSW
index names are regex-checked against [A-Za-z_][A-Za-z0-9_]* before
SQL interpolation. Operator config + DB catalog are not user input
under the current threat model, but the constraint is cheap to gate.
Test + docs:
- test_app_settings_accepts_non_1536_with_any_vector_store_configuration
now actually exercises turbopuffer (was missing); supplies a dummy
TURBOPUFFER_API_KEY to satisfy the model_validator.
- changing-embeddings.mdx: hyphenated "out-of-band" per reviewer style.
* fix: modify conftest to fix ci
* fix: ci tests for typescript server
* fix: Add JSON repair for truncated LLM responses across all providers and Gemini thinking budget support
LengthFinishReasonError from OpenAI-compatible providers (custom, openai, groq) was crashing the deriver
with 14k+ occurrences in production. The vLLM path already had repair logic but it was gated on
provider=="vllm", unreachable when routing through litellm as a custom provider.
- Extract shared _repair_response_model_json() helper for all providers
- Catch LengthFinishReasonError in OpenAI/custom parse() path and repair truncated JSON
- Add repair fallback to Anthropic and Gemini response_model paths
- Add repair fallback to Groq response_model path
- Pass thinking_budget_tokens to Gemini 2.5 models via thinking_config
- Add 14 tests covering repair paths for all providers and Gemini thinking budget
Fixes HONCHO-YC
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: live llm integration tests
* feat: Consistent Model Config Protocol
* fix: migrate the remaining app callers off the legacy llm_settings path
* fix: Docs and regression tests
* fix: refactor llm runtime path to model-config-only API
* fix: refactor config to nested model-config source of truth
* fix: refactor llm streaming and tool dispatch through backends
* fix: cut over llm config to nested model_config only
* fix: collapse vllm and custom into openai_compatible transport
* feat: refactor llm config to explicit transports and bare model ids
* feat: (embed) Add configurability for embedding model
* fix: tests for embedding provider
* fix: Address Review Comments
* fix: (llm) remove Groq backend and per-vendor base URLs
* chore: move llm tests
* fix: (llm) address review findings — config regressions, backend bugs, dead code
* fix: address backend end silly errors
* chore: (docs) update configuration and self-hosting guides
* chore: fix tests
* fix: address code rabbit comments
* fix: add validation to the dream settings
* fix: further address code rabbit comments
* fix: Address Code Rabbit Comments
* fix: Another round of code rabbit
* fix: Address Code Rabbit Nits
* fix: tests
* refactor: rename thinking validator to reflect transport scope
_validate_anthropic_thinking_minimum only enforces the >=1024 rule for
Anthropic and no-ops for other transports, so the name was misleading
now that it's shared across ConfiguredModelSettings, FallbackModelSettings,
and ModelConfig. Renamed to _validate_thinking_constraints with a docstring
clarifying per-transport behavior. No logic change.
* fix(config): drop transport-specific thinking params when env override changes transport
_fill_defaults_for_nested_field previously preserved the default MODEL_CONFIG's
thinking_budget_tokens/thinking_effort across a transport override. This leaked
Gemini-family defaults (e.g. thinking_budget_tokens=1024) into OpenAI-transport
overrides, and the OpenAI backend then correctly rejected the unsupported param
at call time (OpenAI uses reasoning.effort, not a token budget).
The helper now strips thinking_budget_tokens and thinking_effort from the
default dict when the env override supplies a transport different from the
default's. Explicit thinking params in the override are preserved.
* fix(config): apply thinking-param strip to dialectic level merge too
DialecticSettings._merge_level_defaults does its own inline MODEL_CONFIG
merge (parallel to _fill_defaults_for_nested_field), so the previous fix
missed dialectic-level overrides. E.g. flipping
DIALECTIC_LEVELS__minimal__MODEL_CONFIG__TRANSPORT from gemini (default)
to openai still leaked the default thinking_budget_tokens=0 into the
openai config, which the OpenAI backend then rejected at call time.
The level-merge path now applies the same 'strip transport-specific
thinking params when transport changes' rule as the generic helper.
Added a regression test exercising the merge validator directly.
* refactor(llm): wire ModelConfig knobs through, prune clients.py migration leftovers
Three connected fixes to finish carving the LLM stack out of src/utils/clients.py
and into src/llm/:
1. Propagate ModelConfig tuning knobs into backend calls.
honcho_llm_call_inner built extra_params from only {json_mode, verbosity},
silently dropping top_p, top_k, frequency_penalty, presence_penalty, seed,
and operator-supplied provider_params from any ModelConfig. Thread the
selected config through ProviderSelection and merge
build_config_extra_params(selected_config) into extra_params; per-call
kwargs still win over provider_params defaults. Makes
_build_config_extra_params public as build_config_extra_params so
clients.py and request_builder.py share one translation. Adds
TestModelConfigExtraParamsPropagation covering OpenAI/Anthropic knob
propagation, provider_params passthrough, and per-call override
precedence.
2. Drop dead extract_openai_* duplicates in clients.py.
extract_openai_reasoning_content, extract_openai_reasoning_details, and
extract_openai_cache_tokens had no callers outside their own definitions
— the live implementations live in src/llm/backends/openai.py. -103
lines from clients.py.
3. Unify on ModelTransport, delete SupportedProviders.
The "google" vs "gemini" split forced a _provider_for_model_config
translation shim in two places. Replace all SupportedProviders usages
with ModelTransport, rename CLIENTS["google"] → CLIENTS["gemini"],
update provider branches + LLMError labels + reasoning-trace entries
accordingly. Trace JSONL now writes "provider": "gemini" instead of
"google" — consistent with the broader env-var rename cutover.
Also tidies up pre-existing basedpyright findings in tests/llm/test_model_config.py
(pydantic before-validator dict inputs + descriptor-proxy call).
ruff: clean. basedpyright: 0 errors, 0 warnings. Tests: 153/153 pass across
tests/utils/test_clients.py, tests/utils/test_length_finish_reason.py,
tests/llm/, tests/dialectic/, tests/deriver/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(llm): finish the src/utils/clients.py → src/llm/ migration
honcho_llm_call_inner now delegates to request_builder.execute_completion
and execute_stream instead of re-implementing backend call scaffolding
inline. The new _effective_config_for_call helper carries per-call kwargs
(temperature, stop_seqs, thinking_budget_tokens, reasoning_effort) onto
the selected ModelConfig — or synthesizes a minimal config for the
test-only callers that pass provider+model directly. max_output_tokens
is zeroed on the effective config to preserve the current
"per-call max_tokens wins" semantic; honoring ModelConfig.max_output_tokens
is a separable correctness concern.
Side effect of routing through the new path: ConfiguredModelSettings'
thinking_budget_tokens validator now fires on synthesized configs.
test_anthropic_thinking_budget was asserting that a sub-1024 budget
propagated to Anthropic — bumped to 1024 to match what Anthropic actually
accepts.
Unified client construction. Promoted the cached client factories in
src/llm/__init__.py (get_anthropic_client, get_openai_client,
get_gemini_client, get_{anthropic,openai,gemini}_override_client) to
public API and added them to __all__. Promoted
credentials._default_transport_api_key → default_transport_api_key.
Deleted the duplicate _build_client and _default_credentials_for_provider
from clients.py; _client_for_model_config now falls through to the
public factories. CLIENTS dict and _get_backend_for_provider stay as the
mockable seam for the ~50 patch.dict(CLIENTS, {...}) test call sites.
Wired operator-configurable Gemini cached-content reuse end-to-end.
PromptCachePolicy moved from src/llm/caching.py into src/config.py so
ModelConfig can reference it as a field without a circular import;
caching.py re-exports the name for existing imports. Added
cache_policy: PromptCachePolicy | None on ConfiguredModelSettings,
FallbackModelSettings, ResolvedFallbackConfig, and ModelConfig.
resolve_model_config, _resolve_fallback_config, and
_select_model_config_for_attempt copy the field through.
honcho_llm_call_inner passes effective_config.cache_policy into
execute_completion / execute_stream, so operators opt in via
e.g. DERIVER_MODEL_CONFIG__CACHE_POLICY__MODE=gemini_cached_content
and the selection actually fires instead of sitting on a dead path.
New regression test test_cache_policy_reaches_gemini_backend asserts the
PromptCachePolicy object reaches the Gemini backend's extra_params.
ruff + basedpyright: clean. Tests: 154/154 pass across
tests/utils/test_clients.py, tests/utils/test_length_finish_reason.py,
tests/llm/, tests/dialectic/, tests/deriver/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(llm): move all LLM orchestration into src/llm/ and delete clients.py
The 1624-line src/utils/clients.py has been carved up into focused modules
under src/llm/ and deleted. There is now one golden path for LLM
orchestration and no dual entrypoint.
New module layout:
src/llm/
__init__.py thin stable re-export surface
api.py public honcho_llm_call with retry + fallback + tool
loop delegation
executor.py honcho_llm_call_inner (single-call executor); bridges
to request_builder.execute_completion / execute_stream
tool_loop.py execute_tool_loop + stream_final_response, plus
assistant-tool-message and tool-result formatting
runtime.py AttemptPlan dataclass (replaces the loose
ProviderSelection NamedTuple), effective_config_for_call,
plan_attempt, per-retry temperature bump, attempt
ContextVar
registry.py single owner of CLIENTS dict + cached default and
override SDK-client factories + backend/history-adapter
selection + high-level get_backend(config)
conversation.py count_message_tokens, tool-aware message grouping,
truncate_messages_to_fit
types.py HonchoLLMCallResponse, HonchoLLMCallStreamChunk,
StreamingResponseWithMetadata, IterationData,
IterationCallback, ReasoningEffortType, VerbosityType,
ProviderClient
request_builder.py low-level request assembly (ModelConfig → backend
complete/stream); no longer owns credential resolution
credentials.py default_transport_api_key, resolve_credentials
caching.py gemini_cache_store; re-exports PromptCachePolicy
from src.config
backend.py Protocol + normalized result types
history_adapters.py provider-specific assistant/tool message shapes
structured_output.py
backends/ AnthropicBackend, OpenAIBackend, GeminiBackend
handle_streaming_response had no production callers; it is deleted. The
three tests that used it now drive honcho_llm_call_inner(stream=True,
client_override=...) directly, which exercises the same code path the
public API uses.
Dead credential passthrough removed. The ProviderBackend Protocol and
all three concrete backends no longer accept api_key / api_base — those
are baked into the underlying SDK client at registry construction time
and were being del'd everywhere they appeared. request_builder also
stops resolving and forwarding them.
Client construction is unified. The cached default-client factories
(get_anthropic_client, get_openai_client, get_gemini_client) and override
factories (get_*_override_client) are promoted to public API; the
module-level CLIENTS dict populates from them and remains the
patch.dict(CLIENTS, {...}) mocking seam tests rely on. Old duplicate
helpers (_build_client, _default_credentials_for_provider) are gone.
default_transport_api_key is promoted to public.
Application imports now come from src.llm (dreamer, dialectic, deriver,
summarizer, telemetry-adjacent tests). No code imports from
src.utils.clients anywhere in the repo.
ruff: clean. basedpyright: 0 errors, 0 warnings. Tests: 1013/1013 pass
across the entire non-infra test suite (excluding tests/unified,
tests/bench, tests/live_llm, tests/alembic).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(llm): sanitize tool schemas for Gemini's function_declarations validator
Gemini's native-transport function-declarations validator accepts a narrow
subset of JSON-Schema / OpenAPI: type, format, description, nullable, enum,
properties, required, items, minItems, maxItems, minimum, maximum, title.
Anything else — additionalProperties, allOf, if/then/else, $ref, anyOf,
oneOf, $defs, patternProperties — triggers an INVALID_ARGUMENT 400 at call
time.
Our agent tool schemas in src/utils/agent_tools.py use several of those
(additionalProperties: false, allOf + if/then conditionals) because they
were authored for OpenAI strict-mode + Anthropic, which need the richer
vocabulary. GeminiBackend._convert_tools was passing them straight through.
Add _sanitize_schema(): walks the parameters tree and drops unsupported
keywords while preserving semantics for the keywords that hold user data
(properties maps field-name → sub-schema; required / enum are lists of
literals; items is a single sub-schema). Other backends are untouched and
continue to receive the full strict schemas.
Regression tests:
- test_gemini_sanitize_schema_strips_unsupported_keywords: confirms
additionalProperties, allOf + if/then, and $defs are stripped at nested
levels while legitimate fields survive.
- test_gemini_convert_tools_sanitizes_parameters_schema: end-to-end
_convert_tools output has no forbidden keys.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: fix tool calling syntax for gemini
* refactor(llm): normalize defaults, widen OpenAI reasoning-model routing
* chore: fix test
* fix(llm): address post-migration review feedback
* fix(llm): gemini robustness + dreamer specialist ergonomics
* chore: addres review comments
* chore: (docs) unrelease changelog addition
* chore: (docs) merge commit changes
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Erosika <eri@plasticlabs.ai>
* fix: further remove extraneous transactions
* fix: (search) use 2 phase function to reduce un-needed transaction
* fix: refactor agent search to perform external operations before making a transaction
* fix: reduce scope of queue manager transaction
* fix: (bench) add concurrency to test bench
* fix: address review findings for search dedup, webhook idempotency, and bench throttling
* Fix Leakage in non-session-scoped chat call (#526)
* fix: (search) reduce scope for peer based searches
* fix: tests
* fix: (test) address coderabbit comment
* fix: drop db param from deliver_webhook
---------
Co-authored-by: Rajat Ahuja <rahuja445@gmail.com>
* fix: dialectic held connection
* fix: (agent) pre-compute embeddings for agent tools
* fix: (tests) refactor tests to use smaller test db connections
* fix: Embedding client to branch depending on vector store
* fix: reflect dedup-skipped observations in created counts and isolate DB sessions in extract_preferences
* fix: (tests) update tests to match changes
* fix: expunge docs + don't pass in db to query_documents
---------
Co-authored-by: Rajat Ahuja <rahuja445@gmail.com>
* chore: parallelize tests for speed
* fix: truncate all tables after each test
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* chore: 3.0 honcho and 2.0 sdks changelog
fix: use PeerContextResponse in peer.ts
* chore: move docs to /v3/, build SDKs
* chore: code review
* feat: [WIP] migrate away from stainless in typescript sdk
* chore: move api from /v2/ to /v3/
* feat: no-stainless typescript with real tests
* feat: migrate python sdk off of stainless
* feat: clean typescript sdk
* chore: add tests for ts http client
* fix: rewrite entire python sdk in new format, update typescript sdk to use `configuration` not `config` for consistency with API
* fix: clean up SDKs, synchronize
* chore: update sdk examples
* chore: update OpenAPI documentation and SDK examples to reflect changes
* fix: better test
* fix: install deps in test runner, improve robustness of streaming in sdk, coderabbit nits
* fix: standardize around camelCase in TS SDK
* refactor: update configuration handling in SDKs to use typed models for workspace, session, and peer configurations
* docs: clarify queue status usage and remove polling methods from SDKs
add claude skills for migrations
* chore: fix links in docs
* feat: add deriver flush mode to bypass batch token threshold
- Introduced `is_deriver_flush_enabled` function to check if flush mode is active.
- Updated `QueueManager` to conditionally apply batch token thresholds based on flush mode.
- Enhanced `UnifiedTestExecutor` to enable flush mode via Redis.
- Added `flush` parameter to test cases to facilitate testing of flush mode behavior.
- Updated various test cases to utilize the new flush functionality.
* feat: implement schedule_dream functionality in SDKs, use in unified test runner
- Added `schedule_dream` method to both Python and TypeScript SDKs for scheduling dream tasks.
- Updated HTTP routes to include endpoint for scheduling dreams.
- Enhanced test runner to utilize the new `schedule_dream` method for scheduling actions.
- Updated TypeScript client to support the new scheduling functionality with appropriate parameters.
* feat: update single deriver task to support multiple observers
- Changed the `observer` parameter to `observers` as a list in multiple functions across the deriver module.
- Updated the processing logic to handle multiple observers for representation tasks.
- Adjusted related payload and queue management functions to accommodate the new observers structure.
- Modified tests to reflect changes in the representation task handling and ensure proper functionality.
* refactor: update enqueue tests to support deduplication of queue items with multiple observers
- Modified tests in `test_enqueue.py` to reflect changes in the queue item structure, where each message now results in a single queue item containing a list of observers.
- Updated assertions to validate that the `observers` field correctly includes all relevant peers, ensuring proper functionality of the deduplication logic.
- Removed redundant payload matching logic to streamline test cases and improve clarity.
* fix: add backwards compatibility for representation work unit keys and payload observers
* feat: update dialectic configuration and introduce cost calculator
- Adjusted LLM and dialectic settings in `.env.template`, `config.toml.example`, and `src/config.py` to reduce maximum tool output characters and session history tokens for cost efficiency.
- Implemented a new `dialectic_cost_calculator.py` script to estimate costs based on reasoning levels and model pricing.
- Enhanced `DialecticAgent` to utilize minimal tools and adjusted output token settings based on reasoning level to optimize performance and reduce costs.
* feat: add reasoning level to chat input in unified test runner
- Enhanced the `UnifiedTestExecutor` to include a `reasoning_level` parameter in the chat method call.
- Updated the `QueryAction` model to support the new `reasoning_level` attribute, allowing for more nuanced chat interactions.
* feat: run deriver once for multiple observers (#335)
* feat: update single deriver task to support multiple observers
- Changed the `observer` parameter to `observers` as a list in multiple functions across the deriver module.
- Updated the processing logic to handle multiple observers for representation tasks.
- Adjusted related payload and queue management functions to accommodate the new observers structure.
- Modified tests to reflect changes in the representation task handling and ensure proper functionality.
* refactor: update enqueue tests to support deduplication of queue items with multiple observers
- Modified tests in `test_enqueue.py` to reflect changes in the queue item structure, where each message now results in a single queue item containing a list of observers.
- Updated assertions to validate that the `observers` field correctly includes all relevant peers, ensuring proper functionality of the deduplication logic.
- Removed redundant payload matching logic to streamline test cases and improve clarity.
* fix: add backwards compatibility for representation work unit keys and payload observers
* feat: refactor benchmark runners to share common functionality
- Introduced a new `runner_common.py` module containing shared utilities for benchmark test runners, including common argument parsing, client creation, and queue management.
- Updated `BEAMRunner`, `LoCoMoRunner`, and `LongMemEvalRunner` to inherit from `RunnerMixin`, leveraging shared functionality for metrics collection and logging.
- Added `reasoning_level` and `redis_url` parameters to runner constructors for enhanced configuration.
- Streamlined argument parsing by utilizing `add_common_arguments` for shared command-line options across all runners.
* fix: update last_user_message handling to use message content instead of ID
* fix: standardize config vs configuration
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* feat: fix race condition in message sequence batching
* fix: CodeRabbit comments; commit early to release the advisory lock before generating embeddings
* fix: use index + rm unused method
* fix: PR comments
* fix: bug in lock timeout
* fix: patch tracked_db for peers route within conftest.py
* feat: add optional JWT and webhook secrets to honcho instance creation
* chore: ignore spurious warnings
* feat: add response format if using gpt-5 model family
* feat: add response models to all apis except anthropic
* fix: raise NotImplementedError for response models in AsyncAnthropic client
* chore: address review
* [WIP] representation structure + deriver cleanup
* chore: add tests, cleanup
* feat: [WIP: semi-working] representation object
* fix: alignment
* fix: make observations hashable for dedup
* fix: datetime formatting, observation counting
* fix: switch to int for message id, clean up representation
* feat: remove need for metadata working rep
* chore: cleanup
* fix: use tenacity instead of custom fns
* feat: add representation and card to context if desired
* feat: add semantically relevant observations
* fix: pass all params to streaming, nonblocking streaming
* feat: consolidate document saving, make working representation fetching much smarter
* chore: add 100% test coverage of representation util
* feat: basic dream infra
* feat: dream queue item first pass
* chore: fixes & cleanup from coderabbit
* fix: dreams scheduled when new document count reaches a certain threshold
* feat: wip: timed dreams (not working)
* fix: test
* fix: remove useless pyright ignore
* fix: executing dreams
* feat: dreaming
* feat: [WIP] longmemeval bench
* feat: add USE_PEER_CARD setting, fix longmem test driver
* feat: get full working rep for dialectic in one swoop -- fix representation_from_documents to use the proper timestamp!
* fix: timestamps for real, handle assistant qs in longmem
* fix: remove old client, add batching to longmem
* perf: remove duplicate detection, will move to background task
* feat: track perf metrics on evals
* feat: adjust deriver prompt to use peer_id, add question date to question, clean up deriver
* fix: label metrics by task for better perf trace
* chore: code review
* feat: add efficiency score to longmem bench
* chore: tuning and cleaning up eval
* chore: bring in the big prompts
* feat: add support for vllm client
* feat: perf: bundle db calls in deriver and dialectic, increase max conns in docker db
* feat: add merge-sessions flag to longmemeval, add SUMMARY_ENABLED flag
* fix: COLLECT_METRICS default false
* chore: display start/end message ids, don't include in metrics
* fix: break large messages apart for eval
* fix: only get/create collection when needed
* feat: properly attribute documents with message id ranges and add session name column to documents
* fix: revert move of get_or_create_collection (need for fkey)
* fix: always get collection with peer name even if it's none
* chore: coderabbit
* fix: give peer card its own config, expand document schema, refactor get_context to be parallel, various cleanup chores and bugfixes
* chore: refactor: reify observer/observed system across entire codebase, including db migration
* refactor: cleanup code organization, make singletons where desired
* refactor: replace embeddings store with representation manager
* chore: coderabbit cleanup
* feat: multi-db longmem harness
* Merge branch 'main' into ben/multi-db-harness
* [WIP] feat: add delete workspace endpoint, use in bench
* chore: move excess logging to debug
feat: improve metrics block logs to include more data
fix: make longmem db deletion configurable
* fix: [CRITICAL] use async genai client
* chore: update core sdk, fix tests to use aio as well
* fix: rollback prompt changes
* chore: update version
* fix: cleanup, coderabbit, wrap delete op in try/except
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* feat: add optional JWT and webhook secrets to honcho instance creation
* chore: ignore spurious warnings
* feat: add response format if using gpt-5 model family
* feat: add response models to all apis except anthropic
* fix: raise NotImplementedError for response models in AsyncAnthropic client
* chore: address review
* [WIP] representation structure + deriver cleanup
* chore: add tests, cleanup
* feat: [WIP: semi-working] representation object
* fix: alignment
* fix: make observations hashable for dedup
* fix: datetime formatting, observation counting
* fix: switch to int for message id, clean up representation
* feat: remove need for metadata working rep
* chore: cleanup
* fix: use tenacity instead of custom fns
* feat: add representation and card to context if desired
* feat: add semantically relevant observations
* fix: pass all params to streaming, nonblocking streaming
* feat: consolidate document saving, make working representation fetching much smarter
* chore: add 100% test coverage of representation util
* feat: basic dream infra
* feat: dream queue item first pass
* chore: fixes & cleanup from coderabbit
* fix: dreams scheduled when new document count reaches a certain threshold
* feat: wip: timed dreams (not working)
* fix: test
* fix: remove useless pyright ignore
* fix: executing dreams
* feat: dreaming
* feat: [WIP] longmemeval bench
* feat: add USE_PEER_CARD setting, fix longmem test driver
* feat: get full working rep for dialectic in one swoop -- fix representation_from_documents to use the proper timestamp!
* fix: timestamps for real, handle assistant qs in longmem
* fix: remove old client, add batching to longmem
* perf: remove duplicate detection, will move to background task
* feat: track perf metrics on evals
* feat: adjust deriver prompt to use peer_id, add question date to question, clean up deriver
* fix: label metrics by task for better perf trace
* chore: code review
* feat: add efficiency score to longmem bench
* chore: tuning and cleaning up eval
* chore: bring in the big prompts
* feat: add support for vllm client
* feat: perf: bundle db calls in deriver and dialectic, increase max conns in docker db
* feat: add merge-sessions flag to longmemeval, add SUMMARY_ENABLED flag
* fix: COLLECT_METRICS default false
* chore: display start/end message ids, don't include in metrics
* fix: break large messages apart for eval
* fix: only get/create collection when needed
* feat: properly attribute documents with message id ranges and add session name column to documents
* fix: revert move of get_or_create_collection (need for fkey)
* fix: always get collection with peer name even if it's none
* chore: coderabbit
* fix: give peer card its own config, expand document schema, refactor get_context to be parallel, various cleanup chores and bugfixes
* chore: refactor: reify observer/observed system across entire codebase, including db migration
* refactor: cleanup code organization, make singletons where desired
* refactor: replace embeddings store with representation manager
* chore: coderabbit cleanup
* chore: update migration to non-null session param in documents, general review and cleanup
* chore: merge branch 'main' into ben/deriver-tidy
* chore: review fixes
* feat: add optional JWT and webhook secrets to honcho instance creation
* chore: ignore spurious warnings
* feat: add response format if using gpt-5 model family
* feat: add response models to all apis except anthropic
* fix: raise NotImplementedError for response models in AsyncAnthropic client
* chore: address review
* chore: add tests, cleanup
* fix: use tenacity instead of custom fns
* fix: pass all params to streaming, nonblocking streaming
* chore: fix test mock
* fix: long-held connections in queue manager
* fix: CR comments. tests
* fix (deriver): Limit # of work units claimed based on available workers
* fix: detached instance
* fix: batch claim work units
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* type stuff
* add action
* bump python
* Refactor type annotations and update tracking decorators in agent and dependencies modules. Replace ai_track with track from src.utils.types, and enhance type hints for better clarity. Update pyproject.toml to allow untyped libraries.
* type everything basically
* fix migration typing
* type like crazy
* remove usless tests
* Update mocks in tests to use AsyncMock for dialectic_call and dialectic_stream, ensuring proper async behavior in test cases. Adjust mock return values for consistency and clarity.
* Update src/deriver/tom/single_prompt.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Update src/deriver/tom/long_term.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Enhance CLAUDE.md documentation with additional details on core concepts, API structure, and development commands. Update command syntax for running server and tests to use 'uv run' for consistency. Improve clarity in configuration and architectural decisions sections.
* Refactor type annotations in CRUD functions to accept more flexible filter types, changing from dict[str, str] to dict[str, Any]. Clean up logging in agent.py by removing unnecessary timing logs for user representation generation and query execution.
* Remove unused import of ai_track from long_term.py and single_prompt.py to clean up the codebase.
* pass tests
* update some stuff
* fix unused
* ruff
* make stuff work again
* Add LLM_GROQ_API_KEY to GitHub Actions and format tom_inference parameters
* test
* test
* Refactor LLM settings to use 'gemini' provider and update related model parameters; remove unused API keys from GitHub Actions workflow.
* Update LLM settings to use 'anthropic' provider and change model to 'claude-3-5-haiku-20241022'; maintain existing summarization provider.
* test
* llm provider stuff
* update
* revert
* Integrate client management for LLM providers across various modules; remove deprecated environment variable setup for API keys.
* only if key avaialble
* Refactor type hints and improve schema definitions for queue processing; remove unused imports and enhance function signatures for clarity.
* fix test
* model
* test
* Update LLM provider type annotations and enhance client management; replace Provider with Providers for better type handling in config and clients modules.
* Refactor LLM provider handling to default to "openai" for custom providers across multiple modules; update type annotations and improve client management for consistency.
---------
Co-authored-by: Dani Balcells <18307962+danibalcells@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* chore: Update versioning for release
* fix: remove db creation at start and sync migrations and models
* fix: Checkpoint changing metamessage schema
* chore: linter fixes
* fix: session cloning working
* Hybrid long-term memory (#92)
* Add TOM method switching
* Add system prompt and note on format
* Add persistence tweaks
* Specify format for each section of user representation
* Parse XML tags before saving representation metamessage
* Clean up
* Use Claude 3.5 Haiku and refine prompt
* Simplify message processing
* chore: update token limit on dialectic and model for deriver
* Add embedding-based long-term fact retrieval
* Fix bug preventing new documents from being created
* Use multiple queries + tweak prompt
* Fix collection name bug + add duplicate removal
* First implementation of on-demand user rep generation
* WIP debug on-demand user rep changes
* Fixed representations not being stored & deriver issue
* Some speed improvements
* Play with number of facts / queries
* WIP prompt caching for Claude
* WIP fix anthropic caching
* Anthropic prompt caching working but messages too short
* Use Cerebras for small inferences
* Make dialectic responses 1000 tokens max
* Make user representation generation model a constant
* Use llama 3.1 8b for query generation
* Update env template
* Add crud.get_or_create_protected_collection
* rabbit comments
* Fix linter issues
* Add Cerebras to stream router method
* Better handling of default-empty string args
* Change prints to debug logs
* Add error handling to TOM inference
* Handle missing/empty client in model responses
* Handle no messages case in get_chat_history
* Fix indent
* Add error handling to single_prompt methods
* Fix get_or_create_user_protected_collection
* Simplify openAI-compatible model client instantiation
* Remove health endpoint
* Remove LocalEmbeddingStore
* Change prints to debug logs
* Change sentry track
* Code review changes
* Add README to ToM module
* Switch to Groq
* Fix inconsistent openai compatible provider list in stream()
* Update env template to include Groq variables
* Add model_client tests
* fix: Fix unit tests
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* add scoped API keys (#91)
* add AUTH_JWT_SECRET and ADMIN_KEY, use in security middleware (TODO granular keys)
* WIP: convert all API paths to use scoped keys
* add basic unit tests for API keys, ruff formatting
* MVP of route using JWT for payload
* add get_user_from_token
* add key table to postgres, use it to enable key revocation
* add key revocation pt 2 -- fix order of param checks
* finish convenience routes that assume params from JWT
* add tests for key API
* get_keys
* add secrets utility script, add key rotation, fill out tests
* add tiny cache as PoC
* nits, validations, etc
* only create keys table migration if necessary
* fix keys tests to always use auth
* tiny fix to make custom DATABASE_SCHEMA work
* review: add better docs, fix security issue with cache, clear db on rotation, and more
* remove rotation
* remove key database entirely
* Add `/all` path to get all apps (#94)
* add `/all` path for apps
* assert vector extension installed (need this for groudon)
* review
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* add scoped API keys (#91)
* add AUTH_JWT_SECRET and ADMIN_KEY, use in security middleware (TODO granular keys)
* WIP: convert all API paths to use scoped keys
* add basic unit tests for API keys, ruff formatting
* MVP of route using JWT for payload
* add get_user_from_token
* add key table to postgres, use it to enable key revocation
* add key revocation pt 2 -- fix order of param checks
* finish convenience routes that assume params from JWT
* add tests for key API
* get_keys
* add secrets utility script, add key rotation, fill out tests
* add tiny cache as PoC
* nits, validations, etc
* only create keys table migration if necessary
* fix keys tests to always use auth
* tiny fix to make custom DATABASE_SCHEMA work
* review: add better docs, fix security issue with cache, clear db on rotation, and more
* remove rotation
* remove key database entirely
* Add `/all` path to get all apps (#94)
* add `/all` path for apps
* assert vector extension installed (need this for groudon)
* review
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* chore: README and CHANGELOG updates
* add JWT expiry
* fix: Consolidate get methods with JWT token resolution
* chore: Add Annotation to Path, Query, and Body params
* chore: run ruff formatter
* chore: nits & add one exhaustive test of a query route
* fix: undo change to fly.toml
* fix: Langfuse tracing
* Consolidate Get Methods (#96)
* fix: Consolidate get methods with JWT token resolution
* chore: Add Annotation to Path, Query, and Body params
* chore: run ruff formatter
* chore: nits & add one exhaustive test of a query route
* fix: undo change to fly.toml
---------
Co-authored-by: dr-frmr <docterformer@protonmail.com>
* fix: dev-667 fix streaming endpoint
* fix: Anthropic Langfuse Tracing
* fix: add scripts folder to dockerfile
* fix: Remove redundant fields from pydantic schemas
* fix: Add deeper protection on reserved collection
* fix: Consolidate chat and stream methods
* docs: Update Mintlify API Reference and Changelog
* remove langchain guide, update architecture diagram
* honcho mcp server
* chore: Update .env template
* update discord, temporarily remove other guides
* Limit dialectic & deriver context usage with two-scale progressive summarization (#97)
* WIP two tiered summaries
* Move to process_item
* Save user rep metamessage even if no message_id
* Change number of messages per short summary
* Fix broken mock
* Remove prints
* chore: fix test
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* feat: Add Gemini Support, link facts to message, use 8b for dialectic fact queries
* chore: Styling
* chore: coderabbit nitpicks
* keep dialectic guide
* Add streaming guide
* Remove TODO from dialectic guide
* Fix JS snippets that referred to honcho singleton as client
* Add App explanation to architecture page
---------
Co-authored-by: Dani Balcells <18307962+danibalcells@users.noreply.github.com>
Co-authored-by: doria <93405247+dr-frmr@users.noreply.github.com>
Co-authored-by: dr-frmr <docterformer@protonmail.com>
Co-authored-by: vintro <vince@plasticlabs.ai>
Co-authored-by: Daniel Balcells <dbalcells@gmail.com>
* Add TOM method switching
* Add system prompt and note on format
* Specify format for each section of user representation
* Clean up
* Use Claude 3.5 Haiku and refine prompt
* chore: update token limit on dialectic and model for deriver
* chore: Remove healthcheck endpoint
* feat: Fix inconsistent error handling
* fix: remove SQL echo for performance and increase dialectic to 300 tokens on stream
* chore: Update CLAUDE.md
* fix: Update Dialectic 3.7 Sonnet and add to Changelog
* chore: Update Version Number
---------
Co-authored-by: Daniel Balcells <dbalcells@gmail.com>
* fix: remove opentelemtry code and make deriver workers configurable
* feat(storage) change get route for messages and metamessages to use POST for options
* feat(storage) Change remaining paginated gets to POST requests with options
* fix(embedding) use base openai for embedding and remove azure
* feat(models) switch to nanoids with internal and public id system
* feat(pydantic) Updates models and routes to get tests passing
* fix(deriver) fix enqueue method
* Perf Testing Utilities
* feat(ci) Add github action with github service container
* chore(ci) Change hostname for service container
* chore(ci) Add Trust Flag
* chore(ci) Use Defaults
* readability
* Try to use localhost
* chore(ci) Use non-default database
* Try with default credentials but non-standard db
* Try to ignore pwd
* feat(dialectic) Allow for batch questions and load session history
* feat(dialectic) parallelize facts and history queries
* feat(agent) Addresses dev-258 allow specifying additional collections
* feat(uv) switched from poetry to uv
* feat(deriver) Turn off derivations by editing session medatadata with a deriver_disabled flag
* fix(tests) Clean up test logic
---------
Co-authored-by: Vineeth Voruganti <vineeth@macbook-pro.mynetworksettings.com>
* chore: Save Point
* fix(db): change connection logic and remove unnecessary refreshes
* fix(documents): switch to Azure embedding model
* feat: Setup pytest fixtures
* feat(tests): Initial test routes for tranche 1
* feat(test) tranche 2 of tests and associated bug fixes
* feat(test) tranche 3 of tests and associated bug fixes
* fix(tests) Address PR comments and update version and changelog
Aakash Kattelu
Vineeth Voruganti
Eugene Eisenstein
Rajat Ahuja
Anthony Yuan
doria
Ayush Paul