honcho/tests/dialectic
Vineeth Voruganti be7ad9d919
Scopes Phase 2b: `scope` option on chat, representation, context, and search (#897)
* fix: apply session scoping to all working-representation query paths

session_name was only applied to the recent-documents query in
RepresentationManager; the semantic and most-derived paths ignored it,
so limit_to_session leaked cross-session conclusions into perspectives.

- Thread a session allowlist (session_names) uniformly through all
  three query paths; pushed down to pgvector and external vector stores
- Accept a list so the upcoming session-allowlist API reuses this path
- Fail closed on an empty allowlist (downstream stores drop empty IN
  clauses, which would silently widen scope)

Fixes DEV-1994

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: bare-list membership sugar in the filter DSL

{"session_id": ["s1", "s2"]} is now shorthand for
{"session_id": {"in": [...]}} on regular columns, generically
(peer_id, etc.). JSONB metadata columns are excluded — a bare list
there keeps JSONB containment semantics, unchanged.

Previously a bare list on a regular column compiled to a type-mismatched
equality that matched nothing, so this is strictly additive.

Also translates the same shape in the turbopuffer/lancedb filter
builders, and fixes lancedb dropping empty IN clauses (fail-open) —
an empty membership list now emits an always-false condition.

Groundwork for DEV-1995 (session allowlist via the existing filters
DSL, no new API params)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: session allowlist on dialectic and representation via filters

Adds a constrained 'filters' body to peer.chat and /representation —
the same DSL search and conclusions already accept, supporting only the
session_id key (a session id, a bare list, or {"in": [...]}).
Unsupported keys and shapes are rejected with 422, never silently
ignored. Composes with session_id (must be included in the allowlist
when both are given). Capped at 1,000 sessions per request.

Enforcement is uniform at every recall chokepoint, fail-closed:
- dialectic prefetch + search_memory: conclusion recall restricted to
  the allowlist; dream docs (session_name IS NULL) excluded
- message tools (search/grep/date-range/temporal/context/history):
  strict intersection of allowlist and observer session membership
- get_reasoning_chain: unavailable under an allowlist (chains traverse
  provenance across sessions and cannot be scoped without leaking)
- empty allowlist short-circuits to empty results everywhere

Auth: workspace keys pass the allowlist as-given; peer-scoped JWTs must
be a member of every allowlisted session (403 otherwise), mirroring the
existing single-session check.

Fixes DEV-1995

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: reserve scope__ peer namespace with kind flag and guardrails

Introduce the scope peer namespace (scope__<name>) and the authoritative
{"kind": "scope"} configuration flag, plus the server-side guardrails:

- src/utils/scopes.py: single source of truth for the prefix, kind flag,
  and name helpers (scope_peer_name / is_scope_peer_name /
  scope_name_from_peer / validate_no_scope_peer_names)
- reject reserved-prefix names on peer get-or-create (422)
- reject scope peers as message authors in crud.create_messages (422)
- reject scope peers as chat/representation targets (422); a scope peer
  as the path-level observer is deferred to Phase 2b
- reject scope peers on the generic session-peer add/set/remove routes
  and the session-create peers mapping (422, directing to scopes routes)
- peers.list excludes scope peers by default; new PeerGet.kind option
  ("scope" | "all") switches the view via a configuration JSONB filter
- schemas: Scope / ScopeCreate / ScopeSessions(Add) and
  SessionCreate.scopes (unprefixed scope names, validated)

Part of DEV-1997 (Scopes RFC DEV-1970).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: add scopes CRUD routes and session-create scopes wiring

New /v3/workspaces/{workspace_id}/scopes facade (workspace-level auth;
peer- and session-scoped keys are rejected):

  POST   ""                                   create-or-get (201/200)
  POST   /list                                 paginated scope list
  GET    /{scope_id}                           single scope
  POST   /{scope_id}/sessions                  add memberships
  DELETE /{scope_id}/sessions/{session_id}     remove membership
  GET    /{scope_id}/sessions                  list member session ids

- crud/scope.py: get_or_create_scopes stamps the backing peer with
  {"kind": "scope", "observe_me": false} and refuses to adopt a
  legacy peer occupying the reserved name without the flag (409)
- memberships are session_peers rows with observe_others=true /
  observe_me=false — identical to a hand-built observer peer
- SessionCreate.scopes: create-or-get each scope peer and add the
  membership at session creation (the no-backfill common path)
- crud/session.py: public upsert_session_peers wrapper so the facade
  bypasses the route-level guardrails without reaching into privates

Backfill of pre-existing documents and reconciliation on removal land in
DEV-1999; membership only affects messages ingested after the change.

Part of DEV-1997 (Scopes RFC DEV-1970).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: cover scopes facade, guardrails, and observer semantics

- create-or-get idempotency, list/get, name validation, legacy-collision
  rejection (409), auth scoping (workspace key ok, peer/session keys 401)
- reserved prefix rejected on peer create; peers.list kind filtering
- scope peers rejected as message authors, chat/representation targets,
  and on the generic session-peer routes
- membership add/list/remove with observe_others=true / observe_me=false
  row shape asserted via DB, and facade-less equivalence with a
  hand-built observer peer
- end-to-end litmus: after adding a session to a scope, the deriver
  enqueue fan-out includes the scope peer as an observer
- session creation with scopes: [a, b] creates both memberships

Part of DEV-1997 (Scopes RFC DEV-1970).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(scopes): scope resolution helper and read-route schemas

Add resolve_scope_peers (crud/scope.py) mapping unprefixed scope names to
their backing scope peers, 404 when missing and 422 when a non-scope peer
squats the reserved name. Add the `scope` option to DialecticOptions and
PeerRepresentationGet, and a WorkspaceMessageSearchOptions with `scope`.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(scopes): wire the `scope` read option into the routes

Chat and representation: a single scope swaps the observer to the scope
peer (recall confined to the scoped collection and the scope's sessions by
existing observer semantics); a scope list resolves to the union of member
sessions and rides the DEV-1995 dynamic allowlist arm with the path peer as
observer. `scope` is mutually exclusive with `filters`/`session_id` (422)
and requires a workspace/admin key (403 for peer-scoped JWTs).

Session context: `scope` swaps the perspective source for both the working
representation and the peer-card fetch. Workspace search: `scope` injects
the scope's session set into the message-search filter (empty scope -> no
results). Close the carry-over guardrail gap: scope peers are rejected as
peer_target/peer_perspective in session context and as the peer/target in
GET /peers/{id}/context.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(scopes): cover the `scope` read option end-to-end

Validation (404/422/403), single-scope observer swap vs. union allowlist,
scoped representation and session-context (scoped collection + scoped card),
workspace search restricted to a scope's sessions, and guardrail closure for
scope peers on the peer/session context surfaces. Patch the workspaces
tracked_db import site so scope resolution reads the per-test database.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(crud): preserve cache invalidation across get_or_create retry

`get_or_create_peers` and `get_or_create_scopes` mutate existing rows, then
insert new ones inside `db.begin_nested()`. When a concurrent writer creates
one of those rows first, the insert raises IntegrityError and the function
retries.

`begin_nested()` autoflushes the pending UPDATEs *before* opening the
savepoint, so the rollback neither undoes them nor expires the now-clean ORM
state. The retry then compared already-updated values, found no change, and
dropped those peers from `changed_peers` — skipping the cache purge while the
row change committed anyway, leaving entries stale until the 300s TTL.

Carry the mutated names into the retry via `_pending_invalidation` so the
purge cannot be lost.

The scopes facade mirrors `get_or_create_peers`, so both copies carried this.
The peer path is pre-existing and runs on every message ingest.

Also add the missing /v3 prefix to `_SCOPES_ROUTE_GUIDANCE`, which pointed
callers at a 404.

Adds tests/crud/test_get_or_create_retry_invalidation.py, which drives a real
racing session and fails without this change.

* fix(scopes): make scope identity unforgeable, unblock non-pattern peer names

Three coupled changes to the scopes facade.

1. `PeerCreate` no longer gates internal lookups. It exists to validate a new,
   user-supplied peer id at the API boundary, but crud used it as a DTO for
   names that already exist, so any name outside RESOURCE_NAME_PATTERN raised a
   raw pydantic ValidationError — which is not a HonchoException, so it fell
   through to the catch-all handler as an HTTP 500. Adds `PeerSpec` (same
   fields, no charset pattern) as `PeerCreate`'s base, widens
   `get_or_create_peers` to accept it, and changes `get_peer` to take a plain
   str. All 13 construction sites converted; the create route keeps full
   validation.

   This unbreaks the Dreamer: DreamScheduler passes `collection.observer`
   straight into the specialist preflight, and scope peers have
   `observe_others=true`, so every `(scope.x, peer)` dream died there — the
   feature scopes exist to enable. It also fixes a pre-existing bug unrelated
   to scopes: a peer named `alice.smith` (legal before d429de0e5338, which
   validated names by length alone) 500s on message create, session peer add,
   and peer update.

2. The `kind` flag moves from `configuration` to `internal_metadata`.
   `configuration` is user-writable — `PeerCreate`/`PeerUpdate` accept a
   free-form dict and `update_peer` replaces it wholesale — so a legitimate
   `{"observe_me": true}` update silently dropped the flag, and a forged
   `{"kind": "scope"}` injected an ordinary peer into `POST /scopes/list`.
   `internal_metadata` appears in no API schema. `observe_me: false` stays in
   `configuration`, where it belongs.

3. Scope identity requires prefix AND flag, via `is_scope_peer()` and
   `scope_peer_clause()`. Neither half is forgeable: the prefix sits outside
   RESOURCE_NAME_PATTERN, `internal_metadata` is unreachable. Usage-site guards
   become flag-based so a legacy peer merely occupying the namespace keeps
   working rather than 422-ing on its own traffic; peer create and update stay
   name-based, since those must stop new names entering the namespace.
   `update_peer` now returns 422 instead of 500.

Also swaps the reserved prefix from `scope__` to `scope.`: `_` is inside
RESOURCE_NAME_PATTERN, so any tenant could already own a `scope__x` peer.

No DB migration — `internal_metadata` already exists on `peers`, and no scope
peers exist in any deployment yet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(scopes): validate peer names on create, close namespace squatting and upsert race

Addresses three review findings against 48047a6a.

1. `PeerSpec` let API callers create invalid and reserved-prefix peers.
   Widening `get_or_create_peers` to accept a pattern-free schema fixed the
   lookup 500s but also removed validation from the *insert* path, and
   request-controlled names reach it via message authors, session peer maps, and
   the chat observer path — none of which carry a charset pattern of their own.
   Confirmed: `POST /sessions/{id}/messages` with `peer_id: "scope.x"` returned
   201 and minted an unflagged squatter, after which `POST /scopes {id: x}` was
   permanently 409-blocked — namespace denial of service by any caller able to
   post a message. `peer_id: "not a valid name!@#"` was likewise created.

   Fixed by validating only names about to be INSERTed
   (`_validate_new_peer_names`), so already-existing names — legacy dotted
   names, scope peers — still resolve without a spurious 422. That keeps the
   Dreamer fix intact, since it reads through `get_peer`.

2. Existing reserved-prefix squatters could not be updated. The name-based guard
   on `PUT /peers/{peer_id}` refused every `scope.` name, contradicting the
   invariant that an unflagged squatter stays a normal peer. Now flag-based, so
   behavior is three-way: a real scope is refused, an existing unflagged peer
   updates, and a missing reserved-prefix name is refused by (1) rather than
   minted.

3. Scope checks raced with get-or-create and the membership upsert. The
   route-level guards run before peers are resolved, so a scope created
   concurrently in that window would be attached by the generic path with a
   default `SessionPeerConfig()`, clobbering its observer membership config.
   Adds `_reject_resolved_scope_peers`, which runs on the resolved rows in the
   same transaction as the upsert — no window, no extra query. The early checks
   stay for better error messages.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(scopes): guard scope membership config, move checks to the mutation point

Addresses a second review pass against 10655792.

1. Scope membership configuration was directly user-mutable.
   `PUT /sessions/{id}/peers/{peer_id}/config` had no scope guard at all, and
   `crud.set_peer_config` resolved the peer only to discard the row. Confirmed:
   posting `{"observe_others": false, "observe_me": true}` for a scope returned
   204 and persisted, which silently stops all fan-out into the scope and makes
   Honcho form a representation *of* a scope — neither of which is reachable
   through the facade. Deterministic, no race required. Now checked on the row
   `get_peer` already returns, so it costs nothing and cannot race.

2. Empty and over-long names were still 500s. Removing the charset pattern from
   `PeerSpec` fixed one trap but left its length bounds, and request-bound peer
   names carry no length limits of their own — so `peer_id: ""` or a 513-char
   name reached `PeerSpec(...)` and raised a raw pydantic ValidationError that
   the catch-all turned into a 500. `PeerSpec` now carries no constraints at all
   (matching its documented purpose) and every rule for a new name lives in
   `_validate_new_peer_names` on the insert path.

3. Resolved-row protection generalized. The previous pass applied it only to
   membership upserts, leaving check-then-use windows elsewhere: peer update
   could have a concurrently-created scope's configuration replaced wholesale
   (create-path validation does not fire for a peer that now exists), the chat
   observer get-or-create could resolve a fresh scope as its observer, and the
   generic session-peer removal could silently detach a scope from its sessions.
   Each now inspects the resolved peer immediately before acting; the redundant
   name-level guard on the update route is dropped in favor of the race-free one.

`remove_peers_from_session` grows an internal `_allow_scope_peers` flag because
the scopes facade ends membership through that same path and must not be blocked
by its own guard.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(scopes): enumerate every peer-touching route against a scope policy

Three review passes each found the same class of defect: a route nobody had
checked, rather than logic that was subtly wrong. One of them — `PUT
/sessions/{id}/peers/{id}/config`, which let any caller set a scope to
`observe_others=false` and silently stop all fan-out into it — predated this work
entirely, because the guardrail set was assembled guardrail-by-guardrail instead
of derived from the route list. Sampling review cannot close that kind of gap;
enumeration can.

Derives every route through which a peer name can reach the system and requires
each to be classified as GUARDED or EXEMPT-with-a-reason, so a newly added
peer-touching route fails the suite until someone classifies it. Detection is the
union of path shape and a walk of the dependant tree (including sub-dependency
`Form(...)` params and nested request-body models), because neither signal alone
suffices: parameter names miss `POST /sessions/{id}/peers`, whose peer names are
dict keys, and path shape misses `messages/upload`, whose `peer_id` arrives as a
form field behind a parser dependency.

Both invariants are then asserted behaviorally, by calling the routes rather than
inspecting annotations — the guards deliberately live in crud, which is what makes
`messages/upload` guarded for free via `crud.create_messages`:

- a real scope is refused on all 11 guarded routes, and the rejection must name
  the scope, so an unrelated 422 (a malformed body) cannot pass the assertion;
- an *unflagged* peer merely occupying the reserved namespace is unaffected. That
  half regressed once already when `update_peer` used a name-based check.

Mutation-tested all three failure modes: disabling the `set_peer_config` guard
fails the guarded test naming that route; regressing `update_peer` to name-based
fails the squatter test; adding an unclassified peer route fails the enumeration.

Covers the HTTP surface only. Peer names also reach the system through the
deriver, dreamer, and queue, which have no route table to enumerate — noted in
the module docstring rather than implied to be covered.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(scopes): refuse a scope in every observed position; enumerate per position

Addresses a fourth review pass against 62681e4d. The headline finding is that the
previous commit's enumeration test had the wrong *model*, not a missing entry.

1. Manual conclusions could create knowledge about a scope. `POST /conclusions`
   validated only that observer_id and observed_id exist, so a scope as
   `observed_id` persisted a conclusion about a peer carrying observe_me=false and
   created an (observer, scope) collection for it. Confirmed: 201, and it read
   back. `POST /schedule_dream` had the same hole via `observed`.

   The fix is positional, because the invariant is:

       A scope may be an OBSERVER. A scope may never be OBSERVED.

   A scope as `observer_id` is how scoped conclusions are stored and must keep
   working (verified still 201); as `observed_id` it is now refused. Same split
   applied to schedule_dream `observed`, the peer-card `target` (which also covers
   a scope's self-card, since target-omitted collapses observed to peer_id), and
   session-context `peer_target`.

2. Chat target and both representation roles kept check-to-use races. Only the
   chat path-level observer was re-checked on its resolved row; the target was
   checked by name and then resolved without inspecting scope identity. Both are
   now checked at the dialectic preflight, where observer and observed are already
   resolved — an absent name has already failed by then, and an existing squatter
   cannot retroactively become a scope.

3. Generic membership removal was still racy. The adjacent SELECT narrowed the
   window but could not close it under READ COMMITTED. The UPDATE now carries its
   own correlated NOT EXISTS against scope_peer_clause(), so Postgres evaluates
   the exclusion as part of the statement and a scope committed after the advisory
   check still cannot be detached.

4. New-name validation ran after the name reached Postgres. A NUL byte passed the
   request schemas and PeerSpec, then raised psycopg.DataError inside the lookup —
   a 500. Values that cannot correspond to a stored row by construction (NUL
   bytes, over-length names) are now refused before the query.
   (Over-length names already returned 422; only the wasted query was real there.)

The enumeration test is rekeyed from (method, path) to (method, path, position).
A binary per-route verdict cannot express finding 1 at all: `POST /conclusions` is
one route with two positions and opposite verdicts. Detection widens to observer /
observed / target / peer_target / peer_perspective, which surfaced four routes the
previous version never saw — conclusions, schedule_dream, queue/status, and
session context.

Also registers `src.routers.workspaces.tracked_db` in the conftest patch list; the
new guard there would otherwise have run against the real configured database
instead of the per-test one.

Mutation-tested: disabling the conclusions observed-guard fails the positional
test naming that position; adding an unclassified `observed_id` param fails
enumeration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(scopes): refuse future scopes in observed positions, preserve scope membership

Addresses a fifth review pass against 14136e5b. All six findings reproduced
locally before fixing.

1. High — generic peer replacement removed scope memberships.
   `set_peers_for_session` soft-deleted every active SessionPeer row, and the
   request-level guard only inspected names *present* in the replacement map. A
   caller detached a scope by simply omitting it, never naming it — so no
   request-level guard could ever see it. Reproduced: scope sessions went
   ['<id>'] -> [] on a 200. The exclusion now lives in the UPDATE itself
   (correlated NOT EXISTS against scope_peer_clause), so replacement means
   "replace ordinary peers" regardless of request contents or concurrent creation.

2. High — peer cards could be pre-seeded for future scopes.
   `set_peer_card` resolves only the observer and writes a JSONB key derived from
   an unchecked observed name, and the route guard rejected only *existing*
   flagged scopes. Reproduced: PUT card with target=scope.<missing> returned 200,
   creating that scope then returned 201, and the card described the real scope.

3. High — dreams could be queued for future scopes.
   The route checked `observed` in a read-only session that closed before
   `enqueue_dream`, and a missing reserved name passes any is-it-a-scope check.
   Reproduced: 204 with observed=scope.<missing>.

   2 and 3 share a root cause, so they share a fix: a new `reject_scope_observed`
   that is stricter than `reject_scope_peers` in exactly one case — a *missing*
   reserved name is refused, because nothing on these paths creates the peer, so
   nothing else would ever catch it. Existing unflagged squatters still pass.
   Both guards moved to the mutation point: card validation into
   `crud.set_peer_card` (same transaction as the JSONB write, so Dreamer and
   agent-tool callers are covered), dream validation into `enqueue_dream` (same
   transaction as the queue insert). The redundant route-level checks are dropped
   rather than left as weaker duplicates.

4. Medium — prefixed NUL names still reached PostgreSQL.
   `reject_scope_peers` filtered for the reserved prefix and sent matches to a
   text comparison, so "scope.future\0name" raised psycopg.DataError — a 500.
   Both guards now share `_reserved_name_candidates`, which materializes the input
   once and rejects impossible values before any SQL. Materializing matters
   independently: the message-author path passes a generator, and validation
   iterates separately from the prefix filter, so a generator would be
   half-consumed. `_reject_impossible_peer_names` now takes a Collection so the
   type checker enforces that.

5. Medium — representation kept a check-to-use race.
   The previous commit claimed both representation roles were rechecked after
   resolution; that was wrong — only the dialectic preflight got that check, and
   the representation route never goes through it. It now opens one short
   read-only session *after* the embedding call, checks both positions, and passes
   that same session to `get_working_representation`, so no connection is held
   across external work and a scope committed later cannot have conclusions in the
   collection being read.

6. Low — policy coverage was not exhaustive. `sender_id` reaches CRUD as
   `observed` but was missing from the detected parameter set. ALLOW cases could
   also not carry builders, so the suite never proved the other half of the
   contract — that legitimate scope *observers* keep working, which a guard
   rejecting scopes everywhere would satisfy. Both fixed; observer positions on
   conclusions, dreams, cards, session context and queue status are now asserted
   behaviorally.

Deliberately not implemented: the scope-creation backstop scanning for
pre-existing card keys and queue items naming a future backing peer. Reasoning is
recorded in `get_or_create_scopes` — no new such state can be created now, any
pre-existing row is coincidental since `scope.` was never a meaningful namespace,
the consequence is inert, and detecting card keys means a full table scan per
scope creation.

Mutation-tested each new guard: removing the replacement exclusion fails both
membership-preservation tests; weakening either observed guard to existing-only
fails the pre-seeding tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(scopes): exclude scope memberships from the session observer limit

Scope memberships carry observe_others=true, so every scope counted against
SESSION_OBSERVERS_LIMIT (default 10) — capping scopes-per-session at the limit
minus the session's real observers, and reporting the failure as
`400 Cannot create session <name> with 11 observers. ... Observers are peers
with 'observe_others' set to true.` on a membership call. Wrong on three counts:
the ceiling is undocumented and contradicts RFC §5.1 ("sessions belong to any
number of scopes"), the message describes session creation, and it leaks the
word "observer" through a facade whose entire job is hiding observers (RFC
goal 5). The limit exists to bound per-observer deriver fan-out for real peers;
a scope costs document rows, not LLM calls (RFC §5.2), so it does not belong in
that budget.

Excluded from both halves of the check in `_get_or_add_peers_to_session`: the
incoming names via a flag-based lookup, existing memberships via a correlated
NOT EXISTS on `scope_peer_clause()` — the same pattern the replacement and
removal paths already use, so the exclusion holds regardless of concurrent
scope creation. The early `count_observers_in_config(session.peer_names)` check
in `get_or_create_session` is left alone: `peer_names` cannot contain a scope,
and `scopes` is a separate field.

`reject_scope_peers` is split into a `scope_peer_names()` query helper plus a
two-line raiser so the observer count reuses the authoritative name-AND-flag
predicate instead of growing a third copy of it. Still costs nothing on the
common path — no reserved-prefix name in the input means no query at all.

Also caps `SessionCreate.scopes` at 100, matching `ScopeSessionsAdd.session_ids`.
This belongs in the same commit: the observer limit was the only thing bounding
that list, so removing it turns an unbounded `scopes` array into a peer row and
a membership row per element, committed — the single-request path to the
cardinality anti-pattern RFC §8 warns about. Partly answers OQ6: no per-session
cap, 100 per request.

Tests: a session joins SESSION_OBSERVERS_LIMIT + 2 scopes through both the
facade and session creation; real observers over the limit still 400, so the
carve-out cannot quietly disable the limit; 101 scopes is a 422.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(scopes): skip semantic retrieval when the embedding precompute failed

Addresses three open review comments.

1. Major — the representation read could embed inside its DB session. The
   route's precompute is suppressed, and both
   `RepresentationManager.get_working_representation` and
   `crud.query_documents` fall back to embedding when a query arrives without
   one, so a failed precompute meant an external call inside the read session
   this branch opens for the scope re-check — the connection-holding rule the
   route's own comment claimed to satisfy. The innermost fallback also only
   catches ValueError, so a provider outage surfaced as a 500. The semantic
   query is now passed only when an embedding exists, degrading to
   derived+recent retrieval. (`crud.query_documents` embedding inside a caller's
   session predates this branch and is left alone.)

2. Minor — `test_resolved_scope_peer_rejected_at_membership_upsert` described a
   race it does not perform. It creates an already-flagged scope and calls crud
   directly; the unflagged → flagged transition is not simulated. Docstring now
   says what the test actually pins.

3. Minor — `test_empty_replacement_preserves_scope_membership` asserted only
   half its docstring. It passed if the empty PUT left every ordinary
   membership intact; now asserts the ordinary peer's left_at is set.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(scopes): close auth and observed-position gaps, paginate membership

Review response for #884.

Security:
- gate `SessionCreate.scopes` behind a workspace-level key; the session-create
  route is self-authorizing, so a peer- or session-scoped token could mint scope
  peers and join sessions to scopes it had no access to via `POST /scopes`
- refuse a reserved-but-nonexistent name in two observed positions that used the
  permissive guard: chat `target` and session-context `peer_target`. Both let a
  caller act on `scope.X` before it existed, then create the scope

Facade:
- exclude scope peers from `GET /sessions/{id}/peers` and refuse the membership
  -config read for a real scope, matching its write side
- replace `GET /scopes/{id}/sessions` with `POST /scopes/{id}/sessions/list`
  returning `Page[Session]`; the add route now returns 204. Membership was
  unbounded on both, while every other list surface paginates
- rename `crud.get_scope` to `get_scope_or_raise`

Tests:
- add a missing-name axis to the route-policy table (`Case.refuse_missing`), which
  is what surfaced the two guard gaps above
- delete 14 hand-written tests the table now enumerates; 52 -> 39 functions in
  test_scopes.py with more cases covered
- tighten the squatter assertion from `!= 422` to `< 400`, which was passing on 5xx
- assert the FastAPI-internals traversal still derives positions, so a framework
  upgrade can't silently empty the suite

Docs:
- drop internal ticket and RFC references from the published OpenAPI descriptions
  and surrounding comments; state the behavior instead
- move implementation reasoning out of the `PUT /peers/{id}` docstring, which
  FastAPI publishes, into a comment

* test(scopes): assert exact statuses for permissive missing-name cases

Follow-up review pass on #884.

- add `Case.missing_status` so a permissive missing-name position asserts the
  status it should actually get (404, or 200 for the no-op removal) instead of
  `!= 422`, which also passed on a 5xx — the same hole already closed in the
  squatter assertion
- require it whenever `refuse_missing` is False, and require its absence when
  True, so the policy table can't drift from the assertion
- repoint a stale allow-reason at POST /scopes/{scope_id}/sessions/list; the GET
  it named was removed
- document the membership list's ordering under `reverse`

* fix: Address Coderabbit Comments

* fix: Address more coderabbit comments

* chore: remove ai artifacting

* fix: remove unused fakeredis client in lieu of in-memory taskless cache for test suite

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 18:34:55 -04:00
..
__init__.py feat: agentic dreamer and agentic dialectic (#309) 2026-01-12 15:12:17 -05:00
test_model_config_usage.py Refactor clients.py to add modern features and more flexible configuration (#459) 2026-04-20 02:46:37 -04:00
test_scope_preflight.py Scopes Phase 2b: `scope` option on chat, representation, context, and search (#897) 2026-08-12 18:34:55 -04:00
test_structured_output.py feat(dialectic): optional structured outputs with limited schema for Dialectic calls (#896) 2026-07-20 18:46:49 -04:00