Two defects in _build_comparison_conditions, both reachable from any
route that accepts filters:
1. A null operand hit float(None), raising TypeError where only
ValueError was caught — an unhandled 500. A null operand is a null
check, not a value comparison, so {"ne": null} now compiles to
IS NOT NULL and the other operators reject null with a 422. Equality
against null already produced IS NULL via _build_field_condition.
2. Numeric operators float()-cast on every column type, so a string
inequality on a text column ({"session_id": {"ne": "abc"}}) was
rejected as an invalid number. Coercion is now gated on the column
actually being numeric; text columns compare as text. Numeric columns
still validate, and TypeError is caught alongside ValueError.
Existing ne coverage only exercised the JSONB metadata path, which uses
_safe_numeric_cast and handles strings — the scalar column path was
untested. Adds cases for both.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An unrecognized operator dict on a non-JSONB column (e.g.
{"session_id": {"operator": "null"}}) fell through to `column == value`,
binding a dict to a VARCHAR parameter. That compiles, then fails in the
driver at execute time with "cannot adapt type 'dict'" — an unhandled
500 for what is invalid input.
Raise FilterError (422) instead. The guard lives in the shared
_build_field_condition, so every route through apply_filter is covered.
It keys on the actual column type rather than the JSONB_COLUMNS name
list, so dict equality still works on JSONB columns reachable through
Document's raw-key fallback (e.g. source_ids), where the driver adapts
dicts fine.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: apply session scoping to all working-representation query paths
session_name was only applied to the recent-documents query in
RepresentationManager; the semantic and most-derived paths ignored it,
so limit_to_session leaked cross-session conclusions into perspectives.
- Thread a session allowlist (session_names) uniformly through all
three query paths; pushed down to pgvector and external vector stores
- Accept a list so the upcoming session-allowlist API reuses this path
- Fail closed on an empty allowlist (downstream stores drop empty IN
clauses, which would silently widen scope)
Fixes DEV-1994
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: bare-list membership sugar in the filter DSL
{"session_id": ["s1", "s2"]} is now shorthand for
{"session_id": {"in": [...]}} on regular columns, generically
(peer_id, etc.). JSONB metadata columns are excluded — a bare list
there keeps JSONB containment semantics, unchanged.
Previously a bare list on a regular column compiled to a type-mismatched
equality that matched nothing, so this is strictly additive.
Also translates the same shape in the turbopuffer/lancedb filter
builders, and fixes lancedb dropping empty IN clauses (fail-open) —
an empty membership list now emits an always-false condition.
Groundwork for DEV-1995 (session allowlist via the existing filters
DSL, no new API params)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: session allowlist on dialectic and representation via filters
Adds a constrained 'filters' body to peer.chat and /representation —
the same DSL search and conclusions already accept, supporting only the
session_id key (a session id, a bare list, or {"in": [...]}).
Unsupported keys and shapes are rejected with 422, never silently
ignored. Composes with session_id (must be included in the allowlist
when both are given). Capped at 1,000 sessions per request.
Enforcement is uniform at every recall chokepoint, fail-closed:
- dialectic prefetch + search_memory: conclusion recall restricted to
the allowlist; dream docs (session_name IS NULL) excluded
- message tools (search/grep/date-range/temporal/context/history):
strict intersection of allowlist and observer session membership
- get_reasoning_chain: unavailable under an allowlist (chains traverse
provenance across sessions and cannot be scoped without leaking)
- empty allowlist short-circuits to empty results everywhere
Auth: workspace keys pass the allowlist as-given; peer-scoped JWTs must
be a member of every allowlisted session (403 otherwise), mirroring the
existing single-session check.
Fixes DEV-1995
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: fail closed on empty session allowlist across all filter builders
Empty session allowlists relied solely on the early-return guard in
_get_working_representation_internal. The layers below it were
inconsistent, so a future direct caller (the DEV-1995 allowlist API)
would silently widen scope instead of failing closed:
- _build_filter_conditions used a truthiness check; an empty list was
treated like None and dropped the filter. Now uses `is not None`,
matching the recent/most-derived SQL paths.
- turbopuffer emitted a bare `In []` with undocumented (possibly
fail-open) semantics. Now emits an explicit always-false predicate,
mirroring lancedb's `1 = 0`.
Also extract the duplicated JSONB column tuple in filter.py to a
JSONB_COLUMNS constant.
Tests exercise each fail-closed guarantee at the layer it lives, rather
than masking it behind the early-return guard.
* fix: address tests
* fix(crud): fail closed when session_name is outside the allowlist
search/grep/history helpers scoped to a single session_name ignored the
session_names allowlist entirely — a caller could read a session the
allowlist forbids. The API routes guarded this with a 422, but the
dialectic tools call these CRUD functions directly and bypassed it.
Enforce it at the boundary: return [] when session_name is set and not in
the allowlist, across _semantic_search_messages (covers search_messages +
search_messages_temporal), grep_messages, get_messages_by_date_range,
get_recent_history, and get_observation_context.
Also rename the public param allowed_sessions -> session_names for
consistency with representation.py / chat.py / peers.py; the resolved
intersection keeps its distinct name allowed_session_names.
* fix: test
* fix(scopes): tighten and consolidate session allowlist per review
Addresses review feedback on the session allowlist (DEV-1995).
Behavior changes:
- Auth gate on peers.chat now uses active membership (left_at IS NULL)
via get_peer_session_names(active_only=True), matching the adjacent
is_peer_in_session check on options.session_id. Previously a peer that
had left a session was denied when naming it directly but permitted
when naming it in filters.session_id.
- Scoped conclusion recall is restricted to level == "explicit"
(ALLOWLIST_SAFE_LEVELS). Dream-derived conclusions are stamped with a
single session_name but synthesized across all sessions, so that stamp
can't be scoped on. Applied at all four recall paths. Unscoped recall
is unchanged. Follow-up to give conclusions an authoritative
source-session set is tracked in DEV-2201.
- The allowlist gate checks `is not None` rather than truthiness, so
filters={"session_id": []} reaches it instead of being skipped.
Refactors:
- New crud.message.resolve_session_scope replaces four near-identical
copies of the allowlist-membership intersection. Returns
(allowlist, deny) and never returns an empty list, so the None vs []
distinction that external stores fail open on lives in one tested
place. Takes db=None and opens its own short-lived session only when
distinction that external stores fail open on lives in one tested
place. Takes db=None and opens its own short-lived session only when
an observer lookup is needed, preserving external-lookup-first
ordering on the vector-store path.
- extract_session_allowlist takes must_include, collapsing the
session_id-in-allowlist check duplicated across both peer routes.
- DialecticAgent._select_tools dedupes the two toolset-selection blocks
and drops get_reasoning_chain under an allowlist, rather than paying
for the schema plus a wasted turn to return a refusal.
- Rename session_names -> session_allowlist across crud, agent tools,
dialectic and routes, to remove the one-character ambiguity with
session_name. Internal only; the public filters.session_id surface is
unchanged.
Docs:
- session_allowlist documented across all message and recall entry
points, including the None / [] / populated contract.
- session_name marked deprecated for scoping. Not removed and not
aliased: it also pins the query to one session, bypasses observer
scoping, and drives session-history injection into the dialectic
prompt, so it has no drop-in replacement.
- Note at the Document branch in utils/filter.py that the raw-key
fallback is load-bearing for session scoping.
Tests: 20 -> 39 in tests/test_session_allowlist.py, covering the
peer-scoped JWT gate (member, non-member, left-session, workspace-key
bypass, empty allowlist), the resolve_session_scope tri-state including
the no-DB-checkout path, must_include, and the level narrowing.
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix: apply session scoping to all working-representation query paths
session_name was only applied to the recent-documents query in
RepresentationManager; the semantic and most-derived paths ignored it,
so limit_to_session leaked cross-session conclusions into perspectives.
- Thread a session allowlist (session_names) uniformly through all
three query paths; pushed down to pgvector and external vector stores
- Accept a list so the upcoming session-allowlist API reuses this path
- Fail closed on an empty allowlist (downstream stores drop empty IN
clauses, which would silently widen scope)
Fixes DEV-1994
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: bare-list membership sugar in the filter DSL
{"session_id": ["s1", "s2"]} is now shorthand for
{"session_id": {"in": [...]}} on regular columns, generically
(peer_id, etc.). JSONB metadata columns are excluded — a bare list
there keeps JSONB containment semantics, unchanged.
Previously a bare list on a regular column compiled to a type-mismatched
equality that matched nothing, so this is strictly additive.
Also translates the same shape in the turbopuffer/lancedb filter
builders, and fixes lancedb dropping empty IN clauses (fail-open) —
an empty membership list now emits an always-false condition.
Groundwork for DEV-1995 (session allowlist via the existing filters
DSL, no new API params)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: fail closed on empty session allowlist across all filter builders
Empty session allowlists relied solely on the early-return guard in
_get_working_representation_internal. The layers below it were
inconsistent, so a future direct caller (the DEV-1995 allowlist API)
would silently widen scope instead of failing closed:
- _build_filter_conditions used a truthiness check; an empty list was
treated like None and dropped the filter. Now uses `is not None`,
matching the recent/most-derived SQL paths.
- turbopuffer emitted a bare `In []` with undocumented (possibly
fail-open) semantics. Now emits an explicit always-false predicate,
mirroring lancedb's `1 = 0`.
Also extract the duplicated JSONB column tuple in filter.py to a
JSONB_COLUMNS constant.
Tests exercise each fail-closed guarantee at the layer it lives, rather
than masking it behind the early-return guard.
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix: refactor datetime handling in security and utility modules
- Updated `verify_jwt` function to utilize `parse_datetime_iso` for improved expiration time validation.
- Enhanced `_validate_datetime_string` to prioritize timezone-aware formats and streamline parsing logic.
- Modified `created_at` assignments in `ObservationContext` to use UTC timezone for consistency in timestamp handling.
* fix: proper observation extraction
- Replaced direct observation references with the `extract_observation_content` function to enhance clarity and maintainability in the `new_observations` list comprehension.
* refactor: streamline datetime parsing in filter and formatting utilities
- Removed redundant timezone-aware format handling from `_validate_datetime_string` in `filter.py`.
- Enhanced `parse_datetime_iso` in `formatting.py` to ensure consistent conversion of 'Z' suffix to timezone-aware datetime objects, improving clarity and functionality.
* refactor: update JWT verification and enhance datetime parsing
- Changed `verify_jwt` function from asynchronous to synchronous for improved performance and simplicity.
- Enhanced `parse_datetime_iso` to include comprehensive input validation and support for various timezone formats, ensuring consistent and secure datetime parsing.
* chore: coderabbit
* feat: add support for custom message timestamps in API
- Introduced `created_at` parameter for message creation, allowing users to specify custom timestamps.
- **Single source of truth for timestamp string format**
- Updated SDK documentation to reflect this new feature and its use cases.
- Enhanced validation schemas to include the optional `created_at` field.
- Added tests to verify functionality for messages with and without custom timestamps, ensuring correct behavior and default timestamp usage.
* feat: add timestamp option to sdks
* feat: Add get summaries endpoints
* feat: WIP basic SDK implementation blocked until stainless release
* feat: Implement SDKs with honcho-core methods
* fix (sdk): Used release 1.4.0 core sdks
* fix: Code Rabbit
* chore: Pytest errors
---------
Co-authored-by: Benjamin McCormick <docterformer@protonmail.com>
* feat: add complex arbitrary filtering on all objects
* fix: safe numeric casting and application of comparators
* fix: add way more tests, fix bugs with filter parsing
* chore: pass model_class as arugment to apply_filter
* fix: default to not caring about is_active in get_sessions_for_peer
* fix: address coderabbit complaints (valid)
* fix: throw filter errors when necessary, validate inputs and handle edge cases with more tests
* fix: remove all type errors and most type warnings
* fix: don't use db in tests that don't need it
cheat: sprinkle in some pyright: ignore in filter.py
* fix: allowlist for filtering -- no filtering by content, message id, or anything internal
* fix: handle mixed types in metadata, add tests
* chore: refine types
* chore: rename fiter param everywhere
Vineeth Voruganti
doria