diff --git a/.agents/skills b/.agents/skills new file mode 120000 index 00000000..42c5394a --- /dev/null +++ b/.agents/skills @@ -0,0 +1 @@ +../skills \ No newline at end of file diff --git a/.claude/skills b/.claude/skills new file mode 120000 index 00000000..42c5394a --- /dev/null +++ b/.claude/skills @@ -0,0 +1 @@ +../skills \ No newline at end of file diff --git a/.claude/skills/honcho-integration/SKILL.md b/.claude/skills/honcho-integration/SKILL.md deleted file mode 100644 index 9ed50194..00000000 --- a/.claude/skills/honcho-integration/SKILL.md +++ /dev/null @@ -1,554 +0,0 @@ ---- -name: honcho-integration -description: Integrate Honcho memory and social cognition into existing Python or TypeScript codebases. Use when adding Honcho SDK, setting up peers, configuring sessions, implementing the dialectic chat endpoint for AI agents, or wiring Honcho into bot frameworks (nanobot, openclaw, picoclaw, etc). -allowed-tools: Read, Glob, Grep, Bash(uv:*), Bash(bun:*), Bash(npm:*), Edit, Write, WebFetch, AskUserQuestion ---- - -# Honcho Integration Guide - -## What is Honcho - -Honcho is an open source memory library for building stateful agents. It works with any model, framework, or architecture. You send Honcho the messages from your conversations, and custom reasoning models process them in the background — extracting premises, drawing conclusions, and building rich representations of each participant over time. Your agent can then query those representations on-demand ("What does this user care about?", "How technical is this person?") and get grounded, reasoned answers. - -The key mental model: **Peers** are any participant — human or AI. Both are represented the same way. Observation settings (`observe_me`, `observe_others`) control which peers Honcho reasons about. Typically you want Honcho to model your users (`observe_me=True`) but not your AI assistant (`observe_me=False`). **Sessions** scope conversations between peers. **Messages** are the raw data you feed in — Honcho reasons about them asynchronously and stores the results as the peer's **representation**. No messages means no reasoning means no memory. - -Your agent accesses this memory through `peer.chat(query)` (ask a natural language question, get a reasoned answer), `session.context()` (get formatted conversation history + representations), or both. - -## Integration Workflow - -Follow these phases in order: - -### Phase 1: Codebase Exploration - -Before asking the user anything, explore the codebase to understand: - -1. **Language & Framework**: Is this Python or TypeScript? What frameworks are used (FastAPI, Express, Next.js, etc.)? -2. **Existing AI/LLM code**: Search for existing LLM integrations (OpenAI, Anthropic, LangChain, etc.) -3. **Entity structure**: Identify users, agents, bots, or other entities that interact -4. **Session/conversation handling**: How does the app currently manage conversations? -5. **Message flow**: Where are messages sent/received? What's the request/response cycle? - -Use Glob and Grep to find: - -- `**/*.py` or `**/*.ts` files with "openai", "anthropic", "llm", "chat", "message" -- User/session models or types -- API routes handling chat or conversation endpoints - -> **Bot framework detected?** If the codebase is built around an agent loop, tool registry, session manager, and message bus (e.g., nanobot, openclaw, picoclaw), read `{baseDir}/references/bot-frameworks.md` for framework-specific integration guidance and check `{baseDir}/references/bot-frameworks//` for concrete reference implementations. - -### Phase 2: Interview (REQUIRED) - -After exploring the codebase, use the **AskUserQuestion** tool to clarify integration requirements. Ask these questions (adapt based on what you learned in Phase 1): - -#### Question Set 1 - Entities & Peers - -Ask about which entities should be Honcho peers: - -- header: "Peers" -- question: "Which entities should Honcho track and build representations for?" -- options based on what you found (e.g., "End users only", "Users + AI assistant", "Users + multiple AI agents", "All participants including third-party services") -- Include a follow-up if they have multiple AI agents: should any AI peers be observed? - -#### Question Set 2 - Integration Pattern - -Ask how they want to use Honcho context: - -- header: "Pattern" -- question: "How should your AI access Honcho's user context?" -- options: - - "Tool call (Recommended)" - "Agent queries Honcho on-demand via function calling" - - "Pre-fetch" - "Fetch user context before each LLM call with predefined queries" - - "context()" - "Include conversation history and representations in prompt" - - "Multiple patterns" - "Combine approaches for different use cases" - -#### Question Set 3 - Session Structure - -Ask about conversation structure: - -- header: "Sessions" -- question: "How should conversations map to Honcho sessions?" -- options based on their app (e.g., "One session per chat thread", "One session per user", "Multiple users per session (group chat)", "Custom session logic") - -#### Question Set 4 - Specific Queries (if using pre-fetch pattern) - -If they chose pre-fetch, ask what context matters: - -- header: "Context" -- question: "What user context should be fetched for the AI?" -- multiSelect: true -- options: "Communication style", "Expertise level", "Goals/priorities", "Preferences", "Recent activity summary", "Custom queries" - -### Phase 3: Implementation - -Based on interview responses, implement the integration: - -1. Install the SDK -2. Create Honcho client initialization -3. Set up peer creation for identified entities -4. Implement the chosen integration pattern(s) -5. Add message storage after exchanges -6. Update any existing conversation handlers - -### Phase 4: Verification - -- If the Honcho CLI is available, run `honcho doctor` to confirm connectivity before testing the integration code -- Use `honcho peer list` and `honcho peer chat` to verify peers exist and the dialectic endpoint works independently of the integration -- Ensure all message exchanges are stored to Honcho -- Verify AI peers have `observe_me=False` (unless user specifically wants AI observation) -- Check that the workspace ID is consistent across the codebase -- Confirm environment variable for API key is documented - ---- - -## Before You Start - -1. **Check the latest SDK versions** at - - Python SDK: `honcho-ai` - - TypeScript SDK: `@honcho-ai/sdk` - -2. **Get an API key** ask the user to get a Honcho API key from and add it to the environment. - -3. **Verify with the CLI** (optional but recommended). If the user has the Honcho CLI installed (`pip install honcho-cli`), they can validate their setup before writing any integration code: - - ```bash - honcho init # persist API key + URL to ~/.honcho/config.json - honcho doctor # verify connectivity, config, workspace health - honcho peer chat # test the dialectic endpoint interactively - ``` - - This is the fastest way to confirm the API key and URL are correct before debugging SDK code. - -## Installation - -### Python (use uv) - -```bash -uv add honcho-ai -``` - -### TypeScript (use bun) - -```bash -bun add @honcho-ai/sdk -``` - -## Sync vs Async - -**TypeScript** — The SDK is async by default. All methods return promises. No separate sync API. - -**Python** — The SDK provides both sync and async interfaces: - -- **Sync** (default): `from honcho import Honcho` — use in sync frameworks (Flask, Django, CLI scripts) -- **Async**: `from honcho import Honcho` with `.aio` namespace — use in async frameworks (FastAPI, Starlette, async workers) - -```python -# Sync usage (Flask, Django, scripts) -from honcho import Honcho -honcho = Honcho(workspace_id="my-app", api_key=os.environ["HONCHO_API_KEY"]) -peer = honcho.peer("user-123") -response = peer.chat("What does this user prefer?") - -# Async usage (FastAPI, Starlette) -from honcho import Honcho -honcho = Honcho(workspace_id="my-app", api_key=os.environ["HONCHO_API_KEY"]) -peer = await honcho.aio.peer("user-123") -response = await peer.aio.chat("What does this user prefer?") -``` - -Match the client to the framework — check whether the codebase uses `async def` handlers or sync `def` handlers and choose accordingly. The rest of this skill shows sync Python examples; swap to `.aio` equivalents for async codebases. - -## Core Integration Patterns - -### 1. Initialize with a Single Workspace - -Use ONE workspace for your entire application. The workspace name should reflect your app/product. - -**Python:** - -```python -from honcho import Honcho -import os - -# Sync client (Flask, Django, scripts) -honcho = Honcho( - workspace_id="your-app-name", - api_key=os.environ["HONCHO_API_KEY"], - environment="production" -) - -# Async client (FastAPI, Starlette) — use honcho.aio for all operations -# honcho.aio.peer(), honcho.aio.session(), etc. -``` - -**TypeScript:** - -```typescript -import { Honcho } from '@honcho-ai/sdk'; - -// All methods are async by default -const honcho = new Honcho({ - workspaceId: "your-app-name", - apiKey: process.env.HONCHO_API_KEY, - environment: "production" -}); -``` - -### 2. Create Peers for ALL Entities - -Create peers for **every entity** in your business logic - users AND AI assistants. - -**Python:** - -```python -from honcho.api_types import PeerConfig - -# Human users -user = honcho.peer("user-123") - -# AI assistants - set observe_me=False so Honcho doesn't model the AI -assistant = honcho.peer("assistant", configuration=PeerConfig(observe_me=False)) -support_bot = honcho.peer("support-bot", configuration=PeerConfig(observe_me=False)) -``` - -**TypeScript:** - -```typescript -// Human users -const user = await honcho.peer("user-123"); - -// AI assistants - set observeMe=false so Honcho doesn't model the AI -const assistant = await honcho.peer("assistant", { configuration: { observeMe: false } }); -const supportBot = await honcho.peer("support-bot", { configuration: { observeMe: false } }); -``` - -### 3. Multi-Peer Sessions - -Sessions can have multiple participants. Configure observation settings per-peer. - -**Python:** - -```python -from honcho.api_types import SessionPeerConfig - -session = honcho.session("conversation-123") - -# User is observed (Honcho builds a model of them) -user_config = SessionPeerConfig(observe_me=True, observe_others=True) - -# AI is NOT observed (no model built of the AI) -ai_config = SessionPeerConfig(observe_me=False, observe_others=True) - -session.add_peers([ - (user, user_config), - (assistant, ai_config) -]) -``` - -**TypeScript:** - -```typescript -const session = await honcho.session("conversation-123"); - -await session.addPeers([ - [user, { observeMe: true, observeOthers: true }], - [assistant, { observeMe: false, observeOthers: true }] -]); -``` - -### 4. Add Messages to Sessions - -**Python:** - -```python -session.add_messages([ - user.message("I'm having trouble with my account"), - assistant.message("I'd be happy to help. What seems to be the issue?"), - user.message("I can't reset my password") -]) -``` - -**TypeScript:** - -```typescript -await session.addMessages([ - user.message("I'm having trouble with my account"), - assistant.message("I'd be happy to help. What seems to be the issue?"), - user.message("I can't reset my password") -]); -``` - -## Using Honcho for AI Agents - -### Pattern A: Dialectic Chat as a Tool Call (Recommended for Agents) - -Make Honcho's chat endpoint available as a **tool** for your AI agent. This lets the agent query user context on-demand. - -**Python (OpenAI function calling):** - -```python -import openai -from honcho import Honcho - -honcho = Honcho(workspace_id="my-app", api_key=os.environ["HONCHO_API_KEY"]) - -# Define the tool for your agent -honcho_tool = { - "type": "function", - "function": { - "name": "query_user_context", - "description": "Query Honcho to retrieve relevant context about the user based on their history and preferences. Use this when you need to understand the user's background, preferences, past interactions, or goals.", - "parameters": { - "type": "object", - "properties": { - "query": { - "type": "string", - "description": "A natural language question about the user, e.g. 'What are this user's main goals?' or 'What communication style does this user prefer?'" - } - }, - "required": ["query"] - } - } -} - -def handle_honcho_tool_call(user_id: str, query: str) -> str: - """Execute the Honcho chat tool call.""" - peer = honcho.peer(user_id) - return peer.chat(query) - -# Use in your agent loop -def run_agent(user_id: str, user_message: str): - messages = [{"role": "user", "content": user_message}] - - response = openai.chat.completions.create( - model="gpt-4", - messages=messages, - tools=[honcho_tool] - ) - - # Handle tool calls - if response.choices[0].message.tool_calls: - for tool_call in response.choices[0].message.tool_calls: - if tool_call.function.name == "query_user_context": - import json - args = json.loads(tool_call.function.arguments) - result = handle_honcho_tool_call(user_id, args["query"]) - # Continue conversation with tool result... -``` - -**TypeScript (OpenAI function calling):** - -```typescript -import OpenAI from 'openai'; -import { Honcho } from '@honcho-ai/sdk'; - -const honcho = new Honcho({ - workspaceId: "my-app", - apiKey: process.env.HONCHO_API_KEY -}); - -const honchoTool: OpenAI.ChatCompletionTool = { - type: "function", - function: { - name: "query_user_context", - description: "Query Honcho to retrieve relevant context about the user based on their history and preferences.", - parameters: { - type: "object", - properties: { - query: { - type: "string", - description: "A natural language question about the user" - } - }, - required: ["query"] - } - } -}; - -async function handleHonchoToolCall(userId: string, query: string): Promise { - const peer = await honcho.peer(userId); - return await peer.chat(query); -} -``` - -### Pattern B: Pre-fetch Context with Targeted Queries - -For simpler integrations, fetch user context before the LLM call using pre-defined queries. - -**Python:** - -```python -def get_user_context_for_prompt(user_id: str) -> dict: - """Fetch key user attributes via targeted Honcho queries.""" - peer = honcho.peer(user_id) - - return { - "communication_style": peer.chat("What communication style does this user prefer? Be concise."), - "expertise_level": peer.chat("What is this user's technical expertise level? Be concise."), - "current_goals": peer.chat("What are this user's current goals or priorities? Be concise."), - "preferences": peer.chat("What key preferences should I know about this user? Be concise.") - } - -def build_system_prompt(user_context: dict) -> str: - return f"""You are a helpful assistant. Here's what you know about this user: - -Communication style: {user_context['communication_style']} -Expertise level: {user_context['expertise_level']} -Current goals: {user_context['current_goals']} -Key preferences: {user_context['preferences']} - -Tailor your responses accordingly.""" -``` - -**TypeScript:** - -```typescript -async function getUserContextForPrompt(userId: string): Promise> { - const peer = await honcho.peer(userId); - - const [style, expertise, goals, preferences] = await Promise.all([ - peer.chat("What communication style does this user prefer? Be concise."), - peer.chat("What is this user's technical expertise level? Be concise."), - peer.chat("What are this user's current goals or priorities? Be concise."), - peer.chat("What key preferences should I know about this user? Be concise.") - ]); - - return { - communicationStyle: style, - expertiseLevel: expertise, - currentGoals: goals, - preferences: preferences - }; -} -``` - -### Pattern C: Get Context for LLM Integration - -Use `context()` for conversation history with built-in LLM formatting. - -**Python:** - -```python -import openai - -session = honcho.session("conversation-123") -user = honcho.peer("user-123") -assistant = honcho.peer("assistant", configuration=PeerConfig(observe_me=False)) - -# Get context formatted for your LLM -context = session.context( - tokens=2000, - peer_target=user.id, # Include representation of this user - summary=True # Include conversation summaries -) - -# Convert to OpenAI format -messages = context.to_openai(assistant=assistant) - -# Or Anthropic format -# messages = context.to_anthropic(assistant=assistant) - -# Add the new user message -messages.append({"role": "user", "content": "What should I focus on today?"}) - -response = openai.chat.completions.create( - model="gpt-4", - messages=messages -) - -# Store the exchange -session.add_messages([ - user.message("What should I focus on today?"), - assistant.message(response.choices[0].message.content) -]) -``` - -**TypeScript:** - -```typescript -import OpenAI from 'openai'; - -const session = await honcho.session("conversation-123"); -const user = await honcho.peer("user-123"); -const assistant = await honcho.peer("assistant", { configuration: { observeMe: false } }); - -// Get context formatted for your LLM -const context = await session.context({ - tokens: 2000, - peerTarget: user.id, // Include representation of this user - summary: true // Include conversation summaries -}); - -// Convert to OpenAI format -const messages = context.toOpenAI(assistant); - -// Or Anthropic format -// const messages = context.toAnthropic(assistant); - -// Add the new user message -messages.push({ role: "user", content: "What should I focus on today?" }); - -const openai = new OpenAI(); -const response = await openai.chat.completions.create({ - model: "gpt-4", - messages -}); - -// Store the exchange -await session.addMessages([ - user.message("What should I focus on today?"), - assistant.message(response.choices[0].message.content!) -]); -``` - -## Streaming Responses - -**Python:** - -```python -stream = peer.chat_stream("What do we know about this user?") - -for chunk in stream: - print(chunk, end="", flush=True) -``` - -**TypeScript:** - -```typescript -const stream = await peer.chatStream("What do we know about this user?"); - -for await (const chunk of stream) { - process.stdout.write(chunk); -} -``` - -## Integration Checklist - -When integrating Honcho into an existing codebase: - -- [ ] Install SDK with `uv add honcho-ai` (Python) or `bun add @honcho-ai/sdk` (TypeScript) -- [ ] Set up `HONCHO_API_KEY` environment variable -- [ ] Initialize Honcho client with a single workspace ID -- [ ] Create peers for all entities (users AND AI assistants) -- [ ] Set `observe_me=False` for AI peers -- [ ] Configure sessions with appropriate peer observation settings -- [ ] Choose integration pattern: - - [ ] Tool call pattern for agentic systems - - [ ] Pre-fetch pattern for simpler integrations - - [ ] context() for conversation history -- [ ] Store messages after each exchange to build user models -- [ ] (Optional) Run `honcho doctor` to verify connectivity before testing integration code -- [ ] (Optional) Use `honcho peer chat` to test dialectic queries independently - -## Common Mistakes to Avoid - -1. **Multiple workspaces**: Use ONE workspace per application -2. **Forgetting AI peers**: Create peers for AI assistants, not just users -3. **Observing AI peers**: Set `observe_me=False` for AI peers unless you specifically want Honcho to model your AI's behavior -4. **Not storing messages**: Always call `add_messages()` to feed Honcho's reasoning engine -5. **Blocking on processing**: Messages are processed asynchronously — don't poll or wait for reasoning to complete before continuing - -## Resources - -- Documentation: -- Latest SDK versions: -- API Reference: diff --git a/.claude/skills/migrate-honcho-py/DETAILED-CHANGES.md b/.claude/skills/migrate-honcho-py/DETAILED-CHANGES.md deleted file mode 100644 index 74e93a4b..00000000 --- a/.claude/skills/migrate-honcho-py/DETAILED-CHANGES.md +++ /dev/null @@ -1,607 +0,0 @@ -# Detailed API Changes - -## 1. Async Client Architecture (Major Change) - -The separate `AsyncHoncho`, `AsyncPeer`, and `AsyncSession` classes have been removed. Use the `.aio` accessor instead. - -### Before (v1.6.0) - -```python -from honcho import Honcho, AsyncHoncho, AsyncPeer, AsyncSession - -# Sync client -client = Honcho() - -# Async client - separate class -async_client = AsyncHoncho() -peer = await async_client.peer("user-123") -response = await peer.chat("query") -``` - -### After (v2.0.0) - -```python -from honcho import Honcho - -# Single client with .aio accessor for async operations -client = Honcho() - -# Sync operations -peer = client.peer("user-123") -response = peer.chat("query") - -# Async operations via .aio accessor -peer = await client.aio.peer("user-123") -response = await peer.aio.chat("query") - -# Async iteration -async for p in client.aio.peers(): - print(p.id) -``` - -**Migration steps:** - -1. Remove all `AsyncHoncho`, `AsyncPeer`, `AsyncSession` imports -2. Replace `AsyncHoncho()` with `Honcho()` and use `.aio` accessor -3. Replace `AsyncPeer` type hints with `Peer` -4. Replace `AsyncSession` type hints with `Session` -5. Access async methods via `.aio` property on instances - ---- - -## 2. Observations → Conclusions (Terminology Change) - -### Before (v1.6.0) - -```python -from honcho import Observation, ObservationScope, AsyncObservationScope - -# Access observations -scope = peer.observations -scope = peer.observations_of("other-peer") - -# List observations -obs_list = scope.list() - -# Query observations -results = scope.query("preferences") - -# Create observations -scope.create([{"content": "User likes dark mode", "session_id": "sess-1"}]) - -# Get representation from observations -rep = scope.get_representation() -``` - -### After (v2.0.0) - -```python -from honcho import Conclusion, ConclusionScope, ConclusionScopeAio - -# Access conclusions -scope = peer.conclusions -scope = peer.conclusions_of("other-peer") - -# List conclusions (now returns SyncPage, not list) -conclusions_page = scope.list() -for conclusion in conclusions_page: - print(conclusion.content) - -# Query conclusions -results = scope.query("preferences") - -# Create conclusions -scope.create([{"content": "User likes dark mode", "session_id": "sess-1"}]) - -# Get representation from conclusions -rep = scope.representation() # Returns str, not Representation object -``` - ---- - -## 3. Representation Type Change (Major Change) - -The `Representation` class has been removed. Representations are now simple strings. - -### Before (v1.6.0) - -```python -from honcho import Representation, ExplicitObservation, DeductiveObservation - -# Get working representation -rep: Representation = peer.working_rep() - -# Access explicit and deductive observations -for obs in rep.explicit: - print(obs.content, obs.created_at) - -for obs in rep.deductive: - print(obs.conclusion, obs.premises) - -# Check if empty -if rep.is_empty(): - print("No observations") - -# Merge representations -rep.merge_representation(other_rep) - -# Diff representations -diff = rep.diff_representation(other_rep) - -# String formatting -print(str(rep)) -print(rep.str_no_timestamps()) -print(rep.format_as_markdown()) -``` - -### After (v2.0.0) - -```python -# Get representation - now returns str directly -rep: str = peer.representation() - -# It's just a string now -print(rep) - -# Check if empty -if not rep: - print("No conclusions") -``` - -**Removed methods:** - -- `.explicit` property -- `.deductive` property -- `.is_empty()` -- `.merge_representation()` -- `.diff_representation()` -- `.str_no_timestamps()` -- `.format_as_markdown()` - ---- - -## 4. Configuration Parameter Rename - -All `config` parameters have been renamed to `configuration`, and configuration types are now strongly typed. - -### Before (v1.6.0) - -```python -# Creating resources with config -peer = client.peer("user-1", config={"observe_me": True}) -session = client.session("sess-1", config={"some_setting": True}) - -# Getting/setting config -config = peer.get_config() -peer.set_config({"observe_me": False}) - -config = session.get_config() -session.set_config({"some_setting": False}) - -config = client.get_config() -client.set_config({"workspace_setting": True}) - -# Message config parameter -msg = peer.message("Hello", config={"reasoning": {"enabled": True}}) -``` - -### After (v2.0.0) - -```python -from honcho.api_types import PeerConfig, SessionConfiguration, WorkspaceConfiguration - -# Creating resources with configuration (typed) -peer = client.peer("user-1", configuration=PeerConfig(observe_me=True)) -session = client.session("sess-1", configuration=SessionConfiguration()) - -# Getting/setting configuration (returns typed objects) -config: PeerConfig = peer.get_configuration() -peer.set_configuration(PeerConfig(observe_me=False)) - -config: SessionConfiguration = session.get_configuration() -session.set_configuration(SessionConfiguration()) - -config: WorkspaceConfiguration = client.get_configuration() -client.set_configuration(WorkspaceConfiguration()) - -# Message configuration parameter -msg = peer.message("Hello", configuration={"reasoning": {"enabled": True}}) -``` - ---- - -## 5. Streaming Chat API Change - -### Before (v1.6.0) - -```python -# Streaming via parameter -response = peer.chat("query", stream=True) -for chunk in response: - print(chunk, end="") - -final = response.get_final_response() -``` - -### After (v2.0.0) - -```python -# Streaming via separate method -stream = peer.chat_stream("query") -for chunk in stream: - print(chunk, end="") - -final = stream.get_final_response() - -# Non-streaming (no stream parameter needed) -response = peer.chat("query") -``` - ---- - -## 6. Deriver Status → Queue Status - -### Before (v1.6.0) - -```python -from honcho_core.types import DeriverStatus - -# Get status -status: DeriverStatus = client.get_deriver_status() -status = session.get_deriver_status() - -# Poll until complete -status = client.poll_deriver_status(timeout=300.0) -status = session.poll_deriver_status(timeout=300.0) - -# Access fields -print(status.pending_work_units) -print(status.in_progress_work_units) -``` - -### After (v2.0.0) - -```python -from honcho.api_types import QueueStatusResponse - -# Get status -status: QueueStatusResponse = client.queue_status() -status = session.queue_status() - -# Access fields (same as before) -print(status.pending_work_units) -print(status.in_progress_work_units) - -# poll_deriver_status has been removed - implement polling manually if needed: -import time - -def poll_until_complete(client, timeout=300.0): - start = time.time() - while time.time() - start < timeout: - status = client.queue_status() - if status.pending_work_units == 0 and status.in_progress_work_units == 0: - return status - time.sleep(1) - raise TimeoutError("Queue processing did not complete in time") -``` - ---- - -## 7. PeerContext Changes - -### Before (v1.6.0) - -```python -from honcho import PeerContext - -context: PeerContext = peer.get_context() - -# Access representation (was Representation object) -rep: Representation = context.representation -if rep: - print(rep.explicit) - print(rep.deductive) -``` - -### After (v2.0.0) - -```python -from honcho.api_types import PeerContextResponse - -context: PeerContextResponse = peer.context() - -# Access representation (now str) -rep: str | None = context.representation -if rep: - print(rep) -``` - ---- - -## 8. Card Method Return Type Change - -### Before (v1.6.0) - -```python -# card() returned str (joined with newlines) -card: str = peer.card() -print(card) # "line1\nline2\nline3" -``` - -### After (v2.0.0) - -```python -# card() returns list[str] | None -card: list[str] | None = peer.card() -if card: - print("\n".join(card)) # Join manually if needed -``` - ---- - -## 9. Message Update Location Change - -### Before (v1.6.0) - -```python -# Update message via client -updated = client.update_message( - message=msg, - metadata={"key": "value"}, - session="session-id" # Required if message is string ID -) -``` - -### After (v2.0.0) - -```python -# Update message via session -updated = session.update_message( - message=msg, - metadata={"key": "value"} -) -``` - ---- - -## 10. Removed: `core` Property - -### Before (v1.6.0) - -```python -# Access underlying Stainless-generated client -core_client = client.core -workspace = client.core.workspaces.get_or_create(id="custom-workspace") -``` - -### After (v2.0.0) - -```python -# The `core` property has been removed -# The SDK no longer uses a Stainless-generated client internally -# Use the SDK's public API directly -``` - ---- - -## 11. Environment Changes - -### Before (v1.6.0) - -```python -# Three environments available -client = Honcho(environment="local") -client = Honcho(environment="production") -client = Honcho(environment="demo") -``` - -### After (v2.0.0) - -```python -# Only two environments -client = Honcho(environment="local") -client = Honcho(environment="production") -# "demo" environment has been removed -``` - ---- - -## 12. Reasoning Level Parameter (New Feature) - -The chat method now supports a `reasoning_level` parameter: - -```python -# New in v2.0.0 -response = peer.chat( - "complex query", - reasoning_level="high" # "minimal", "low", "medium", "high", "max" -) - -stream = peer.chat_stream( - "complex query", - reasoning_level="max" -) -``` - ---- - -## 13. Import Changes Summary - -### Removed Imports - -```python -# These no longer exist in v2.0.0 -from honcho import AsyncHoncho # Use Honcho with .aio accessor -from honcho import AsyncPeer # Use Peer with .aio accessor -from honcho import AsyncSession # Use Session with .aio accessor -from honcho import Observation # Renamed to Conclusion -from honcho import ObservationScope # Renamed to ConclusionScope -from honcho import AsyncObservationScope # Renamed to ConclusionScopeAio -from honcho import Representation # Removed (now str) -from honcho import ExplicitObservation # Removed -from honcho import DeductiveObservation # Removed -from honcho import PeerContext # Use PeerContextResponse from api_types -``` - -### New Imports - -```python -from honcho import Conclusion, ConclusionScope -from honcho import ConclusionScopeAio -from honcho import HonchoAio, PeerAio, SessionAio # For type hints -from honcho import MessageCreateParams, Message - -# Typed configuration classes -from honcho.api_types import ( - PeerConfig, - SessionConfiguration, - WorkspaceConfiguration, - SessionPeerConfig, - QueueStatusResponse, - PeerContextResponse, -) -``` - -### Message Type Import Changes - -```python -# Before -from honcho_core.types.workspaces.sessions import MessageCreateParam -from honcho_core.types.workspaces.sessions.message import Message -from honcho.session import SessionPeerConfig - -# After -from honcho import Message, MessageCreateParams # Note: plural "Params" -from honcho.api_types import SessionPeerConfig -``` - -**Note:** `MessageCreateParam` (singular) is now `MessageCreateParams` (plural). - ---- - -## 14. Card Method Deprecation and set_card (v2.0.1) - -### Before (v2.0.0) - -```python -card: list[str] | None = peer.card() -``` - -### After (v2.0.1+) - -```python -# get_card() is the preferred method -card: list[str] | None = peer.get_card() - -# card() still works but emits a deprecation warning -card = peer.card() # Deprecated - -# New: set_card() -updated = peer.set_card(["Fact 1", "Fact 2"]) -updated = peer.set_card(["Fact 1"], target="other-peer") - -# Async variants -card = await peer.aio.get_card() -await peer.aio.set_card(["Fact 1"]) -``` - ---- - -## 15. Strict Input Validation (v2.0.2) - -All Pydantic input models now use `extra="forbid"`, raising `ValidationError` for unknown fields. - -```python -from honcho.api_types import PeerConfig - -# This now raises ValidationError instead of silently ignoring the typo -PeerConfig(observe_mee=True) # ValidationError: extra fields not permitted -``` - ---- - -## 16. peer() and session() Always Make API Calls (v2.1.0) - -### Before (v2.0.x) - -```python -# Without options: lazy object, no API call -peer = client.peer("user-123") -# peer.created_at was None - -# With options: made API call -peer = client.peer("user-123", metadata={"key": "value"}) -``` - -### After (v2.1.0+) - -```python -# Always makes a get-or-create API call -peer = client.peer("user-123") -# peer.created_at is now always populated - -# Async -peer = await client.aio.peer("user-123") -``` - -All Peer/Session objects now have `created_at` populated immediately after construction. - ---- - -## 17. New Properties: created_at, is_active (v2.1.0) - -```python -# Peer -peer = client.peer("user-123") -print(peer.created_at) # datetime | None - -# Session -session = client.session("sess-1") -print(session.created_at) # datetime | None -print(session.is_active) # bool | None - -# These are refreshed by get_metadata(), get_configuration(), and refresh() -peer.refresh() -session.refresh() -``` - ---- - -## 18. get_message() on Session (v2.1.0) - -```python -# Fetch a single message by ID -msg = session.get_message("msg-abc123") -print(msg.content, msg.created_at) - -# Async -msg = await session.aio.get_message("msg-abc123") -``` - ---- - -## 19. Pagination Parameters (v2.1.0) - -All list methods now accept `page`, `size`, and `reverse`: - -```python -# Defaults: page=1, size=50, reverse=False -peers_page = client.peers(page=2, size=25, reverse=True) - -# Returns SyncPage / AsyncPage with: -print(peers_page.total) # Total items -print(peers_page.pages) # Total pages -print(peers_page.has_next_page()) - -# Works on: -# client.peers(), client.sessions() -# peer.sessions() -# session.messages() -# scope.list() -``` - ---- - -## 20. Broader HTTP Retry Logic (v2.1.1) - -The SDK now catches `httpx.NetworkError` and `httpx.RemoteProtocolError` for retry in addition to `httpx.TimeoutException` and `httpx.ConnectError`. This is transparent — no code changes needed. diff --git a/.claude/skills/migrate-honcho-py/MIGRATION-CHECKLIST.md b/.claude/skills/migrate-honcho-py/MIGRATION-CHECKLIST.md deleted file mode 100644 index ef3254da..00000000 --- a/.claude/skills/migrate-honcho-py/MIGRATION-CHECKLIST.md +++ /dev/null @@ -1,155 +0,0 @@ -# Migration Checklist - -Use this checklist to track migration progress. Copy into your working notes and check off items as completed. - -## Dependencies - -- [ ] Update `honcho` package to v2.1.1 -- [ ] Remove any `honcho-core` imports - -## Async Architecture Changes - -- [ ] Remove `AsyncHoncho` imports → use `Honcho` with `.aio` accessor -- [ ] Remove `AsyncPeer` imports → use `Peer` with `.aio` accessor -- [ ] Remove `AsyncSession` imports → use `Session` with `.aio` accessor -- [ ] Update all async client usage to use `.aio` accessor pattern -- [ ] Update type hints: `AsyncPeer` → `Peer`, `AsyncSession` → `Session` - -## Terminology: Observations → Conclusions - -- [ ] Replace `Observation` import with `Conclusion` -- [ ] Replace `ObservationScope` import with `ConclusionScope` -- [ ] Replace `AsyncObservationScope` import with `ConclusionScopeAio` -- [ ] Replace `.observations` property with `.conclusions` -- [ ] Replace `.observations_of()` method with `.conclusions_of()` -- [ ] Replace `.get_representation()` with `.representation()` - -## Representation Changes - -- [ ] Remove `Representation` import (now returns `str`) -- [ ] Remove `ExplicitObservation` import -- [ ] Remove `DeductiveObservation` import -- [ ] Replace `working_rep()` with `representation()` -- [ ] Update type hints from `Representation` to `str` -- [ ] Remove `.explicit` property access -- [ ] Remove `.deductive` property access -- [ ] Replace `.is_empty()` checks with `not rep` -- [ ] Remove `.merge_representation()` calls -- [ ] Remove `.diff_representation()` calls -- [ ] Remove `.str_no_timestamps()` calls -- [ ] Remove `.format_as_markdown()` calls - -## Configuration Changes - -- [ ] Replace all `config=` parameters with `configuration=` -- [ ] Replace `.get_config()` with `.get_configuration()` -- [ ] Replace `.set_config()` with `.set_configuration()` -- [ ] Rename `.get_peer_config()` → `.get_peer_configuration()` -- [ ] Rename `.set_peer_config()` → `.set_peer_configuration()` -- [ ] Import typed config classes from `honcho.api_types` if needed: - - [ ] `PeerConfig` - - [ ] `SessionConfiguration` - - [ ] `WorkspaceConfiguration` - -## Method Renames - -### Peer Methods - -- [ ] `peer.working_rep()` → `peer.representation()` -- [ ] `peer.get_context()` → `peer.context()` -- [ ] `peer.get_sessions()` → `peer.sessions()` -- [ ] `peer.chat(stream=True)` → `peer.chat_stream()` - -### Session Methods - -- [ ] `session.get_context()` → `session.context()` -- [ ] `session.get_summaries()` → `session.summaries()` -- [ ] `session.get_messages()` → `session.messages()` -- [ ] `session.get_peers()` → `session.peers()` -- [ ] `session.get_peer_config()` → `session.get_peer_configuration()` -- [ ] `session.set_peer_config()` → `session.set_peer_configuration()` -- [ ] `session.working_rep()` → `session.representation()` -- [ ] `session.get_deriver_status()` → `session.queue_status()` -- [ ] Remove `session.poll_deriver_status()` calls - -### Client Methods - -- [ ] `client.get_peers()` → `client.peers()` -- [ ] `client.get_sessions()` → `client.sessions()` -- [ ] `client.get_workspaces()` → `client.workspaces()` -- [ ] `client.get_deriver_status()` → `client.queue_status()` -- [ ] Remove `client.poll_deriver_status()` calls -- [ ] Move `client.update_message()` → `session.update_message()` - -## Parameter Renames - -- [ ] `include_most_derived=` → `include_most_frequent=` -- [ ] `max_observations=` → `max_conclusions=` -- [ ] `last_user_message=` → `search_query=` - -## Return Type Changes - -- [ ] Handle `card()` returning `list[str] | None` instead of `str` -- [ ] Handle `.list()` on conclusions returning `SyncPage` instead of `list` - -## Removed Features - -- [ ] Remove any usage of `client.core` property -- [ ] Remove usage of `"demo"` environment (only `"local"` and `"production"` remain) -- [ ] Implement custom polling if you were using `poll_deriver_status()` - -## Type Import Updates - -- [ ] Replace `PeerContext` import with `PeerContextResponse` from `honcho.api_types` -- [ ] Replace `DeriverStatus` import with `QueueStatusResponse` from `honcho.api_types` -- [ ] Replace `MessageCreateParam` with `MessageCreateParams` (plural) -- [ ] Move `SessionPeerConfig` import from `honcho.session` to `honcho.api_types` - -## Exception Handling (Optional) - -- [ ] Update exception handling to use new exception types if needed: - - `HonchoError`, `APIError`, `BadRequestError`, `AuthenticationError` - - `PermissionDeniedError`, `NotFoundError`, `ConflictError` - - `UnprocessableEntityError`, `RateLimitError`, `ServerError` - - `TimeoutError`, `ConnectionError` - -## Card Method Updates (v2.0.1) - -- [ ] Replace `peer.card()` with `peer.get_card()` (card() is deprecated) -- [ ] Use `peer.set_card(list[str])` if setting peer cards - -## Strict Validation (v2.0.2) - -- [ ] Verify no input models pass unknown/misspelled fields (now raises `ValidationError`) -- [ ] Check for typos in `PeerConfig`, `SessionConfiguration`, `WorkspaceConfiguration` fields - -## peer() / session() API Call Change (v2.1.0) - -- [ ] Update code that relied on lazy `peer()` / `session()` — they now always make API calls -- [ ] Add `await` if using async and previously didn't need it for lazy construction - -## New Properties (v2.1.0) - -- [ ] Use `peer.created_at` / `session.created_at` where creation time is needed -- [ ] Use `session.is_active` where session active status is needed - -## New Methods (v2.1.0) - -- [ ] Use `session.get_message(message_id)` to fetch single messages by ID - -## Pagination Parameters (v2.1.0) - -- [ ] Add `page`, `size`, `reverse` parameters to list calls where needed: - - [ ] `client.peers()` - - [ ] `client.sessions()` - - [ ] `peer.sessions()` - - [ ] `session.messages()` - - [ ] `scope.list()` - -## Final Verification - -- [ ] Run type checker (mypy/pyright) with no errors -- [ ] Run tests -- [ ] Verify async operations work with `.aio` accessor -- [ ] Verify streaming functionality works with `chat_stream()` -- [ ] Verify configuration changes take effect diff --git a/.claude/skills/migrate-honcho-py/SKILL.md b/.claude/skills/migrate-honcho-py/SKILL.md deleted file mode 100644 index c9f8320e..00000000 --- a/.claude/skills/migrate-honcho-py/SKILL.md +++ /dev/null @@ -1,358 +0,0 @@ ---- -name: migrate-honcho -description: Migrates Honcho Python SDK code from v1.6.0 to v2.1.1. Use when upgrading honcho package, fixing breaking changes after upgrade, or when errors mention AsyncHoncho, observations, Representation class, .core property, or get_config methods. ---- - -# Honcho Python SDK Migration (v1.6.0 → v2.1.1) - -## Overview - -This skill migrates code from `honcho` Python SDK v1.6.0 to v2.1.1 (required for Honcho 3.0.0+). - -**Key breaking changes:** - -- `AsyncHoncho`/`AsyncPeer`/`AsyncSession` removed → use `.aio` accessor -- "Observation" → "Conclusion" terminology -- `Representation` class removed (returns `str` now) -- `get_config`/`set_config` → `get_configuration`/`set_configuration` -- Streaming via `chat_stream()` instead of `chat(stream=True)` -- `poll_deriver_status()` removed -- `.core` property removed - -## Quick Migration - -### 1. Update async architecture - -```python -# Before -from honcho import AsyncHoncho, AsyncPeer, AsyncSession - -async_client = AsyncHoncho() -peer = await async_client.peer("user-123") -response = await peer.chat("query") - -# After -from honcho import Honcho - -client = Honcho() -peer = await client.aio.peer("user-123") -response = await peer.aio.chat("query") - -# Async iteration -async for p in client.aio.peers(): - print(p.id) -``` - -### 2. Replace observations with conclusions - -```python -# Before -from honcho import Observation, ObservationScope, AsyncObservationScope - -scope = peer.observations -scope = peer.observations_of("other-peer") -rep = scope.get_representation() - -# After -from honcho import Conclusion, ConclusionScope, ConclusionScopeAio - -scope = peer.conclusions -scope = peer.conclusions_of("other-peer") -rep = scope.representation() # Returns str -``` - -### 3. Update representation handling - -```python -# Before -from honcho import Representation, ExplicitObservation, DeductiveObservation - -rep: Representation = peer.working_rep() -print(rep.explicit) -print(rep.deductive) -if rep.is_empty(): - print("No observations") - -# After -rep: str = peer.representation() -print(rep) # Just a string now -if not rep: - print("No conclusions") -``` - -### 4. Rename configuration methods - -```python -# Before -config = peer.get_config() -peer.set_config({"observe_me": False}) -session.get_config() -client.get_config() - -# After -from honcho.api_types import PeerConfig, SessionConfiguration, WorkspaceConfiguration - -config = peer.get_configuration() -peer.set_configuration(PeerConfig(observe_me=False)) -session.get_configuration() -client.get_configuration() -``` - -### 5. Update method names - -```python -# Before -peer.working_rep() -peer.get_context() -peer.get_sessions() -session.get_context() -session.get_summaries() -session.get_messages() -session.get_peers() -session.get_peer_config() -client.get_peers() -client.get_sessions() -client.get_workspaces() - -# After -peer.representation() -peer.context() -peer.sessions() -session.context() -session.summaries() -session.messages() -session.peers() -session.get_peer_configuration() -client.peers() -client.sessions() -client.workspaces() -``` - -### 6. Update streaming - -```python -# Before -response = peer.chat("query", stream=True) -for chunk in response: - print(chunk, end="") - -# After -stream = peer.chat_stream("query") -for chunk in stream: - print(chunk, end="") -``` - -### 7. Update queue status (formerly deriver) - -```python -# Before -from honcho_core.types import DeriverStatus - -status = client.get_deriver_status() -status = client.poll_deriver_status(timeout=300.0) # Removed! - -# After -from honcho.api_types import QueueStatusResponse - -status = client.queue_status() -# poll_deriver_status removed - implement polling manually if needed -``` - -### 8. Update representation parameters - -```python -# Before -rep = peer.working_rep( - include_most_derived=True, - max_observations=50 -) - -# After -rep = peer.representation( - include_most_frequent=True, - max_conclusions=50 -) -``` - -### 9. Move update_message to session - -```python -# Before -updated = client.update_message(message=msg, metadata={"key": "value"}, session="sess-id") - -# After -updated = session.update_message(message=msg, metadata={"key": "value"}) -``` - -### 10. Update card() return type and method name - -```python -# Before -card: str = peer.card() # Returns str - -# After (v2.0.0+) -card: list[str] | None = peer.get_card() # Returns list[str] | None -if card: - print("\n".join(card)) - -# peer.card() still works but is deprecated — use get_card() - -# New in v2.0.1: set_card() -peer.set_card(["Prefers dark mode", "Located in US"]) -``` - -### 11. Strict input validation (v2.0.2+) - -All input models now reject unknown fields via `extra="forbid"` Pydantic validation. Previously, misspelled or extraneous fields were silently ignored. - -```python -# Before (v2.0.1 and earlier) — silently ignored -peer = client.peer("user-1", configuration=PeerConfig(observe_mee=True)) # typo silently ignored - -# After (v2.0.2+) — raises ValidationError -peer = client.peer("user-1", configuration=PeerConfig(observe_mee=True)) # ValidationError! -``` - -### 12. peer() and session() always make API calls (v2.1.0+) - -**Breaking**: `peer()` and `session()` now always make a get-or-create API call. Previously, calling without metadata/configuration returned a lazy object with no API call. - -```python -# Before (v2.0.x) — no API call without options -peer = client.peer("user-123") # Lazy, no network request - -# After (v2.1.0+) — always hits the API -peer = client.peer("user-123") # Makes POST to /peers (get-or-create) - -# Async -peer = await client.aio.peer("user-123") # Also always hits API -``` - -### 13. New properties and methods (v2.1.0+) - -```python -# created_at on Peer and Session -peer = client.peer("user-123") -print(peer.created_at) # datetime | None - -session = client.session("sess-1") -print(session.created_at) # datetime | None - -# is_active on Session -print(session.is_active) # bool | None - -# get_message() on Session -msg = session.get_message("msg-id") -# Async: msg = await session.aio.get_message("msg-id") -``` - -### 14. Pagination parameters on list methods (v2.1.0+) - -All list methods now accept `page`, `size`, and `reverse` parameters: - -```python -# Before (v2.0.x) — only filters -peers_page = client.peers(filters={"metadata": {"role": "admin"}}) - -# After (v2.1.0+) — pagination controls -peers_page = client.peers( - filters={"metadata": {"role": "admin"}}, - page=2, - size=25, - reverse=True -) - -# Works on: client.peers(), client.sessions(), peer.sessions(), -# session.messages(), scope.list() -``` - -### 15. Broader HTTP retry logic (v2.1.1+) - -The SDK now retries on `httpx.TimeoutException`, `httpx.NetworkError`, and `httpx.RemoteProtocolError` (previously only `httpx.TimeoutException` and `httpx.ConnectError`). These are mapped to the SDK's `TimeoutError` and `ConnectionError` respectively. No code changes needed — this is transparent. - -## Quick Reference Table - -| v1.6.0 | v2.0.0 | -|--------|--------| -| `AsyncHoncho()` | `Honcho()` + `.aio` accessor | -| `AsyncPeer` | `Peer` + `.aio` accessor | -| `AsyncSession` | `Session` + `.aio` accessor | -| `Observation` | `Conclusion` | -| `ObservationScope` | `ConclusionScope` | -| `AsyncObservationScope` | `ConclusionScopeAio` | -| `Representation` | `str` | -| `.observations` | `.conclusions` | -| `.observations_of()` | `.conclusions_of()` | -| `.get_config()` | `.get_configuration()` | -| `.set_config()` | `.set_configuration()` | -| `.working_rep()` | `.representation()` | -| `.get_context()` | `.context()` | -| `.get_sessions()` | `.sessions()` | -| `.get_peers()` | `.peers()` | -| `.get_messages()` | `.messages()` | -| `.get_summaries()` | `.summaries()` | -| `.get_deriver_status()` | `.queue_status()` | -| `.poll_deriver_status()` | *(removed)* | -| `.get_peer_config()` | `.get_peer_configuration()` | -| `.set_peer_config()` | `.set_peer_configuration()` | -| `client.update_message()` | `session.update_message()` | -| `peer.card()` | `peer.get_card()` *(card() deprecated)* | -| *(new)* | `peer.set_card(list[str])` | -| `chat(stream=True)` | `chat_stream()` | -| `include_most_derived=` | `include_most_frequent=` | -| `max_observations=` | `max_conclusions=` | -| `last_user_message=` | `search_query=` | -| `config=` | `configuration=` | -| `PeerContext` | `PeerContextResponse` | -| `DeriverStatus` | `QueueStatusResponse` | -| `client.core` | *(removed)* | -| *(new v2.1.0)* | `peer.created_at` / `session.created_at` | -| *(new v2.1.0)* | `session.is_active` | -| *(new v2.1.0)* | `session.get_message(id)` | -| *(new v2.1.0)* | `page=`, `size=`, `reverse=` on list methods | - -## Detailed Reference - -For comprehensive details on each change, see: - -- [DETAILED-CHANGES.md](DETAILED-CHANGES.md) - Full API change documentation -- [MIGRATION-CHECKLIST.md](MIGRATION-CHECKLIST.md) - Step-by-step checklist - -## New Exception Types - -```python -from honcho import ( - HonchoError, - APIError, - BadRequestError, - AuthenticationError, - PermissionDeniedError, - NotFoundError, - ConflictError, - UnprocessableEntityError, - RateLimitError, - ServerError, - TimeoutError, - ConnectionError, -) -``` - -## New Import Locations - -```python -# Configuration types -from honcho.api_types import ( - PeerConfig, - SessionConfiguration, - WorkspaceConfiguration, - SessionPeerConfig, - QueueStatusResponse, - PeerContextResponse, -) - -# Async type hints -from honcho import HonchoAio, PeerAio, SessionAio - -# Message types (note: Params is plural now) -from honcho import Message, MessageCreateParams -``` diff --git a/.claude/skills/migrate-honcho-ts/DETAILED-CHANGES.md b/.claude/skills/migrate-honcho-ts/DETAILED-CHANGES.md deleted file mode 100644 index 9133756e..00000000 --- a/.claude/skills/migrate-honcho-ts/DETAILED-CHANGES.md +++ /dev/null @@ -1,583 +0,0 @@ -# Detailed API Changes - -## Client Changes - -### `.core` Property Removed - -The `.core` property (which exposed the raw `@honcho-ai/core` client) has been removed. Use `.http` for advanced HTTP access. - -```typescript -// Before -const workspace = await client.core.workspaces.getOrCreate({ id: 'my-workspace' }) - -// After - SDK handles workspace creation automatically -// For advanced usage: -const response = await client.http.post('/v3/workspaces', { body: { id: 'my-workspace' } }) -``` - -### Listing Methods Return Type Changes - -- `workspaces()` now returns `Page` instead of `string[]` -- `session.peers()` now returns `Peer[]` instead of `Page` - -```typescript -const workspacePage = await honcho.workspaces() -for (const id of workspacePage.items) { - console.log(id) -} -``` - -### `updateMessage()` Moved to Session - -```typescript -// Before -await honcho.updateMessage(message, { key: 'value' }, session) - -// After -await session.updateMessage(message, { key: 'value' }) -``` - -### `config` Option Renamed to `configuration` - -```typescript -// Before -const peer = await honcho.peer('user-id', { config: { observe_me: true } }) -const session = await honcho.session('session-id', { config: { ... } }) - -// After -const peer = await honcho.peer('user-id', { configuration: { observeMe: true } }) -const session = await honcho.session('session-id', { configuration: { reasoning: { enabled: true } } }) -``` - ---- - -## Peer Changes - -### Streaming API - -The `stream` option on `chat()` has been removed. Use `chatStream()` instead. - -```typescript -// Before -const stream = await peer.chat('Hello', { stream: true }) -for await (const chunk of stream) { - process.stdout.write(chunk) -} - -// After -const stream = await peer.chatStream('Hello') -for await (const chunk of stream) { - process.stdout.write(chunk) -} -``` - -Non-streaming `chat()` now only returns `string | null`: - -```typescript -const response = await peer.chat('Hello') // Returns string | null -``` - -### New `reasoningLevel` Option - -```typescript -const response = await peer.chat('Complex question', { - reasoningLevel: 'high' // 'minimal' | 'low' | 'medium' | 'high' | 'max' -}) -``` - -### `workingRep()` Renamed to `representation()` - -```typescript -// Before -const rep = await peer.workingRep(session, target, options) -console.log(rep.toString()) -console.log(rep.explicit) -console.log(rep.deductive) - -// After -const rep = await peer.representation({ - session, - target, - searchQuery: options?.searchQuery, - maxConclusions: options?.maxObservations, - includeMostFrequent: options?.includeMostDerived, -}) -console.log(rep) // Returns string directly -``` - -### `getContext()` Renamed to `context()` - -Options are now passed as a single object: - -```typescript -// Before -const ctx = await peer.getContext(target, options) - -// After -const ctx = await peer.context({ target, ...options }) -``` - -### `card()` Return Type Changed - -```typescript -// Before -const card = await peer.card(target) // Returns string - -// After -const card = await peer.card(target) // Returns string[] | null -``` - -### `message()` Options Changed - -```typescript -// Before -const msg = peer.message('Hello', { - metadata: { key: 'value' }, - configuration: { deriver: { enabled: true } }, - created_at: '2024-01-01T00:00:00Z' -}) -// Returns ValidatedMessageCreate with peer_id, created_at - -// After -const msg = peer.message('Hello', { - metadata: { key: 'value' }, - configuration: { reasoning: { enabled: true } }, - createdAt: '2024-01-01T00:00:00Z' -}) -// Returns MessageInput with peerId, createdAt -``` - -### `PeerContext.representation` Type Changed - -```typescript -// Before -const ctx = await peer.getContext() -if (ctx.representation) { - console.log(ctx.representation.explicit) // Representation object - console.log(ctx.representation.deductive) -} - -// After -const ctx = await peer.context() -if (ctx.representation) { - console.log(ctx.representation) // Now a string -} -``` - ---- - -## Session Changes - -### `getPeers()` Return Type Changed - -```typescript -// Before -const peers = await session.getPeers() // Returns Page - -// After -const peers = await session.peers() // Returns Peer[] -``` - -### `getContext()` Renamed to `context()` - -```typescript -// Before -const ctx = await session.getContext({ - summary: true, - peerTarget: user, - peerPerspective: assistant, - lastUserMessage: "What are my preferences?", - representationOptions: { - maxObservations: 50, - includeMostDerived: true - } -}) - -// After -const ctx = await session.context({ - summary: true, - peerTarget: user, - peerPerspective: assistant, - searchQuery: "What are my preferences?", - representationOptions: { - maxConclusions: 50, - includeMostFrequent: true - } -}) -``` - -### `SessionPeerConfig` Uses camelCase and Methods Renamed - -```typescript -// Before -await session.setPeerConfig(peer, { - observe_me: true, - observe_others: false -}) -const config = await session.peerConfig(peer) - -// After -await session.setPeerConfiguration(peer, { - observeMe: true, - observeOthers: false -}) -const config = await session.getPeerConfiguration(peer) -``` - ---- - -## Message Changes - -### Message Properties Use camelCase - -```typescript -// Before (from @honcho-ai/core) -message.peer_id -message.session_id -message.workspace_id -message.created_at -message.token_count - -// After -message.peerId -message.sessionId -message.workspaceId -message.createdAt -message.tokenCount -``` - -### MessageInput Type - -```typescript -// Before -interface ValidatedMessageCreate { - peer_id: string - content: string - metadata?: Record - configuration?: Record - created_at?: string -} - -// After -interface MessageInput { - peerId: string - content: string - metadata?: Record - configuration?: MessageConfiguration - createdAt?: string -} -``` - ---- - -## Streaming Changes - -### `DialecticStreamDelta` Removed - -```typescript -// Before -import { DialecticStreamDelta, DialecticStreamChunk } from '@honcho-ai/sdk' - -// After -import { DialecticStreamChunk, DialecticStreamResponse } from '@honcho-ai/sdk' -``` - ---- - -## Configuration Changes - -### Workspace Configuration - -Configurations are now strongly typed objects instead of `Record`. - -```typescript -// Before -await honcho.setConfig({ - deriver: { enabled: true }, - some_custom_key: 'value' -}) - -// After -await honcho.setConfiguration({ - reasoning: { - enabled: true, - customInstructions: 'Be concise' - }, - peerCard: { - use: true, - create: true - }, - summary: { - enabled: true, - messagesPerShortSummary: 20, - messagesPerLongSummary: 60 - }, - dream: { - enabled: true - } -}) -``` - -### Peer Configuration - -```typescript -// Before -await peer.setConfig({ observe_me: false }) - -// After -await peer.setConfiguration({ observeMe: false }) -``` - -### Message Configuration - -```typescript -// Before -peer.message('Hello', { - configuration: { - deriver: { enabled: true } - } -}) - -// After -peer.message('Hello', { - configuration: { - reasoning: { - enabled: true, - customInstructions: 'Focus on emotions' - } - } -}) -``` - ---- - -## Type Changes - -### Removed Exports - -- `Observation` (use `Conclusion`) -- `ObservationScope` (use `ConclusionScope`) -- `ObservationData`, `ObservationCreateParam`, `ObservationQueryParams` -- `Representation`, `RepresentationData`, `RepresentationOptions` (class removed) -- `ExplicitObservation`, `DeductiveObservation` -- `DialecticStreamDelta` -- `DeriverStatusOptions` (use `QueueStatusOptions`) -- `MessageCreate` (use `MessageInput`) -- `WorkingRepParams` - -### New Exports - -```typescript -import { - // Domain classes - Conclusion, - ConclusionScope, - ConclusionCreateParams, - - // Error types - HonchoError, - AuthenticationError, - BadRequestError, - NotFoundError, - PermissionDeniedError, - RateLimitError, - ConflictError, - UnprocessableEntityError, - ServerError, - ConnectionError, - TimeoutError, - - // Message types - Message, - MessageInput, - - // Configuration types - WorkspaceConfig, - SessionConfig, - PeerConfig, - SessionPeerConfig, - MessageConfiguration, - ReasoningConfig, - PeerCardConfig, - SummaryConfig, - DreamConfig, - - // API response types - QueueStatus, - QueueStatusOptions, - RepresentationOptions, - ConclusionQueryParams, - ConclusionResponse, -} from '@honcho-ai/sdk' -``` - -### SummaryData Type Changed - -```typescript -// Before -interface SummaryData { - content: string - message_id: string - summary_type: string - created_at: string - token_count: number -} - -// After -interface SummaryData { - content: string - messageId: string - summaryType: string - createdAt: string - tokenCount: number -} -``` - ---- - -## Post-v2.0.0 Changes - ---- - -## Card Method Deprecation and setCard (v2.0.1) - -### Before (v2.0.0) - -```typescript -const card = await peer.card(target) // string[] | null -``` - -### After (v2.0.1+) - -```typescript -// getCard() is the preferred method -const card = await peer.getCard(target) // string[] | null - -// card() still works but is deprecated -const card = await peer.card(target) // Deprecated - -// New: setCard() -const updated = await peer.setCard(['Fact 1', 'Fact 2']) -const updated = await peer.setCard(['Fact 1'], targetPeer) -``` - ---- - -## Strict Input Validation (v2.0.2) - -Client constructor and all input schemas now use `.strict()` Zod validation. - -```typescript -// Before (v2.0.1) — silently ignored -const honcho = new Honcho({ baseUrl: 'http://...' }) // typo fell back to default - -// After (v2.0.2+) — ZodError thrown -const honcho = new Honcho({ baseUrl: 'http://...' }) // ZodError: Unrecognized key "baseUrl" -``` - ---- - -## peer() and session() Always Make API Calls (v2.1.0) - -### Before (v2.0.x) - -```typescript -// Without options: lazy object, no API call -const peer = honcho.peer('user-123') - -// With options: made API call -const peer = await honcho.peer('user-123', { metadata: { key: 'value' } }) -``` - -### After (v2.1.0+) - -```typescript -// Always makes a get-or-create API call -const peer = await honcho.peer('user-123') -// peer.createdAt is now always populated -``` - ---- - -## New Properties: createdAt, isActive (v2.1.0) - -```typescript -// Peer -const peer = await honcho.peer('user-123') -console.log(peer.createdAt) // string | undefined - -// Session -const session = await honcho.session('sess-1') -console.log(session.createdAt) // string | undefined -console.log(session.isActive) // boolean | undefined - -// Refreshed by getMetadata(), getConfiguration(), and refresh() -await session.refresh() -``` - ---- - -## getMessage() on Session (v2.1.0) - -```typescript -// Fetch a single message by ID -const msg = await session.getMessage('msg-abc123') -console.log(msg.content, msg.createdAt) -``` - ---- - -## Pagination Parameters (v2.1.0) - -All list methods now accept `page`, `size`, and `reverse`: - -```typescript -// Defaults: page=1, size=50, reverse=false -const peersPage = await honcho.peers({ - filters: { metadata: { role: 'admin' } }, - page: 2, - size: 25, - reverse: true -}) - -// Page properties: -console.log(peersPage.total) // Total items -console.log(peersPage.pages) // Total pages -console.log(peersPage.hasNextPage) // boolean - -// Works on: -// honcho.peers(), honcho.sessions(), honcho.workspaces() -// peer.sessions() -// session.messages() -// scope.list() -``` - ---- - -## searchQuery Moved in context() (v2.1.0) - -### Before (v2.0.x) - -```typescript -const ctx = await session.context({ - searchQuery: 'What are my preferences?', - representationOptions: { maxConclusions: 50 } -}) -``` - -### After (v2.1.0+) - -```typescript -const ctx = await session.context({ - representationOptions: { - searchQuery: 'What are my preferences?', - maxConclusions: 50 - } -}) -``` - ---- - -## Broader Fetch Retry Logic (v2.1.1) - -The SDK now retries on all `TypeError` network failures (connection resets, DNS errors, etc.) instead of only those containing `'fetch'` in the error message. This is transparent — no code changes needed. diff --git a/.claude/skills/migrate-honcho-ts/MIGRATION-CHECKLIST.md b/.claude/skills/migrate-honcho-ts/MIGRATION-CHECKLIST.md deleted file mode 100644 index 7abdd277..00000000 --- a/.claude/skills/migrate-honcho-ts/MIGRATION-CHECKLIST.md +++ /dev/null @@ -1,147 +0,0 @@ -# Migration Checklist - -Use this checklist to track migration progress. Copy into your working notes and check off items as completed. - -## Dependencies - -- [ ] Remove `@honcho-ai/core` from dependencies -- [ ] Update `@honcho-ai/sdk` to v2.1.1 - -## Client-Level Changes - -- [ ] Replace all `.core` usages with `.http` or remove -- [ ] Rename `getConfig()` → `getConfiguration()` -- [ ] Rename `setConfig()` → `setConfiguration()` -- [ ] Rename `getPeers()` → `peers()` -- [ ] Rename `getSessions()` → `sessions()` -- [ ] Rename `getWorkspaces()` → `workspaces()` (returns `Page` now) -- [ ] Rename `getDeriverStatus()` → `queueStatus()` -- [ ] Remove `pollDeriverStatus()` calls entirely (no replacement—do not rely on queue being empty) -- [ ] Move `updateMessage()` calls from client to session - -## Peer-Level Changes - -- [ ] Replace `peer.chat(q, { stream: true })` with `peer.chatStream(q)` -- [ ] Rename `getSessions()` → `sessions()` -- [ ] Rename `getConfig()` → `getConfiguration()` -- [ ] Rename `setConfig()` → `setConfiguration()` -- [ ] Rename `peerConfig()` → `getPeerConfiguration()` -- [ ] Rename `setPeerConfig()` → `setPeerConfiguration()` -- [ ] Rename `workingRep()` → `representation()` (returns string now) -- [ ] Rename `getContext()` → `context()` -- [ ] Replace `observations` → `conclusions` -- [ ] Replace `observationsOf()` → `conclusionsOf()` -- [ ] Handle `card()` returning `string[] | null` instead of `string` - -## Session-Level Changes - -- [ ] Rename `getPeers()` → `peers()` (returns `Peer[]` now, not `Page`) -- [ ] Rename `getMessages()` → `messages()` -- [ ] Rename `getConfig()` → `getConfiguration()` -- [ ] Rename `setConfig()` → `setConfiguration()` -- [ ] Rename `getContext()` → `context()` -- [ ] Rename `getSummaries()` → `summaries()` -- [ ] Rename `getDeriverStatus()` → `queueStatus()` -- [ ] Remove `pollDeriverStatus()` calls entirely (no replacement—do not rely on queue being empty) -- [ ] Rename `workingRep()` → `representation()` (returns string now) - -## Terminology Changes - -- [ ] Rename `maxObservations` → `maxConclusions` -- [ ] Rename `includeMostDerived` → `includeMostFrequent` -- [ ] Rename `lastUserMessage` → `searchQuery` -- [ ] Rename `Observation` type → `Conclusion` -- [ ] Rename `ObservationScope` type → `ConclusionScope` - -## snake_case → camelCase - -- [ ] Update all `{ config: ... }` to `{ configuration: ... }` -- [ ] Update `observe_me` → `observeMe` -- [ ] Update `observe_others` → `observeOthers` -- [ ] Update `created_at` → `createdAt` -- [ ] Update message property access: - - [ ] `peer_id` → `peerId` - - [ ] `session_id` → `sessionId` - - [ ] `workspace_id` → `workspaceId` - - [ ] `created_at` → `createdAt` - - [ ] `token_count` → `tokenCount` -- [ ] Update summary property access: - - [ ] `message_id` → `messageId` - - [ ] `summary_type` → `summaryType` - -## Configuration Objects - -- [ ] Update workspace configuration to typed structure -- [ ] Update session configuration to typed structure -- [ ] Update peer configuration to typed structure -- [ ] Replace `deriver` config with `reasoning` config - -## Error Handling - -- [ ] Update error handling to use new error types if needed - -## Type Imports - -- [ ] Remove imports of deleted types: - - `Observation`, `ObservationScope`, `ObservationData` - - `Representation`, `RepresentationData` - - `ExplicitObservation`, `DeductiveObservation` - - `DialecticStreamDelta` - - `DeriverStatusOptions` - - `MessageCreate`, `ValidatedMessageCreate` - - `WorkingRepParams` -- [ ] Add imports of new types as needed: - - `Conclusion`, `ConclusionScope` - - `MessageInput` - - `QueueStatusOptions` - - Error types - -## Representation Handling - -- [ ] Remove usage of `Representation` class methods (`.explicit`, `.deductive`, `.isEmpty()`, `.diff()`) -- [ ] Handle representation as plain string - -## Card Method Updates (v2.0.1) - -- [ ] Replace `peer.card()` with `peer.getCard()` (card() is deprecated) -- [ ] Use `peer.setCard(string[])` if setting peer cards - -## Strict Validation (v2.0.2) - -- [ ] Verify no constructor options or input schemas pass unknown/misspelled fields (now throws `ZodError`) -- [ ] Check for `baseUrl` vs `baseURL` typo in Honcho constructor - -## peer() / session() API Call Change (v2.1.0) - -- [ ] Update code that relied on lazy `peer()` / `session()` — they now always make API calls -- [ ] Ensure all `peer()` and `session()` calls are `await`ed - -## New Properties (v2.1.0) - -- [ ] Use `peer.createdAt` / `session.createdAt` where creation time is needed -- [ ] Use `session.isActive` where session active status is needed - -## New Methods (v2.1.0) - -- [ ] Use `session.getMessage(messageId)` to fetch single messages by ID - -## Pagination Parameters (v2.1.0) - -- [ ] Add `page`, `size`, `reverse` parameters to list calls where needed: - - [ ] `honcho.peers()` - - [ ] `honcho.sessions()` - - [ ] `honcho.workspaces()` - - [ ] `peer.sessions()` - - [ ] `session.messages()` - - [ ] `scope.list()` - -## searchQuery Location Change (v2.1.0) - -- [ ] Move `searchQuery` from top-level `context()` options to `representationOptions.searchQuery` - -## Final Verification - -- [ ] Run TypeScript compiler with no errors -- [ ] Run tests -- [ ] Verify streaming functionality works -- [ ] Verify configuration changes take effect diff --git a/.claude/skills/migrate-honcho-ts/SKILL.md b/.claude/skills/migrate-honcho-ts/SKILL.md deleted file mode 100644 index 5de5e973..00000000 --- a/.claude/skills/migrate-honcho-ts/SKILL.md +++ /dev/null @@ -1,330 +0,0 @@ ---- -name: migrate-honcho-ts -description: Migrates Honcho TypeScript SDK code from v1.6.0 to v2.1.1. Use when upgrading @honcho-ai/sdk, fixing breaking changes after upgrade, or when errors mention removed APIs like .core, getConfig, observations, or snake_case properties. ---- - -# Honcho TypeScript SDK Migration (v1.6.0 → v2.1.1) - -## Overview - -This skill migrates code from `@honcho-ai/sdk` v1.6.0 to v2.1.1 (required for Honcho 3.0.0+). - -**Key breaking changes:** - -- `@honcho-ai/core` dependency removed -- "Observation" → "Conclusion" terminology -- "Deriver" → "Queue" terminology -- `getConfig`/`setConfig` → `getConfiguration`/`setConfiguration` -- `snake_case` → `camelCase` throughout -- Streaming via `chatStream()` instead of `chat({ stream: true })` -- `Representation` class removed (returns string now) - -## Quick Migration - -### 1. Update dependencies - -Remove `@honcho-ai/core` from package.json. The SDK now has its own HTTP client. - -### 2. Replace `.core` with `.http` - -```typescript -// Before -const workspace = await client.core.workspaces.getOrCreate({ id: 'my-workspace' }) - -// After -const response = await client.http.post('/v3/workspaces', { body: { id: 'my-workspace' } }) -``` - -### 3. Rename configuration methods - -```typescript -// Before -await honcho.getConfig() -await honcho.setConfig({ key: 'value' }) -await peer.getConfig() -await session.getConfig() - -// After -await honcho.getConfiguration() -await honcho.setConfiguration({ reasoning: { enabled: true } }) -await peer.getConfiguration() -await session.getConfiguration() -``` - -### 4. Rename listing methods - -```typescript -// Before -const peers = await honcho.getPeers() -const sessions = await honcho.getSessions() -const workspaces = await honcho.getWorkspaces() // string[] - -// After -const peers = await honcho.peers() -const sessions = await honcho.sessions() -const workspaces = await honcho.workspaces() // Page -``` - -### 5. Update streaming - -```typescript -// Before -const stream = await peer.chat('Hello', { stream: true }) - -// After -const stream = await peer.chatStream('Hello') -``` - -### 6. Update observations → conclusions - -```typescript -// Before -peer.observations -peer.observationsOf('bob') -maxObservations: 50 -includeMostDerived: true - -// After -peer.conclusions -peer.conclusionsOf('bob') -maxConclusions: 50 -includeMostFrequent: true -``` - -### 7. Update queue status methods - -```typescript -// Before -await honcho.getDeriverStatus({ observer: peer }) -await honcho.pollDeriverStatus({ timeoutMs: 60000 }) // REMOVE - see note below - -// After -await honcho.queueStatus({ observer: peer }) -// pollDeriverStatus() has no replacement - see note below -``` - -**Important:** `pollDeriverStatus()` and its polling pattern have been removed entirely. Do not rely on the queue ever being empty. The queue is a continuous processing system—new messages may arrive at any time, and waiting for "completion" is not a valid pattern. If your code previously polled for queue completion, redesign it to work without that assumption. - -### 8. Convert snake_case to camelCase - -```typescript -// Before -message.peer_id -message.session_id -message.created_at -message.token_count -{ observe_me: true, observe_others: false } -{ created_at: '2024-01-01' } - -// After -message.peerId -message.sessionId -message.createdAt -message.tokenCount -{ observeMe: true, observeOthers: false } -{ createdAt: '2024-01-01' } -``` - -### 9. Update representation calls - -```typescript -// Before -const rep = await peer.workingRep(session, target, options) -console.log(rep.explicit) // ExplicitObservation[] -console.log(rep.deductive) // DeductiveObservation[] - -// After -const rep = await peer.representation({ session, target, ...options }) -console.log(rep) // string -``` - -### 10. Move updateMessage to session - -```typescript -// Before -await honcho.updateMessage(message, metadata, session) - -// After -await session.updateMessage(message, metadata) -``` - -### 11. Update card() to getCard() (v2.0.1+) - -```typescript -// Before -const card = await peer.card(target) - -// After (v2.0.1+) -const card = await peer.getCard(target) // Returns string[] | null - -// peer.card() still works but is deprecated — use getCard() - -// New: setPeerCard / setCard -await peer.setCard(['Prefers dark mode', 'Located in US']) -``` - -### 12. Strict input validation (v2.0.2+) - -Client constructor and all input schemas now reject unknown options via `.strict()` Zod validation. - -```typescript -// Before (v2.0.1 and earlier) — silently ignored -const honcho = new Honcho({ baseUrl: 'http://...' }) // typo: baseUrl vs baseURL — silently fell back to default - -// After (v2.0.2+) — throws ZodError -const honcho = new Honcho({ baseUrl: 'http://...' }) // ZodError! Use baseURL -``` - -### 13. peer() and session() always make API calls (v2.1.0+) - -**Breaking**: `peer()` and `session()` now always make a get-or-create API call. Previously, calling without metadata/configuration returned a lazy object with no API call. - -```typescript -// Before (v2.0.x) — no API call without options -const session = honcho.session('my-session') // Lazy, no network request - -// After (v2.1.0+) — always hits the API -const session = await honcho.session('my-session') // Makes POST to /sessions (get-or-create) -``` - -### 14. New properties and methods (v2.1.0+) - -```typescript -// createdAt on Peer and Session -const peer = await honcho.peer('user-123') -console.log(peer.createdAt) // string | undefined - -const session = await honcho.session('sess-1') -console.log(session.createdAt) // string | undefined - -// isActive on Session -console.log(session.isActive) // boolean | undefined - -// getMessage() on Session -const msg = await session.getMessage('msg-id') -``` - -### 15. Pagination parameters on list methods (v2.1.0+) - -All list methods now accept `page`, `size`, and `reverse` parameters: - -```typescript -// Before (v2.0.x) — only filters -const peers = await honcho.peers({ metadata: { role: 'admin' } }) - -// After (v2.1.0+) — pagination controls via options object -const peers = await honcho.peers({ - filters: { metadata: { role: 'admin' } }, - page: 2, - size: 25, - reverse: true -}) - -// Legacy raw-filter form still works: -const peers = await honcho.peers({ metadata: { role: 'admin' } }) - -// Works on: honcho.peers(), honcho.sessions(), honcho.workspaces(), -// peer.sessions(), session.messages(), scope.list() -``` - -### 16. searchQuery moved in context() (v2.1.0+) - -**Breaking**: `searchQuery` removed from top-level `context()` options. Use `representationOptions.searchQuery` instead. - -```typescript -// Before (v2.0.x) -await session.context({ searchQuery: '...' }) - -// After (v2.1.0+) -await session.context({ representationOptions: { searchQuery: '...' } }) -``` - -### 17. Broader fetch retry logic (v2.1.1+) - -The SDK now retries on all `TypeError` network failures (connection resets, DNS errors, etc.) instead of only those with `'fetch'` in the message. No code changes needed — this is transparent. - -## Quick Reference Table - -| v1.6.0 | v2.0.0 | -|--------|--------| -| `client.core` | `client.http` | -| `getConfig()` | `getConfiguration()` | -| `setConfig()` | `setConfiguration()` | -| `getPeers()` | `peers()` | -| `getSessions()` | `sessions()` | -| `getWorkspaces()` | `workspaces()` | -| `getDeriverStatus()` | `queueStatus()` | -| `pollDeriverStatus()` | *Removed - do not poll* | -| `peer.chat(q, { stream: true })` | `peer.chatStream(q)` | -| `peer.workingRep()` | `peer.representation()` | -| `peer.getContext()` | `peer.context()` | -| `peer.observations` | `peer.conclusions` | -| `peer.observationsOf()` | `peer.conclusionsOf()` | -| `session.getPeers()` | `session.peers()` | -| `session.getMessages()` | `session.messages()` | -| `session.getSummaries()` | `session.summaries()` | -| `session.getContext()` | `session.context()` | -| `session.workingRep()` | `session.representation()` | -| `session.peerConfig()` | `session.getPeerConfiguration()` | -| `session.setPeerConfig()` | `session.setPeerConfiguration()` | -| `{ timeoutMs: 60000 }` | `{ timeout: 60000 }` | -| `{ maxObservations: 50 }` | `{ maxConclusions: 50 }` | -| `{ includeMostDerived }` | `{ includeMostFrequent }` | -| `{ lastUserMessage }` | `{ searchQuery }` | -| `{ config: ... }` | `{ configuration: ... }` | -| `message.peer_id` | `message.peerId` | -| `message.created_at` | `message.createdAt` | -| `peer.card()` | `peer.getCard()` *(card() deprecated)* | -| *(new)* | `peer.setCard(string[])` | -| `Observation` | `Conclusion` | -| `ObservationScope` | `ConclusionScope` | -| *(new v2.1.0)* | `peer.createdAt` / `session.createdAt` | -| *(new v2.1.0)* | `session.isActive` | -| *(new v2.1.0)* | `session.getMessage(id)` | -| *(new v2.1.0)* | `page`, `size`, `reverse` on list methods | -| `context({ searchQuery })` | `context({ representationOptions: { searchQuery } })` | - -## Detailed Reference - -For comprehensive details on each change, see: - -- [DETAILED-CHANGES.md](DETAILED-CHANGES.md) - Full API change documentation -- [MIGRATION-CHECKLIST.md](MIGRATION-CHECKLIST.md) - Step-by-step checklist - -## New Error Types - -```typescript -import { - HonchoError, - AuthenticationError, - BadRequestError, - NotFoundError, - PermissionDeniedError, - RateLimitError, - ConflictError, - UnprocessableEntityError, - ServerError, - ConnectionError, - TimeoutError -} from '@honcho-ai/sdk' -``` - -## New Configuration Types - -Configurations are now strongly typed: - -```typescript -await honcho.setConfiguration({ - reasoning: { - enabled: true, - customInstructions: 'Be concise' - }, - peerCard: { use: true, create: true }, - summary: { - enabled: true, - messagesPerShortSummary: 20, - messagesPerLongSummary: 60 - }, - dream: { enabled: true } -}) -``` diff --git a/.env.template b/.env.template index 92bdf93d..be2d6fed 100644 --- a/.env.template +++ b/.env.template @@ -21,6 +21,7 @@ PERFORMANCE_LOG_FORMAT=compact # compact|rich # EMBEDDING_MAX_TOKENS_PER_REQUEST=300000 # EMBEDDING_MODEL_CONFIG__TRANSPORT=openai # EMBEDDING_MODEL_CONFIG__MODEL=text-embedding-3-small +# EMBEDDING_MODEL_CONFIG__MAX_BATCH_SIZE=10 # EMBEDDING_MODEL_CONFIG__OVERRIDES__BASE_URL= # EMBEDDING_MODEL_CONFIG__OVERRIDES__API_KEY_ENV= diff --git a/.github/workflows/fly-deploy-prod.yml b/.github/workflows/fly-deploy-prod.yml deleted file mode 100644 index b13da23e..00000000 --- a/.github/workflows/fly-deploy-prod.yml +++ /dev/null @@ -1,61 +0,0 @@ -# See https://fly.io/docs/app-guides/continuous-deployment-with-github-actions/ - -name: Fly Deploy (Production Environment) -permissions: - contents: read -on: - push: - tags: - - v* - workflow_dispatch: - inputs: - version: - description: "Version to deploy (without v prefix)" - required: true - type: string - default: 'manual' - -jobs: - deploy-honcho-prod-image: - name: Deploy Honcho Image (Production Environment) - runs-on: ubuntu-latest - concurrency: - group: deploy-prod-group - cancel-in-progress: true - steps: - - uses: actions/checkout@v4 - - uses: superfly/flyctl-actions/setup-flyctl@1.5 - - run: | - # Determine the image label based on trigger type - if [[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then - IMAGE_LABEL="deployment-${{ github.event.inputs.version }}" - else - IMAGE_LABEL="deployment-${{ github.ref_name }}" - fi - flyctl deploy -a honcho-prod-image --remote-only --build-only --push --no-cache --image-label "$IMAGE_LABEL" - env: - FLY_API_TOKEN: ${{ secrets.FLY_PROD_API_TOKEN }} - - prompt-service: - name: Push to Service (Production Environment) - needs: deploy-honcho-prod-image - runs-on: ubuntu-latest - steps: - - name: Send POST request - env: - GITHUB_REF_NAME: ${{ github.ref_name }} - run: | - # Determine version and image label based on trigger type - if [[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then - TAG="${{ github.event.inputs.version }}" - IMAGE_LABEL="honcho-prod-image:deployment-${{ github.event.inputs.version }}" - else - TAG=${GITHUB_REF_NAME#v} - IMAGE_LABEL="honcho-prod-image:deployment-${GITHUB_REF_NAME}" - fi - - curl --fail -X POST \ - -H "Content-Type: application/json" \ - -H "Authorization: Bearer ${{ secrets.PROD_ENV_WEBHOOK_SECRET }}" \ - -d "{\"version\":\"$TAG\",\"image_label\":\"$IMAGE_LABEL\"}" \ - "${{ secrets.PROD_ENV_URL }}/webhooks/v1/add_honcho_version" diff --git a/.github/workflows/fly-deploy.yml b/.github/workflows/fly-deploy.yml deleted file mode 100644 index 2a3cbc8c..00000000 --- a/.github/workflows/fly-deploy.yml +++ /dev/null @@ -1,61 +0,0 @@ -# See https://fly.io/docs/app-guides/continuous-deployment-with-github-actions/ - -name: Fly Deploy (Test Environment) -permissions: - contents: read -on: - push: - tags: - - v* - workflow_dispatch: - inputs: - version: - description: "Version to deploy (without v prefix)" - required: true - type: string - default: 'manual' - -jobs: - deploy-honcho-image: - name: Deploy Honcho Image (Test Environment) - runs-on: ubuntu-latest - concurrency: - group: deploy-test-group - cancel-in-progress: true - steps: - - uses: actions/checkout@v4 - - uses: superfly/flyctl-actions/setup-flyctl@1.5 - - run: | - # Determine the image label based on trigger type - if [[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then - IMAGE_LABEL="deployment-${{ github.event.inputs.version }}" - else - IMAGE_LABEL="deployment-${{ github.ref_name }}" - fi - flyctl deploy -a honcho-image --remote-only --build-only --push --no-cache --image-label "$IMAGE_LABEL" - env: - FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }} - - prompt-service: - name: Push to Service (Test Environment) - runs-on: ubuntu-latest - needs: deploy-honcho-image - steps: - - name: Send POST request - env: - GITHUB_REF_NAME: ${{ github.ref_name }} - run: | - # Determine version and image label based on trigger type - if [[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then - TAG="${{ github.event.inputs.version }}" - IMAGE_LABEL="honcho-image:deployment-${{ github.event.inputs.version }}" - else - TAG=${GITHUB_REF_NAME#v} - IMAGE_LABEL="honcho-image:deployment-${GITHUB_REF_NAME}" - fi - - curl --fail -X POST \ - -H "Content-Type: application/json" \ - -H "Authorization: Bearer ${{ secrets.TEST_ENV_WEBHOOK_SECRET }}" \ - -d "{\"version\":\"$TAG\",\"image_label\":\"$IMAGE_LABEL\"}" \ - "${{ secrets.TEST_ENV_URL }}/webhooks/v1/add_honcho_version" diff --git a/.github/workflows/push-gcp-registry-prod.yml b/.github/workflows/push-gcp-registry-prod.yml index d5d9fc42..81195928 100644 --- a/.github/workflows/push-gcp-registry-prod.yml +++ b/.github/workflows/push-gcp-registry-prod.yml @@ -7,13 +7,6 @@ on: push: tags: - v* - workflow_dispatch: - inputs: - version: - description: "Version to deploy (without v prefix)" - required: true - type: string - default: "manual" env: GCP_PROJECT_ID: ${{ secrets.PROD_GCP_PROJECT_ID }} @@ -25,9 +18,26 @@ env: jobs: build-and-push: runs-on: ubuntu-latest + outputs: + version: ${{ steps.version.outputs.version }} steps: - name: Checkout uses: actions/checkout@v4 + with: + persist-credentials: false + + - name: Resolve and verify version + id: version + run: | + VERSION="${GITHUB_REF_NAME#v}" + # A running instance serves this version at /openapi.json, so it must + # match the version being deployed. + PYPROJECT_VERSION="$(grep -m1 '^version = ' pyproject.toml | cut -d'"' -f2)" + if [[ "$VERSION" != "$PYPROJECT_VERSION" ]]; then + echo "::error::pyproject.toml version '$PYPROJECT_VERSION' does not match tag '$GITHUB_REF_NAME'. Bump pyproject.toml before tagging." + exit 1 + fi + echo "version=$VERSION" >> "$GITHUB_OUTPUT" - name: Authenticate to GCP uses: google-github-actions/auth@v2 @@ -42,15 +52,27 @@ jobs: - name: Build and push image env: - VERSION: ${{ github.event.inputs.version }} + VERSION: ${{ steps.version.outputs.version }} run: | - # Determine the image label based on trigger type - if [[ "$GITHUB_EVENT_NAME" == "workflow_dispatch" ]]; then - IMAGE_LABEL="deployment-${VERSION}" - else - IMAGE_LABEL="deployment-${GITHUB_REF_NAME}" - fi BASE="${{ env.GCP_AR_LOCATION }}-docker.pkg.dev/${{ env.GCP_PROJECT_ID }}/${{ env.GCP_AR_REPO }}/${{ env.IMAGE_NAME }}" - TAG="$BASE:$IMAGE_LABEL" + TAG="$BASE:deployment-v${VERSION}" docker build -t "$TAG" . docker push "$TAG" + + prompt-service: + name: Push to Service (Production Environment) + runs-on: ubuntu-latest + needs: build-and-push + steps: + - name: Send POST request + env: + VERSION: ${{ needs.build-and-push.outputs.version }} + run: | + # Name and tag only; the registry path is supplied downstream. + IMAGE_LABEL="${{ env.IMAGE_NAME }}:deployment-v${VERSION}" + + curl --fail --connect-timeout 10 --max-time 60 -X POST \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer ${{ secrets.PROD_ENV_WEBHOOK_SECRET }}" \ + -d "{\"version\":\"$VERSION\",\"image_label\":\"$IMAGE_LABEL\"}" \ + "${{ secrets.PROD_ENV_URL }}/webhooks/v1/add_honcho_version" diff --git a/.github/workflows/push-gcp-registry-staging.yml b/.github/workflows/push-gcp-registry-staging.yml index f41a38de..636f9a4c 100644 --- a/.github/workflows/push-gcp-registry-staging.yml +++ b/.github/workflows/push-gcp-registry-staging.yml @@ -7,13 +7,6 @@ on: push: tags: - v* - workflow_dispatch: - inputs: - version: - description: "Version to deploy (without v prefix)" - required: true - type: string - default: "manual" env: GCP_PROJECT_ID: ${{ secrets.STAGING_GCP_PROJECT_ID }} @@ -25,9 +18,26 @@ env: jobs: build-and-push: runs-on: ubuntu-latest + outputs: + version: ${{ steps.version.outputs.version }} steps: - name: Checkout uses: actions/checkout@v4 + with: + persist-credentials: false + + - name: Resolve and verify version + id: version + run: | + VERSION="${GITHUB_REF_NAME#v}" + # A running instance serves this version at /openapi.json, so it must + # match the version being deployed. + PYPROJECT_VERSION="$(grep -m1 '^version = ' pyproject.toml | cut -d'"' -f2)" + if [[ "$VERSION" != "$PYPROJECT_VERSION" ]]; then + echo "::error::pyproject.toml version '$PYPROJECT_VERSION' does not match tag '$GITHUB_REF_NAME'. Bump pyproject.toml before tagging." + exit 1 + fi + echo "version=$VERSION" >> "$GITHUB_OUTPUT" - name: Authenticate to GCP uses: google-github-actions/auth@v2 @@ -42,15 +52,27 @@ jobs: - name: Build and push image env: - VERSION: ${{ github.event.inputs.version }} + VERSION: ${{ steps.version.outputs.version }} run: | - # Determine the image label based on trigger type - if [[ "$GITHUB_EVENT_NAME" == "workflow_dispatch" ]]; then - IMAGE_LABEL="deployment-${VERSION}" - else - IMAGE_LABEL="deployment-${GITHUB_REF_NAME}" - fi BASE="${{ env.GCP_AR_LOCATION }}-docker.pkg.dev/${{ env.GCP_PROJECT_ID }}/${{ env.GCP_AR_REPO }}/${{ env.IMAGE_NAME }}" - TAG="$BASE:$IMAGE_LABEL" + TAG="$BASE:deployment-v${VERSION}" docker build -t "$TAG" . docker push "$TAG" + + prompt-service: + name: Push to Service (Staging Environment) + runs-on: ubuntu-latest + needs: build-and-push + steps: + - name: Send POST request + env: + VERSION: ${{ needs.build-and-push.outputs.version }} + run: | + # Name and tag only; the registry path is supplied downstream. + IMAGE_LABEL="${{ env.IMAGE_NAME }}:deployment-v${VERSION}" + + curl --fail --connect-timeout 10 --max-time 60 -X POST \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer ${{ secrets.TEST_ENV_WEBHOOK_SECRET }}" \ + -d "{\"version\":\"$VERSION\",\"image_label\":\"$IMAGE_LABEL\"}" \ + "${{ secrets.TEST_ENV_URL }}/webhooks/v1/add_honcho_version" diff --git a/.github/workflows/staticanalysis.yml b/.github/workflows/staticanalysis.yml index 6578b506..18bb37fd 100644 --- a/.github/workflows/staticanalysis.yml +++ b/.github/workflows/staticanalysis.yml @@ -1,5 +1,9 @@ name: Static Analysis -on: [push] +on: + push: + branches: [main] + pull_request: + branches: [main] permissions: contents: read diff --git a/CHANGELOG.md b/CHANGELOG.md index 7a12356e..ed77a80b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,53 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](http://keepachangelog.com/) and this project adheres to [Semantic Versioning](http://semver.org/). +## [3.0.12] - 2026-08-10 + +### Added + +- Session allowlist on the Dialectic and representation via a constrained `filters` body on `POST /peers/{peer_id}/chat` and `/representation`, supporting only the `session_id` key (a session id, a bare list, or `{"in": [...]}`). Unsupported keys and shapes are rejected with 422 rather than silently ignored, it composes with `session_id` (which must be included in the allowlist when both are given), and it is capped at 1,000 sessions per request. Enforcement is uniform and fail-closed at every recall chokepoint: scoped conclusion recall is restricted to `level == "explicit"` (dream-derived conclusions carry a single `session_name` but are synthesized across all sessions, so that stamp can't be scoped on), `get_reasoning_chain` is unavailable under an allowlist, and an empty allowlist short-circuits to empty results everywhere. Workspace keys pass the allowlist as-given; peer-scoped JWTs must be an active member of every allowlisted session (401 otherwise) (#882) +- Bare-list membership sugar in the filter DSL: `{"session_id": ["s1", "s2"]}` is now shorthand for `{"session_id": {"in": [...]}}` on regular columns generically. JSONB metadata columns are excluded and keep containment semantics. Strictly additive, since a bare list on a regular column previously compiled to a type-mismatched equality that matched nothing (#881) +- Optional structured outputs on the Dialectic: `response_format` (a JSON Schema with root type `object`) on peer chat makes `content` a JSON string conforming to that schema. Only a conservative subset of JSON Schema is supported, with DoS guards and non-recursive `$ref` support (#896) +- Combined tool calling and structured output in the LLM transport layer, with per-backend request shaping: OpenAI routes tool-carrying structured requests through `create()` with an explicit `json_schema` response format (`parse()` 500s on non-strict function tools), Anthropic skips the `{` JSON prefill when tools are present so `tool_use` blocks stay reachable, and Gemini injects a schema instruction into the final turn instead of using native `response_schema` (rejected alongside function calling before Gemini 3). All backends skip structured-output parsing on tool-call turns, which carry no consumable content (#907) +- `card_refresh` dream type: a lightweight dream that runs only the peer-card update, for event-driven refreshes such as membership changes and cold starts. Handled by a new `CardRefreshSpecialist` restricted to `get_recent_observations`, `search_memory`, and `update_peer_card` (no observation-mutating tools) with a tool-iteration cap of `min(6, DREAM.MAX_TOOL_ITERATIONS)`. `POST /v3/workspaces/{workspace_id}/schedule_dream` accepts `dream_type=card_refresh` plus a `rebuild` flag, which omits the existing card from the prompt so the specialist rebuilds it solely from observations present in the collection. Card refreshes never advance the omni dream guard pair (`last_dream_at` / `last_dream_document_count`) (#883) +- Full-fidelity LLM trace stream, with Langfuse as one projection over it: each call is captured once (`CapturedLLMCall`) and fanned out to a CloudEvents trace stream (`llm.call.traced` / `trace.content`) and a Langfuse exporter, both reconstructing trace → run → step → generation from the same source of truth. Adds `TELEMETRY_TRACE_PAYLOADS_ENABLED` (default `false`), `TELEMETRY_TRACE_MAX_BYTES` (default 262144, per-message cap with oversized content clipped), `TELEMETRY_TRACE_PURPOSES` (JSON list of `CallPurpose` values; empty means all), and `LANGFUSE_EXPORTER_MODE` (`exporter` by default; `inline` is kept for one release for side-by-side validation). Embedding calls are traced, dreamer branches nest under one dream trace, tool calls become spans under their step, and high-volume events are sampled deterministically. `TRACE_ENDPOINT` is dropped (#845) +- Redis Cluster support via `CACHE_CLUSTER` (for example GCP Memorystore for Redis Cluster), alongside a new `CACHE_LOCK_WAIT_CHECK_INTERVAL_SECONDS` (#905) +- `EMBEDDING_MODEL_CONFIG__MAX_BATCH_SIZE` caps texts per embedding request for OpenAI-compatible providers with smaller limits than OpenAI's, such as DashScope `text-embedding-v4` (10) and Alibaba Bailian `qwen3.7-text-embedding` (20). When unset, native provider defaults are preserved (OpenAI 2048, Gemini 100) (#983) +- Per-request provider timeouts via `provider_params.timeout` on any model config, validated at config load so a bad value fails at startup with the exact config path instead of surfacing per-request as a retried 500. Good values normalize to float seconds; Gemini's is converted to milliseconds (#832) +- `RepresentationCompletedEvent` now reports deduplication counts: `exact_dup_in_batch_count`, `exact_dup_existing_count`, `semantic_dup_rejected_count`, and `semantic_dup_replaced_count` (#910) +- OAuth discovery for MCP clients: the MCP worker serves `/.well-known/oauth-protected-resource` (RFC 9728) without auth so clients can discover the authorization server, and a 401 now carries `WWW-Authenticate: Bearer resource_metadata="..."` (exposed cross-origin) to start the flow (#923) +- Prometheus metrics for the immediate-embed fast path: tasks shed because `EMBEDDING_MAX_PENDING_EMBED_TASKS` was reached, and the current in-flight task count (#892) +- Docs: a detailed system architecture diagram, a Codex integration guide (#879), a structured-outputs page (#896), a section on filtering conclusions by reasoning level (#851), a health-check endpoint reference, and SDK updates (#867) + +### Changed + +- **Breaking config change:** `DERIVER_REPRESENTATION_BATCH_MAX_TOKENS` is split into two settings that were previously conflated — `DERIVER_REPRESENTATION_BATCH_WORK_UNIT_TARGET_TOKENS` (default 512), the producer-side minimum a work unit accumulates before the deriver claims it, where `0` disables the gate; and `DERIVER_REPRESENTATION_BATCH_TARGET_INPUT_TOKENS` (default 1024), the consumer-side maximum context-window tokens per deriver LLM call. Deployments setting the old name must migrate (#889) +- The immediate-embed fast path now applies backpressure: `EMBEDDING_MAX_PENDING_EMBED_TASKS` (default 50) caps in-flight embed tasks, and once saturated, message creation skips the fast path entirely and the reconciler embeds on its next cycle. `0` disables the fast path (#892) +- Explicit-level documents are now kept session-pure, so memory can be built by copying explicit documents between collections. Enforcement refuses rather than rewrites: `create_documents` rejects explicit documents with a null `session_name`, exact dedup keys on (content, level, session-for-explicit), semantic dedup scopes candidate search to the same level and — for explicit documents — the same session, and the generic `create_observations` tool rejects `level='explicit'` outside message-ingestion (deriver) context. Derived levels keep cross-session consolidation (#883) +- Sentry's `before_send` filter is centralized as `default_before_send` in `src/telemetry/sentry.py` instead of living only in the API's `main.py`, so the deriver gets the same non-actionable-exception filtering. All Sentry events also carry a `namespace` tag for correlation (#934, #870) +- The minimal deriver's extraction examples no longer teach inferences its own output schema forbids. The `EXAMPLES` block demonstrated deriving a specific birthday from "I just had my 25th birthday last Saturday", deriving residence from a single visit ("I took my dog for a walk in NYC" → "alice lives in NYC"), and a "+ general knowledge" deductive output the deriver has no channel for. The replacements stay inside the schema's contract and teach the boundary: the dog/NYC message is kept and shown extracting correctly, and a separate example shows "lives in NYC" is valid when actually stated (#985) +- Dreamer specialists are instructed not to output summaries (#894) +- `session_name` is deprecated for scoping in favor of the session allowlist. It is not removed and not aliased: it also pins the query to one session, bypasses observer scoping, and drives session-history injection into the dialectic prompt, so it has no drop-in replacement (#882) +- The MCP worker no longer requires the `X-Honcho-User-Name` or `X-Honcho-Assistant-Name` headers (#923) + +### Fixed + +- Session scoping was applied to only one of the three working-representation query paths: `session_name` reached the recent-documents query, but the semantic and most-derived paths ignored it, so `limit_to_session` leaked cross-session conclusions into perspectives. The allowlist is now threaded uniformly through all three paths and pushed down to pgvector and external vector stores (#881) +- Empty membership lists failed open in the vector-store filter builders, silently widening scope: LanceDB dropped empty `IN` clauses and Turbopuffer emitted a bare `In []` with undocumented semantics. Both now emit an explicit always-false predicate, and `_build_filter_conditions` checks `is not None` rather than truthiness so an empty list is no longer treated like `None` (#881, #882) +- Session-scoped CRUD helpers ignored the session allowlist entirely, so a caller could read a session the allowlist forbids. The API routes guarded this with a 422, but the dialectic tools call these CRUD functions directly and bypassed it. `_semantic_search_messages` (covering `search_messages` and `search_messages_temporal`), `grep_messages`, `get_messages_by_date_range`, `get_recent_history`, and `get_observation_context` now return `[]` when `session_name` is set and outside the allowlist (#882) +- The cache client logged the full Redis URL — including the password — at INFO and WARNING on every connection attempt and failure, exposing the live credential in container logs and downstream aggregation. Credentials are now redacted across userinfo, the `?password=` (redis-py) and `?secret=` (cashews) query params, scheme-less URLs whose password is invisible to `.port`/`.password` parsing, and malformed URLs, whose fallback previously echoed the raw input verbatim (#869) +- A `top_k` of `0` reached the vector store, where Turbopuffer rejects it with a 400 (`top_k must be between 1 and 10000`). A non-positive `top_k` now returns `[]` before the embedding call, and the semantic budget floors at 1 so an explicitly requested search isn't silently allocated zero (#970) +- Gemini clients had no HTTP timeout, so a stalled socket wedged the deriver worker's uvloop event loop, which the in-process reconciler shares. A 10-minute timeout is now set on both the Gemini LLM client and the Gemini embedding client (#903) +- Dreamer conclusions were dated to ingestion time rather than their latest source observation, and their timestamps are now normalized (#890) +- Langfuse I/O annotation was gated on `LANGFUSE_PUBLIC_KEY` instead of `langfuse_inline_enabled`, so in the default `exporter` mode it called `update_current_generation()` with no active span — logging "No active span in current context" roughly 14 times per dialectic run and building throwaway `model_dump` payloads on every LLM call. Separately, `AgentToolSummaryCreatedEvent` hardcoded `run_id="deriver"` / `iteration=0`, polluting `run_id` grouping in the CloudEvents stream with a phantom run; both fields are now optional and the resource id is keyed on `message_id:summary_type` (schema_version 2 → 3) (#845) +- Assistant tool calls were dropped from the captured trace stream for OpenAI and Gemini: `build_captured_messages` read only `{role, content, tool_call_id}`, but those providers keep tool calls outside `content`, so replayed tool-call turns landed as empty content and Gemini lost its text and tool results entirely. Tool calls are now normalized per provider into a unified `tool_calls` field and folded into the content hash. Gemini's `thought_signature` is bytes, so `model_dump(mode="json")` raised `UnicodeDecodeError` inside `emit_trace`, silently dropping whole tool-calling iterations from the trace stream (billing and Langfuse were unaffected); it is now base64-encoded on the telemetry path while replay keeps the raw bytes (#845) +- `EmbeddingClient.encoding` forced full client construction, raising "OpenAI API key is required" even though tiktoken needs no credentials. The document dedup tie-break only needs `.encoding` for token counting, so any test hitting that path failed in environments without embedding keys — notably CI for pull requests from forks. The encoding is now resolved from the configured model directly, falling back to `cl100k_base`, and the underlying client's encoding is reused only when it has already been constructed (#955) +- The Docker build failed under Podman because the uv build inputs weren't copied (#878) +- LanceDB was installed on macOS Intel, where it doesn't work. A PEP 508 marker excludes `darwin/x86_64` and the LanceDB vector-store import is wrapped so a misconfiguration surfaces as a clear config error (#496) +- Prompt checks requiring the literal token "json" for `json_object` mode are now satisfied in lowercase (#887) +- Reverted an unintended `RepresentationCompletedEvent` schema-version increment +- Documented preinstalling pgvector as a privileged role for deployments where the `DB_CONNECTION_URI` role deliberately cannot create extensions (managed Postgres, Kubernetes operators, NixOS). `CREATE EXTENSION IF NOT EXISTS vector` does not help there, because Postgres checks the privilege before checking whether the extension exists. Docker Compose is unaffected, since the bundled stack connects as the `postgres` superuser (#984) + ## [3.0.11] - 2026-06-24 ### Added @@ -31,6 +78,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/). - Fixed a `create_tree` keyword-argument mismatch in the Dreamer's surprisal tree construction (#749) - Providers that omit output-token counts (observed with Gemini on tool-loop completions) returned `output_tokens=None`, which raised a Pydantic validation error that aborted the call and crashed the Dreamer's induction phase before inductive conclusions were persisted. `None` is now coerced to `0` so token accounting degrades gracefully (#809) - Document creation now performs exact (case-insensitive, whitespace-trimmed) content deduplication before the existing semantic dedup step: exact duplicates within a batch collapse to a single insert, and an exact match against a live document reinforces it (atomic `times_derived` increment) instead of creating a new row (#861) +- The OpenAI backend passed `tool_choice` through raw while the Anthropic and Gemini backends translate Honcho's canonical vocabulary to their native form, so on a mixed-provider fallback chain (for example Gemini primary → OpenAI backup) a canonical `"any"` reached OpenAI unchanged and was rejected as an invalid param. The OpenAI backend now converts it, mirroring the others: `any`/`required` → `required`, `auto`/`none` pass through, and a tool-name string or `{"name": ...}` dict becomes a function selection (#850) +- Langfuse `@observe` auto-capture serialized every argument of `honcho_llm_call_inner` into the generation span input, including `client_override` (a live `AsyncOpenAI`/`genai` client) and `selected_config` (which carries `api_key`). Auto-capture deep-copied the client into a half-constructed object whose teardown raised (`AsyncHttpxClientWrapper ... no attribute '_state'` on OpenAI, flooding stderr; `BaseApiClient ... no attribute '_http_options'` on Gemini), and it leaked `ModelConfig.api_key` into traces. Capture is now an explicit allowlist: `capture_input`/`capture_output` are disabled and curated, serializable input and output are stamped instead, with tuning knobs surfaced as `model_parameters` via a secret-bearing denylist and per-call token usage mirrored as `usage_details` (#849) ## [3.0.10] - 2026-06-15 diff --git a/Dockerfile b/Dockerfile index 18f23a85..8fd9be47 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,10 +1,11 @@ +# syntax=docker/dockerfile:1 + # https://pythonspeed.com/articles/base-image-python-docker-images/ # https://testdriven.io/blog/docker-best-practices/ -FROM python:3.13-slim-bookworm +FROM python:3.13-slim-bookworm AS builder COPY --from=ghcr.io/astral-sh/uv:0.9.24 /uv /bin/uv -# Set Working directory WORKDIR /app # Enable bytecode compilation @@ -17,26 +18,45 @@ ENV UV_LINK_MODE=copy ENV PYTHONDONTWRITEBYTECODE=1 ENV PYTHONUNBUFFERED=1 -# Install the project's dependencies using the lockfile and settings -RUN --mount=type=cache,target=/root/.cache/uv \ - --mount=type=bind,source=uv.lock,target=uv.lock \ - --mount=type=bind,source=pyproject.toml,target=pyproject.toml \ - uv sync --frozen --no-install-project --no-group dev - # Copy only requirements to cache them in docker layer COPY uv.lock pyproject.toml /app/ -# Sync the project +# Optionall include lancedb with: +# docker build --build-arg INSTALL_LANCEDB=true . +ARG INSTALL_LANCEDB=false RUN --mount=type=cache,target=/root/.cache/uv \ - uv sync --frozen --no-group dev + if [ "$INSTALL_LANCEDB" = "true" ]; then \ + uv sync --frozen --no-install-project --no-group dev --extra lancedb; \ + elif [ "$INSTALL_LANCEDB" = "false" ]; then \ + uv sync --frozen --no-install-project --no-group dev; \ + else \ + echo "INSTALL_LANCEDB must be 'true' or 'false'" >&2; \ + exit 2; \ + fi + +FROM python:3.13-slim-bookworm AS runtime + +WORKDIR /app + +ENV PYTHONDONTWRITEBYTECODE=1 +ENV PYTHONUNBUFFERED=1 + +# Create the runtime user before copying dependencies with their final owner. +# A recursive chown in a later layer would copy the whole virtualenv and nearly +# double the image size. +RUN addgroup --system app \ + && adduser --system --group app \ + && chown app:app /app \ + # Pre-create the LanceDB dir so a named volume mounted here inherits app + # ownership instead of defaulting to root. + && mkdir /app/lancedb_data \ + && chown app:app /app/lancedb_data + +COPY --from=builder --chown=app:app /app/.venv /app/.venv # Place executables in the environment at the front of the path ENV PATH="/app/.venv/bin:$PATH" ENV HOME=/app -ENV UV_CACHE_DIR=/tmp/uv-cache - -# Create non-root user and set ownership -RUN addgroup --system app && adduser --system --group app && mkdir -p /tmp/uv-cache && chown -R app:app /app /tmp/uv-cache COPY --chown=app:app src/ /app/src/ COPY --chown=app:app migrations/ /app/migrations/ diff --git a/README.md b/README.md index 3ecd6f7f..705ab846 100644 --- a/README.md +++ b/README.md @@ -226,7 +226,7 @@ For wiring the Honcho SDK into an existing application, install the integration npx skills add plastic-labs/honcho ``` -Then invoke `/honcho-integration` in Claude Code (or `/honcho-dev:integrate` via the plugin marketplace). Details: [agentic development guide](https://honcho.dev/docs/v3/documentation/introduction/vibecoding). +Then invoke `/honcho-integration` in Claude Code (or `/honcho-dev:integrate` via the plugin marketplace). The same command also installs the memory skills — `honcho-memory` (concepts: the recall/record loop, session and peer strategy, plus how to connect and drive an MCP-connected Honcho) and `honcho-cli` (inspecting and debugging a deployment). Details: [agentic development guide](https://honcho.dev/docs/v3/documentation/introduction/vibecoding). ### Other MCP clients diff --git a/config.toml.example b/config.toml.example index aa4ddb41..4a955eda 100644 --- a/config.toml.example +++ b/config.toml.example @@ -74,6 +74,9 @@ MAX_TOKENS_PER_REQUEST = 300000 [embedding.model_config] transport = "openai" model = "text-embedding-3-small" +# Optional provider request input cap. Useful for OpenAI-compatible embedding +# APIs with smaller limits, such as DashScope text-embedding-v4. +# max_batch_size = 10 # Optional module-level endpoint overrides # [embedding.model_config.overrides] @@ -138,6 +141,7 @@ model = "gpt-5.4-mini" # api_key_env = "DERIVER_CUSTOM_BACKUP_API_KEY" # [deriver.model_config.overrides.provider_params] # verbosity = "low" +# timeout = 3600.0 # Peer card settings [peer_card] diff --git a/docker-compose.yml.example b/docker-compose.yml.example index 51b279e7..ee201e6f 100644 --- a/docker-compose.yml.example +++ b/docker-compose.yml.example @@ -4,6 +4,7 @@ # cp docker-compose.yml.example docker-compose.yml # cp .env.template .env # edit with your provider config # docker compose up -d --build +# INSTALL_LANCEDB=true docker compose up -d --build # optional local vector store # # By default, ports are bound to 127.0.0.1 (localhost only). # For development, uncomment the source mounts and monitoring services below. @@ -13,6 +14,8 @@ services: build: context: . dockerfile: Dockerfile + args: + INSTALL_LANCEDB: ${INSTALL_LANCEDB:-false} entrypoint: ["sh", "docker/entrypoint.sh"] depends_on: database: @@ -33,10 +36,12 @@ services: timeout: 5s retries: 5 start_period: 10s - # -- Development: mount source for live reload -- - # volumes: - # - .:/app - # - venv:/app/.venv + volumes: + # Shared LanceDB data (used when VECTOR_STORE_TYPE=lancedb) + - lancedb-data:/app/lancedb_data + # -- Development: mount source for live reload -- + # - .:/app + # - venv:/app/.venv environment: - DB_CONNECTION_URI=postgresql+psycopg://postgres:postgres@database:5432/postgres - CACHE_URL=redis://redis:6379/0?suppress=true @@ -50,6 +55,8 @@ services: build: context: . dockerfile: Dockerfile + args: + INSTALL_LANCEDB: ${INSTALL_LANCEDB:-false} entrypoint: ["/app/.venv/bin/python", "-m", "src.deriver"] depends_on: api: @@ -58,10 +65,12 @@ services: condition: service_healthy redis: condition: service_healthy - # -- Development: mount source for live reload -- - # volumes: - # - .:/app - # - venv:/app/.venv + volumes: + # Shared LanceDB data (used when VECTOR_STORE_TYPE=lancedb) + - lancedb-data:/app/lancedb_data + # -- Development: mount source for live reload -- + # - .:/app + # - venv:/app/.venv environment: - DB_CONNECTION_URI=postgresql+psycopg://postgres:postgres@database:5432/postgres - CACHE_URL=redis://redis:6379/0?suppress=true @@ -137,6 +146,7 @@ services: volumes: pgdata: redis-data: + lancedb-data: # -- Development: uncomment if using source mounts -- # venv: # prometheus-data: diff --git a/docs/changelog/introduction.mdx b/docs/changelog/introduction.mdx index 6e741563..69e25d14 100644 --- a/docs/changelog/introduction.mdx +++ b/docs/changelog/introduction.mdx @@ -27,7 +27,54 @@ Welcome to the Honcho changelog! This section documents all notable changes to t ### Honcho API and SDK Changelogs - + + ### Added + + - Session allowlist on the Dialectic and representation via a constrained `filters` body on `POST /peers/{peer_id}/chat` and `/representation`, supporting only the `session_id` key (a session id, a bare list, or `{"in": [...]}`). Unsupported keys and shapes are rejected with 422 rather than silently ignored, it composes with `session_id` (which must be included in the allowlist when both are given), and it is capped at 1,000 sessions per request. Enforcement is uniform and fail-closed at every recall chokepoint: scoped conclusion recall is restricted to `level == "explicit"` (dream-derived conclusions carry a single `session_name` but are synthesized across all sessions, so that stamp can't be scoped on), `get_reasoning_chain` is unavailable under an allowlist, and an empty allowlist short-circuits to empty results everywhere. Workspace keys pass the allowlist as-given; peer-scoped JWTs must be an active member of every allowlisted session (401 otherwise) (#882) + - Bare-list membership sugar in the filter DSL: `{"session_id": ["s1", "s2"]}` is now shorthand for `{"session_id": {"in": [...]}}` on regular columns generically. JSONB metadata columns are excluded and keep containment semantics. Strictly additive, since a bare list on a regular column previously compiled to a type-mismatched equality that matched nothing (#881) + - Optional structured outputs on the Dialectic: `response_format` (a JSON Schema with root type `object`) on peer chat makes `content` a JSON string conforming to that schema. Only a conservative subset of JSON Schema is supported, with DoS guards and non-recursive `$ref` support (#896) + - Combined tool calling and structured output in the LLM transport layer, with per-backend request shaping: OpenAI routes tool-carrying structured requests through `create()` with an explicit `json_schema` response format (`parse()` 500s on non-strict function tools), Anthropic skips the `{` JSON prefill when tools are present so `tool_use` blocks stay reachable, and Gemini injects a schema instruction into the final turn instead of using native `response_schema` (rejected alongside function calling before Gemini 3). All backends skip structured-output parsing on tool-call turns, which carry no consumable content (#907) + - `card_refresh` dream type: a lightweight dream that runs only the peer-card update, for event-driven refreshes such as membership changes and cold starts. Handled by a new `CardRefreshSpecialist` restricted to `get_recent_observations`, `search_memory`, and `update_peer_card` (no observation-mutating tools) with a tool-iteration cap of `min(6, DREAM.MAX_TOOL_ITERATIONS)`. `POST /v3/workspaces/{workspace_id}/schedule_dream` accepts `dream_type=card_refresh` plus a `rebuild` flag, which omits the existing card from the prompt so the specialist rebuilds it solely from observations present in the collection. Card refreshes never advance the omni dream guard pair (`last_dream_at` / `last_dream_document_count`) (#883) + - Full-fidelity LLM trace stream, with Langfuse as one projection over it: each call is captured once (`CapturedLLMCall`) and fanned out to a CloudEvents trace stream (`llm.call.traced` / `trace.content`) and a Langfuse exporter, both reconstructing trace → run → step → generation from the same source of truth. Adds `TELEMETRY_TRACE_PAYLOADS_ENABLED` (default `false`), `TELEMETRY_TRACE_MAX_BYTES` (default 262144, per-message cap with oversized content clipped), `TELEMETRY_TRACE_PURPOSES` (JSON list of `CallPurpose` values; empty means all), and `LANGFUSE_EXPORTER_MODE` (`exporter` by default; `inline` is kept for one release for side-by-side validation). Embedding calls are traced, dreamer branches nest under one dream trace, tool calls become spans under their step, and high-volume events are sampled deterministically. `TRACE_ENDPOINT` is dropped (#845) + - Redis Cluster support via `CACHE_CLUSTER` (for example GCP Memorystore for Redis Cluster), alongside a new `CACHE_LOCK_WAIT_CHECK_INTERVAL_SECONDS` (#905) + - `EMBEDDING_MODEL_CONFIG__MAX_BATCH_SIZE` caps texts per embedding request for OpenAI-compatible providers with smaller limits than OpenAI's, such as DashScope `text-embedding-v4` (10) and Alibaba Bailian `qwen3.7-text-embedding` (20). When unset, native provider defaults are preserved (OpenAI 2048, Gemini 100) (#983) + - Per-request provider timeouts via `provider_params.timeout` on any model config, validated at config load so a bad value fails at startup with the exact config path instead of surfacing per-request as a retried 500. Good values normalize to float seconds; Gemini's is converted to milliseconds (#832) + - `RepresentationCompletedEvent` now reports deduplication counts: `exact_dup_in_batch_count`, `exact_dup_existing_count`, `semantic_dup_rejected_count`, and `semantic_dup_replaced_count` (#910) + - OAuth discovery for MCP clients: the MCP worker serves `/.well-known/oauth-protected-resource` (RFC 9728) without auth so clients can discover the authorization server, and a 401 now carries `WWW-Authenticate: Bearer resource_metadata="..."` (exposed cross-origin) to start the flow (#923) + - Prometheus metrics for the immediate-embed fast path: tasks shed because `EMBEDDING_MAX_PENDING_EMBED_TASKS` was reached, and the current in-flight task count (#892) + - Docs: a detailed system architecture diagram, a Codex integration guide (#879), a structured-outputs page (#896), a section on filtering conclusions by reasoning level (#851), a health-check endpoint reference, and SDK updates (#867) + + ### Changed + + - **Breaking config change:** `DERIVER_REPRESENTATION_BATCH_MAX_TOKENS` is split into two settings that were previously conflated — `DERIVER_REPRESENTATION_BATCH_WORK_UNIT_TARGET_TOKENS` (default 512), the producer-side minimum a work unit accumulates before the deriver claims it, where `0` disables the gate; and `DERIVER_REPRESENTATION_BATCH_TARGET_INPUT_TOKENS` (default 1024), the consumer-side maximum context-window tokens per deriver LLM call. Deployments setting the old name must migrate (#889) + - The immediate-embed fast path now applies backpressure: `EMBEDDING_MAX_PENDING_EMBED_TASKS` (default 50) caps in-flight embed tasks, and once saturated, message creation skips the fast path entirely and the reconciler embeds on its next cycle. `0` disables the fast path (#892) + - Explicit-level documents are now kept session-pure, so memory can be built by copying explicit documents between collections. Enforcement refuses rather than rewrites: `create_documents` rejects explicit documents with a null `session_name`, exact dedup keys on (content, level, session-for-explicit), semantic dedup scopes candidate search to the same level and — for explicit documents — the same session, and the generic `create_observations` tool rejects `level='explicit'` outside message-ingestion (deriver) context. Derived levels keep cross-session consolidation (#883) + - Sentry's `before_send` filter is centralized as `default_before_send` in `src/telemetry/sentry.py` instead of living only in the API's `main.py`, so the deriver gets the same non-actionable-exception filtering. All Sentry events also carry a `namespace` tag for correlation (#934, #870) + - The minimal deriver's extraction examples no longer teach inferences its own output schema forbids. The `EXAMPLES` block demonstrated deriving a specific birthday from "I just had my 25th birthday last Saturday", deriving residence from a single visit ("I took my dog for a walk in NYC" → "alice lives in NYC"), and a "+ general knowledge" deductive output the deriver has no channel for. The replacements stay inside the schema's contract and teach the boundary: the dog/NYC message is kept and shown extracting correctly, and a separate example shows "lives in NYC" is valid when actually stated (#985) + - Dreamer specialists are instructed not to output summaries (#894) + - `session_name` is deprecated for scoping in favor of the session allowlist. It is not removed and not aliased: it also pins the query to one session, bypasses observer scoping, and drives session-history injection into the dialectic prompt, so it has no drop-in replacement (#882) + - The MCP worker no longer requires the `X-Honcho-User-Name` or `X-Honcho-Assistant-Name` headers (#923) + + ### Fixed + + - Session scoping was applied to only one of the three working-representation query paths: `session_name` reached the recent-documents query, but the semantic and most-derived paths ignored it, so `limit_to_session` leaked cross-session conclusions into perspectives. The allowlist is now threaded uniformly through all three paths and pushed down to pgvector and external vector stores (#881) + - Empty membership lists failed open in the vector-store filter builders, silently widening scope: LanceDB dropped empty `IN` clauses and Turbopuffer emitted a bare `In []` with undocumented semantics. Both now emit an explicit always-false predicate, and `_build_filter_conditions` checks `is not None` rather than truthiness so an empty list is no longer treated like `None` (#881, #882) + - Session-scoped CRUD helpers ignored the session allowlist entirely, so a caller could read a session the allowlist forbids. The API routes guarded this with a 422, but the dialectic tools call these CRUD functions directly and bypassed it. `_semantic_search_messages` (covering `search_messages` and `search_messages_temporal`), `grep_messages`, `get_messages_by_date_range`, `get_recent_history`, and `get_observation_context` now return `[]` when `session_name` is set and outside the allowlist (#882) + - The cache client logged the full Redis URL — including the password — at INFO and WARNING on every connection attempt and failure, exposing the live credential in container logs and downstream aggregation. Credentials are now redacted across userinfo, the `?password=` (redis-py) and `?secret=` (cashews) query params, scheme-less URLs whose password is invisible to `.port`/`.password` parsing, and malformed URLs, whose fallback previously echoed the raw input verbatim (#869) + - A `top_k` of `0` reached the vector store, where Turbopuffer rejects it with a 400 (`top_k must be between 1 and 10000`). A non-positive `top_k` now returns `[]` before the embedding call, and the semantic budget floors at 1 so an explicitly requested search isn't silently allocated zero (#970) + - Gemini clients had no HTTP timeout, so a stalled socket wedged the deriver worker's uvloop event loop, which the in-process reconciler shares. A 10-minute timeout is now set on both the Gemini LLM client and the Gemini embedding client (#903) + - Dreamer conclusions were dated to ingestion time rather than their latest source observation, and their timestamps are now normalized (#890) + - Langfuse I/O annotation was gated on `LANGFUSE_PUBLIC_KEY` instead of `langfuse_inline_enabled`, so in the default `exporter` mode it called `update_current_generation()` with no active span — logging "No active span in current context" roughly 14 times per dialectic run and building throwaway `model_dump` payloads on every LLM call. Separately, `AgentToolSummaryCreatedEvent` hardcoded `run_id="deriver"` / `iteration=0`, polluting `run_id` grouping in the CloudEvents stream with a phantom run; both fields are now optional and the resource id is keyed on `message_id:summary_type` (schema_version 2 → 3) (#845) + - Assistant tool calls were dropped from the captured trace stream for OpenAI and Gemini: `build_captured_messages` read only `{role, content, tool_call_id}`, but those providers keep tool calls outside `content`, so replayed tool-call turns landed as empty content and Gemini lost its text and tool results entirely. Tool calls are now normalized per provider into a unified `tool_calls` field and folded into the content hash. Gemini's `thought_signature` is bytes, so `model_dump(mode="json")` raised `UnicodeDecodeError` inside `emit_trace`, silently dropping whole tool-calling iterations from the trace stream (billing and Langfuse were unaffected); it is now base64-encoded on the telemetry path while replay keeps the raw bytes (#845) + - `EmbeddingClient.encoding` forced full client construction, raising "OpenAI API key is required" even though tiktoken needs no credentials. The document dedup tie-break only needs `.encoding` for token counting, so any test hitting that path failed in environments without embedding keys — notably CI for pull requests from forks. The encoding is now resolved from the configured model directly, falling back to `cl100k_base`, and the underlying client's encoding is reused only when it has already been constructed (#955) + - The Docker build failed under Podman because the uv build inputs weren't copied (#878) + - LanceDB was installed on macOS Intel, where it doesn't work. A PEP 508 marker excludes `darwin/x86_64` and the LanceDB vector-store import is wrapped so a misconfiguration surfaces as a clear config error (#496) + - Prompt checks requiring the literal token "json" for `json_object` mode are now satisfied in lowercase (#887) + - Reverted an unintended `RepresentationCompletedEvent` schema-version increment + - Documented preinstalling pgvector as a privileged role for deployments where the `DB_CONNECTION_URI` role deliberately cannot create extensions (managed Postgres, Kubernetes operators, NixOS). `CREATE EXTENSION IF NOT EXISTS vector` does not help there, because Postgres checks the privilege before checking whether the extension exists. Docker Compose is unaffected, since the bundled stack connects as the `postgres` superuser (#984) + + + ### Added - `api_request_duration_seconds` Prometheus histogram tracking per-route request latency, labeled by method and endpoint (#837) @@ -52,6 +99,8 @@ Welcome to the Honcho changelog! This section documents all notable changes to t - Fixed a `create_tree` keyword-argument mismatch in the Dreamer's surprisal tree construction (#749) - Providers that omit output-token counts (observed with Gemini on tool-loop completions) returned `output_tokens=None`, which raised a Pydantic validation error that aborted the call and crashed the Dreamer's induction phase before inductive conclusions were persisted. `None` is now coerced to `0` so token accounting degrades gracefully (#809) - Document creation now performs exact (case-insensitive, whitespace-trimmed) content deduplication before the existing semantic dedup step: exact duplicates within a batch collapse to a single insert, and an exact match against a live document reinforces it (atomic `times_derived` increment) instead of creating a new row (#861) + - The OpenAI backend passed `tool_choice` through raw while the Anthropic and Gemini backends translate Honcho's canonical vocabulary to their native form, so on a mixed-provider fallback chain (for example Gemini primary → OpenAI backup) a canonical `"any"` reached OpenAI unchanged and was rejected as an invalid param. The OpenAI backend now converts it, mirroring the others: `any`/`required` → `required`, `auto`/`none` pass through, and a tool-name string or `{"name": ...}` dict becomes a function selection (#850) + - Langfuse `@observe` auto-capture serialized every argument of `honcho_llm_call_inner` into the generation span input, including `client_override` (a live `AsyncOpenAI`/`genai` client) and `selected_config` (which carries `api_key`). Auto-capture deep-copied the client into a half-constructed object whose teardown raised (`AsyncHttpxClientWrapper ... no attribute '_state'` on OpenAI, flooding stderr; `BaseApiClient ... no attribute '_http_options'` on Gemini), and it leaked `ModelConfig.api_key` into traces. Capture is now an explicit allowlist: `capture_input`/`capture_output` are disabled and curated, serializable input and output are stamped instead, with tuning knobs surfaced as `model_parameters` via a secret-bearing denylist and per-call token usage mirrored as `usage_details` (#849) @@ -698,6 +747,12 @@ Welcome to the Honcho changelog! This section documents all notable changes to t [Python SDK](https://pypi.org/project/honcho-ai/) + + ### Added + + - `response_format` on `Peer.chat()` / `PeerAio.chat()` and `Peer.chat_stream()` / `PeerAio.chat_stream()`, for constraining a dialectic answer to a schema. Pass a Pydantic model class to get a validated instance back (parsed via `model_validate_json`), or a raw JSON Schema dict to get the JSON string as-is. Overloads type the return precisely, so a model class narrows to that model and a dict narrows to `str`. On the streaming variants, chunks stay raw text that accumulates to a JSON string — parse it after the stream completes. Requires a Honcho server with the matching API support (Honcho v3.0.12+). + - `response_format` field on `DialecticParams`. + ### Added @@ -860,6 +915,11 @@ Welcome to the Honcho changelog! This section documents all notable changes to t [TypeScript SDK](https://www.npmjs.com/package/@honcho-ai/sdk) + + ### Added + + - `responseFormat` option on `peer.chat()` and `peer.chatStream()`, for constraining a dialectic answer to a schema. Pass a Zod schema to get a parsed, validated result back, or a raw JSON Schema object to get the JSON string as-is. Overloads type the return precisely, so a Zod schema narrows to its inferred type and a plain object narrows to `string`. On `chatStream()`, chunks stay raw text that accumulates to a JSON string — parse it after the stream completes. Requires a Honcho server with the matching API support (Honcho v3.0.12+). + ### Added @@ -1048,6 +1108,34 @@ Welcome to the Honcho changelog! This section documents all notable changes to t - Simplified Honcho client import path + + [Honcho CLI](https://pypi.org/project/honcho-cli/) + + ### Added + + - Device-code OAuth login for managed Honcho servers. `honcho init` now offers browser-based login (RFC 8628 device authorization grant) when the host advertises the device grant in its OAuth authorization-server metadata; tokens are persisted to `~/.honcho/config.json` and auto-refreshed (#891) + - `HONCHO_CONFIG_DIR` environment variable for pointing the CLI at an alternate config directory (#891) + + ### Changed + + - An OAuth grant now records the host it was minted against and is ignored — neither used nor refreshed — when `base_url` points elsewhere, so a staging grant is never sent to production. A live OAuth token takes precedence over a stored `apiKey`, and a dead grant degrades to the saved key with a warning instead of aborting. Device login no longer deletes the shared `apiKey`, which sibling tools read from the same config file (#891) + + + ### Fixed + + - Declare `click` as an explicit dependency. The CLI imported `click` directly but relied on it being pulled in transitively, so installs without it on the path could fail at runtime (#787) + + + ### Added + + - Initial release of `honcho-cli` — a terminal for inspecting and managing a Honcho deployment (#424) + - `workspace`, `peer`, `session`, `message`, `conclusion`, and `config` command groups for managing resources against any Honcho server + - `init` onboarding flow that prompts for and persists connection settings, with flag/env-var pre-seeding for non-interactive use + - Per-command flags, environment variables, and a config file for pointing the CLI at different servers (local, self-hosted, or hosted) + - Rich terminal output and an agent-usage mode for scripting against the CLI + - Documentation and an agent skill for the CLI (#589) + + ## Getting Help diff --git a/docs/docs.json b/docs/docs.json index e5bb31e8..36fef957 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -24,7 +24,7 @@ "navigation": { "versions": [ { - "version": "v3.0.11", + "version": "v3.0.12", "api": { "openapi": ["v3/openapi.json"] }, @@ -65,6 +65,7 @@ "v3/documentation/features/advanced/representation-scopes", "v3/documentation/features/advanced/dreaming", "v3/documentation/features/advanced/queue-status", + "v3/documentation/features/advanced/webhooks", "v3/documentation/features/advanced/search", "v3/documentation/features/advanced/using-filters", "v3/documentation/features/advanced/structured-outputs", diff --git a/docs/snippets/cli-commands.mdx b/docs/snippets/cli-commands.mdx index 4eb9d2d3..800d45a7 100644 --- a/docs/snippets/cli-commands.mdx +++ b/docs/snippets/cli-commands.mdx @@ -305,7 +305,7 @@ honcho peer set-metadata ## honcho session -List, inspect, create, delete, and manage conversation sessions and their peers. +List, inspect, view, create, delete, and manage conversation sessions and their peers. @@ -461,6 +461,48 @@ honcho session summaries [] + +View a session transcript as a chat log. + +Modes (pick one): + +- default / --last N: tail of the conversation (most recent N) +- --page N [--size M]: page through the full transcript +- --all: every message + +Paging follows the requested order: --page 1 starts at the oldest message, +or the newest with --reverse. + +Human mode prints a row-delimited table. JSON mode emits the message list +(same shape as message list). + +```bash +honcho session view [] +``` + + + + Show only the N most recent messages (default when no --page/--all: 50). + + + 1-indexed page of the full transcript. Use for page 2+. + + + Messages per page; requires --page (1-100, default: 50). + + + Show the full transcript (every page). + + + Newest first (default is chronological: oldest at top). + + + Include message IDs in the transcript. + + + Filter by peer ID. Short alias: `-p`. + + ## honcho workspace diff --git a/docs/v3/contributing/configuration.mdx b/docs/v3/contributing/configuration.mdx index 8f4c3eb6..1c9f2629 100644 --- a/docs/v3/contributing/configuration.mdx +++ b/docs/v3/contributing/configuration.mdx @@ -189,8 +189,29 @@ Each model config supports an `overrides.provider_params` dict for passing arbit [deriver.model_config.overrides.provider_params] # These are passed directly to the provider SDK verbosity = "low" +# Per-request timeout in seconds; useful for queued workers that can wait longer +timeout = 3600.0 ``` +Because provider params live on each model config, background workers such as +the Deriver and Dreamer can use longer request timeouts while synchronous +chat paths keep tighter defaults. + +`timeout` gotchas: + +- The value is validated **at config load**: it must coerce to a positive, + finite number of seconds (numbers or numeric strings like `"3600"`), or the + process refuses to start with an error naming the offending config path. + This applies to both the primary model config and its `fallback.overrides`. +- The unit is always **seconds**, regardless of transport. OpenAI and + Anthropic receive it as the SDK's `timeout` kwarg; Gemini has no such + kwarg, so Honcho converts it to milliseconds on `http_options.timeout`. +- When unset, nothing is forwarded and each SDK's default applies — adding + this key is opt-in and changes no existing behavior. +- A too-tight timeout doesn't fail once: the aborted request goes through the + normal retry/fallback chain before the caller sees an error, so the + observed latency is several multiples of the timeout. + #### Transport passthrough keys Three keys inside `provider_params` are recognized as request-level escape hatches and forwarded to the underlying transport. Where a transport actually validates and merges one of these keys, its value must be a mapping — a non-mapping value raises a configuration error (see the per-transport behavior below; a key a transport ignores is not validated): @@ -245,18 +266,33 @@ EMBEDDING_MAX_TOKENS_PER_REQUEST=300000 # Embedding transport/model selection EMBEDDING_MODEL_CONFIG__TRANSPORT=openai # openai, gemini EMBEDDING_MODEL_CONFIG__MODEL=text-embedding-3-small +EMBEDDING_MODEL_CONFIG__MAX_BATCH_SIZE=10 # optional per-request input cap # Optional endpoint overrides EMBEDDING_MODEL_CONFIG__OVERRIDES__BASE_URL=http://localhost:8000/v1 EMBEDDING_MODEL_CONFIG__OVERRIDES__API_KEY_ENV=EMBEDDING_CUSTOM_API_KEY ``` +`EMBEDDING_MODEL_CONFIG__MAX_BATCH_SIZE` defaults to 2048 for OpenAI. For +Gemini the client applies a conservative default of 100 — Gemini does not +document a per-request limit. Set it when an OpenAI-compatible embedding +provider accepts fewer inputs per request, such as DashScope +`text-embedding-v4` with a limit of 10. + Forwarding `dimensions=` to OpenAI-compatible providers is controlled by `EMBEDDING_MODEL_CONFIG__DIMENSIONS_MODE`: - `auto` (default): forwards `dimensions=` when **the operator has explicitly set `EMBEDDING_VECTOR_DIMENSIONS`** — provenance, not value — and the configured model is not on the known-rejecting list (currently `text-embedding-ada-002`). Explicit `EMBEDDING_VECTOR_DIMENSIONS=1536` *does* trigger the forward; this is how `text-embedding-3-large` truncation to 1536 is expressed. Deployments that leave the setting unset get their existing behavior (`dimensions=` is not forwarded). - `always`: always forward, regardless of whether `EMBEDDING_VECTOR_DIMENSIONS` was set. Use for OpenAI-compatible self-hosted providers that require it. Do not pick `always` *just* for same-as-default truncation — `auto` handles that case correctly as long as you set `EMBEDDING_VECTOR_DIMENSIONS=1536` explicitly in your environment. `always` is the right answer when your config layer might strip explicit "default-valued" envs, or when you want defense-in-depth. - `never`: never forward. Explicit opt-out for providers that reject the parameter (e.g. `text-embedding-ada-002` if it slips past the known-rejecting allowlist). +The embedding wire format is controlled by `EMBEDDING_MODEL_CONFIG__ENCODING_FORMAT_MODE`. The `openai` SDK sends `encoding_format=base64` when the caller passes nothing, and some OpenAI-compatible providers answer that with an error or with empty data, so Honcho always sends the format explicitly: + +- `auto` (default): `base64` when no `EMBEDDING_MODEL_CONFIG__OVERRIDES__BASE_URL` is set or it points at `api.openai.com`, `float` otherwise. base64 is roughly 3.6x smaller on the wire than JSON floats, so this keeps the compact format for real OpenAI and only pays the larger payload where compatibility requires it. +- `float`: always request floats. Use for a provider that rejects base64 but sits behind a host `auto` reads as OpenAI-compatible-but-capable. +- `base64`: always request base64. Use for a proxy that fronts real OpenAI (Azure OpenAI, LiteLLM) where `auto` cannot tell from the host that base64 is safe, and you want the smaller payload. + +Both formats decode to identical vectors, so switching modes does not require re-embedding. + #### Bootstrapping non-default dimensions `EMBEDDING_VECTOR_DIMENSIONS` is treated as immutable for the life of a deployment. The pgvector schema is dim-pinned by Alembic at `1536` by default; if you want a different dim, you must ALTER the empty columns once at bootstrap time. @@ -569,6 +605,8 @@ VECTOR_STORE_QDRANT_PREFIX= # optional, for reverse-proxy path prefix VECTOR_STORE_QDRANT_TIMEOUT= # optional, request timeout in seconds ``` +LanceDB is an optional extra and is not included in the default Docker image. Build with `docker build --build-arg INSTALL_LANCEDB=true .` (or `INSTALL_LANCEDB=true docker compose up -d --build`), or run `uv sync --extra lancedb` for manual setups. Note the extra is unavailable on Intel macOS. + ## Monitoring ### Prometheus Metrics diff --git a/docs/v3/contributing/self-hosting.mdx b/docs/v3/contributing/self-hosting.mdx index b062a766..41a439f2 100644 --- a/docs/v3/contributing/self-hosting.mdx +++ b/docs/v3/contributing/self-hosting.mdx @@ -95,6 +95,8 @@ docker compose up -d --build The first build takes a few minutes (compiling from source). Subsequent starts are fast. +The default image does not include LanceDB. To use `VECTOR_STORE_TYPE=lancedb`, build with `INSTALL_LANCEDB=true docker compose up -d --build`. + This starts four services: **api** (port 8000), **deriver** (background worker), **database** (PostgreSQL with pgvector, port 5432), and **redis** (port 6379). All ports are bound to `127.0.0.1`. Redis caching is enabled by default. For development, uncomment the source mount and monitoring sections inside `docker-compose.yml` to enable live reload, Prometheus, and Grafana. @@ -178,6 +180,12 @@ CREATE EXTENSION IF NOT EXISTS vector; \q ``` + +**Least-privilege database roles.** Honcho runs `CREATE EXTENSION IF NOT EXISTS vector` before migrations and again at startup, so if the role in `DB_CONNECTION_URI` can't create extensions, both fail with a privilege error — `IF NOT EXISTS` doesn't help, because Postgres checks the privilege first. + +Run the statement above once as a superuser (or `rds_superuser`) and the application role needs no extension privileges. See [Troubleshooting](/v3/contributing/troubleshooting) if you hit this. + + ### 4. Configure Environment Create a `.env` file with your settings: diff --git a/docs/v3/contributing/troubleshooting.mdx b/docs/v3/contributing/troubleshooting.mdx index d9b745a1..ffb73417 100644 --- a/docs/v3/contributing/troubleshooting.mdx +++ b/docs/v3/contributing/troubleshooting.mdx @@ -204,6 +204,20 @@ uv run alembic history uv run alembic upgrade head ``` +### "permission denied to create extension \"vector\"" + +**Cause:** The role in `DB_CONNECTION_URI` is not a superuser and lacks privileges to create extensions. Honcho issues `CREATE EXTENSION IF NOT EXISTS vector` both before running migrations and again at server startup, so this fails in both places. `IF NOT EXISTS` does not help — Postgres checks the privilege before checking whether the extension exists. + +**Fix:** Preinstall pgvector once with a privileged role (superuser, or `rds_superuser` on RDS) in the Honcho database: + +```sql +CREATE EXTENSION IF NOT EXISTS vector; +``` + +The `IF NOT EXISTS` calls then short-circuit and the application role needs no extension privileges. This is the supported path for deployments where the platform, not the application, owns extension management. + +You won't hit this with `docker compose` — the bundled stack connects as `postgres`, a superuser, and its database service also creates the extension from `database/init.sql` on first boot. It comes up on managed Postgres, Kubernetes operators, and other setups where you bring your own database and its role is deliberately not a superuser. + ## Cache & Redis ### Redis is optional diff --git a/docs/v3/documentation/features/advanced/representation-scopes.mdx b/docs/v3/documentation/features/advanced/representation-scopes.mdx index d7c478d7..30e2ae32 100644 --- a/docs/v3/documentation/features/advanced/representation-scopes.mdx +++ b/docs/v3/documentation/features/advanced/representation-scopes.mdx @@ -22,7 +22,7 @@ You can retrieve a subset of conclusions from a peer's representation using `rep alice_rep = session.representation("alice") # Or via chat -response = alice.chat("What are Alice's main interests?", session_id=session.id) +response = alice.chat("What are Alice's main interests?", session=session.id) ``` This is sufficient for most applications—Honcho reasons over every message written to the peer, storing conclusions that any part of your system can retrieve. @@ -160,13 +160,13 @@ The `target` parameter also works with the chat endpoint: # Query using conclusions from Honcho's representation (across all sessions) honcho_answer = alice.chat( "What did Bob say about breakfast?", - session_id=session.id + session=session.id ) # Query using conclusions from Alice's representation of Bob (from Alice's sessions only) alice_answer = alice.chat( "What did Bob say about breakfast?", - session_id=session.id, + session=session.id, target="bob" ) ``` @@ -175,13 +175,13 @@ alice_answer = alice.chat( // Query using conclusions from Honcho's representation (across all sessions) const honchoAnswer = await alice.chat( "What did Bob say about breakfast?", - { sessionId: session.id } + { session: session.id } ); // Query using conclusions from Alice's representation of Bob (from Alice's sessions only) const aliceAnswer = await alice.chat( "What did Bob say about breakfast?", - { sessionId: session.id, target: "bob" } + { session: session.id, target: "bob" } ); ``` @@ -225,7 +225,7 @@ This architecture enables: ## Semantic Search Parameters -Both `representation()` and `chat()` support semantic filtering to retrieve a subset of relevant conclusions. You can optionally filter by session to retrieve only conclusions from specific session context: +Both `representation()` and `chat()` support semantic filtering to retrieve a subset of relevant conclusions. You can optionally filter by session — pass `session` to scope to a single session, or use the REST-only [session allowlist](/v3/documentation/features/advanced/using-filters#scoping-recall-to-sessions) to scope to a set of sessions: | Parameter | Type | Description | |-----------|------|-------------| diff --git a/docs/v3/documentation/features/advanced/using-filters.mdx b/docs/v3/documentation/features/advanced/using-filters.mdx index 2a015c03..7d6caacd 100644 --- a/docs/v3/documentation/features/advanced/using-filters.mdx +++ b/docs/v3/documentation/features/advanced/using-filters.mdx @@ -304,6 +304,33 @@ sessions = honcho.sessions(filters={ ### List Membership +A bare list is shorthand for `in`, so `{"peer_id": ["alice", "bob"]}` and +`{"peer_id": {"in": ["alice", "bob"]}}` are equivalent: + + +```python Python +# Shorthand: a bare list means "any of these" +messages = session.messages(filters={ + "peer_id": ["alice", "bob", "charlie"] +}) +``` + +```typescript TypeScript +(async () => { + // Shorthand: a bare list means "any of these" + const messages = await session.messages({ + filters: { peer_id: ["alice", "bob", "charlie"] } + }); +})(); +``` + + + +Bare lists behave differently inside metadata — use `{"in": [...]}` there for OR matching. + + +The explicit form, plus the other comparison operators: + ```python Python # Find messages from specific peers in a session @@ -673,6 +700,93 @@ bob_explicit = peer.conclusions_of("bob").list(filters={"level": "explicit"}) ``` +## Scoping Recall to Sessions + +The [chat endpoint](/v3/documentation/features/chat) and the representation +endpoint accept a `filters` body too, but a deliberately narrow one: it defines +a **session allowlist**, restricting what the request can recall to the sessions +you name — conclusions on both endpoints, and on chat the messages the agent +reads as well. + +This is how you scope recall to more than one session. The `session_id` +parameter pins a request to exactly one session; an allowlist accepts a set. + +Only the `session_id` key is supported here, in three shapes: + +```json +{"filters": {"session_id": "support-chat-1"}} +{"filters": {"session_id": ["support-chat-1", "support-chat-2"]}} +{"filters": {"session_id": {"in": ["support-chat-1", "support-chat-2"]}}} +``` + + +```bash Chat +curl -X POST "$HONCHO_URL/v3/workspaces/my-app/peers/user-123/chat" \ + -H "Authorization: Bearer $HONCHO_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{ + "query": "What did the user ask about billing?", + "filters": { "session_id": ["support-chat-1", "support-chat-2"] } + }' +``` + +```bash Representation +curl -X POST "$HONCHO_URL/v3/workspaces/my-app/peers/user-123/representation" \ + -H "Authorization: Bearer $HONCHO_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{ + "filters": { "session_id": ["support-chat-1", "support-chat-2"] } + }' +``` + + + +The session allowlist is REST-only today. The SDKs cover the single-session case +with `session`, but do not yet expose the allowlist — call the endpoint directly +when you need a set of sessions. + + +### Rules + +Unlike the list endpoints above, this filter **fails closed**: an unrecognized +key or shape is rejected with `422` rather than ignored, because a silently +dropped filter here would widen recall instead of narrowing it. + +| Rule | Behavior | +|------|----------| +| Any key other than `session_id` | `422` | +| A shape other than a string, a list of strings, or `{"in": [...]}` | `422` | +| More than 1,000 sessions | `422` | +| `session_id` set alongside `filters` | The `session_id` must appear in the allowlist, else `422` | +| An empty allowlist (`[]`) | Valid, and recalls nothing | +| A peer-scoped key naming a session its peer isn't an active member of | `401` on chat — see below | + + +On chat, a peer-scoped key must be an active member of every session it names — +the allowlist reaches message recall there — and the request is rejected with +`401` otherwise. The representation endpoint runs no membership check: key scope +already confines the caller to its own peer's representation, which an allowlist +can only narrow. + + +### What Changes Under an Allowlist + +Scoping recall by session narrows what the reasoning agent can draw on: + +- **Only `explicit` conclusions are recalled.** Dream-derived conclusions + (`deductive`, `inductive`) are synthesized across sessions, so they can't be + attributed to one session and are excluded. +- **Reasoning-chain traversal is unavailable**, since it walks into those + derived conclusions. +- **Message recall is restricted to the allowlisted sessions** across every + search path — semantic, keyword, and date-range. + + +Because of this, an allowlisted request answers from directly-stated facts +rather than higher-order inferences. If you want the full representation, omit +`filters` and let the agent search everything. + + ## Error Handling Handle filter errors gracefully: diff --git a/docs/v3/documentation/features/advanced/webhooks.mdx b/docs/v3/documentation/features/advanced/webhooks.mdx new file mode 100644 index 00000000..8976750a --- /dev/null +++ b/docs/v3/documentation/features/advanced/webhooks.mdx @@ -0,0 +1,217 @@ +--- +title: 'Webhooks' +description: 'Receive push notifications when Honcho finishes background work' +icon: 'satellite-dish' +--- + +Honcho's reasoning runs in the background, so a message you just created is not +immediately reflected in the peer's representation. Instead of polling +[queue status](/v3/documentation/features/advanced/queue-status), you can +register a webhook endpoint and have Honcho notify you when the work it queued +for a session has drained. + +Webhooks are registered per workspace. Every event for that workspace is +delivered to every endpoint registered on it. + +## Registering an Endpoint + + +```bash Register +curl -X POST "$HONCHO_URL/v3/workspaces/my-app/webhooks" \ + -H "Authorization: Bearer $HONCHO_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{"url": "https://example.com/honcho/webhook"}' +``` + +```bash List +curl -X GET "$HONCHO_URL/v3/workspaces/my-app/webhooks" \ + -H "Authorization: Bearer $HONCHO_API_KEY" +``` + +```bash Test +curl -X GET "$HONCHO_URL/v3/workspaces/my-app/webhooks/test" \ + -H "Authorization: Bearer $HONCHO_API_KEY" +``` + +```bash Delete +curl -X DELETE "$HONCHO_URL/v3/workspaces/my-app/webhooks/$ENDPOINT_ID" \ + -H "Authorization: Bearer $HONCHO_API_KEY" +``` + + +Registration is get-or-create: a URL already registered on the workspace +returns `200` with the existing endpoint, a new one returns `201`. The test +route emits a `test.event` to every endpoint on the workspace, which is the +quickest way to confirm your receiver and signature check work end to end. + +Webhook routes accept an admin key or a workspace-scoped key for that +workspace. Peer- and session-scoped keys cannot manage webhooks. + + +Webhook management is also available in the dashboard on the +[Webhooks](https://app.honcho.dev/webhooks) page. + + +### URL Requirements + +A webhook URL must be absolute and use `http` or `https`. URLs whose host is an +IP literal in a private, loopback, link-local, reserved, multicast, or +unspecified range are rejected with `422`. + + +This check inspects IP literals only — hostnames are accepted without +resolution. If you self-host, treat network-level egress controls, not this +validation, as your defense against internal-address delivery. + + +Each workspace can register up to `WEBHOOK_MAX_WORKSPACE_LIMIT` endpoints +(default 10). Exceeding the limit returns `409`. + +## Events + +| Event | When it fires | `data` fields | +|-------|---------------|---------------| +| `queue.empty` | A unit of queued background work finished draining | `workspace_id`, `queue_type` (`representation` or `summary`), `session_id`, `observer`, `observed` | +| `test.event` | You called `GET /webhooks/test` | `workspace_id` | + + +`queue.empty` is scoped to a single unit of work — one task type for one +session and observer/observed pair — not to the workspace as a whole. Other +work may still be queued elsewhere in the workspace when it fires. A session +whose messages produce both representation and summary work emits one event per +task type. + + +## Payload + +Every delivery is a `POST` with a `Content-Type: application/json` body in this +envelope: + +```json +{ + "type": "queue.empty", + "data": { + "workspace_id": "my-app", + "queue_type": "representation", + "session_id": "support-chat-1", + "observer": "assistant", + "observed": "user-123" + }, + "timestamp": "2026-08-10T18:24:05.123456Z" +} +``` + +**`data` is event-specific — its keys differ by event type.** A `test.event` +carries only `workspace_id`: + +```json +{ + "type": "test.event", + "data": { "workspace_id": "my-app" }, + "timestamp": "2026-08-10T18:24:05.123456Z" +} +``` + +Within one event type, an optional field with no value is sent as an explicit +`null` — on `queue.empty`, that's `session_id`, `observer`, and `observed` for +work that isn't tied to a session or an observer pair. Across event types the key +is simply absent. + +Parse defensively: branch on `type` as the discriminator, treat every `data` key +as optional rather than required, and tolerate new event types and new fields. +A parser that requires the `queue.empty` keys on every event will break on a +`test.event`. + +## Verifying Signatures + +Each delivery carries an `X-Honcho-Signature` header: the hex-encoded +HMAC-SHA256 of the **raw request body**, keyed with your deployment's +`WEBHOOK_SECRET`. Always compare with a constant-time function, and always sign +the bytes you received — Honcho serializes the body compactly with sorted keys, +so re-serializing your parsed JSON will not reliably reproduce it. + + +```python Python +import hashlib +import hmac +import json +import os + +def verify(raw_body: bytes, signature: str) -> bool: + expected = hmac.new( + os.environ["WEBHOOK_SECRET"].encode(), + raw_body, + hashlib.sha256, + ).hexdigest() + return hmac.compare_digest(expected, signature) + +# FastAPI — read the raw body, not a parsed model +@app.post("/honcho/webhook") +async def handle(request: Request): + raw = await request.body() + if not verify(raw, request.headers.get("X-Honcho-Signature", "")): + raise HTTPException(status_code=401) + event = json.loads(raw) + ... +``` + +```typescript TypeScript +import crypto from 'node:crypto'; + +function verify(rawBody: Buffer, signature: string): boolean { + const expected = crypto + .createHmac('sha256', process.env.WEBHOOK_SECRET!) + .update(rawBody) + .digest('hex'); + const a = Buffer.from(expected); + const b = Buffer.from(signature); + return a.length === b.length && crypto.timingSafeEqual(a, b); +} + +// Express — note express.raw(), not express.json() +app.post('/honcho/webhook', express.raw({ type: 'application/json' }), (req, res) => { + if (!verify(req.body, req.header('X-Honcho-Signature') ?? '')) { + return res.sendStatus(401); + } + const event = JSON.parse(req.body.toString()); + res.sendStatus(200); +}); +``` + + +## Delivery Semantics + +Delivery is best-effort and fire-and-forget: + +- Events fan out to all of the workspace's endpoints concurrently. +- Each request has a 30-second timeout. +- **There are no retries.** A non-2xx response, a timeout, or a connection + error is logged on the server and the event is dropped. + +Design your receiver accordingly: treat the event as a hint to re-read state +from the API rather than as the state itself, and fall back to +[queue status](/v3/documentation/features/advanced/queue-status) polling if you +need a guarantee. + +## Self-Hosting Requirements + + +`WEBHOOK_SECRET` must be set, or nothing is delivered. Honcho signs every +payload before sending it; with no secret configured, signing fails and the +event is dropped after being logged. Registration still succeeds, so a missing +secret looks like silence rather than an error. + + +Webhook delivery is queued work handled by the deriver process, so a deriver +worker must be running for events to be sent. See +[Configuration](/v3/contributing/configuration#webhooks) for +`WEBHOOK_SECRET` and `WEBHOOK_MAX_WORKSPACE_LIMIT`. + + + + Poll background processing state instead of waiting for a push + + + Full request and response schemas for the webhook endpoints + + diff --git a/docs/v3/documentation/features/chat.mdx b/docs/v3/documentation/features/chat.mdx index 6aab6996..b6e7e96c 100644 --- a/docs/v3/documentation/features/chat.mdx +++ b/docs/v3/documentation/features/chat.mdx @@ -94,6 +94,27 @@ for await (const chunk of responseStream.iter_text()) { Streaming is useful for displaying real-time responses in chat interfaces or when asking complex questions that require longer answers. +## Scoping to Sessions + +By default the chat endpoint reasons over everything Honcho knows about the +peer. Pass `session` (`session_id` on the REST body) to restrict it to one +session: + + +```python Python +answer = peer.chat("What did the user ask about?", session=session.id) +``` + +```typescript TypeScript +const answer = await peer.chat("What did the user ask about?", { session: session.id }); +``` + + +To scope a request to a *set* of sessions, use the session allowlist — a +constrained `filters` body on the endpoint. See +[Scoping Recall to Sessions](/v3/documentation/features/advanced/using-filters#scoping-recall-to-sessions) +for the accepted shapes and for what an allowlist changes about the answer. + ## Structured Outputs When your application needs a machine-readable answer instead of prose, pass a schema as `response_format` and the answer is guaranteed to conform to it: diff --git a/docs/v3/documentation/introduction/vibecoding.mdx b/docs/v3/documentation/introduction/vibecoding.mdx index 36e6661d..cd5e8ac6 100644 --- a/docs/v3/documentation/introduction/vibecoding.mdx +++ b/docs/v3/documentation/introduction/vibecoding.mdx @@ -122,7 +122,9 @@ npx skills add plastic-labs/honcho ``` ```bash Install as Claude Skill Manually -curl -o ~/.claude/skills/honcho-integration.md https://raw.githubusercontent.com/plastic-labs/honcho/main/docs/SKILL.md +mkdir -p ~/.claude/skills/honcho-integration +curl -o ~/.claude/skills/honcho-integration/SKILL.md \ + https://raw.githubusercontent.com/plastic-labs/honcho/main/.claude/skills/honcho-integration/SKILL.md ``` @@ -139,24 +141,16 @@ curl -o ~/.claude/skills/honcho-integration.md https://raw.githubusercontent.com Invoke with `/honcho-integration` in your coding agent. +#### honcho-memory + +**Concepts & strategy for using Honcho at runtime.** The hub skill: it teaches the recall → respond → record loop and session and peer design — the durable model behind using Honcho as memory, independent of how you're connected — plus how to connect via MCP and drive the [MCP tools](#mcp-server). Use this when your agent already has Honcho available and you want it to remember the user across conversations — as opposed to `honcho-integration`, which adds the SDK to a codebase. (An MCP-connected agent also receives usage guidance directly from the server on connect.) + #### honcho-cli **For inspection & debugging.** Teaches your coding agent the right commands and flags for the [honcho CLI](#cli) — peer memory, session context, queue status, dialectic quality. Invoke implicitly when you ask your agent to inspect a Honcho deployment. -#### migrate-honcho-py / migrate-honcho-ts - -**For SDK upgrades.** Migrates code from v1.6.0 to v2.0.0 (required for Honcho 3.0.0+). Use when upgrading the SDK or seeing errors about removed APIs like `observations`, `Representation`, `.core`, or `get_config`. - -Both skills handle: terminology changes (`Observation` → `Conclusion`), `Representation` class removal, method renames, and streaming API updates. - -| Python | TypeScript | -|--------|------------| -| `/migrate-honcho-py` | `/migrate-honcho-ts` | -| `AsyncHoncho` → `.aio` accessor | `@honcho-ai/core` removal | -| | `snake_case` → `camelCase` | - --- ## Universal Starter Prompt diff --git a/docs/v3/documentation/reference/cli.mdx b/docs/v3/documentation/reference/cli.mdx index 7b89bfeb..2b460d48 100644 --- a/docs/v3/documentation/reference/cli.mdx +++ b/docs/v3/documentation/reference/cli.mdx @@ -142,7 +142,7 @@ When you pick up a workspace and need to orient — start broad, narrow to the p ```bash honcho session inspect --json - honcho message list --last 20 --json + honcho session view --last 20 honcho session context --json honcho session summaries --json ``` @@ -150,7 +150,7 @@ When you pick up a workspace and need to orient — start broad, narrow to the p - `honcho session context` shows exactly what an agent would receive at inference time — check it before `honcho peer chat` if a response surprises you. + `honcho session context` shows exactly what an agent would receive at inference time — check it before `honcho peer chat` if a response surprises you. `honcho session view` shows the raw transcript that context was built from; it prints content verbatim, so tag-delimited and multi-line messages appear exactly as stored. ### A peer isn't learning @@ -176,7 +176,7 @@ When an agent's responses don't reflect what you expect it to know. ```bash honcho session context --json honcho session summaries --json -honcho message list --last 50 --json +honcho session view --last 50 ``` ### Dialectic returns bad answers diff --git a/docs/v3/documentation/reference/platform.mdx b/docs/v3/documentation/reference/platform.mdx index abf3c646..3ed501d5 100644 --- a/docs/v3/documentation/reference/platform.mdx +++ b/docs/v3/documentation/reference/platform.mdx @@ -67,6 +67,7 @@ Scoped keys are authorized by their narrowest claim and never widen to the whole - A **peer-scoped** key acts on its own peer, plus **read-only** access to the sessions its peer is an active member of (context, summaries, peers, its own per-session config, search, and message reads). It cannot write to those sessions or act on other peers. - A **session-scoped** key is confined to its own session and cannot reach peer routes. - Peer- and session-scoped keys **must carry their parent workspace** — creating one without a workspace is rejected. +- On the chat endpoint, a peer-scoped key can only name sessions its peer is an active member of — both the `session_id` and every session in a [session allowlist](/v3/documentation/features/advanced/using-filters#scoping-recall-to-sessions). Naming any other session returns `401`. Workspace and admin keys pass the allowlist through as given. API Key Management Dashboard diff --git a/docs/v3/guides/integrations/mcp.mdx b/docs/v3/guides/integrations/mcp.mdx index 10bfdfd0..d6beeca2 100644 --- a/docs/v3/guides/integrations/mcp.mdx +++ b/docs/v3/guides/integrations/mcp.mdx @@ -37,13 +37,10 @@ Edit `%APPDATA%\Claude\claude_desktop_config.json`: "mcp-remote", "https://mcp.honcho.dev", "--header", - "Authorization:${AUTH_HEADER}", - "--header", - "X-Honcho-User-Name:${USER_NAME}" + "Authorization:${AUTH_HEADER}" ], "env": { - "AUTH_HEADER": "Bearer hch-your-key-here", - "USER_NAME": "YourName" + "AUTH_HEADER": "Bearer hch-your-key-here" } } } @@ -62,8 +59,7 @@ For best results, create a project and paste these [instructions](https://raw.gi claude mcp add honcho \ --transport http \ --url "https://mcp.honcho.dev" \ - --header "Authorization: Bearer hch-your-key-here" \ - --header "X-Honcho-User-Name: YourName" + --header "Authorization: Bearer hch-your-key-here" ``` Or if you prefer the [Claude Code Honcho plugin](/v3/guides/integrations/claudecode) for a deeper integration with persistent memory, git awareness, and agent skills: @@ -83,9 +79,7 @@ args = [ "mcp-remote", "https://mcp.honcho.dev", "--header", - "Authorization:Bearer hch-your-key-here", - "--header", - "X-Honcho-User-Name:YourName" + "Authorization:Bearer hch-your-key-here" ] ``` @@ -103,8 +97,7 @@ Cursor supports MCP servers natively via HTTP. Add to your global config at `~/. "honcho": { "url": "https://mcp.honcho.dev", "headers": { - "Authorization": "Bearer hch-your-key-here", - "X-Honcho-User-Name": "YourName" + "Authorization": "Bearer hch-your-key-here" } } } @@ -123,8 +116,7 @@ Add to `~/.codeium/windsurf/mcp_config.json`: "honcho": { "serverUrl": "https://mcp.honcho.dev", "headers": { - "Authorization": "Bearer hch-your-key-here", - "X-Honcho-User-Name": "YourName" + "Authorization": "Bearer hch-your-key-here" } } } @@ -146,8 +138,7 @@ Add to your workspace `.vscode/mcp.json`: "type": "http", "url": "https://mcp.honcho.dev", "headers": { - "Authorization": "Bearer hch-your-key-here", - "X-Honcho-User-Name": "YourName" + "Authorization": "Bearer hch-your-key-here" } } } @@ -164,8 +155,7 @@ Or add to your User Settings JSON (`Cmd+Shift+P` → "Preferences: Open User Set "type": "http", "url": "https://mcp.honcho.dev", "headers": { - "Authorization": "Bearer hch-your-key-here", - "X-Honcho-User-Name": "YourName" + "Authorization": "Bearer hch-your-key-here" } } } @@ -192,8 +182,7 @@ Cline supports remote MCP servers natively. Open Cline's MCP settings at: "honcho": { "url": "https://mcp.honcho.dev", "headers": { - "Authorization": "Bearer hch-your-key-here", - "X-Honcho-User-Name": "YourName" + "Authorization": "Bearer hch-your-key-here" } } } @@ -212,8 +201,7 @@ Add to `~/.config/zed/settings.json`: "honcho": { "url": "https://mcp.honcho.dev", "headers": { - "Authorization": "Bearer hch-your-key-here", - "X-Honcho-User-Name": "YourName" + "Authorization": "Bearer hch-your-key-here" } } } @@ -228,7 +216,7 @@ Zed uses `context_servers` instead of `mcpServers`. Native HTTP support requires [Goose](https://goose-docs.ai/) supports remote MCP servers natively over Streamable HTTP. -The easiest way is to run `goose configure`, choose **Add Extension → Remote Extension (Streamable HTTP)**, and enter the name `honcho`, the URI `https://mcp.honcho.dev`, and the headers `Authorization: Bearer hch-your-key-here` and `X-Honcho-User-Name: YourName`. +The easiest way is to run `goose configure`, choose **Add Extension → Remote Extension (Streamable HTTP)**, and enter the name `honcho`, the URI `https://mcp.honcho.dev`, and the header `Authorization: Bearer hch-your-key-here`. Or edit your `config.yaml` directly (on Linux, `~/.config/goose/config.yaml`): @@ -242,7 +230,6 @@ extensions: uri: https://mcp.honcho.dev headers: Authorization: "Bearer hch-your-key-here" - X-Honcho-User-Name: "YourName" timeout: 60 ``` @@ -254,13 +241,11 @@ To teach Goose the recommended memory flow, save the [instructions](https://raw. ## Optional Configuration -You can customize the assistant name and workspace ID by adding extra headers. Both are optional. +You can target a specific workspace by adding an extra header. It's optional. | Header | Default | Description | |--------|---------|-------------| | `Authorization` | *required* | `Bearer hch-your-key-here` | -| `X-Honcho-User-Name` | *required* | What the AI should call you | -| `X-Honcho-Assistant-Name` | `"Assistant"` | Name for the AI peer | | `X-Honcho-Workspace-ID` | `"default"` | Isolate memory per project | Example with all headers (Claude Desktop format): @@ -276,16 +261,10 @@ Example with all headers (Claude Desktop format): "--header", "Authorization:${AUTH_HEADER}", "--header", - "X-Honcho-User-Name:${USER_NAME}", - "--header", - "X-Honcho-Assistant-Name:${ASSISTANT_NAME}", - "--header", "X-Honcho-Workspace-ID:${WORKSPACE_ID}" ], "env": { "AUTH_HEADER": "Bearer hch-your-key-here", - "USER_NAME": "YourName", - "ASSISTANT_NAME": "Claude", "WORKSPACE_ID": "my-project" } } @@ -295,19 +274,11 @@ Example with all headers (Claude Desktop format): --- -## Available Tools +## Using the Tools -The recommended flow for a standard conversation uses `create_session` + `add_messages_to_session` + `chat`. See the [full instructions](https://raw.githubusercontent.com/plastic-labs/honcho/refs/heads/main/mcp/instructions.md) for a complete walkthrough. +Once connected, the Honcho MCP server tells your assistant how to use the tools automatically — it ships usage instructions (the recall → respond → record loop, the full tool list, and reasoning levels) on connect, so there's nothing extra to configure. -**Workspace** — `inspect_workspace`, `list_workspaces`, `search`, `get_metadata`, `set_metadata` - -**Peers** — `create_peer`, `list_peers`, `chat`, `get_peer_card`, `set_peer_card`, `get_peer_context`, `get_representation` - -**Sessions** — `create_session`, `list_sessions`, `delete_session`, `clone_session`, `add_peers_to_session`, `remove_peers_from_session`, `get_session_peers`, `inspect_session`, `add_messages_to_session`, `get_session_messages`, `get_session_message`, `get_session_context` - -**Conclusions** — `list_conclusions`, `query_conclusions`, `create_conclusions`, `delete_conclusion` - -**System** — `schedule_dream`, `get_queue_status` +If you want to read that guidance yourself, it's the [full instructions](https://raw.githubusercontent.com/plastic-labs/honcho/refs/heads/main/mcp/instructions.md). --- diff --git a/docs/v3/openapi.json b/docs/v3/openapi.json index 85e4891a..7fa1d194 100644 --- a/docs/v3/openapi.json +++ b/docs/v3/openapi.json @@ -9,7 +9,7 @@ "url": "https://honcho.dev/", "email": "hello@plasticlabs.ai" }, - "version": "3.0.11" + "version": "3.0.12" }, "servers": [ { @@ -2909,6 +2909,14 @@ "title": "Session Id", "description": "ID of the session to scope the representation to" }, + "filters": { + "anyOf": [ + { "additionalProperties": true, "type": "object" }, + { "type": "null" } + ], + "title": "Filters", + "description": "Optional filters to scope recall. This endpoint supports only the 'session_id' key: a session id, a list of session ids, or {\"in\": [...]}. Recall (conclusions and messages) is restricted to the allowlist; unsupported keys are rejected. When session_id is also set, it must be included in the allowlist." + }, "target": { "anyOf": [{ "type": "string" }, { "type": "null" }], "title": "Target", @@ -2928,6 +2936,14 @@ "title": "Reasoning Level", "description": "Level of reasoning to apply: minimal, low, medium, high, or max", "default": "low" + }, + "response_format": { + "anyOf": [ + { "additionalProperties": true, "type": "object" }, + { "type": "null" } + ], + "title": "Response Format", + "description": "Optional JSON Schema (root type 'object') the response must conform to. When provided, `content` is a JSON string matching this schema. Only a conservative subset of JSON Schema is supported; unsupported schemas are rejected with 422. Constraint keywords (minItems, maxLength, ...) are hints to the model, not enforced server-side." } }, "type": "object", @@ -2947,7 +2963,7 @@ }, "DreamType": { "type": "string", - "enum": ["omni"], + "enum": ["omni", "card_refresh"], "title": "DreamType", "description": "Types of dreams that can be triggered." }, @@ -3352,6 +3368,14 @@ "title": "Session Id", "description": "Optional session ID within which to scope the representation" }, + "filters": { + "anyOf": [ + { "additionalProperties": true, "type": "object" }, + { "type": "null" } + ], + "title": "Filters", + "description": "Optional filters to scope the representation. This endpoint supports only the 'session_id' key: a session id, a list of session ids, or {\"in\": [...]}. When session_id is also set, it must be included in the allowlist." + }, "target": { "anyOf": [{ "type": "string" }, { "type": "null" }], "title": "Target", @@ -3506,6 +3530,12 @@ "anyOf": [{ "type": "string" }, { "type": "null" }], "title": "Session Id", "description": "Session ID to scope the dream to if specified" + }, + "rebuild": { + "type": "boolean", + "title": "Rebuild", + "description": "card_refresh dreams only: rebuild the peer card solely from observations currently in the collection, without injecting the existing card (use after removals)", + "default": false } }, "type": "object", diff --git a/honcho-cli/CHANGELOG.md b/honcho-cli/CHANGELOG.md index 3c2c383a..7973341a 100644 --- a/honcho-cli/CHANGELOG.md +++ b/honcho-cli/CHANGELOG.md @@ -5,11 +5,26 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](http://keepachangelog.com/) and this project adheres to [Semantic Versioning](http://semver.org/). +## [Unreleased] + +### Added + +- `honcho session view` — session transcript table (`--last N`, `--page N --size M`, `--all`, `--reverse`, `--ids`, peer filter via `-p`). Content is shown verbatim, timestamps are normalized to UTC, and the command is read-only: unlike the other session commands it never get-or-creates the session + +### Fixed + +- `honcho message list --last N` no longer stops at the first page of 50 — it walks pages to fill the requested window + ## [0.1.2] - 2026-07-20 ### Added - Device-code OAuth login for managed Honcho servers. `honcho init` now offers browser-based login (RFC 8628 device authorization grant) when the host advertises the device grant in its OAuth authorization-server metadata; tokens are persisted to `~/.honcho/config.json` and auto-refreshed (#891) +- `HONCHO_CONFIG_DIR` environment variable for pointing the CLI at an alternate config directory (#891) + +### Changed + +- An OAuth grant now records the host it was minted against and is ignored — neither used nor refreshed — when `base_url` points elsewhere, so a staging grant is never sent to production. A live OAuth token takes precedence over a stored `apiKey`, and a dead grant degrades to the saved key with a warning instead of aborting. Device login no longer deletes the shared `apiKey`, which sibling tools read from the same config file (#891) ## [0.1.1] - 2026-06-15 diff --git a/honcho-cli/README.md b/honcho-cli/README.md index 804e2285..b191bcdb 100644 --- a/honcho-cli/README.md +++ b/honcho-cli/README.md @@ -71,6 +71,7 @@ Per-command scoping (workspace / peer / session) is handled via `-w` / `-p` / `- | `honcho session list` | List sessions in the workspace (filter with `--peer/-p`) | | `honcho session create ` | Create or get a session (optionally `--peers` to add peers, `--metadata`) | | `honcho session inspect ` | Peers, message count, summaries, config | +| `honcho session view ` | Transcript table (`--last N`, `--page N --size M`, `--all`, `--reverse`, `--ids`, `-p`) | | `honcho session context ` | What an agent would see | | `honcho session summaries ` | Short + long summaries | | `honcho session peers ` / `add-peers` / `remove-peers` | Peer management | diff --git a/honcho-cli/scripts/generate_cli_docs.py b/honcho-cli/scripts/generate_cli_docs.py index 8fa4e223..a632b069 100644 --- a/honcho-cli/scripts/generate_cli_docs.py +++ b/honcho-cli/scripts/generate_cli_docs.py @@ -230,7 +230,7 @@ def build() -> str: body: list[str] = [] for name in sorted(root.commands): body.extend(_render_top(root.commands[name], ["honcho", name])) - return HEADER + "\n".join(body) + "\n" + return HEADER + "\n".join(body).rstrip("\n") + "\n" def main() -> int: diff --git a/honcho-cli/src/honcho_cli/_help.py b/honcho-cli/src/honcho_cli/_help.py index d5dcb976..d47d344d 100644 --- a/honcho-cli/src/honcho_cli/_help.py +++ b/honcho-cli/src/honcho_cli/_help.py @@ -76,7 +76,7 @@ def print_welcome(console: Console) -> None: ("workspace", "list · create · search · delete · inspect · queue-status"), ("peer", "list · create · search · inspect · card · chat"), ("", "get-metadata · set-metadata · representation"), - ("session", "list · create · search · delete · inspect · add-peers"), + ("session", "list · create · search · delete · inspect · view · add-peers"), ("", "context · get-metadata · set-metadata · peers"), ("", "remove-peers · representation · summaries"), ("message", "list · create · get"), diff --git a/honcho-cli/src/honcho_cli/commands/message.py b/honcho-cli/src/honcho_cli/commands/message.py index b810917a..7a773fcc 100644 --- a/honcho-cli/src/honcho_cli/commands/message.py +++ b/honcho-cli/src/honcho_cli/commands/message.py @@ -10,7 +10,7 @@ import typer from honcho.api_types import MessageCreateParams -from honcho_cli.commands.session import _get_session_id +from honcho_cli.commands.session import _fetch_recent_messages, _get_session_id from honcho_cli.commands.workspace import _handle_error from honcho_cli.output import print_error, print_result, status from honcho_cli.validation import validate_resource_id @@ -37,15 +37,18 @@ def list_messages( handle_cmd_flags(json_output=json_output, workspace=workspace, peer=peer, session=session) sid = _get_session_id(session_id) + if last < 1: + print_error("INVALID_FLAGS", "--last must be >= 1", {"last": last}) + raise typer.Exit(1) client, config = get_client() sess = client.session(sid) try: filters = {"peer_id": config.peer_id} if config.peer_id else None - # Fetch newest-first so [:last] always gives the most recent N messages, - # then flip to oldest-at-top / newest-at-bottom for readable display. + # Fetch newest-first so we always get the most recent N messages, then + # flip to oldest-at-top / newest-at-bottom for readable display. # --reverse keeps the raw server order (oldest first, descending in table). - msgs = sess.messages(filters=filters, reverse=True).items[:last] + msgs, _ = _fetch_recent_messages(sess, filters, last) if not reverse: msgs = list(reversed(msgs)) diff --git a/honcho-cli/src/honcho_cli/commands/session.py b/honcho-cli/src/honcho_cli/commands/session.py index 2c5d811b..08dd796c 100644 --- a/honcho-cli/src/honcho_cli/commands/session.py +++ b/honcho-cli/src/honcho_cli/commands/session.py @@ -1,22 +1,29 @@ -"""Session commands: list, inspect, context, summaries, peers, search, representation, metadata.""" +"""Session commands: list, inspect, view, context, summaries, peers, search, representation, metadata.""" from __future__ import annotations import json +import shlex from typing import List, Optional import typer -from honcho import HonchoError +from honcho import HonchoError, Session from honcho_cli.commands.workspace import _config_to_dict, _handle_error, _raw_list -from honcho_cli.output import print_error, print_result, status, use_json +from honcho_cli.output import print_error, print_result, print_transcript, status, use_json from honcho_cli.validation import validate_resource_id from honcho_cli._help import HonchoTyperGroup -from honcho_cli.common import add_common_options, get_client, get_resolved_config, handle_cmd_flags +from honcho_cli.common import ( + add_common_options, + get_client, + get_flag_overrides, + get_resolved_config, + handle_cmd_flags, +) -app = typer.Typer(cls=HonchoTyperGroup, help="List, inspect, create, delete, and manage conversation sessions and their peers.") +app = typer.Typer(cls=HonchoTyperGroup, help="List, inspect, view, create, delete, and manage conversation sessions and their peers.") add_common_options(app) @@ -135,6 +142,240 @@ def inspect( _handle_error(e, "session", sid) +# Server-side ceiling on page size (fastapi-pagination's default ``Params`` +# declares ``size`` as ``Query(50, ge=1, le=100)``). +MAX_PAGE_SIZE = 100 +DEFAULT_PAGE_SIZE = 50 + + +def _fetch_recent_messages(sess, filters: dict | None, last: int) -> tuple[list, int | None]: + """Fetch the ``last`` most recent messages, newest first. + + Walks as many newest-first server pages as it takes to fill the window. + Returns the messages plus the session's total message count (if reported). + """ + page = sess.messages( + filters=filters, + reverse=True, + size=min(max(last, 1), MAX_PAGE_SIZE), + ) + total = page.total + msgs = list(page.items) + while len(msgs) < last and page.has_next_page(): + page = page.get_next_page() + if page is None: + break + msgs.extend(page.items) + return msgs[:last], total + + +def _next_page_command( + session_id: str, + next_page: int, + size: int, + *, + reverse: bool, + show_ids: bool, + workspace: str | None, + peer: str | None, +) -> str: + """Continuation command for the next page, carrying this invocation's scope. + + Scoping flags are echoed only when passed as flags; anything resolved from + the environment or config file resolves the same way on the next run. IDs + are shell-quoted — they may contain spaces and metacharacters, and this + string is meant to be pasted into a shell. + """ + parts = [ + "honcho", + "session", + "view", + session_id, + "--page", + str(next_page), + "--size", + str(size), + ] + if reverse: + parts.append("--reverse") + if show_ids: + parts.append("--ids") + if workspace: + parts += ["-w", workspace] + if peer: + parts += ["-p", peer] + return shlex.join(parts) + + +def _fetch_all_messages(sess, filters: dict | None) -> tuple[list, int | None]: + """Fetch every message in the session, oldest first.""" + page = sess.messages(filters=filters, reverse=False, size=MAX_PAGE_SIZE) + total = page.total + msgs = list(page.items) + while page.has_next_page(): + page = page.get_next_page() + if page is None: + break + msgs.extend(page.items) + return msgs, total + + +@app.command() +def view( + session_id: Optional[str] = typer.Argument(None, help="Session ID (uses default if omitted)"), + last: Optional[int] = typer.Option( + None, + "--last", + help=f"Show only the N most recent messages (default when no --page/--all: {DEFAULT_PAGE_SIZE})", + ), + page_number: Optional[int] = typer.Option( + None, + "--page", + help="1-indexed page of the full transcript. Use for page 2+.", + ), + size: Optional[int] = typer.Option( + None, + "--size", + help=f"Messages per page; requires --page (1-{MAX_PAGE_SIZE}, default: {DEFAULT_PAGE_SIZE})", + ), + all_messages: bool = typer.Option(False, "--all", help="Show the full transcript (every page)"), + reverse: bool = typer.Option( + False, + "--reverse", + help="Newest first (default is chronological: oldest at top)", + ), + show_ids: bool = typer.Option(False, "--ids", help="Include message IDs in the transcript"), + workspace: Optional[str] = typer.Option(None, "--workspace", "-w", help="Override workspace ID"), + peer: Optional[str] = typer.Option(None, "--peer", "-p", help="Filter by peer ID"), + session: Optional[str] = typer.Option(None, "--session", "-s", help="Override session ID"), + json_output: bool = typer.Option(False, "--json", help="Force JSON output"), +) -> None: + """View a session transcript as a chat log. + + Modes (pick one): + + - default / --last N: tail of the conversation (most recent N) + - --page N [--size M]: page through the full transcript + - --all: every message + + Paging follows the requested order: --page 1 starts at the oldest message, + or the newest with --reverse. + + Human mode prints a row-delimited table. JSON mode emits the message list + (same shape as message list). + """ + handle_cmd_flags(json_output=json_output, workspace=workspace, peer=peer, session=session) + sid = _get_session_id(session_id) + + # Validate every flag before touching the network. + modes = sum([ + last is not None, + page_number is not None, + all_messages, + ]) + if modes > 1: + print_error( + "INVALID_FLAGS", + "--last, --page, and --all are mutually exclusive", + {"last": last, "page": page_number, "all": all_messages}, + ) + raise typer.Exit(1) + + if page_number is not None and page_number < 1: + print_error("INVALID_FLAGS", "--page must be >= 1", {"page": page_number}) + raise typer.Exit(1) + if size is not None and page_number is None: + print_error("INVALID_FLAGS", "--size only applies with --page", {"size": size}) + raise typer.Exit(1) + if size is not None and not 1 <= size <= MAX_PAGE_SIZE: + print_error( + "INVALID_FLAGS", + f"--size must be between 1 and {MAX_PAGE_SIZE}", + {"size": size}, + ) + raise typer.Exit(1) + if last is not None and last < 1: + print_error("INVALID_FLAGS", "--last must be >= 1", {"last": last}) + raise typer.Exit(1) + + # Default: tail of conversation (most recent 50). + mode = "page" if page_number is not None else ("all" if all_messages else "last") + tail = last if last is not None else DEFAULT_PAGE_SIZE + page_size = size if size is not None else DEFAULT_PAGE_SIZE + + client, config = get_client() + # Read-only: client.session() is a get-or-create POST, so build the Session directly. + sess = Session(sid, client) + + try: + filters = {"peer_id": config.peer_id} if config.peer_id else None + page_meta: int | None = None + pages_meta: int | None = None + + if mode == "page": + # Page in the order the caller asked for. + result_page = sess.messages( + filters=filters, + page=page_number, + size=page_size, + reverse=reverse, + ) + msgs = list(result_page.items) + total = result_page.total + page_meta = result_page.page if result_page.page is not None else page_number + pages_meta = result_page.pages + elif mode == "all": + msgs, total = _fetch_all_messages(sess, filters) + if reverse: + msgs = list(reversed(msgs)) + else: + # Tail window: fetched newest-first, flipped to chronological unless --reverse. + msgs, total = _fetch_recent_messages(sess, filters, tail) + if not reverse: + msgs = list(reversed(msgs)) + + items = [ + { + "id": m.id, + "peer_id": m.peer_id, + "content": m.content, + "token_count": m.token_count, + "metadata": m.metadata, + "created_at": str(m.created_at), + } + for m in msgs + ] + except Exception as e: + _handle_error(e, "session", sid) + raise # unreachable: _handle_error always exits + + next_page_hint = None + if page_meta is not None and pages_meta is not None and page_meta < pages_meta: + # Effective overrides, not the command-level params: -w/-p also parse at + # group and top level. + overrides = get_flag_overrides() + next_page_hint = _next_page_command( + sid, + page_meta + 1, + page_size, + reverse=reverse, + show_ids=show_ids, + workspace=overrides["workspace"], + peer=overrides["peer"], + ) + + # Rendered outside the try: output failures aren't session API errors. + print_transcript( + items, + session_id=sid, + total=total, + page=page_meta, + pages=pages_meta, + show_ids=show_ids, + next_page_hint=next_page_hint, + ) + + @app.command() def context( session_id: Optional[str] = typer.Argument(None, help="Session ID (uses default if omitted)"), diff --git a/honcho-cli/src/honcho_cli/common.py b/honcho-cli/src/honcho_cli/common.py index d87a4be7..ed1f54ae 100644 --- a/honcho-cli/src/honcho_cli/common.py +++ b/honcho-cli/src/honcho_cli/common.py @@ -52,6 +52,15 @@ def get_resolved_config(): return config +def get_flag_overrides() -> dict[str, str | None]: + """Workspace/peer/session as supplied by ``-w``/``-p``/``-s`` at any level. + + Unlike :func:`get_resolved_config`, this excludes values coming from the + environment or config file. + """ + return dict(_global_overrides) + + def maybe_refresh_token(config: CLIConfig) -> None: """Refresh an expired OAuth access token in place and persist it. diff --git a/honcho-cli/src/honcho_cli/output.py b/honcho-cli/src/honcho_cli/output.py index de7902b8..2f0ec5e9 100644 --- a/honcho-cli/src/honcho_cli/output.py +++ b/honcho-cli/src/honcho_cli/output.py @@ -8,10 +8,12 @@ from __future__ import annotations import json import os import sys +from datetime import datetime, timezone from typing import Any from rich.console import Console from rich.table import Table +from rich.text import Text console = Console(stderr=True) stdout_console = Console() @@ -102,3 +104,112 @@ def print_error(code: str, message: str, details: dict | None = None) -> None: def status(msg: str) -> None: """Print a status message to stderr.""" console.print(f"[dim]{msg}[/dim]") + + +# Stable peer-color palette for transcript rendering. Brand blue first so the +# primary peer lands on brand when there's only one speaker. +_PEER_COLORS = ( + "#B6DAFD", # brand + "#9ccfd8", # foam + "#c4a7e7", # iris + "#ebbcba", # rose + "#f6c177", # gold + "#a3be8c", # pine-ish green + "#ea9a97", # love +) + + +#: Rendered width of :func:`_format_timestamp` output. +TIMESTAMP_WIDTH = len("2026-01-01T00:00:00.000Z") + + +def _format_timestamp(value: Any) -> str: + """Compact UTC timestamp: ``YYYY-MM-DDTHH:MM:SS.mmmZ``. + + Offsets are converted to UTC; naive values are assumed UTC. Unparseable + values pass through verbatim. + """ + if value is None: + return "" + if isinstance(value, datetime): + parsed = value + else: + try: + parsed = datetime.fromisoformat(str(value).strip()) + except ValueError: + return str(value).strip() + if parsed.tzinfo is not None: + parsed = parsed.astimezone(timezone.utc) + return f"{parsed:%Y-%m-%dT%H:%M:%S}.{parsed.microsecond // 1000:03d}Z" + + +def print_transcript( + messages: list[dict[str, Any]], + *, + session_id: str, + total: int | None = None, + page: int | None = None, + pages: int | None = None, + show_ids: bool = False, + next_page_hint: str | None = None, +) -> None: + """Render a session transcript as a row-delimited table, or JSON. + + Each message dict must have ``peer_id``, ``content``, ``created_at``; + ``id`` is optional and only shown when ``show_ids`` is set. + ``next_page_hint`` is printed below the table when given. + """ + if use_json(): + print_json(messages) + return + + shown = len(messages) + parts = [f"session {session_id}"] + if page is not None and pages is not None: + parts.append(f"page {page}/{pages}") + elif page is not None: + parts.append(f"page {page}") + if total is not None and shown != total: + parts.append(f"showing {shown} of {total}") + else: + parts.append(f"{shown} message{'s' if shown != 1 else ''}") + title = " · ".join(parts) + + if not messages: + stdout_console.print(f"[dim]── {title} ──[/dim]") + stdout_console.print("[dim] (empty)[/dim]") + return + + table = Table( + title=title, + show_header=True, + header_style="bold", + show_lines=True, # delimiters between rows + expand=True, + pad_edge=False, + ) + # time is fixed-width ISO-UTC; ids and peers wrap rather than truncate; + # content takes the rest. + table.add_column("time", style="dim", no_wrap=True, width=TIMESTAMP_WIDTH) + if show_ids: + table.add_column("id", style="dim", no_wrap=True) + table.add_column("peer", overflow="fold", max_width=24) + table.add_column("content", overflow="fold", ratio=1, min_width=40) + + peer_color: dict[str, str] = {} + for msg in messages: + peer = str(msg.get("peer_id") or "?") + if peer not in peer_color: + peer_color[peer] = _PEER_COLORS[len(peer_color) % len(_PEER_COLORS)] + + # Text, not Markdown or console markup: content renders verbatim. + row: list[Any] = [_format_timestamp(msg.get("created_at"))] + if show_ids: + row.append(Text(str(msg.get("id") or ""))) + row.append(Text(peer, style=f"bold {peer_color[peer]}")) + row.append(Text(str(msg.get("content") or ""))) + table.add_row(*row) + + stdout_console.print(table) + if next_page_hint: + status(f"more: {next_page_hint}") diff --git a/honcho-cli/tests/test_commands.py b/honcho-cli/tests/test_commands.py index 61de7d47..2fb29256 100644 --- a/honcho-cli/tests/test_commands.py +++ b/honcho-cli/tests/test_commands.py @@ -9,11 +9,13 @@ from __future__ import annotations import json import os +from contextlib import ExitStack, contextmanager from unittest.mock import MagicMock, patch import pytest from typer.testing import CliRunner +from honcho_cli.commands.session import _next_page_command from honcho_cli.main import app @@ -34,6 +36,62 @@ def runner(): return CliRunner() +# --------------------------------------------------------------------------- # +# Helpers for `session view` — a fake SDK message, page, and Session + +def _view_msg(i: int) -> MagicMock: + return MagicMock( + id=f"m{i}", + peer_id="alice" if i % 2 == 0 else "bob", + content=f"msg-{i}", + token_count=i, + metadata={}, + created_at=f"2026-01-01T00:00:00.{i:03d}Z", + ) + + +def _fake_page( + items: list, + *, + total: int | None = None, + page: int | None = None, + pages: int | None = None, + has_next: bool = False, + next_page: MagicMock | None = None, +) -> MagicMock: + """A stand-in for the SDK's ``SyncPage`` with explicit (non-mock) metadata.""" + fake = MagicMock() + fake.items = items + fake.total = total + fake.page = page + fake.pages = pages + fake.has_next_page.return_value = has_next + fake.get_next_page.return_value = next_page + return fake + + +def _fake_session(page: MagicMock) -> MagicMock: + session = MagicMock() + session.messages.return_value = page + return session + + +def _patch_view(session: MagicMock, *, peer_id: str = ""): + """Patch `session view`'s client + read-only Session construction.""" + client = MagicMock() + config = MagicMock(session_id="sess1", workspace_id="ws1", peer_id=peer_id) + return _nested( + patch("honcho_cli.commands.session.get_client", return_value=(client, config)), + patch("honcho_cli.commands.session.Session", return_value=session), + ) + + +@contextmanager +def _nested(*managers): + with ExitStack() as stack: + yield [stack.enter_context(m) for m in managers] + + # --------------------------------------------------------------------------- # # 1. `honcho init` end-to-end @@ -169,6 +227,233 @@ class TestJsonContract: "created_at": "2026-01-01T00:00:00Z", } + @pytest.mark.parametrize("last", ["0", "-5"]) + def test_message_list_rejects_non_positive_last(self, cfg, runner, last): + """Non-positive --last silently returned an empty list via slice semantics.""" + cfg.write_text(json.dumps({"apiKey": "k", "environmentUrl": "http://localhost:8000"})) + config = MagicMock(session_id="sess1", workspace_id="ws1", peer_id="") + with patch("honcho_cli.commands.message.get_client", return_value=(MagicMock(), config)) as get_client: + result = runner.invoke( + app, + ["message", "list", "sess1", "--last", last, "-w", "ws1"], + ) + assert result.exit_code == 1 + assert json.loads(result.stderr)["error"]["code"] == "INVALID_FLAGS" + get_client.assert_not_called() + + def test_session_view_json_is_chronological_window(self, cfg, runner): + """`session view` returns the most recent N messages oldest→newest by default.""" + cfg.write_text(json.dumps({"apiKey": "k", "environmentUrl": "http://localhost:8000"})) + + # Server returns newest-first when reverse=True (m4, m3, m2, m1, m0). + page = _fake_page([_view_msg(i) for i in range(4, -1, -1)], total=5) + session = _fake_session(page) + + with _patch_view(session): + result = runner.invoke( + app, + ["session", "view", "sess1", "--last", "3", "-w", "ws1", "--json"], + ) + + assert result.exit_code == 0, result.stderr + payload = json.loads(result.stdout) + # Most recent 3 (m4,m3,m2) flipped to chronological: m2, m3, m4. + assert [m["id"] for m in payload] == ["m2", "m3", "m4"] + assert [m["content"] for m in payload] == ["msg-2", "msg-3", "msg-4"] + session.messages.assert_called_once() + assert session.messages.call_args.kwargs["reverse"] is True + + def test_session_view_rejects_all_with_last(self, cfg, runner): + cfg.write_text(json.dumps({"apiKey": "k", "environmentUrl": "http://localhost:8000"})) + config = MagicMock(session_id="sess1", workspace_id="ws1", peer_id="") + with patch("honcho_cli.commands.session.get_client", return_value=(MagicMock(), config)): + result = runner.invoke( + app, + ["session", "view", "sess1", "--all", "--last", "10", "-w", "ws1"], + ) + assert result.exit_code == 1 + assert json.loads(result.stderr)["error"]["code"] == "INVALID_FLAGS" + + def test_session_view_page_fetches_exact_server_page(self, cfg, runner): + """`--page N --size M` hits the API page directly (oldest-first).""" + cfg.write_text(json.dumps({"apiKey": "k", "environmentUrl": "http://localhost:8000"})) + + # Page 2 contents, already chronological. + page = _fake_page( + [_view_msg(i) for i in (50, 51, 52)], + total=120, + page=2, + pages=3, + has_next=True, + ) + session = _fake_session(page) + + with _patch_view(session): + result = runner.invoke( + app, + ["session", "view", "sess1", "--page", "2", "--size", "50", "-w", "ws1", "--json"], + ) + + assert result.exit_code == 0, result.stderr + payload = json.loads(result.stdout) + assert [m["id"] for m in payload] == ["m50", "m51", "m52"] + session.messages.assert_called_once_with( + filters=None, + page=2, + size=50, + reverse=False, + ) + + def test_session_view_page_with_reverse_pages_from_newest(self, cfg, runner): + """`--reverse --page N` pages from the newest end, not the oldest one flipped.""" + cfg.write_text(json.dumps({"apiKey": "k", "environmentUrl": "http://localhost:8000"})) + page = _fake_page([_view_msg(i) for i in (9, 8, 7)], total=30, page=1, pages=10) + session = _fake_session(page) + + with _patch_view(session): + result = runner.invoke( + app, + ["session", "view", "sess1", "--page", "1", "--reverse", "-w", "ws1", "--json"], + ) + + assert result.exit_code == 0, result.stderr + assert session.messages.call_args.kwargs["reverse"] is True + # Server order is preserved: no local flip on top of a reversed fetch. + assert [m["id"] for m in json.loads(result.stdout)] == ["m9", "m8", "m7"] + + def test_session_view_rejects_page_with_last(self, cfg, runner): + cfg.write_text(json.dumps({"apiKey": "k", "environmentUrl": "http://localhost:8000"})) + config = MagicMock(session_id="sess1", workspace_id="ws1", peer_id="") + with patch("honcho_cli.commands.session.get_client", return_value=(MagicMock(), config)): + result = runner.invoke( + app, + ["session", "view", "sess1", "--page", "2", "--last", "10", "-w", "ws1"], + ) + assert result.exit_code == 1 + assert json.loads(result.stderr)["error"]["code"] == "INVALID_FLAGS" + + @pytest.mark.parametrize( + "args", + [ + ["--size", "10"], # --size requires --page + ["--page", "1", "--size", "500"], # over the server's 100 ceiling + ["--page", "0"], + ["--last", "0"], + ], + ) + def test_session_view_rejects_bad_flags_before_any_api_call(self, cfg, runner, args): + """Flag validation runs before the client is built, so nothing reaches the API.""" + cfg.write_text(json.dumps({"apiKey": "k", "environmentUrl": "http://localhost:8000"})) + with patch("honcho_cli.commands.session.get_client") as get_client: + result = runner.invoke(app, ["session", "view", "sess1", *args, "-w", "ws1"]) + assert result.exit_code == 1 + assert json.loads(result.stderr)["error"]["code"] == "INVALID_FLAGS" + get_client.assert_not_called() + + def test_session_view_does_not_create_the_session(self, cfg, runner): + """`view` is read-only: it must not use the get-or-create client.session().""" + cfg.write_text(json.dumps({"apiKey": "k", "environmentUrl": "http://localhost:8000"})) + session = _fake_session(_fake_page([_view_msg(1)], total=1)) + client = MagicMock() + config = MagicMock(session_id="sess1", workspace_id="ws1", peer_id="") + + with patch("honcho_cli.commands.session.get_client", return_value=(client, config)), \ + patch("honcho_cli.commands.session.Session", return_value=session) as session_cls: + result = runner.invoke(app, ["session", "view", "sess1", "-w", "ws1", "--json"]) + + assert result.exit_code == 0, result.stderr + client.session.assert_not_called() + session_cls.assert_called_once_with("sess1", client) + + @pytest.mark.parametrize( + ("kwargs", "expected"), + [ + ( + {}, + "honcho session view s1 --page 2 --size 50", + ), + ( + {"reverse": True, "show_ids": True}, + "honcho session view s1 --page 2 --size 50 --reverse --ids", + ), + ( + {"workspace": "ws2", "peer": "alice"}, + "honcho session view s1 --page 2 --size 50 -w ws2 -p alice", + ), + ], + ) + def test_next_page_command_carries_the_invocation_scope(self, kwargs, expected): + """A copied hint must land on the same workspace, peer, and ordering.""" + opts = {"reverse": False, "show_ids": False, "workspace": None, "peer": None, **kwargs} + assert _next_page_command("s1", 2, 50, **opts) == expected + + @pytest.mark.parametrize( + ("session_id", "workspace", "expected_fragment"), + [ + ("has space", None, "'has space'"), + ("a;rm -rf x", None, "'a;rm -rf x'"), + ("s1", "ws$(id)", "'ws$(id)'"), + ("s1", "ws|tee", "'ws|tee'"), + ], + ) + def test_next_page_command_shell_quotes_identifiers( + self, session_id, workspace, expected_fragment + ): + """IDs only reject ?#%/\\ and control chars, so spaces and metacharacters reach here.""" + hint = _next_page_command( + session_id, + 2, + 50, + reverse=False, + show_ids=False, + workspace=workspace, + peer=None, + ) + assert expected_fragment in hint + + def test_session_view_hint_carries_group_level_scope(self, cfg, runner): + """-w/-p also parse at group level, where the command-level params are None.""" + cfg.write_text(json.dumps({"apiKey": "k", "environmentUrl": "http://localhost:8000"})) + session = _fake_session(_fake_page([_view_msg(1)], total=10, page=1, pages=5)) + + with _patch_view(session), patch("honcho_cli.output.use_json", return_value=False): + result = runner.invoke( + app, + ["session", "-w", "ws2", "-p", "alice", "view", "sess1", "--page", "1"], + ) + + assert result.exit_code == 0, result.stderr + assert "-w ws2" in result.stderr + assert "-p alice" in result.stderr + + def test_session_view_last_walks_pages_past_the_page_cap(self, cfg, runner): + """`--last N` above the 100-item server cap keeps walking instead of truncating.""" + cfg.write_text(json.dumps({"apiKey": "k", "environmentUrl": "http://localhost:8000"})) + + # Newest-first pages of 100: m149..m50, then m49..m0. + second = _fake_page([_view_msg(i) for i in range(49, -1, -1)], total=150) + first = _fake_page( + [_view_msg(i) for i in range(149, 49, -1)], + total=150, + has_next=True, + next_page=second, + ) + session = _fake_session(first) + + with _patch_view(session): + result = runner.invoke( + app, + ["session", "view", "sess1", "--last", "120", "-w", "ws1", "--json"], + ) + + assert result.exit_code == 0, result.stderr + payload = json.loads(result.stdout) + assert len(payload) == 120 + # Oldest of the 120-message tail first, newest last. + assert payload[0]["id"] == "m30" + assert payload[-1]["id"] == "m149" + assert session.messages.call_args.kwargs["size"] == 100 + # --------------------------------------------------------------------------- # # 4. Exit codes on error diff --git a/honcho-cli/tests/test_output.py b/honcho-cli/tests/test_output.py new file mode 100644 index 00000000..50352d04 --- /dev/null +++ b/honcho-cli/tests/test_output.py @@ -0,0 +1,139 @@ +"""Transcript rendering: timestamp normalization and content fidelity. + +`session view` is a debugging surface, so the human-mode table must show what +was actually stored — no Markdown reflow, no truncated identifiers. +""" + +from __future__ import annotations + +from datetime import datetime, timezone + +import pytest + +from honcho_cli import output +from honcho_cli.output import _format_timestamp, print_transcript + + +@pytest.fixture +def render(monkeypatch, capsys): + """Render a transcript in human mode at a fixed width and return stdout.""" + monkeypatch.setattr(output, "is_tty", lambda: True) + monkeypatch.setattr(output, "_force_json", False) + + def _render(messages, width: int = 120, **kwargs): + monkeypatch.setattr(output, "stdout_console", output.Console(width=width, no_color=True)) + print_transcript(messages, **kwargs) + return capsys.readouterr().out + + return _render + + +def _msg(content: str = "hi", **overrides) -> dict: + return { + "id": "V1StGXR8_Z5jdHi6B-myT", + "peer_id": "alice", + "content": content, + "created_at": "2026-01-01T00:00:00Z", + **overrides, + } + + +class TestFormatTimestamp: + @pytest.mark.parametrize( + ("value", "expected"), + [ + ("2026-01-01T00:00:00Z", "2026-01-01T00:00:00.000Z"), + ("2026-01-01T00:00:00+00:00", "2026-01-01T00:00:00.000Z"), + ("2026-01-01 00:00:00+00:00", "2026-01-01T00:00:00.000Z"), + ("2026-01-01 00:00:00", "2026-01-01T00:00:00.000Z"), + ("2026-01-01T00:00:00.080000Z", "2026-01-01T00:00:00.080Z"), + ], + ) + def test_normalizes_utc_shapes(self, value, expected): + assert _format_timestamp(value) == expected + + @pytest.mark.parametrize( + ("value", "expected"), + [ + ("2026-01-01T14:30:00-05:00", "2026-01-01T19:30:00.000Z"), + ("2026-01-01T14:30:00+02:00", "2026-01-01T12:30:00.000Z"), + ], + ) + def test_converts_offsets_instead_of_relabelling_them(self, value, expected): + """An offset must be converted to UTC, not dropped and stamped `Z`.""" + assert _format_timestamp(value) == expected + + def test_keeps_sub_second_precision(self): + """Messages inside the same second must stay distinguishable.""" + a = _format_timestamp("2026-01-01T00:00:03.000Z") + b = _format_timestamp("2026-01-01T00:00:03.080Z") + assert a != b + assert (a, b) == ("2026-01-01T00:00:03.000Z", "2026-01-01T00:00:03.080Z") + + def test_accepts_datetime_objects(self): + value = datetime(2026, 1, 1, 12, 0, 0, tzinfo=timezone.utc) + assert _format_timestamp(value) == "2026-01-01T12:00:00.000Z" + + @pytest.mark.parametrize("value", [None, ""]) + def test_empty_values_render_blank(self, value): + assert _format_timestamp(value) == "" + + def test_unparseable_values_pass_through(self): + assert _format_timestamp("not-a-date") == "not-a-date" + + def test_output_matches_the_declared_column_width(self): + assert len(_format_timestamp("2026-01-01T00:00:00Z")) == output.TIMESTAMP_WIDTH + + +class TestTranscriptFidelity: + def test_newlines_are_not_reflowed_into_a_paragraph(self, render): + out = render([_msg("line one\nline two\nline three")], session_id="s1") + assert "line one line two line three" not in out + for line in ("line one", "line two", "line three"): + assert line in out + + def test_tagged_content_is_not_stripped(self, render): + """Agent transcripts are full of ``-style tags; they must survive.""" + out = render([_msg("reasoning answer")], session_id="s1") + assert "" in out + assert "" in out + + def test_console_markup_is_not_interpreted(self, render): + out = render([_msg("literal [bold]not markup[/bold] text")], session_id="s1") + assert "[bold]" in out + + def test_ids_are_shown_in_full(self, render): + """A displayed ID must be usable with `honcho message get`.""" + out = render([_msg()], session_id="s1", show_ids=True) + assert "V1StGXR8_Z5jdHi6B-myT" in out + assert "…" not in out + + def test_long_peer_ids_stay_distinguishable(self, render): + out = render( + [ + _msg(peer_id="user_1234567890abcdef"), + _msg(peer_id="user_1234567890abcXYZ"), + ], + session_id="s1", + ) + assert "abcdef" in out + assert "abcXYZ" in out + + def test_empty_transcript_reports_the_session(self, render): + out = render([], session_id="s1") + assert "s1" in out + assert "(empty)" in out + + +class TestNextPageHint: + def test_given_hint_is_printed(self, render, monkeypatch): + printed: list[str] = [] + monkeypatch.setattr(output, "status", printed.append) + render([_msg()], session_id="s1", page=1, pages=3, next_page_hint="honcho ... --page 2") + assert printed == ["more: honcho ... --page 2"] + + def test_no_hint_when_none_given(self, render, monkeypatch): + printed: list[str] = [] + monkeypatch.setattr(output, "status", printed.append) + render([_msg()], session_id="s1", page=3, pages=3) + assert printed == [] diff --git a/mcp/instructions.md b/mcp/instructions.md index 10cae48d..2abc05ff 100644 --- a/mcp/instructions.md +++ b/mcp/instructions.md @@ -70,6 +70,16 @@ add_messages_to_session --- +## Best Practices + +- **Group messages into coherent context buckets** — give each distinct context its own `session_id` (a chat thread, a project, a channel) and reuse that same `session_id` for every turn within it, rather than minting a new one per turn. Honcho reasons over the messages in a session together, so keeping a context's messages in one bucket produces a coherent representation; scattering them across sessions fragments it. +- **Use one stable `peer_id` per real person**, reused across every session and channel. A fresh or per-channel ID (`user-web` vs. `user-discord`) builds separate, weaker representations instead of one. +- **`observe_me: false` skips building a model of a peer** — reserve it for deterministic bots (nothing meaningful to model). For a real AI assistant it's fine to leave observation on. +- **Reasoning is asynchronous** — don't poll or wait for it to finish before responding. A brand-new or low-volume peer legitimately has little to show yet. +- **Reach for reads before `chat`** — `get_session_context` / `get_peer_context` / `get_representation` / `search` are near-instant; `chat` runs live reasoning and takes a few seconds. Use `chat` only when you need a reasoned answer. + +--- + ## General Tools The full API for advanced use cases. diff --git a/mcp/server.json b/mcp/server.json new file mode 100644 index 00000000..704e788b --- /dev/null +++ b/mcp/server.json @@ -0,0 +1,31 @@ +{ + "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json", + "name": "io.github.plastic-labs/honcho", + "title": "Honcho", + "description": "Memory that reasons: continual learning for stateful agents. Better context, fewer tokens.", + "version": "3.0.0", + "repository": { + "url": "https://github.com/plastic-labs/honcho", + "source": "github" + }, + "remotes": [ + { + "type": "streamable-http", + "url": "https://mcp.honcho.dev", + "headers": [ + { + "name": "Authorization", + "description": "Authorization header for Honcho. Use either an OAuth access token or a Honcho API key from https://app.honcho.dev (API keys start with hch-). Send as: Bearer .", + "isRequired": true, + "isSecret": true + }, + { + "name": "X-Honcho-Workspace-ID", + "description": "Optional. Target Honcho workspace; defaults to 'default' when omitted.", + "isRequired": false, + "isSecret": false + } + ] + } + ] +} diff --git a/mcp/src/instructions.d.ts b/mcp/src/instructions.d.ts new file mode 100644 index 00000000..1cde2fab --- /dev/null +++ b/mcp/src/instructions.d.ts @@ -0,0 +1,5 @@ +// Markdown files are bundled as text strings via the wrangler `Text` rule. +declare module "*.md" { + const content: string; + export default content; +} diff --git a/mcp/src/server.ts b/mcp/src/server.ts index 6bbd5e08..363cf9c0 100644 --- a/mcp/src/server.ts +++ b/mcp/src/server.ts @@ -5,12 +5,16 @@ import { register as registerPeerTools } from "./tools/peers.js"; import { register as registerSessionTools } from "./tools/sessions.js"; import { register as registerConclusionTools } from "./tools/conclusions.js"; import { register as registerSystemTools } from "./tools/system.js"; +import instructions from "../instructions.md"; export function createServer(ctx: ToolContext): McpServer { - const server = new McpServer({ - name: "Honcho MCP Server", - version: "3.0.0", - }); + const server = new McpServer( + { + name: "Honcho MCP Server", + version: "3.0.0", + }, + { instructions }, + ); registerWorkspaceTools(server, ctx); registerPeerTools(server, ctx); diff --git a/mcp/wrangler.toml b/mcp/wrangler.toml index c83d240f..987b0b6b 100644 --- a/mcp/wrangler.toml +++ b/mcp/wrangler.toml @@ -3,6 +3,11 @@ main = "src/index.ts" compatibility_date = "2024-12-09" compatibility_flags = ["nodejs_compat"] +# Bundle Markdown (e.g. instructions.md) as text strings so it can be imported. +rules = [ + { type = "Text", globs = ["**/*.md"], fallthrough = true }, +] + [env.production] name = "honcho-mcp" diff --git a/pyproject.toml b/pyproject.toml index 9427cd88..1c7cc240 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "honcho" -version = "3.0.11" +version = "3.0.12" description = "Honcho Server" authors = [ {name = "Plastic Labs", email = "hello@plasticlabs.ai"}, @@ -8,7 +8,7 @@ authors = [ readme = "README.md" requires-python = ">=3.10" dependencies = [ - "fastapi[standard]>=0.131.0", + "fastapi[standard-no-fastapi-cloud-cli]>=0.131.0", "python-dotenv>=1.0.0", "sqlalchemy>=2.0.30", "fastapi-pagination>=0.14.2", @@ -33,14 +33,17 @@ dependencies = [ "json-repair>=0.49.0", "turbopuffer>=1.8.1", "qdrant-client>=1.18.0", - "lancedb>=0.25.3; sys_platform != \"darwin\" or platform_machine != \"x86_64\"", - "pyarrow>=19.0.0", "redis>=7.0.0,<8.0.0", "cashews[redis]==7.5.0", "scikit-learn>=1.6.0", "prometheus_client>=0.21.0", "cloudevents>=1.12.0,<2.0", ] +[project.optional-dependencies] +lancedb = [ + "lancedb>=0.25.3; sys_platform != \"darwin\" or platform_machine != \"x86_64\"", + "pyarrow>=19.0.0", +] [dependency-groups] dev = [ "pytest>=8.2.2", @@ -54,7 +57,6 @@ dev = [ "pre-commit>=4.2.0", "pytest-cov>=6.2.1", "honcho-ai", - "fakeredis>=2.32.0", "scipy>=1.15.3", "boto3>=1.42.5", "pytest-xdist>=3.8.0", diff --git a/sdks/python/CHANGELOG.md b/sdks/python/CHANGELOG.md index 84843c55..751d9005 100644 --- a/sdks/python/CHANGELOG.md +++ b/sdks/python/CHANGELOG.md @@ -5,6 +5,13 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](http://keepachangelog.com/) and this project adheres to [Semantic Versioning](http://semver.org/). +## [2.3.0] - 2026-08-10 + +### Added + +- `response_format` on `Peer.chat()` / `PeerAio.chat()` and `Peer.chat_stream()` / `PeerAio.chat_stream()`, for constraining a dialectic answer to a schema. Pass a Pydantic model class to get a validated instance back (parsed via `model_validate_json`), or a raw JSON Schema dict to get the JSON string as-is. Overloads type the return precisely, so a model class narrows to that model and a dict narrows to `str`. On the streaming variants, chunks stay raw text that accumulates to a JSON string — parse it after the stream completes. Requires a Honcho server with the matching API support (Honcho v3.0.12+). +- `response_format` field on `DialecticParams`. + ## [2.2.0] - 2026-07-02 ### Added diff --git a/sdks/python/pyproject.toml b/sdks/python/pyproject.toml index 6fe801ff..f2d30457 100644 --- a/sdks/python/pyproject.toml +++ b/sdks/python/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "honcho-ai" -version = "2.2.0" +version = "2.3.0" description = "Official DX Optimized Python SDK for Honcho" dynamic = ["readme"] license = "Apache-2.0" diff --git a/sdks/typescript/CHANGELOG.md b/sdks/typescript/CHANGELOG.md index 179cca72..8d0e5ec1 100644 --- a/sdks/typescript/CHANGELOG.md +++ b/sdks/typescript/CHANGELOG.md @@ -5,6 +5,12 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](http://keepachangelog.com/) and this project adheres to [Semantic Versioning](http://semver.org/). +## [2.3.0] - 2026-08-10 + +### Added + +- `responseFormat` option on `peer.chat()` and `peer.chatStream()`, for constraining a dialectic answer to a schema. Pass a Zod schema to get a parsed, validated result back, or a raw JSON Schema object to get the JSON string as-is. Overloads type the return precisely, so a Zod schema narrows to its inferred type and a plain object narrows to `string`. On `chatStream()`, chunks stay raw text that accumulates to a JSON string — parse it after the stream completes. Requires a Honcho server with the matching API support (Honcho v3.0.12+). + ## [2.2.0] - 2026-07-02 ### Added diff --git a/sdks/typescript/package.json b/sdks/typescript/package.json index 9819ddf9..7db0696c 100644 --- a/sdks/typescript/package.json +++ b/sdks/typescript/package.json @@ -1,6 +1,6 @@ { "name": "@honcho-ai/sdk", - "version": "2.2.0", + "version": "2.3.0", "description": "Official DX Optimized TypeScript SDK for Honcho", "author": "Plastic Labs ", "license": "Apache-2.0", diff --git a/.claude/skills/honcho-cli/SKILL.md b/skills/honcho-cli/SKILL.md similarity index 90% rename from .claude/skills/honcho-cli/SKILL.md rename to skills/honcho-cli/SKILL.md index e2276669..e773cfa0 100644 --- a/.claude/skills/honcho-cli/SKILL.md +++ b/skills/honcho-cli/SKILL.md @@ -1,6 +1,6 @@ --- name: honcho-cli -description: Inspect and debug Honcho workspaces via the `honcho` CLI. Use when investigating peer representations, memory state, session context, queue status, or dialectic quality — any task that requires introspection of a Honcho deployment. +description: Inspect and debug Honcho workspaces via the `honcho` CLI. Use when investigating peer representations, memory state, session context, or dialectic quality — any task that requires introspection of a Honcho deployment, including verifying that a recall/record memory loop is actually working. allowed-tools: Bash(honcho:*), Bash(jq:*), Read, Grep --- @@ -20,7 +20,7 @@ allowed-tools: Bash(honcho:*), Bash(jq:*), Read, Grep - `honcho config` — CLI configuration - `honcho workspace` — inspect, delete, search - `honcho peer` — inspect, card, chat, search -- `honcho session` — inspect, messages, context, summaries +- `honcho session` — inspect, view (transcript), context, summaries - `honcho message` — list and get - `honcho conclusion` — list, search, create, delete @@ -30,7 +30,6 @@ allowed-tools: Bash(honcho:*), Bash(jq:*), Read, Grep - Run `honcho peer inspect` before `honcho peer chat` to understand context. - Use `honcho session context` to see exactly what an agent receives. - Never run `honcho workspace delete` without `honcho workspace inspect` first. -- Check queue status when derivation seems stalled. - Compare peer card with conclusions to understand memory state. ## Inspection tour @@ -62,6 +61,8 @@ honcho conclusion search "topic" --observer --json ```bash honcho session inspect --json +honcho session view --last 20 --json +honcho session view --page 2 --size 50 --json honcho message list --last 20 --json honcho session context --json honcho session summaries --json @@ -82,9 +83,6 @@ honcho peer search "query" --json # Is observation enabled? honcho peer inspect --json | jq '.configuration' -# Is the deriver queue processing messages? -honcho workspace queue-status --json - # What conclusions exist? honcho conclusion list --observer --json honcho conclusion search "expected topic" --observer --json diff --git a/skills/honcho-integration/SKILL.md b/skills/honcho-integration/SKILL.md new file mode 100644 index 00000000..6b6706a2 --- /dev/null +++ b/skills/honcho-integration/SKILL.md @@ -0,0 +1,179 @@ +--- +name: honcho-integration +description: Integrate Honcho memory into existing Python or TypeScript codebases. Use when adding Honcho SDK, setting up peers, configuring sessions, and accessing Honcho's representation. +allowed-tools: Read, Glob, Grep, Bash(uv:*), Bash(bun:*), Bash(npm:*), Edit, Write, WebFetch, AskUserQuestion +--- + +# Honcho Integration Guide + +## What is Honcho + +Honcho is an open source memory library for building stateful agents. It works with any model, framework, or architecture. You send Honcho the messages from your conversations, and custom reasoning models process them in the background — extracting premises, drawing conclusions, and building rich representations of each participant over time. Your agent can then query those representations on-demand ("What does this user care about?", "How technical is this person?") and get grounded, reasoned answers. + +The key mental model: **Peers** are any participant — human or AI. Both are represented the same way. `observe_me` is a peer-level flag (`PeerConfig`) controlling whether Honcho forms a representation of *that* peer; typically you want Honcho to model your users (`observe_me=True`) but not anything with deterministic behavior (`observe_me=False`). `observe_others` is a separate per-peer `SessionPeerConfig` setting that controls whether that peer forms representations of the *other* participants in a session. **Sessions** scope conversations between peers. **Messages** are the raw data you feed in — Honcho reasons about them asynchronously and stores the results as the peer's **representation**. No messages means no reasoning means no memory. + +Your agent accesses this memory through `peer.chat(query)` (ask a natural language question, get a reasoned answer — a few seconds of live reasoning) or `session.context()` (near-instant read of formatted history + representation). Prefer `context()` for per-turn grounding; use `chat()` when you need a reasoned answer. + +## Reference map + +Follow the workflow below. Read a reference file only when you reach the step that needs it: + +| When you're… | Read | +| --- | --- | +| Writing the client/peer/session setup (init, peers, sessions, add messages) | `references/core-patterns.md` | +| Wiring how the AI reads context (tool call, pre-fetch, `context()`, streaming) | `references/agent-patterns.md` | +| Integrating into a bot framework (nanobot, openclaw, picoclaw, …) | `references/bot-frameworks.md` + `references/bot-frameworks//` | + +## Integration Workflow + +Follow these phases in order: + +### Phase 1: Codebase Exploration + +Before asking the user anything, explore the codebase to understand: + +1. **Language & Framework**: Is this Python or TypeScript? What frameworks are used (FastAPI, Express, Next.js, etc.)? +2. **Existing AI/LLM code**: Search for existing LLM integrations (OpenAI, Anthropic, LangChain, etc.) +3. **Entity structure**: Identify users, agents, bots, or other entities that interact +4. **Session/conversation handling**: How does the app currently manage conversations? +5. **Message flow**: Where are messages sent/received? What's the request/response cycle? + +Use Glob and Grep to find: + +- `**/*.py` or `**/*.ts` files with "openai", "anthropic", "llm", "chat", "message" +- User/session models or types +- API routes handling chat or conversation endpoints + +> **Bot framework detected?** If the codebase is built around an agent loop, tool registry, session manager, and message bus (e.g., nanobot, openclaw, picoclaw), read `references/bot-frameworks.md` for framework-specific integration guidance and check `references/bot-frameworks//` for concrete reference implementations. + +### Phase 2: Interview (REQUIRED) + +After exploring the codebase, use the **AskUserQuestion** tool to clarify integration requirements. Ask these questions (adapt based on what you learned in Phase 1): + +#### Question Set 1 - Entities & Peers + +Ask about which entities should be Honcho peers: + +- header: "Peers" +- question: "Which entities should Honcho track and build representations for?" +- options based on what you found (e.g., "End users only", "Users + AI assistant", "Users + multiple AI agents", "All participants including third-party services") +- Include a follow-up if they have multiple AI agents: should any AI peers be observed? + +#### Question Set 2 - Integration Pattern + +Ask how they want to use Honcho context (see `references/agent-patterns.md` for the implementation of each): + +- header: "Pattern" +- question: "How should your AI access Honcho's user context?" +- options: + - "Tool call (Recommended)" - "Agent queries Honcho on-demand via function calling" + - "Pre-fetch" - "Fetch user context before each LLM call with predefined queries" + - "context()" - "Include conversation history and representations in prompt" + - "Multiple patterns" - "Combine approaches for different use cases" + +#### Question Set 3 - Session Structure + +Ask about conversation structure: + +- header: "Sessions" +- question: "How should conversations map to Honcho sessions?" +- options based on their app (e.g., "One session per chat thread", "One session per user", "Multiple users per session (group chat)", "Custom session logic") + +#### Question Set 4 - Specific Queries (if using pre-fetch pattern) + +If they chose pre-fetch, ask what context matters: + +- header: "Context" +- question: "What user context should be fetched for the AI?" +- multiSelect: true +- options: "Communication style", "Expertise level", "Goals/priorities", "Preferences", "Recent activity summary", "Custom queries" + +### Phase 3: Implementation + +Based on interview responses, implement the integration: + +1. Install the SDK (see [Installation](#installation)) +2. Create Honcho client initialization — `references/core-patterns.md` §1 +3. Set up peer creation for identified entities — `references/core-patterns.md` §2–3 +4. Implement the chosen integration pattern(s) — `references/agent-patterns.md` +5. Add message storage after exchanges — `references/core-patterns.md` §4 +6. Update any existing conversation handlers + +### Phase 4: Verification + +- If the Honcho CLI is available, run `honcho doctor` to confirm connectivity before testing the integration code +- Use `honcho peer list` and `honcho peer chat` to verify peers exist and the dialectic endpoint works independently of the integration +- Ensure all message exchanges are stored to Honcho +- Verify deterministic bot peers have `observe_me=False`; AI-assistant peers can keep observation on (it's fine to model them) +- Check that the workspace ID is consistent across the codebase +- Confirm environment variable for API key is documented + +--- + +## Before You Start + +1. **Check the latest SDK versions** at + - Python SDK: `honcho-ai` + - TypeScript SDK: `@honcho-ai/sdk` + +2. **Get an API key** ask the user to get a Honcho API key from and add it to the environment. + +3. **Verify with the CLI** (optional but recommended). If the user has the Honcho CLI installed (`uv install honcho-cli`), they can validate their setup before writing any integration code: + + ```bash + honcho init # persist API key + URL to ~/.honcho/config.json + honcho doctor # verify connectivity, config, workspace health + honcho peer chat # test the dialectic endpoint interactively + ``` + + This is the fastest way to confirm the API key and URL are correct before debugging SDK code. + +## Installation + +### Python (use uv) + +```bash +uv add honcho-ai +``` + +### TypeScript (use bun) + +```bash +bun add @honcho-ai/sdk +``` + +The SDK is sync-by-default in Python (with an `.aio` async namespace) and async-only in TypeScript — match the client to your framework. Full sync/async guidance and the base client/peer/session/message code are in `references/core-patterns.md`. + +## Integration Checklist + +When integrating Honcho into an existing codebase: + +- [ ] Install SDK with `uv add honcho-ai` (Python) or `bun add @honcho-ai/sdk` (TypeScript) +- [ ] Set up `HONCHO_API_KEY` environment variable +- [ ] Initialize Honcho client with a single workspace ID +- [ ] Create peers for all entities (users AND AI assistants) +- [ ] Set `observe_me=False` for deterministic bot peers (optional for AI assistants — fine to leave observation on) +- [ ] Configure sessions with appropriate peer observation settings +- [ ] Choose integration pattern: + - [ ] Tool call pattern for agentic systems + - [ ] Pre-fetch pattern for simpler integrations + - [ ] context() for conversation history +- [ ] Store messages after each exchange to build user models +- [ ] (Optional) Run `honcho doctor` to verify connectivity before testing integration code +- [ ] (Optional) Use `honcho peer chat` to test dialectic queries independently + +## Common Mistakes to Avoid + +1. **Multiple workspaces**: Use ONE workspace per application +2. **Forgetting AI peers**: Create peers for AI assistants, not just users +3. **Modeling bots**: Set `observe_me=False` for deterministic bots (scripted output — nothing meaningful to model). For AI assistants it's fine to leave observation on; turning it off is an optional optimization when you only care about the user. +4. **Not storing messages**: Always call `add_messages()` to feed Honcho's reasoning engine +5. **Blocking on processing**: Messages are processed asynchronously — don't poll or wait for reasoning to complete before continuing + +## Resources + +- Documentation (LLM-friendly index): +- Latest SDK versions: +- API Reference: + +> Tip: append `.md` to any Honcho docs URL to fetch the raw Markdown version. diff --git a/skills/honcho-integration/references/agent-patterns.md b/skills/honcho-integration/references/agent-patterns.md new file mode 100644 index 00000000..ca1865d9 --- /dev/null +++ b/skills/honcho-integration/references/agent-patterns.md @@ -0,0 +1,257 @@ +# Agent Access Patterns + +How your AI accesses Honcho's user context. Pick based on the interview answer to Question Set 2. These build on the client/peer/session setup in `core-patterns.md`. + +- **Pattern A — Dialectic chat as a tool call**: the agent decides when to query context on-demand. +- **Pattern B — Pre-fetch with targeted queries**: fetch a fixed set of attributes before each LLM call. +- **Pattern C — `context()` for LLM integration**: inject conversation history + representation into the prompt. + +> **Speed note.** `chat()` runs live dialectic reasoning (a few seconds) — Patterns A and B call it. `context()` (Pattern C) is a near-instant read. Prefer `context()` for per-turn grounding; reach for `chat()` when you genuinely need a reasoned answer. + +## Pattern A: Dialectic Chat as a Tool Call + +Make Honcho's chat endpoint available as a **tool** for your AI agent. This lets the agent query user context on-demand. + +**Python (OpenAI function calling):** + +```python +import openai +from honcho import Honcho + +honcho = Honcho(workspace_id="my-app", api_key=os.environ["HONCHO_API_KEY"]) + +# Define the tool for your agent +honcho_tool = { + "type": "function", + "function": { + "name": "query_user_context", + "description": "Query Honcho to retrieve relevant context about the user based on their history and preferences. Use this when you need to understand the user's background, preferences, past interactions, or goals.", + "parameters": { + "type": "object", + "properties": { + "query": { + "type": "string", + "description": "A natural language question about the user, e.g. 'What are this user's main goals?' or 'What communication style does this user prefer?'" + } + }, + "required": ["query"] + } + } +} + +def handle_honcho_tool_call(user_id: str, query: str) -> str: + """Execute the Honcho chat tool call.""" + peer = honcho.peer(user_id) + return peer.chat(query) + +# Use in your agent loop +def run_agent(user_id: str, user_message: str): + messages = [{"role": "user", "content": user_message}] + + response = openai.chat.completions.create( + model="gpt-4", + messages=messages, + tools=[honcho_tool] + ) + + # Handle tool calls + if response.choices[0].message.tool_calls: + for tool_call in response.choices[0].message.tool_calls: + if tool_call.function.name == "query_user_context": + import json + args = json.loads(tool_call.function.arguments) + result = handle_honcho_tool_call(user_id, args["query"]) + # Continue conversation with tool result... +``` + +**TypeScript (OpenAI function calling):** + +```typescript +import OpenAI from 'openai'; +import { Honcho } from '@honcho-ai/sdk'; + +const honcho = new Honcho({ + workspaceId: "my-app", + apiKey: process.env.HONCHO_API_KEY +}); + +const honchoTool: OpenAI.ChatCompletionTool = { + type: "function", + function: { + name: "query_user_context", + description: "Query Honcho to retrieve relevant context about the user based on their history and preferences.", + parameters: { + type: "object", + properties: { + query: { + type: "string", + description: "A natural language question about the user" + } + }, + required: ["query"] + } + } +}; + +async function handleHonchoToolCall(userId: string, query: string): Promise { + const peer = await honcho.peer(userId); + return await peer.chat(query); +} +``` + +## Pattern B: Pre-fetch Context with Targeted Queries + +For simpler integrations, fetch user context before the LLM call using pre-defined queries. + +**Python:** + +```python +def get_user_context_for_prompt(user_id: str) -> dict: + """Fetch key user attributes via targeted Honcho queries.""" + peer = honcho.peer(user_id) + + return { + "communication_style": peer.chat("What communication style does this user prefer? Be concise."), + "expertise_level": peer.chat("What is this user's technical expertise level? Be concise."), + "current_goals": peer.chat("What are this user's current goals or priorities? Be concise."), + "preferences": peer.chat("What key preferences should I know about this user? Be concise.") + } + +def build_system_prompt(user_context: dict) -> str: + return f"""You are a helpful assistant. Here's what you know about this user: + +Communication style: {user_context['communication_style']} +Expertise level: {user_context['expertise_level']} +Current goals: {user_context['current_goals']} +Key preferences: {user_context['preferences']} + +Tailor your responses accordingly.""" +``` + +**TypeScript:** + +```typescript +async function getUserContextForPrompt(userId: string): Promise> { + const peer = await honcho.peer(userId); + + const [style, expertise, goals, preferences] = await Promise.all([ + peer.chat("What communication style does this user prefer? Be concise."), + peer.chat("What is this user's technical expertise level? Be concise."), + peer.chat("What are this user's current goals or priorities? Be concise."), + peer.chat("What key preferences should I know about this user? Be concise.") + ]); + + return { + communicationStyle: style, + expertiseLevel: expertise, + currentGoals: goals, + preferences: preferences + }; +} +``` + +## Pattern C: Get Context for LLM Integration + +Use `context()` for conversation history with built-in LLM formatting. This is a near-instant read so it's the cheapest way to ground each turn. + +**Python:** + +```python +import openai + +session = honcho.session("conversation-123") +user = honcho.peer("user-123") +assistant = honcho.peer("assistant") + +# Get context formatted for your LLM +context = session.context( + tokens=2000, + peer_target=user.id, # Include representation of this user + summary=True # Include conversation summaries +) + +# Convert to OpenAI format +messages = context.to_openai(assistant=assistant) + +# Or Anthropic format +# messages = context.to_anthropic(assistant=assistant) + +# Add the new user message +messages.append({"role": "user", "content": "What should I focus on today?"}) + +response = openai.chat.completions.create( + model="gpt-4", + messages=messages +) + +# Store the exchange +session.add_messages([ + user.message("What should I focus on today?"), + assistant.message(response.choices[0].message.content) +]) +``` + +**TypeScript:** + +```typescript +import OpenAI from 'openai'; + +const session = await honcho.session("conversation-123"); +const user = await honcho.peer("user-123"); +const assistant = await honcho.peer("assistant"); + +// Get context formatted for your LLM +const context = await session.context({ + tokens: 2000, + peerTarget: user.id, // Include representation of this user + summary: true // Include conversation summaries +}); + +// Convert to OpenAI format +const messages = context.toOpenAI(assistant); + +// Or Anthropic format +// const messages = context.toAnthropic(assistant); + +// Add the new user message +messages.push({ role: "user", content: "What should I focus on today?" }); + +const openai = new OpenAI(); +const response = await openai.chat.completions.create({ + model: "gpt-4", + messages +}); + +// Store the exchange +await session.addMessages([ + user.message("What should I focus on today?"), + assistant.message(response.choices[0].message.content!) +]); +``` + +### What `context()` returns + +`session.context()` bundles the session-local view you can drop straight into an LLM call: + +- **Recent messages** from the session, trimmed to the `tokens` budget. +- **Conversation summaries** when `summary=True` — the two-tier short/long summaries so older turns still count without spending the full token budget. +- **The target peer's representation** when you pass `peer_target` — Honcho's synthesized understanding of that user, folded in. Omit it and you get session-local context only (no cross-session memory). + +The `to_openai()` / `to_anthropic()` helpers format all of that as a `messages` array for the respective provider. Pass your `assistant` peer so its turns are tagged as the assistant role. + +## Streaming Responses + +```python +stream = peer.chat_stream("What do we know about this user?") + +for chunk in stream: + print(chunk, end="", flush=True) +``` + +```typescript +const stream = await peer.chatStream("What do we know about this user?"); + +for await (const chunk of stream) { + process.stdout.write(chunk); +} +``` diff --git a/.claude/skills/honcho-integration/references/bot-frameworks.md b/skills/honcho-integration/references/bot-frameworks.md similarity index 97% rename from .claude/skills/honcho-integration/references/bot-frameworks.md rename to skills/honcho-integration/references/bot-frameworks.md index 16e7c24b..874217d0 100644 --- a/.claude/skills/honcho-integration/references/bot-frameworks.md +++ b/skills/honcho-integration/references/bot-frameworks.md @@ -127,7 +127,7 @@ Key patterns (shared): - IDs sanitized to `^[a-zA-Z0-9_-]+` (Honcho requirement) - User peer: `observe_me=True, observe_others=True` -- Assistant peer: `observe_me=False, observe_others=True` +- Assistant/bot peer: `observe_others=True`; set `observe_me=False` only for deterministic bots (scripted output — nothing to model). AI-assistant bots can keep `observe_me=True`. If references exist for this framework, use them directly from `{baseDir}/references/bot-frameworks//`. diff --git a/.claude/skills/honcho-integration/references/bot-frameworks/nanobot/client.py b/skills/honcho-integration/references/bot-frameworks/nanobot/client.py similarity index 100% rename from .claude/skills/honcho-integration/references/bot-frameworks/nanobot/client.py rename to skills/honcho-integration/references/bot-frameworks/nanobot/client.py diff --git a/.claude/skills/honcho-integration/references/bot-frameworks/nanobot/honcho_tool.py b/skills/honcho-integration/references/bot-frameworks/nanobot/honcho_tool.py similarity index 100% rename from .claude/skills/honcho-integration/references/bot-frameworks/nanobot/honcho_tool.py rename to skills/honcho-integration/references/bot-frameworks/nanobot/honcho_tool.py diff --git a/.claude/skills/honcho-integration/references/bot-frameworks/nanobot/session.py b/skills/honcho-integration/references/bot-frameworks/nanobot/session.py similarity index 98% rename from .claude/skills/honcho-integration/references/bot-frameworks/nanobot/session.py rename to skills/honcho-integration/references/bot-frameworks/nanobot/session.py index b5ebc081..f340f2a8 100644 --- a/.claude/skills/honcho-integration/references/bot-frameworks/nanobot/session.py +++ b/skills/honcho-integration/references/bot-frameworks/nanobot/session.py @@ -137,7 +137,10 @@ class HonchoSessionManager: session = self.honcho.session(session_id) - # Configure peer observation settings + # Configure peer observation settings. + # observe_me=False on the assistant skips modeling it — optional for an + # AI assistant (fine to leave observation on); it's really only needed + # for deterministic bots, where there's nothing meaningful to model. from honcho.api_types import SessionPeerConfig user_config = SessionPeerConfig(observe_me=True, observe_others=True) ai_config = SessionPeerConfig(observe_me=False, observe_others=True) diff --git a/skills/honcho-integration/references/core-patterns.md b/skills/honcho-integration/references/core-patterns.md new file mode 100644 index 00000000..2c6a84df --- /dev/null +++ b/skills/honcho-integration/references/core-patterns.md @@ -0,0 +1,157 @@ +# Core Integration Patterns + +The base SDK boilerplate for any Honcho integration: choosing sync vs async, initializing the client, creating peers, configuring sessions, and adding messages. Read this once you've chosen your entities and session structure (Phases 1–2). For the agent-facing recall patterns (tool call, pre-fetch, `context()`), see `agent-patterns.md`. + +## Sync vs Async + +**TypeScript** — The SDK is async by default. All methods return promises. No separate sync API. + +**Python** — The SDK provides both sync and async interfaces: + +- **Sync** (default): `from honcho import Honcho` — use in sync frameworks (Flask, Django, CLI scripts) +- **Async**: `from honcho import Honcho` with `.aio` namespace — use in async frameworks (FastAPI, Starlette, async workers) + +```python +# Sync usage (Flask, Django, scripts) +from honcho import Honcho +honcho = Honcho(workspace_id="my-app", api_key=os.environ["HONCHO_API_KEY"]) +peer = honcho.peer("user-123") +response = peer.chat("What does this user prefer?") + +# Async usage (FastAPI, Starlette) +from honcho import Honcho +honcho = Honcho(workspace_id="my-app", api_key=os.environ["HONCHO_API_KEY"]) +peer = await honcho.aio.peer("user-123") +response = await peer.aio.chat("What does this user prefer?") +``` + +Match the client to the framework — check whether the codebase uses `async def` handlers or sync `def` handlers and choose accordingly. The examples below show sync Python; swap to `.aio` equivalents for async codebases. + +## 1. Initialize with a Single Workspace + +Use ONE workspace for your entire application. The workspace name should reflect your app/product. + +**Python:** + +```python +from honcho import Honcho +import os + +# Sync client (Flask, Django, scripts) +honcho = Honcho( + workspace_id="your-app-name", + api_key=os.environ["HONCHO_API_KEY"], + environment="production" +) + +# Async client (FastAPI, Starlette) — use honcho.aio for all operations +# honcho.aio.peer(), honcho.aio.session(), etc. +``` + +**TypeScript:** + +```typescript +import { Honcho } from '@honcho-ai/sdk'; + +// All methods are async by default +const honcho = new Honcho({ + workspaceId: "your-app-name", + apiKey: process.env.HONCHO_API_KEY, + environment: "production" +}); +``` + +## 2. Create Peers for ALL Entities + +Create peers for **every entity** in your business logic - users AND AI assistants. + +**Python:** + +```python +from honcho.api_types import PeerConfig + +# Human users (observed by default) +user = honcho.peer("user-123") + +# AI assistants can be observed too — leave observe_me on (the default) if you +# want a model of the assistant. +assistant = honcho.peer("assistant") + +# Deterministic bots (scripted/rule-based) - set observe_me=False; there's +# nothing meaningful for Honcho to model. +notification_bot = honcho.peer("notification-bot", configuration=PeerConfig(observe_me=False)) +``` + +**TypeScript:** + +```typescript +// Human users (observed by default) +const user = await honcho.peer("user-123"); + +// AI assistants can be observed too — leave observeMe on (the default) if you +// want a model of the assistant. +const assistant = await honcho.peer("assistant"); + +// Deterministic bots (scripted/rule-based) - set observeMe=false; there's +// nothing meaningful for Honcho to model. +const notificationBot = await honcho.peer("notification-bot", { configuration: { observeMe: false } }); +``` + +## 3. Multi-Peer Sessions + +Sessions can have multiple participants. Configure observation settings per-peer. + +**Python:** + +```python +from honcho.api_types import SessionPeerConfig + +session = honcho.session("conversation-123") + +# User is observed (Honcho builds a model of them) +user_config = SessionPeerConfig(observe_me=True, observe_others=True) + +# A deterministic bot is NOT observed (no model built of it). An AI assistant +# could stay observed instead — observe_me defaults to True. +bot_config = SessionPeerConfig(observe_me=False, observe_others=True) + +session.add_peers([ + (user, user_config), + (notification_bot, bot_config) +]) +``` + +**TypeScript:** + +```typescript +const session = await honcho.session("conversation-123"); + +await session.addPeers([ + // A deterministic bot isn't observed; an AI assistant could stay observed + // instead (observeMe defaults to true). + [user, { observeMe: true, observeOthers: true }], + [notificationBot, { observeMe: false, observeOthers: true }] +]); +``` + +## 4. Add Messages to Sessions + +**Python:** + +```python +session.add_messages([ + user.message("I'm having trouble with my account"), + assistant.message("I'd be happy to help. What seems to be the issue?"), + user.message("I can't reset my password") +]) +``` + +**TypeScript:** + +```typescript +await session.addMessages([ + user.message("I'm having trouble with my account"), + assistant.message("I'd be happy to help. What seems to be the issue?"), + user.message("I can't reset my password") +]); +``` diff --git a/skills/honcho-memory/SKILL.md b/skills/honcho-memory/SKILL.md new file mode 100644 index 00000000..4c19decc --- /dev/null +++ b/skills/honcho-memory/SKILL.md @@ -0,0 +1,89 @@ +--- +name: honcho-memory +description: Concepts and strategy for using a connected Honcho as persistent memory of the user — the recall/record loop and session and peer design. Start here to understand how Honcho memory works, then connect — via a first-class integration for your environment if one exists (preferred), or raw MCP tools (covered here) or the honcho-cli skill (CLI). For embedding the SDK into a codebase, use honcho-integration. +--- + +# Using Honcho as Memory + +Honcho is a memory layer for agents. You feed it the messages from your conversations; in the background it reasons over them and builds a **representation** of each participant. At any point you can ask it natural-language questions about the user ("How technical are they?", "What are they trying to do?") and get grounded, reasoned answers. + +This skill is for when Honcho is **already connected** to you and you want to use it. If you're instead adding Honcho to a codebase from scratch, use the `honcho-integration` skill. + +> **What's durable vs. what to look up.** The concepts and the recall/record loop below change rarely — rely on them. Specifics that change often — the exact set of integrations, tool names, install commands, headers, and defaults — are illustrative here; treat the linked docs (and your own live tool list) as authoritative and fetch them when the details matter. + +## The mental model + +- **Peer** — any participant, human or AI. You and the user are both peers. Honcho builds a representation of peers it observes (typically the user, not you). +- **Session** — one conversation thread; messages live in sessions. Honcho reasons over the messages in a session together, so scope each session to one coherent context (a conversation, channel, task, or project) and keep that context's turns in the same session rather than fragmenting them across many thin ones. For low-volume or trickle inputs, append to one ongoing session rather than spinning up a new one each time. See [design patterns](https://honcho.dev/docs/v3/documentation/core-concepts/design-patterns.md) and [reasoning](https://honcho.dev/docs/v3/documentation/core-concepts/reasoning.md). +- **Message** — the raw turns you feed in. No messages → no reasoning → no memory. +- **Conclusion** — a fact Honcho derived (or you stored) about a peer. Conclusions power the representation. +- **Representation / peer card** — the synthesized understanding of a peer, queryable via `chat`. A peer's representation **accumulates across every session** it appears in — that's the cross-conversation memory. Session-scoped data (recent messages, summaries) stays local to one session. + +Reasoning happens **asynchronously**. After you record a turn, don't poll or wait — the representation updates in the background and is richer next time you ask. + +## The loop: recall → respond → record + +Do this every conversation. It's the whole skill. + +1. **Once per conversation** — make sure there's a session with you and the user as peers (observe the user, don't observe yourself). +2. **Before responding, when personalization helps** — pull the user's current context (`get_session_context` / `get_representation`) or search past messages (`search`) — these are fast reads. For a reasoned answer to a specific question, ask the dialectic (`chat`) — that one takes a few seconds, so use it when it earns its keep. +3. **After every exchange** — record both the user's message and your reply. This is what makes Honcho learn. Don't skip it. + +Optionally, when you learn a durable fact you don't want to wait for background reasoning to surface, **store a conclusion** directly. + +## What you get back when you recall + +Three ways to pull memory, cheapest first: + +- **Representation** (`get_representation`) — Honcho's synthesized understanding of the user as text, ready to drop straight into a system prompt. Near-instant read. +- **Context** (`get_session_context`) — the fuller session view: a session summary + recent messages covering the conversation, and — *only if you target a peer* — that peer's representation folded in. Without a peer target it's session-local (recent turns + summary) and carries no cross-conversation memory. Near-instant read. +- **Dialectic** (`chat`) — a *reasoned* natural-language answer to a specific question ("How does this user like to receive feedback?"). Runs live reasoning, so it takes a few seconds. Use it when a plain read won't answer the question. + +The dialectic (`chat`) also takes a **reasoning level** that trades speed for depth — from `minimal` (fast factual lookup) through `low` (the default balance) to `max` (deep synthesis for the hardest questions). Pick the lowest level that answers the question; higher levels are slower and cost more. The full level-by-level table and model routing are in the [chat docs](https://honcho.dev/docs/v3/documentation/features/chat.md). + +--- + +## Pick your access path + +The loop is the same; the mechanics depend on how you reach Honcho. **Prefer a purpose-built integration over wiring up raw MCP yourself** — they handle sessions, peers, and the record loop for you, stay current, and are tuned per environment. + +1. **A first-class integration exists for your environment? Use it.** In Claude Code, install the [Claude Code plugin](https://honcho.dev/docs/v3/guides/integrations/claude-code.md) (`/plugin marketplace add plastic-labs/claude-honcho`) for persistent memory out of the box; there are also plugins/integrations for [OpenCode](https://honcho.dev/docs/v3/guides/integrations/opencode.md), LangGraph, CrewAI, Discord, and more. Browse the always-current list: . +2. **No integration, but you have MCP tools** (`create_session`, `add_messages_to_session`, `chat`, …) → drive them with the loop above. The MCP server injects its own usage guide on connect, so there's nothing extra to load; to connect a client yourself, see [Setup](#setup-if-not-connected-yet) below. This is the fallback for connected agents. +3. **`honcho` CLI available** in a terminal → use the **`honcho-cli`** skill — for the recall/record loop, and for verifying that memory is actually building (did messages land? is the representation growing? why doesn't it remember me?). +4. **Embedding Honcho into your own codebase** (not just using a connected instance) → use the **`honcho-integration`** skill. + +If you're unsure, list your available tools and look for Honcho memory tools (an MCP connection) before falling back to the CLI. Even on the MCP path, the `honcho-cli` skill is the best way to **verify the loop is working** if memory seems off. + +--- + +## Setup (if not connected yet) + +You need a Honcho API key — get one free at (starts with `hch-`). Then connect via the path you picked above — a purpose-built integration (recommended), or a raw connection: + +- **MCP** — point your client at `https://mcp.honcho.dev` with two headers: `Authorization: Bearer hch-your-key-here` and `X-Honcho-User-Name: YourName` (what Honcho should call the user). Optional: `X-Honcho-Assistant-Name` (default `Assistant`) and `X-Honcho-Workspace-ID` (default `default`; set it to isolate memory per project). Restart the client fully after adding config. Per-client config snippets (Claude Desktop, Cursor, Codex, Windsurf, VS Code, Cline, Zed) are in the [MCP integration guide](https://honcho.dev/docs/v3/guides/integrations/mcp.md). Once connected, the server tells your assistant how to use the tools automatically. +- **CLI** — use the `honcho-cli` skill. + +--- + +## Rules of thumb + +- **Always record turns.** Memory only grows from messages you feed in. Recording is the one non-optional step. +- **Modeling the assistant is optional.** Setting `observe_me: false` on the assistant peer skips building a model of it — required only for deterministic bots (scripted output, nothing meaningful to model). For an AI assistant it's fine to leave observation on if you also want a model of the agent. +- **One stable peer ID per entity.** Reuse the same `peer_id` for a person across every session and channel; splitting them (`user`, `user-web`, `user-discord`) builds separate representations and fragments memory. +- **Scope sessions to coherent context buckets.** Honcho reasons over a session's messages together. Scope a session to one active interaction (per-conversation, per-channel, per-task, per-project); create a new one when context genuinely resets (new topic, new day), reuse it while context should keep accumulating. Keeping a context's turns in one session produces a coherent representation; scattering them fragments it. +- **Don't block on reasoning.** It's asynchronous. Respond now; the representation will be richer next time. +- **Reads are cheap; reasoning isn't.** Fetching the representation/context (`get_session_context`, `get_representation`, `search`) is a near-instant read — use it freely. The dialectic (`chat`) runs live reasoning and takes a few seconds, so save it for when you genuinely need a reasoned answer, not every turn. +- **Check before you store.** Background reasoning derives most conclusions automatically. Store a conclusion manually only for a durable fact you want available immediately; `list`/`query` first to avoid duplicates. +- **One workspace per app/user-context.** Don't scatter the same user's memory across multiple workspaces. +- **Unify memory across tools with a shared workspace + peer ID.** To give one user continuous memory across several apps or agents (e.g. Claude Code, Cursor, your own app), point them at the same workspace and reuse the same peer ID — that shared ID is what links the representation. See [Unified Memory Setup](https://honcho.dev/docs/v3/guides/recipes/unified-memory-setup.md). + +## Resources + +These are the LLM-friendly Markdown versions (append `.md` to any Honcho docs URL to get the raw Markdown; the full machine-readable index is at ). + +- Full docs index (for agents): +- All integrations & plugins: +- MCP server & client setup: +- Full MCP usage walkthrough: +- Agent development overview: +- CLI reference: diff --git a/skills/verify/SKILL.md b/skills/verify/SKILL.md new file mode 100644 index 00000000..0a83fa1a --- /dev/null +++ b/skills/verify/SKILL.md @@ -0,0 +1,103 @@ +--- +name: verify +description: Build, launch, and drive a local Honcho stack to verify a change at its runtime surface (the /v3 HTTP API and the deriver queue). Use when verifying a diff or confirming a change works in the running app. +--- + +# Verifying changes in Honcho + +## Prerequisites + +Docker Compose is the preferred way to run the stack — `docker-compose.yml` at +the repo root brings up Postgres (pgvector), Redis, the API server, and the +deriver worker together. If Docker Compose isn't available, the stack can also +run directly on the host (see Launch below); you'll need Postgres with pgvector +and Redis reachable, plus a `.env` with connection strings and an LLM provider +key for any flow that hits a model (deriver, dialectic, dreamer). + +Working in a worktree? It carries neither `.env` nor `node_modules`. Copy +`.env` from the main checkout (that's where the provider keys live), and run +`bun install` in `sdks/typescript` if you'll run the full test suite — +otherwise the pre-push gate fails on a phantom `Cannot find package 'zod'`. + +## Launch + +First check whether the stack is already running via Docker Compose: + +```bash +docker compose ps # look for api / deriver / database / redis +docker compose up -d # start it if not +``` + +A running stack is not the same as *your branch's code* running — check the +CREATED column; the images may be weeks old. For verifying a diff, the cheap +path is to reuse the stack's Postgres/Redis containers but run the branch's +API as a host process on a spare port: + +```bash +uv run uvicorn src.main:app --port 8901 # branch code, stack's DB/Redis +uv run python -m src.deriver # if the diff touches the worker +``` + +This avoids both an image rebuild and the port/project-name conflicts a second +compose stack in a worktree would cause. Without Docker at all, run the two +processes the same way against host Postgres + Redis (API default port 8000 +via `uv run fastapi dev src/main.py`). + +When reading server logs, note that with the main `.env` the telemetry emitter +spams connection warnings at an unreachable endpoint — `grep -v +telemetry.emitter` before looking for the real error. + +## Drive it + +Prefer driving through the `honcho-cli` skill; fall back to the SDKs +(`sdks/python`, `sdks/typescript`), then raw REST against `/v3`, in that order +when a method isn't available at the higher level. The `honcho-integration` +skill covers how to use the SDKs. + +When a change updates endpoints or makes schema changes, verify across all +three surfaces — CLI, SDKs, and REST — since they can drift independently. + +For LLM-path changes, the fastest synchronous surface is dialectic at the +`minimal` reasoning level: send a couple of messages, then hit +`/v3/.../peers/{peer_id}/chat` and observe the response. + +## Configuration + +Configuration is central to both driving the app and running tests: it's how +API keys reach the server and how a config-related change gets exercised at +all. Settings come from environment variables or files, with precedence +env > `.env` > `config.toml` > defaults. To verify a configuration change, +set the relevant option through one of these layers, restart the affected +process, and observe the behavioral difference at the surface — the same +mechanism lets you point provider base URLs, model choices, and timeouts at +credentials and proxies you actually have. + +Two concrete levers: deep-nested settings (e.g. +`[dialectic.levels.minimal.model_config.overrides.provider_params]`) are +miserable as env vars — drop a partial `config.toml` in the repo root instead. +And for load-time config behavior, `uv run python -c "import src.config; ..."` +is faster than booting the server. + +## Test suites + +Three test types matter here. All run in CI, but they're also runnable locally +with whatever API keys and configuration you have — Honcho's config surface is +large, so options like per-agent timeouts and provider base URLs can be pointed +at your own keys/proxies to exercise a change: + +```bash +# Unit tests (pytest; spins up its own infra via fixtures) +uv run pytest tests/ + +# Unified tests — step-based end-to-end flows defined in JSON +# (config hierarchy, multi-turn interactions, LLM-as-judge assertions) +uv run python -m tests.unified.run +uv run python -m tests.unified.run --test-dir tests/unified/test_cases + +# Live LLM tests — real provider calls, for testing specific backends. +# Needs provider API keys AND model vars — without LIVE_LLM_*_MODELS the +# tests silently deselect that provider. See tests/live_llm/README.md. +export LLM_ANTHROPIC_API_KEY=... +export LIVE_LLM_ANTHROPIC_45_PLUS_MODELS=claude-sonnet-4-5 +uv run pytest tests/live_llm -n 0 --live-llm --no-header -q +``` diff --git a/src/config.py b/src/config.py index 91e5c84d..c6514d48 100644 --- a/src/config.py +++ b/src/config.py @@ -1,7 +1,9 @@ import logging +import math import os from pathlib import Path from typing import Annotated, Any, ClassVar, Literal, cast +from urllib.parse import urlparse import tomllib from dotenv import load_dotenv @@ -25,12 +27,17 @@ logger = logging.getLogger(__name__) ModelTransport = Literal["anthropic", "openai", "gemini"] EmbeddingTransport = Literal["openai", "gemini"] EmbeddingDimensionsMode = Literal["auto", "always", "never"] +EmbeddingEncodingFormat = Literal["float", "base64"] +EmbeddingEncodingFormatMode = Literal["auto", "float", "base64"] # OpenAI-compatible models that reject the `dimensions=` request parameter. _EMBEDDING_KNOWN_REJECTING_MODELS: frozenset[str] = frozenset( {"text-embedding-ada-002"} ) +# Hosts known to serve base64 embeddings, which are ~3.6x smaller on the wire. +_EMBEDDING_BASE64_CAPABLE_HOSTS: frozenset[str] = frozenset({"api.openai.com"}) + def _default_embedding_model_for_transport(transport: EmbeddingTransport) -> str: if transport == "gemini": @@ -66,6 +73,37 @@ ThinkingEffortLevel = Literal[ StructuredOutputMode = Literal["json_schema", "json_object"] +PROVIDER_TIMEOUT_ERROR_TEXT = ( + "provider_params.timeout must be a positive number of seconds" +) + + +def coerce_provider_timeout(value: Any) -> float: + """Coerce a `provider_params.timeout` value to positive, finite seconds. + + Canonical implementation shared by config-load validation (here) and + per-request validation (`src.llm.request_builder.request_timeout_from_extra_params`, + which translates the ValueError into a ValidationException). Lives in + config.py because src.exceptions imports src.config, so config validators + cannot raise Honcho exception types. + """ + if isinstance(value, bool): + raise ValueError(PROVIDER_TIMEOUT_ERROR_TEXT) + if isinstance(value, int | float): + timeout = float(value) + elif isinstance(value, str): + try: + timeout = float(value.strip()) + except ValueError as exc: + raise ValueError(PROVIDER_TIMEOUT_ERROR_TEXT) from exc + else: + raise ValueError(PROVIDER_TIMEOUT_ERROR_TEXT) + + if not math.isfinite(timeout) or timeout <= 0: + raise ValueError(PROVIDER_TIMEOUT_ERROR_TEXT) + return timeout + + class ModelOverrideSettings(BaseModel): """Advanced module-level transport overrides.""" @@ -91,6 +129,14 @@ class ModelOverrideSettings(BaseModel): ), ) + @field_validator("provider_params") + @classmethod + def _validate_provider_timeout(cls, v: dict[str, Any]) -> dict[str, Any]: + """Reject bad `timeout` values at config load; normalize good ones to float.""" + if "timeout" not in v: + return v + return {**v, "timeout": coerce_provider_timeout(v["timeout"])} + class PromptCachePolicy(BaseModel): """Per-call prompt-caching configuration. @@ -346,6 +392,8 @@ class ConfiguredEmbeddingModelSettings(BaseModel): transport: EmbeddingTransport = "openai" overrides: ModelOverrideSettings = Field(default_factory=ModelOverrideSettings) dimensions_mode: EmbeddingDimensionsMode = "auto" + encoding_format_mode: EmbeddingEncodingFormatMode = "auto" + max_batch_size: Annotated[int, Field(gt=0)] | None = None @model_validator(mode="before") @classmethod @@ -382,6 +430,7 @@ class EmbeddingModelConfig(BaseModel): transport: EmbeddingTransport = "openai" api_key: str | None = None base_url: str | None = None + max_batch_size: Annotated[int, Field(gt=0)] | None = None @model_validator(mode="before") @classmethod @@ -506,6 +555,7 @@ def resolve_embedding_model_config( transport=configured.transport, api_key=api_key, base_url=configured.overrides.base_url, + max_batch_size=configured.max_batch_size, ) @@ -787,6 +837,22 @@ class EmbeddingSettings(HonchoSettings): return False return "VECTOR_DIMENSIONS" in self.model_fields_set + def resolve_encoding_format(self) -> EmbeddingEncodingFormat: + """Pick the ``encoding_format`` for OpenAI embedding calls. + + ``auto`` keeps the compact base64 wire format on hosts known to support + it and falls back to float elsewhere, since OpenAI-compatible providers + may answer a base64 request with an error or empty data. + """ + mode = self.MODEL_CONFIG.encoding_format_mode + if mode != "auto": + return mode + base_url = self.MODEL_CONFIG.overrides.base_url + if not base_url: + return "base64" + host = urlparse(base_url).hostname + return "base64" if host in _EMBEDDING_BASE64_CAPABLE_HOSTS else "float" + class DeriverSettings(HonchoSettings): model_config = SettingsConfigDict( # pyright: ignore diff --git a/src/crud/__init__.py b/src/crud/__init__.py index b34e4d17..7ce4c249 100644 --- a/src/crud/__init__.py +++ b/src/crud/__init__.py @@ -39,12 +39,23 @@ from .peer import ( get_peer, get_peers, get_sessions_for_peer, + reject_scope_observed, + reject_scope_peers, update_peer, ) from .peer_card import get_peer_card, set_peer_card from .representation import ( get_working_representation, ) +from .scope import ( + add_sessions_to_scope, + get_or_create_scopes, + get_scope_or_raise, + get_scope_sessions, + get_scopes, + remove_session_from_scope, + resolve_scope_peers, +) from .session import ( SessionDeletionResult, clone_session, @@ -114,6 +125,8 @@ __all__ = [ # Peer "get_or_create_peers", "get_peer", + "reject_scope_observed", + "reject_scope_peers", "get_peers", "update_peer", "get_sessions_for_peer", @@ -122,6 +135,14 @@ __all__ = [ "set_peer_card", # Representation "get_working_representation", + # Scope + "add_sessions_to_scope", + "get_or_create_scopes", + "get_scope_or_raise", + "get_scope_sessions", + "get_scopes", + "remove_session_from_scope", + "resolve_scope_peers", # Session "SessionDeletionResult", "get_sessions", diff --git a/src/crud/document.py b/src/crud/document.py index 6b7810ca..ba80712d 100644 --- a/src/crud/document.py +++ b/src/crud/document.py @@ -15,7 +15,7 @@ from sqlalchemy.sql.functions import func from src import models, schemas from src.config import settings from src.crud.collection import get_or_create_collection -from src.crud.peer import get_peer +from src.crud.peer import get_peer, reject_scope_observed from src.crud.session import get_session from src.dependencies import tracked_db from src.embedding_client import embedding_client @@ -355,6 +355,9 @@ async def query_documents( Returns: Sequence of matching documents """ + if top_k <= 0: + return [] + # Use provided embedding or generate one if embedding is None: try: @@ -952,7 +955,25 @@ async def create_observations( # Validate all peers exist for peer_name in peers_to_validate: - await get_peer(db, workspace_name, schemas.PeerCreate(name=peer_name)) + await get_peer(db, workspace_name, peer_name) + + # A scope may be an *observer* — that is how scoped conclusions are stored — + # but it must never be *observed*: scope peers carry observe_me=false and no + # representation is ever formed of one. Without this, a conclusion about a + # scope persists and a (observer, scope) collection is created for it. + # + # The strict variant because this is an observed position, though defence in + # depth rather than the active guard: the loop above resolves every peer, so a + # reserved name that does not exist yet already 404s before reaching here. If + # that validation ever stops covering observed_id, this still refuses the + # pre-seeding case instead of persisting a conclusion that a later-created + # scope would retroactively own. + await reject_scope_observed( + db, + workspace_name, + {obs.observed_id for obs in observations}, + action="No conclusion is ever formed about a scope.", + ) # Get or create all collections for observer, observed in collection_pairs: diff --git a/src/crud/message.py b/src/crud/message.py index 684ddb4d..fbfb1698 100644 --- a/src/crud/message.py +++ b/src/crud/message.py @@ -17,6 +17,7 @@ from src.utils.formatting import ILIKE_ESCAPE_CHAR, escape_ilike_pattern from src.utils.types import embedding_call_purpose from src.vector_store import get_external_vector_store +from .peer import reject_scope_peers from .session import get_or_create_session logger = getLogger(__name__) @@ -312,7 +313,22 @@ async def create_messages( Returns: List of created message objects + + Raises: + ValidationException: If a message is authored by a scope peer """ + # Scope peers are silent observers — they can never author messages. Keyed + # off name+flag so a legacy peer merely occupying the reserved namespace + # keeps ingesting. Must stay *before* get_or_create_session below: that call + # would create the scope peer and add it with a default SessionPeerConfig(), + # clobbering its observe_others=True/observe_me=False membership config. + await reject_scope_peers( + db, + workspace_name, + (message.peer_name for message in messages), + action="Scope peers cannot author messages.", + ) + # Get or create session with peers in messages list peers = {message.peer_name: schemas.SessionPeerConfig() for message in messages} await get_or_create_session( diff --git a/src/crud/peer.py b/src/crud/peer.py index f7936761..2f81f938 100644 --- a/src/crud/peer.py +++ b/src/crud/peer.py @@ -1,10 +1,12 @@ """CRUD helpers for peer records and peer-scoped session queries.""" +import re +from collections.abc import Collection, Iterable from logging import getLogger -from typing import Any +from typing import Any, Literal from cashews import NOT_NONE -from sqlalchemy import Select, select +from sqlalchemy import ColumnElement, Select, and_, select from sqlalchemy.exc import IntegrityError from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.orm import make_transient_to_detached @@ -13,13 +15,22 @@ from src import models, schemas from src.cache.client import cache, get_cache_namespace, safe_cache_delete from src.config import settings from src.crud.workspace import get_or_create_workspace -from src.exceptions import ConflictException, ResourceNotFoundException +from src.exceptions import ( + ConflictException, + ResourceNotFoundException, + ValidationException, +) from src.models import Peer +from src.schemas.api import RESOURCE_NAME_PATTERN +from src.utils import scopes as scopes_util from src.utils.filter import apply_filter from src.utils.types import GetOrCreateResult logger = getLogger(__name__) +# Matches the peers.name CHECK constraint and PeerCreate's max_length. +PEER_NAME_MAX_LENGTH = 512 + PEER_CACHE_KEY_TEMPLATE = "v2:workspace:{workspace_name}:peer:{peer_name}" PEER_LOCK_PREFIX = f"{get_cache_namespace()}:lock:v2" @@ -36,12 +47,214 @@ def peer_cache_key(workspace_name: str, peer_name: str) -> str: ) +def _reject_impossible_peer_names(names: Collection[str]) -> None: + """Reject names that cannot correspond to any stored row, before querying. + + ``PeerSpec`` accepts anything so existing names can be looked up, and the + full new-name rules run later on the insert path — but a couple of values + cannot be a legacy row *by construction*, and sending them to Postgres first + fails before that 422 can happen: + + - NUL bytes: Postgres text cannot hold them, so psycopg raises DataError + during the lookup itself, surfacing as a 500. + - Over-length names: the ``peers.name`` CHECK caps them at + ``PEER_NAME_MAX_LENGTH``, so no stored row can exceed it. + + Takes a ``Collection`` rather than an ``Iterable`` on purpose: it inspects the + input twice, so a generator would be half-consumed and the second check would + silently see nothing. + + Raises: + ValidationException: On a NUL byte or an over-length name. + """ + if any("\x00" in name for name in names): + raise ValidationException("Peer name(s) must not contain NUL (0x00) bytes") + too_long = sorted({n for n in names if len(n) > PEER_NAME_MAX_LENGTH}) + if too_long: + raise ValidationException( + f"Peer name(s) {too_long} must be at most " + + f"{PEER_NAME_MAX_LENGTH} characters" + ) + + +def _validate_new_peer_names(names: list[str]) -> None: + """Validate peer names that are about to be created. + + Mirrors ``PeerCreate``'s contract for peers arriving through crud rather than + the peers route. The reserved prefix is reported separately because it is + also outside ``RESOURCE_NAME_PATTERN``, so the charset check would otherwise + mask the real problem. + + Raises: + ValidationException: On a reserved-prefix or non-conforming name. + """ + scopes_util.validate_no_scope_peer_names( + names, action="Use the scopes routes to create scopes." + ) + # Length and NUL bytes are already refused before the lookup by + # _reject_impossible_peer_names; RESOURCE_NAME_PATTERN's `+` rejects empty. + offenders = sorted({n for n in names if not re.fullmatch(RESOURCE_NAME_PATTERN, n)}) + if offenders: + raise ValidationException( + f"Peer name(s) {offenders} must match pattern {RESOURCE_NAME_PATTERN}" + ) + + +def scope_peer_clause() -> ColumnElement[bool]: + """SQL form of ``is_scope_peer()``: reserved name prefix AND the internal kind flag. + + Lives here rather than in ``crud/scope.py`` because that module already imports + from this one, and ``get_peers`` below needs the clause — the other direction + would be a cycle. + + ``autoescape=True`` is future-proofing: '.' is not a LIKE wildcard, but '_' is, + so under a ``scope__``-style prefix an unescaped ``startswith`` would also match + ``scopeXY...``. Both columns are NOT NULL with defaults, so the negation + ``~scope_peer_clause()`` has no NULL-semantics trap. + """ + return and_( + models.Peer.name.startswith(scopes_util.SCOPE_PEER_PREFIX, autoescape=True), + models.Peer.internal_metadata.contains({"kind": scopes_util.SCOPE_KIND}), + ) + + +def _reserved_name_candidates(names: Iterable[str]) -> list[str]: + """Materialize ``names`` once and return the reserved-prefix ones, sorted. + + Materializing up front matters: callers pass generators (the message-author + path does), and validating impossible names iterates the input separately from + the prefix filter — a generator would be silently half-consumed. + + Impossible values are refused here, before any SQL, because a reserved-prefix + name containing a NUL byte would otherwise reach the text comparison below and + raise ``psycopg.DataError`` inside the query — a 500 instead of the 422 the + caller should get. + + Raises: + ValidationException: On a NUL byte or an over-length name. + """ + materialized = tuple(names) + _reject_impossible_peer_names(materialized) + return sorted({n for n in materialized if scopes_util.is_scope_peer_name(n)}) + + +async def reject_scope_observed( + db: AsyncSession, + workspace_name: str, + names: Iterable[str], + *, + action: str, +) -> None: + """Reject any name that is — or could later become — an observed scope. + + Stricter than ``reject_scope_peers`` in exactly one case: a **missing** + reserved name is refused. Use this for the *observed* position, where nothing + creates the peer and so nothing else would ever catch it. Without it a caller + can pre-seed state about ``scope.future`` while that peer does not exist, then + create the scope and have the state retroactively describe it. + + Three-way on the reserved namespace: + + ============================== ====== + State Result + ============================== ====== + Existing flagged scope reject + Missing reserved name reject + Existing unflagged squatter allow + ============================== ====== + + Non-reserved names are left entirely to the caller's own existence semantics. + + Raises: + ValidationException: On a real scope or a missing reserved name. + """ + candidates = _reserved_name_candidates(names) + if not candidates: + return + + rows = ( + await db.execute( + select(models.Peer.name, scope_peer_clause()) + .where(models.Peer.workspace_name == workspace_name) + .where(models.Peer.name.in_(candidates)) + ) + ).all() + flagged = {name for name, is_scope in rows if is_scope} + existing = {name for name, _ in rows} + + scopes = sorted(flagged) + if scopes: + raise ValidationException(f"Peer name(s) {scopes} are scopes. {action}") + + missing = sorted(set(candidates) - existing) + if missing: + raise ValidationException( + f"Peer name(s) {missing} are in the reserved scope namespace and do" + + f" not exist, so they may become scopes later. {action}" + ) + + +async def scope_peer_names( + db: AsyncSession, + workspace_name: str, + names: Iterable[str], +) -> set[str]: + """Return the subset of ``names`` that are really scope peers (name AND flag). + + Unlike a pure name check, a legacy peer that merely occupies the reserved + namespace (names were length-only validated before migration + ``d429de0e5338``, so ``scope.production`` is a possible user name) is not + reported, so it keeps its ordinary semantics instead of being locked out of + its own data. A *missing* reserved name is likewise not reported. + + Costs nothing on the common path: with no reserved-prefix name in ``names`` + there is no query at all. + + Raises: + ValidationException: On a NUL byte or an over-length name. + """ + candidates = _reserved_name_candidates(names) + if not candidates: + return set() + + result = await db.execute( + select(models.Peer.name) + .where(models.Peer.workspace_name == workspace_name) + .where(models.Peer.name.in_(candidates)) + .where(scope_peer_clause()) + ) + return {row[0] for row in result.all()} + + +async def reject_scope_peers( + db: AsyncSession, + workspace_name: str, + names: Iterable[str], + *, + action: str, +) -> None: + """Reject peers that really are scopes, keyed off name AND flag. + + A *missing* reserved name passes here — the create paths this guards + (`get_or_create_peers`) refuse it themselves. Positions where nothing creates + the peer need ``reject_scope_observed`` instead. See ``scope_peer_names`` for + the name-vs-flag semantics. + + Raises: + ValidationException: If any name resolves to a real scope peer. + """ + offenders = sorted(await scope_peer_names(db, workspace_name, names)) + if offenders: + raise ValidationException(f"Peer name(s) {offenders} are scopes. {action}") + + async def get_or_create_peers( db: AsyncSession, workspace_name: str, - peers: list[schemas.PeerCreate], + peers: list[schemas.PeerSpec], *, _retry: bool = False, + _pending_invalidation: list[str] | None = None, ) -> GetOrCreateResult[list[models.Peer]]: """ Get an existing list of peers or create new peers if they don't exist. @@ -52,16 +265,23 @@ async def get_or_create_peers( workspace_name: Name of the workspace peers: List of peer creation schemas _retry: Whether to retry the operation + _pending_invalidation: Names of peers already mutated by a prior attempt, + whose cache keys must still be purged. See the retry branch below. Returns: GetOrCreateResult containing the list of peers and whether any were created Raises: ConflictException: If we fail to get or create the peers + ValidationException: On an impossible name (NUL byte, over-length), or a + reserved-prefix or non-conforming name on the create path """ await get_or_create_workspace(db, schemas.WorkspaceCreate(name=workspace_name)) peer_names = [p.name for p in peers] + # Before the lookup: these values cannot match a stored row and would fail + # inside the query itself rather than as a clean 422. + _reject_impossible_peer_names(peer_names) stmt = ( select(models.Peer) .where(models.Peer.workspace_name == workspace_name) @@ -104,6 +324,17 @@ async def get_or_create_peers( existing_names = {p.name for p in existing_peers} peers_to_create = [p for p in peers if p.name not in existing_names] + # Names are validated on the *create* path only. `PeerSpec` deliberately + # carries no charset pattern so already-existing names (legacy dotted names, + # scope peers) can be looked up without a spurious 422 — but a name we are + # about to INSERT is a new peer, and new peers must obey the public contract. + # Without this, request-controlled names reach here unvalidated via message + # authors, session peer maps, and the chat observer path, letting a caller + # mint `scope.x` squatters (permanently 409-blocking that scope) or peers + # that violate RESOURCE_NAME_PATTERN outright. + if peers_to_create: + _validate_new_peer_names([p.name for p in peers_to_create]) + # Create new peers new_peers = [ models.Peer( @@ -122,11 +353,26 @@ async def get_or_create_peers( raise ConflictException( f"Unable to create or get peers: {peer_names}" ) from None - return await get_or_create_peers(db, workspace_name, peers, _retry=True) + # `begin_nested()` autoflushes the mutations above *before* opening the + # savepoint, so they are already committed-in-transaction and the rollback + # doesn't undo them — nor does it expire the now-clean ORM state. The retry + # would therefore compare already-updated values, find no change, and skip + # the purge. Carry the names forward so the invalidation can't be lost. + return await get_or_create_peers( + db, + workspace_name, + peers, + _retry=True, + _pending_invalidation=(_pending_invalidation or []) + + [p.name for p in changed_peers], + ) # Capture peer names eagerly so the closure holds plain strings, not ORM objects _cache_keys_to_invalidate = [ - peer_cache_key(workspace_name, p.name) for p in changed_peers + new_peers + peer_cache_key(workspace_name, name) + for name in dict.fromkeys( + (_pending_invalidation or []) + [p.name for p in changed_peers + new_peers] + ) ] async def _invalidate_peer_cache(): @@ -181,15 +427,20 @@ async def _fetch_peer( async def get_peer( db: AsyncSession, workspace_name: str, - peer: schemas.PeerCreate, + peer_name: str, ) -> models.Peer: """ Get an existing peer. + Takes a plain name, not a create schema: this is a pure read, and validating + an already-existing name against ``PeerCreate``'s charset pattern turns a + lookup into a raw pydantic ValidationError (an HTTP 500) for legacy dotted + names and every ``scope.``-prefixed peer. + Args: db: Database session workspace_name: Name of the workspace - peer: Peer creation schema + peer_name: Name of the peer Returns: The peer if found @@ -197,10 +448,10 @@ async def get_peer( Raises: ResourceNotFoundException: If the peer does not exist """ - data = await _fetch_peer(db, workspace_name, peer.name) + data = await _fetch_peer(db, workspace_name, peer_name) if data is None: raise ResourceNotFoundException( - f"Peer {peer.name} not found in workspace {workspace_name}" + f"Peer {peer_name} not found in workspace {workspace_name}" ) # Reconstruct ORM object from cached dict and merge into session @@ -215,10 +466,26 @@ async def get_peers( workspace_name: str, filters: dict[str, Any] | None = None, reverse: bool = False, + kind: Literal["scope", "all"] | None = None, ) -> Select[tuple[models.Peer]]: - """Build a filtered peer list query ordered by creation time.""" + """Build a filtered peer list query ordered by creation time. + + Args: + workspace_name: Name of the workspace + filters: Filter peers by metadata + reverse: Whether to reverse the default creation order + kind: Which kinds of peers to include. None (default) excludes scope + peers (see ``scope_peer_clause``: reserved name prefix AND the + ``{"kind": "scope"}`` internal_metadata flag), "scope" returns only + scope peers, and "all" returns everything. + """ stmt = select(models.Peer).where(models.Peer.workspace_name == workspace_name) + if kind is None: + stmt = stmt.where(~scope_peer_clause()) + elif kind == "scope": + stmt = stmt.where(scope_peer_clause()) + stmt = apply_filter(stmt, models.Peer, filters) if reverse: @@ -250,10 +517,21 @@ async def update_peer( the peer """ peers_result = await get_or_create_peers( - db, workspace_name, [schemas.PeerCreate(name=peer_name)] + db, workspace_name, [schemas.PeerSpec(name=peer_name)] ) honcho_peer = peers_result.resource[0] + # Refuse a real scope on the row just resolved, not on the name beforehand: + # this route replaces `configuration` wholesale, and a name-level check leaves + # a window in which a concurrently-created scope is resolved as existing (so + # create-path validation never fires) and then overwritten. An existing + # *unflagged* peer in the reserved namespace is an ordinary peer and passes. + if scopes_util.is_scope_peer(honcho_peer.name, honcho_peer.internal_metadata): + raise ValidationException( + f"Peer '{peer_name}' is a scope." + + " Use the scopes routes to manage scopes." + ) + needs_update = False if peer.metadata is not None and honcho_peer.h_metadata != peer.metadata: diff --git a/src/crud/peer_card.py b/src/crud/peer_card.py index 6697130f..38208a37 100644 --- a/src/crud/peer_card.py +++ b/src/crud/peer_card.py @@ -9,7 +9,12 @@ from sqlalchemy.ext.asyncio import AsyncSession from src import exceptions, models, schemas from src.cache.client import safe_cache_delete -from src.crud.peer import get_or_create_peers, get_peer, peer_cache_key +from src.crud.peer import ( + get_or_create_peers, + get_peer, + peer_cache_key, + reject_scope_observed, +) logger = logging.getLogger(__name__) @@ -38,7 +43,7 @@ async def get_peer_card( Raises: ResourceNotFoundException: If the peer does not exist. """ - peer = await get_peer(db, workspace_name, schemas.PeerCreate(name=observer)) + peer = await get_peer(db, workspace_name, observer) return cast( list[str] | None, peer.internal_metadata.get( @@ -68,9 +73,24 @@ async def set_peer_card( observer: Peer name of the observer """ + # A scope may be the card's *observer* — the Dreamer writes (scope, observed) + # cards — but never its subject. Authoritative here rather than only in the + # route, so the Dreamer and agent-tool paths are covered too, and in the same + # transaction as the JSONB write below. + # + # A *missing* reserved name is refused as well: only the observer is resolved + # below, so a card keyed on `scope.future` would otherwise persist while that + # peer does not exist and retroactively describe the scope once created. + await reject_scope_observed( + db, + workspace_name, + [observed], + action="No peer card is ever formed about a scope.", + ) + # Ensure the peer exists (get-or-create) peers_result = await get_or_create_peers( - db, workspace_name, [schemas.PeerCreate(name=observer)] + db, workspace_name, [schemas.PeerSpec(name=observer)] ) stmt = ( diff --git a/src/crud/representation.py b/src/crud/representation.py index d4b86ffb..57e6e8e1 100644 --- a/src/crud/representation.py +++ b/src/crud/representation.py @@ -318,11 +318,12 @@ class RepresentationManager: total = max_observations - # Calculate how many observations to get from each source + # Calculate how many observations to get from each source. + # Floor of 1 when a semantic query was explicitly requested. semantic_observations = ( min( max( - 0, + 1, semantic_search_top_k if semantic_search_top_k is not None else total // 3, diff --git a/src/crud/scope.py b/src/crud/scope.py new file mode 100644 index 00000000..a2ef1588 --- /dev/null +++ b/src/crud/scope.py @@ -0,0 +1,438 @@ +"""CRUD helpers for scopes. + +A scope is a named grouping of sessions, implemented as a peer named +``scope.`` carrying ``{"kind": "scope"}`` in ``internal_metadata`` (the +authoritative, user-unwritable flag) and ``{"observe_me": false}`` in +``configuration``, that observes its member sessions (``observe_others=true``) +and never speaks. +See ``src/utils/scopes.py`` for the namespace helpers. + +Membership only affects messages ingested *after* a session is added to a +scope. Conclusions already derived are neither backfilled on add nor +reconciled on removal. +""" + +from collections.abc import Sequence +from logging import getLogger + +from sqlalchemy import Select, select +from sqlalchemy.exc import IntegrityError +from sqlalchemy.ext.asyncio import AsyncSession + +from src import models, schemas +from src.cache.client import safe_cache_delete +from src.exceptions import ( + ConflictException, + ResourceNotFoundException, + ValidationException, +) +from src.utils.scopes import ( + SCOPE_KIND, + is_scope_peer, + scope_peer_name, +) +from src.utils.types import GetOrCreateResult + +from .peer import peer_cache_key, scope_peer_clause +from .workspace import get_or_create_workspace + +logger = getLogger(__name__) + +# Internal metadata stamped on every scope peer at creation. `kind` is the +# authoritative scope flag and lives here — NOT in `configuration` — because +# `configuration` is user-writable (`PeerCreate`/`PeerUpdate` accept a free-form +# dict, and `update_peer` replaces it wholesale), so a user could forge or clear +# the flag. `internal_metadata` appears in no API schema at all. +SCOPE_PEER_INTERNAL_METADATA: dict[str, str] = { + "kind": SCOPE_KIND, +} + +# Peer-level configuration stamped on every scope peer at creation. +# `observe_me: false` ensures no representation is ever formed *of* a scope peer. +# This one stays user-visible: `observe_me` is a legitimate config knob. +SCOPE_PEER_CONFIGURATION: dict[str, str | bool] = { + "observe_me": False, +} + +# Session-level configuration for a scope peer's membership in a session. +SCOPE_MEMBERSHIP_CONFIG = schemas.SessionPeerConfig( + observe_others=True, observe_me=False +) + + +async def get_or_create_scopes( + db: AsyncSession, + workspace_name: str, + scopes: list[schemas.ScopeCreate], + *, + _retry: bool = False, + _pending_invalidation: list[str] | None = None, +) -> GetOrCreateResult[list[models.Peer]]: + """ + Get existing scopes or create new ones if they don't exist. + + Existing scope peers have their metadata updated when provided. A + pre-existing peer that occupies a scope's reserved name *without* the + authoritative ``kind`` flag (a legacy collision) is never adopted. + + Note: does not commit; the caller owns the transaction (mirror of + ``get_or_create_peers``). Run ``result.post_commit()`` after committing. + + Deliberately does NOT scan for pre-existing state naming the backing peer + (peer-card keys, pending dream queue items). ``reject_scope_observed`` now + refuses writes against a not-yet-existing reserved name, so no new such state + can be created; only data written before that guard existed could collide, and + since ``scope.`` was never a meaningful namespace then, any such row is + coincidental. The consequence would also be inert — a card or queue item + describing a scope, which nothing reads, because no representation is formed of + a scope. Detecting card keys means scanning every peer's ``internal_metadata`` + for a label containing this name, i.e. a full table scan per scope creation: + disproportionate to that risk. Revisit if scope names ever become guessable + across tenants. + + Args: + db: Database session + workspace_name: Name of the workspace + scopes: List of scope creation schemas (unprefixed names) + _retry: Whether this is the retry attempt + _pending_invalidation: Names of scope peers already mutated by a prior + attempt, whose cache keys must still be purged. See the retry branch. + + Returns: + GetOrCreateResult containing the backing peers and whether any were + created + + Raises: + ConflictException: If a peer already occupies a scope's reserved name + without the scope kind flag, or if we fail to get or create the + scope peers + """ + await get_or_create_workspace(db, schemas.WorkspaceCreate(name=workspace_name)) + + peer_names = {scope_peer_name(s.name): s for s in scopes} + stmt = ( + select(models.Peer) + .where(models.Peer.workspace_name == workspace_name) + .where(models.Peer.name.in_(peer_names.keys())) + ) + result = await db.execute(stmt) + existing_peers: list[models.Peer] = list(result.scalars().all()) + + changed_peers: list[models.Peer] = [] + for existing_peer in existing_peers: + if not is_scope_peer(existing_peer.name, existing_peer.internal_metadata): + raise ConflictException( + f"A peer named '{existing_peer.name}' already exists in workspace " + + f"{workspace_name} but is not a scope. Rename or delete that " + + "peer before creating this scope." + ) + scope_schema = peer_names[existing_peer.name] + if ( + scope_schema.metadata is not None + and existing_peer.h_metadata != scope_schema.metadata + ): + existing_peer.h_metadata = scope_schema.metadata + changed_peers.append(existing_peer) + + existing_names = {p.name for p in existing_peers} + new_peers = [ + models.Peer( + workspace_name=workspace_name, + name=name, + h_metadata=scope_schema.metadata or {}, + internal_metadata=dict(SCOPE_PEER_INTERNAL_METADATA), + configuration=dict(SCOPE_PEER_CONFIGURATION), + ) + for name, scope_schema in peer_names.items() + if name not in existing_names + ] + try: + async with db.begin_nested(): + db.add_all(new_peers) + except IntegrityError: + if _retry: + raise ConflictException( + f"Unable to create or get scopes: {sorted(peer_names)}" + ) from None + # `begin_nested()` autoflushes the mutations above *before* opening the + # savepoint, so they survive the rollback and leave the ORM state clean — + # the retry would compare already-updated values, find no change, and skip + # the purge. Carry the names forward so the invalidation can't be lost. + return await get_or_create_scopes( + db, + workspace_name, + scopes, + _retry=True, + _pending_invalidation=(_pending_invalidation or []) + + [p.name for p in changed_peers], + ) + + _cache_keys_to_invalidate = [ + peer_cache_key(workspace_name, name) + for name in dict.fromkeys( + (_pending_invalidation or []) + [p.name for p in changed_peers + new_peers] + ) + ] + + async def _invalidate_peer_cache(): + for cache_key in _cache_keys_to_invalidate: + await safe_cache_delete(cache_key) + + return GetOrCreateResult( + existing_peers + new_peers, + created=len(new_peers) > 0, + on_commit=_invalidate_peer_cache if _cache_keys_to_invalidate else None, + ) + + +async def get_scopes( + workspace_name: str, + reverse: bool = False, +) -> Select[tuple[models.Peer]]: + """Build a scope list query, ordered by creation time. + + Requires both halves via ``scope_peer_clause`` (reserved name prefix AND the + internal kind flag), so a peer carrying a forged ``configuration`` cannot + inject itself into the scope list. + """ + stmt = ( + select(models.Peer) + .where(models.Peer.workspace_name == workspace_name) + .where(scope_peer_clause()) + ) + if reverse: + return stmt.order_by(models.Peer.created_at.desc(), models.Peer.id.desc()) + return stmt.order_by(models.Peer.created_at.asc(), models.Peer.id.asc()) + + +async def get_scope_or_raise( + db: AsyncSession, + workspace_name: str, + scope_name: str, +) -> models.Peer: + """ + Get an existing scope's backing peer by its unprefixed scope name. + + Args: + db: Database session + workspace_name: Name of the workspace + scope_name: Unprefixed scope name + + Returns: + The backing peer if found and flagged as a scope + + Raises: + ResourceNotFoundException: If no scope with that name exists (a peer + occupying the reserved name without the kind flag does not count) + """ + peer = await db.scalar( + select(models.Peer) + .where(models.Peer.workspace_name == workspace_name) + .where(models.Peer.name == scope_peer_name(scope_name)) + ) + if peer is None or not is_scope_peer(peer.name, peer.internal_metadata): + raise ResourceNotFoundException( + f"Scope {scope_name} not found in workspace {workspace_name}" + ) + return peer + + +async def resolve_scope_peers( + db: AsyncSession, + workspace_name: str, + scope_names: Sequence[str], +) -> list[str]: + """ + Resolve unprefixed scope names to their backing scope-peer names. + + Used by the read routes that accept a ``scope`` option (chat, + representation, session context, workspace search) to turn user-facing + scope names into the observer peers that implement them. + + Args: + db: Database session + workspace_name: Name of the workspace + scope_names: Unprefixed scope names (duplicates are collapsed, + preserving first-seen order) + + Returns: + The backing scope-peer names, in first-requested order + + Raises: + ResourceNotFoundException: If any named scope does not exist + ValidationException: If a peer occupies a scope's reserved name + without the authoritative kind flag (a legacy collision) + """ + requested: list[str] = [] + seen: set[str] = set() + for name in scope_names: + if name not in seen: + seen.add(name) + requested.append(name) + + peer_names = [scope_peer_name(name) for name in requested] + if not peer_names: + return [] + + result = await db.execute( + select(models.Peer) + .where(models.Peer.workspace_name == workspace_name) + .where(models.Peer.name.in_(peer_names)) + ) + peers_by_name = {peer.name: peer for peer in result.scalars().all()} + + resolved: list[str] = [] + for name, peer_name in zip(requested, peer_names, strict=True): + peer = peers_by_name.get(peer_name) + if peer is None: + raise ResourceNotFoundException( + f"Scope {name} not found in workspace {workspace_name}" + ) + # The kind flag is authoritative and lives in internal_metadata, so a + # legacy peer merely occupying the reserved name is refused rather than + # silently treated as a scope. + if not is_scope_peer(peer.name, peer.internal_metadata): + raise ValidationException( + f"'{name}' does not name a scope: a non-scope peer occupies " + + "its reserved name." + ) + resolved.append(peer_name) + return resolved + + +async def get_scope_sessions( + workspace_name: str, + scope_name: str, + reverse: bool = False, +) -> Select[tuple[models.Session]]: + """ + Build a query for the active sessions that are members of a scope. + + Membership is unbounded — a scope may span every session in a workspace — so + this returns a query for the caller to paginate rather than a materialized + list. Callers must check the scope exists themselves (``get_scope_or_raise``); + an unknown scope yields an empty page here, not a 404. + + Ordered by membership age, with the session id as a unique tiebreaker: + ``session_peers`` has a composite primary key and no id of its own, so + ``joined_at`` alone is not a stable pagination key. + + Args: + workspace_name: Name of the workspace + scope_name: Unprefixed scope name + reverse: Whether to return newest memberships first + + Returns: + Select for the scope's member sessions + """ + stmt = ( + select(models.Session) + .join( + models.SessionPeer, + (models.Session.name == models.SessionPeer.session_name) + & (models.Session.workspace_name == models.SessionPeer.workspace_name), + ) + .where(models.SessionPeer.workspace_name == workspace_name) + .where(models.SessionPeer.peer_name == scope_peer_name(scope_name)) + .where(models.SessionPeer.left_at.is_(None)) + .where(models.Session.is_active == True) # noqa: E712 + ) + if reverse: + return stmt.order_by( + models.SessionPeer.joined_at.desc(), models.Session.id.desc() + ) + return stmt.order_by(models.SessionPeer.joined_at.asc(), models.Session.id.asc()) + + +async def add_sessions_to_scope( + db: AsyncSession, + workspace_name: str, + scope_name: str, + session_names: list[str], +) -> None: + """ + Add sessions to a scope by creating observer memberships for its peer. + + Each membership is a ``session_peers`` row for the scope peer with + ``observe_others=true, observe_me=false`` — exactly what a hand-built + observer peer would carry. No backfill happens here: membership only affects + messages ingested after this call, and conclusions already derived are left + as they are. + + Args: + db: Database session + workspace_name: Name of the workspace + scope_name: Unprefixed scope name + session_names: Names of existing sessions to add + + Raises: + ResourceNotFoundException: If the scope or any named session does not + exist + """ + # Imported lazily: crud.session imports this module for the session-create + # `scopes` path, so a module-level import would be circular. + from .session import upsert_session_peers + + await get_scope_or_raise(db, workspace_name, scope_name) + + requested = set(session_names) + result = await db.execute( + select(models.Session.name) + .where(models.Session.workspace_name == workspace_name) + .where(models.Session.name.in_(requested)) + .where(models.Session.is_active == True) # noqa: E712 + ) + found = {row[0] for row in result.all()} + missing = sorted(requested - found) + if missing: + raise ResourceNotFoundException( + f"Session(s) {missing} not found in workspace {workspace_name}" + ) + + for session_name in sorted(requested): + await upsert_session_peers( + db, + workspace_name=workspace_name, + session_name=session_name, + peer_names={scope_peer_name(scope_name): SCOPE_MEMBERSHIP_CONFIG}, + fetch_after_upsert=False, + ) + + await db.commit() + + +async def remove_session_from_scope( + db: AsyncSession, + workspace_name: str, + scope_name: str, + session_name: str, +) -> None: + """ + Remove a session from a scope by ending the scope peer's membership. + + Ends the membership the same way the generic remove-peer path does (sets + ``left_at``). Conclusions derived while the session was a member are left in + place — nothing reconciles them. + + Args: + db: Database session + workspace_name: Name of the workspace + scope_name: Unprefixed scope name + session_name: Name of the session to remove + + Raises: + ResourceNotFoundException: If the scope or session does not exist + """ + # Lazy import for the same circular-import reason as add_sessions_to_scope. + from .session import remove_peers_from_session + + await get_scope_or_raise(db, workspace_name, scope_name) + + await remove_peers_from_session( + db, + workspace_name=workspace_name, + session_name=session_name, + peer_names={scope_peer_name(scope_name)}, + # This *is* the supported path for ending scope membership. + _allow_scope_peers=True, + ) diff --git a/src/crud/session.py b/src/crud/session.py index 4310cb3a..1fc884ec 100644 --- a/src/crud/session.py +++ b/src/crud/session.py @@ -7,7 +7,18 @@ from typing import cast as typing_cast from cashews import NOT_NONE from nanoid import generate as generate_nanoid -from sqlalchemy import Select, and_, case, cast, delete, func, insert, select, update +from sqlalchemy import ( + Select, + and_, + case, + cast, + delete, + exists, + func, + insert, + select, + update, +) from sqlalchemy.dialects.postgresql import insert as pg_insert from sqlalchemy.engine import CursorResult from sqlalchemy.exc import IntegrityError @@ -27,12 +38,21 @@ from src.exceptions import ( ConflictException, ObserverException, ResourceNotFoundException, + ValidationException, ) from src.utils.filter import apply_filter +from src.utils.scopes import is_scope_peer, scope_peer_name from src.utils.types import GetOrCreateResult from src.vector_store import get_external_vector_store -from .peer import get_or_create_peers, get_peer +from .peer import ( + get_or_create_peers, + get_peer, + reject_scope_peers, + scope_peer_clause, + scope_peer_names, +) +from .scope import SCOPE_MEMBERSHIP_CONFIG, get_or_create_scopes from .workspace import get_or_create_workspace logger = getLogger(__name__) @@ -99,6 +119,30 @@ async def _fetch_session( } +def _reject_resolved_scope_peers(peers: list[models.Peer]) -> None: + """Reject scope peers among rows already resolved for a membership upsert. + + The route-level guards check names *before* peers are resolved, which leaves a + check-then-upsert window: if a scope is created concurrently between that + check and the upsert below, the generic path would attach the now-flagged + scope peer with a default ``SessionPeerConfig()``, clobbering its + ``observe_others=True/observe_me=False`` membership config. This runs on the + resolved rows inside the same transaction as the upsert, so there is no + window and no extra query. + + Raises: + ValidationException: If any resolved peer is a scope. + """ + offenders = sorted( + p.name for p in peers if is_scope_peer(p.name, p.internal_metadata) + ) + if offenders: + raise ValidationException( + f"Peer name(s) {offenders} are scopes." + + " Scope membership is managed via the scopes routes." + ) + + def count_observers_in_config( peer_configs: dict[str, schemas.SessionPeerConfig], ) -> int: @@ -262,14 +306,39 @@ async def get_or_create_session( db, workspace_name=workspace_name, peers=[ - schemas.PeerCreate(name=peer_name) for peer_name in session.peer_names + schemas.PeerSpec(name=peer_name) for peer_name in session.peer_names + ], + ) + _reject_resolved_scope_peers(peers_result.resource) + await _get_or_add_peers_to_session( + db, + workspace_name=workspace_name, + session_name=session.name, + peer_names=session.peer_names, + fetch_after_upsert=False, + ) + + # Add the session to any requested scopes: create-or-get each scope peer + # and record an observer membership (observe_others=true, observe_me=false). + # No backfill happens here — membership only affects messages ingested + # after this point. + scopes_result = None + if session.scopes: + scopes_result = await get_or_create_scopes( + db, + workspace_name=workspace_name, + scopes=[ + schemas.ScopeCreate(name=scope_name) for scope_name in session.scopes ], ) await _get_or_add_peers_to_session( db, workspace_name=workspace_name, session_name=session.name, - peer_names=session.peer_names, + peer_names={ + scope_peer_name(scope_name): SCOPE_MEMBERSHIP_CONFIG + for scope_name in session.scopes + }, fetch_after_upsert=False, ) @@ -280,6 +349,8 @@ async def get_or_create_session( await ws_result.post_commit() if peers_result is not None: await peers_result.post_commit() + if scopes_result is not None: + await scopes_result.post_commit() # Only update cache if session data changed or was newly created if needs_cache_update: @@ -768,6 +839,8 @@ async def remove_peers_from_session( workspace_name: str, session_name: str, peer_names: set[str], + *, + _allow_scope_peers: bool = False, ) -> bool: """ Remove specified peers from a session. @@ -777,16 +850,32 @@ async def remove_peers_from_session( workspace_name: Name of the workspace session_name: Name of the session peer_names: Set of peer names to remove from the session + _allow_scope_peers: Internal. Set only by the scopes facade, which ends + scope membership through this same path and must not be blocked by + the guard below. Returns: True if peers were removed successfully Raises: ResourceNotFoundException: If the session does not exist + ValidationException: If any named peer is a scope """ # Verify session exists await get_session(db, session_name, workspace_name) + # Scope membership is ended through the scopes routes, which also reconcile + # the scope's copies. Rejected up front for a clear 422 rather than a silent + # no-op — but this check alone is only advisory: under READ COMMITTED a scope + # can be created between it and the UPDATE below. + if not _allow_scope_peers: + await reject_scope_peers( + db, + workspace_name, + peer_names, + action="Scope membership is managed via the scopes routes.", + ) + # Soft delete specified session peers by setting left_at timestamp update_stmt = ( update(models.SessionPeer) @@ -798,6 +887,20 @@ async def remove_peers_from_session( ) .values(left_at=func.now()) ) + if not _allow_scope_peers: + # Closes the window the advisory check above cannot: the exclusion is + # evaluated by Postgres as part of the UPDATE, so a scope committed after + # that check still cannot be detached here. Correlated rather than a join + # so the statement stays a plain UPDATE. + update_stmt = update_stmt.where( + ~exists( + select(models.Peer.id) + .where(models.Peer.workspace_name == workspace_name) + .where(models.Peer.name == models.SessionPeer.peer_name) + .where(scope_peer_clause()) + .correlate(models.SessionPeer) + ) + ) await db.execute(update_stmt) await db.commit() @@ -816,6 +919,13 @@ async def get_peers_from_session( workspace_name: Name of the workspace session_name: Name of the session + Scope peers are excluded: a scope's membership is the facade's internal + observer wiring, and this is the generic peer surface. Listing them here + would show a caller a peer named ``scope.`` with ``observe_others`` + set, which is exactly the mechanic the facade exists to hide. Mirrors the + ``kind``-less default of ``crud.peer.get_peers``; the scopes routes expose + membership from the other direction. + Returns: Paginated list of Peer objects in the session """ @@ -832,6 +942,10 @@ async def get_peers_from_session( .where(models.SessionPeer.session_name == session_name) .where(models.Peer.workspace_name == workspace_name) .where(models.SessionPeer.left_at.is_(None)) # Only active peers + # models.Peer is already in the FROM via the join above, so the clause + # composes directly — no correlated exists() as in the SessionPeer-only + # UPDATE statements elsewhere in this module. + .where(~scope_peer_clause()) ) @@ -946,13 +1060,27 @@ async def set_peers_for_session( f"Session {session_name} not found in workspace {workspace_name}" ) - # Soft delete specified session peers by setting left_at timestamp + # Soft delete every *ordinary* active membership. Scope memberships are + # deliberately preserved: this route replaces the peers the caller names, and a + # caller detaches a scope by simply *omitting* it from an otherwise valid + # replacement map — never naming it, so no request-level guard can see it. + # Without the exclusion a plain replacement silently bypasses the facade that + # owns scope membership and its removal reconciliation. Being part of the + # UPDATE, this holds regardless of the request body or concurrent scope + # creation. update_stmt = ( update(models.SessionPeer) .where( models.SessionPeer.session_name == session_name, models.SessionPeer.workspace_name == workspace_name, models.SessionPeer.left_at.is_(None), # Only update active peers + ~exists( + select(models.Peer.id) + .where(models.Peer.workspace_name == workspace_name) + .where(models.Peer.name == models.SessionPeer.peer_name) + .where(scope_peer_clause()) + .correlate(models.SessionPeer) + ), ) .values(left_at=func.now()) ) @@ -962,8 +1090,9 @@ async def set_peers_for_session( peers_result = await get_or_create_peers( db, workspace_name=workspace_name, - peers=[schemas.PeerCreate(name=peer_name) for peer_name in peer_names], + peers=[schemas.PeerSpec(name=peer_name) for peer_name in peer_names], ) + _reject_resolved_scope_peers(peers_result.resource) # Add new peers to session peers = await _get_or_add_peers_to_session( @@ -978,6 +1107,30 @@ async def set_peers_for_session( return peers +async def upsert_session_peers( + db: AsyncSession, + workspace_name: str, + session_name: str, + peer_names: dict[str, schemas.SessionPeerConfig], + *, + fetch_after_upsert: bool = True, +) -> list[models.SessionPeer]: + """Public wrapper around the session-peer membership upsert. + + Exists for other crud modules (currently the scopes facade in + ``src/crud/scope.py``) that manage memberships directly, bypassing the + route-level scope-peer guardrails. See ``_get_or_add_peers_to_session`` + for semantics. + """ + return await _get_or_add_peers_to_session( + db, + workspace_name=workspace_name, + session_name=session_name, + peer_names=peer_names, + fetch_after_upsert=fetch_after_upsert, + ) + + async def _get_or_add_peers_to_session( db: AsyncSession, workspace_name: str, @@ -1020,8 +1173,17 @@ async def _get_or_add_peers_to_session( result = await db.execute(select_stmt) return list(result.scalars().all()) - # Only validate observer limit if we're adding peers with observe_others=True - new_observer_count = count_observers_in_config(peer_names) + # Scope memberships carry observe_others=True but do not count against the + # limit. The limit bounds per-observer deriver fan-out for real peers; a scope + # costs document rows, not LLM calls, and counting them would + # cap scopes-per-session at SESSION_OBSERVERS_LIMIT and surface as an + # observer-shaped 400 through a facade that hides observers entirely. + scopes_being_added = await scope_peer_names(db, workspace_name, peer_names.keys()) + + # Only validate observer limit if we're adding non-scope peers with observe_others=True + new_observer_count = count_observers_in_config( + {n: c for n, c in peer_names.items() if n not in scopes_being_added} + ) if new_observer_count > 0: # Use a single efficient query to count existing observers not being updated @@ -1036,6 +1198,14 @@ async def _get_or_add_peers_to_session( models.SessionPeer.configuration["observe_others"].astext.cast( Boolean ), # Only observers + # Existing scope memberships are excluded for the same reason as above. + ~exists( + select(models.Peer.id) + .where(models.Peer.workspace_name == workspace_name) + .where(models.Peer.name == models.SessionPeer.peer_name) + .where(scope_peer_clause()) + .correlate(models.SessionPeer) + ), ) result = await db.execute(existing_observers_stmt) existing_observer_count = result.scalar() or 0 @@ -1111,7 +1281,14 @@ async def get_peer_config( Raises: ResourceNotFoundException: If the session or peer does not exist + ValidationException: If the peer is a scope """ + # A scope's membership config belongs to the facade, not the caller — the + # write path refuses it in set_peer_config below, and reading it back is the + # same internal wiring by another route. Checked on the resolved row, so a + # legacy peer merely occupying the reserved name keeps working. + _reject_resolved_scope_peers([await get_peer(db, workspace_name, peer_id)]) + # Get row from session_peer table stmt = select(models.SessionPeer).where( models.SessionPeer.workspace_name == workspace_name, @@ -1148,10 +1325,18 @@ async def set_peer_config( Raises: ObserverException: If the update would exceed the observer limit + ValidationException: If the peer is a scope """ # First, get the session and peer to ensure they exist await get_session(db, session_name, workspace_name) - await get_peer(db, workspace_name, schemas.PeerCreate(name=peer_name)) + peer = await get_peer(db, workspace_name, peer_name) + + # A scope's membership config is the facade's, not the caller's: setting + # observe_others=false silently stops all fan-out into the scope, and + # observe_me=true makes Honcho form a representation *of* a scope, which + # never happens by design. Checked on the row just resolved above, so there + # is no check-then-use window and no extra query. + _reject_resolved_scope_peers([peer]) # Check if a SessionPeer entry already exists stmt = ( diff --git a/src/deriver/enqueue.py b/src/deriver/enqueue.py index 7cd42aa3..f62a3db3 100644 --- a/src/deriver/enqueue.py +++ b/src/deriver/enqueue.py @@ -477,6 +477,22 @@ async def enqueue_dream( rebuild: card_refresh only — rebuild the card without the prior card """ async with tracked_db("dream_enqueue") as db_session: + # Authoritative scope check, in the same transaction as the queue insert. + # A route-level precheck cannot be relied on: it runs in its own session, + # and a *missing* reserved name passes it (nothing has flagged that peer + # yet) — so the dream would be enqueued and the scope created before the + # worker picked it up, letting the Dreamer run with a real scope as + # observed. A scope as `observer` stays allowed: consolidating scoped + # collections is exactly what the Dreamer does. + await crud.reject_scope_observed( + db_session, + workspace_name, + [observed], + action=( + "No representation is formed of a scope, so a scope cannot be the" + " observed peer of a dream." + ), + ) try: dream_record = create_dream_record( workspace_name, diff --git a/src/deriver/prompts.py b/src/deriver/prompts.py index 402bcae7..a4811f1d 100644 --- a/src/deriver/prompts.py +++ b/src/deriver/prompts.py @@ -72,9 +72,9 @@ RULES: - Contextualize each observation sufficiently (e.g. "Ann is nervous about the job interview at the pharmacy" not just "Ann is nervous") EXAMPLES (using `alice` as the target peer id): -- EXPLICIT: "I just had my 25th birthday last Saturday" → "alice is 25 years old", "alice's birthday is June 21st" -- EXPLICIT: "I took my dog for a walk in NYC" → "alice has a dog", "alice lives in NYC" -- EXPLICIT: "alice attended college" + general knowledge → "alice completed high school or equivalent" +- EXPLICIT: "I just turned 25" → "alice is 25 years old" +- EXPLICIT: "I took my dog for a walk in NYC" → "alice has a dog", "alice walked her dog in NYC" +- EXPLICIT: "I've lived in NYC for six years" → "alice lives in NYC", "alice has lived in NYC for six years" {custom_instructions_section} diff --git a/src/dialectic/chat.py b/src/dialectic/chat.py index d2f7e741..ba066741 100644 --- a/src/dialectic/chat.py +++ b/src/dialectic/chat.py @@ -10,15 +10,41 @@ from collections.abc import AsyncIterator from pydantic import BaseModel -from src import crud, schemas +from src import crud, models from src.config import ReasoningLevel from src.dependencies import tracked_db from src.dialectic.core import DialecticAgent +from src.exceptions import ValidationException from src.utils.config_helpers import get_configuration +from src.utils.scopes import is_scope_peer logger = logging.getLogger(__name__) +def _reject_scope_observed(peer: models.Peer) -> None: + """Refuse a dialectic run whose *observed* peer is a scope. + + A scope is a silent observer with ``observe_me=false``: no representation of + one exists to query, so it can never be the subject. + + The observer position is deliberately NOT checked here. A single `scope` on + chat swaps the observer to the scope peer — answering from a scope's + perspective is the entire point of that option — so a guard here would reject + every scoped chat. The raw path peer is still refused as an observer, by the + route (``routers/peers.py``), where the distinction between "the caller named + a scope" and "the `scope` option resolved to one" is still visible. + + Raises: + ValidationException: If the observed peer is a scope. + """ + if is_scope_peer(peer.name, peer.internal_metadata): + raise ValidationException( + f"Peer name '{peer.name}' is a scope." + + " No representation is formed of a scope, so a scope cannot be a" + + " dialectic target." + ) + + async def agentic_chat( workspace_name: str, session_name: str | None, @@ -48,9 +74,18 @@ async def agentic_chat( """ # Short-lived DB session for validation + config async with tracked_db("dialectic.preflight", read_only=True) as db: - await crud.get_peer(db, workspace_name, schemas.PeerCreate(name=observer)) + observer_peer = await crud.get_peer(db, workspace_name, observer) + observed_peer = observer_peer if observer != observed: - await crud.get_peer(db, workspace_name, schemas.PeerCreate(name=observed)) + observed_peer = await crud.get_peer(db, workspace_name, observed) + + # Resolved-row scope check, not a name check. The routes reject a scope + # target up front for a clear error, but that runs before resolution: a + # scope created in between would otherwise be answered about here. + # Checking the row we just resolved closes that window — an absent name + # already failed above, and an existing unflagged squatter cannot + # retroactively become a scope. + _reject_scope_observed(observed_peer) session = None if session_name: @@ -120,9 +155,18 @@ async def agentic_chat_stream( """ # Short-lived DB session for validation + config async with tracked_db("dialectic.preflight", read_only=True) as db: - await crud.get_peer(db, workspace_name, schemas.PeerCreate(name=observer)) + observer_peer = await crud.get_peer(db, workspace_name, observer) + observed_peer = observer_peer if observer != observed: - await crud.get_peer(db, workspace_name, schemas.PeerCreate(name=observed)) + observed_peer = await crud.get_peer(db, workspace_name, observed) + + # Resolved-row scope check, not a name check. The routes reject a scope + # target up front for a clear error, but that runs before resolution: a + # scope created in between would otherwise be answered about here. + # Checking the row we just resolved closes that window — an absent name + # already failed above, and an existing unflagged squatter cannot + # retroactively become a scope. + _reject_scope_observed(observed_peer) session = None if session_name: diff --git a/src/dreamer/specialists.py b/src/dreamer/specialists.py index 61e8a9d3..28ff4577 100644 --- a/src/dreamer/specialists.py +++ b/src/dreamer/specialists.py @@ -20,7 +20,7 @@ from typing import Any, cast from nanoid import generate as generate_nanoid -from src import crud, schemas +from src import crud from src.config import ConfiguredModelSettings, settings from src.dependencies import tracked_db from src.exceptions import ValidationException @@ -266,13 +266,9 @@ If you update it, send the full deduplicated list and remove stale entries. try: # Short-lived DB session for preflight operations async with tracked_db("dream.specialist.preflight") as db: - await crud.get_peer( - db, workspace_name, schemas.PeerCreate(name=observer) - ) + await crud.get_peer(db, workspace_name, observer) if observer != observed: - await crud.get_peer( - db, workspace_name, schemas.PeerCreate(name=observed) - ) + await crud.get_peer(db, workspace_name, observed) # Determine if peer card tools should be included. Specialists that # cannot write to the peer card (e.g., induction) skip the fetch and diff --git a/src/embedding_client.py b/src/embedding_client.py index 07197f43..08a4ff04 100644 --- a/src/embedding_client.py +++ b/src/embedding_client.py @@ -1,18 +1,26 @@ +from __future__ import annotations + import asyncio import logging import threading import time from collections import defaultdict from collections.abc import Awaitable, Callable -from typing import Any, Literal, NamedTuple, TypeVar +from typing import TYPE_CHECKING, Any, Literal, NamedTuple, TypeVar, cast import tiktoken -from google import genai -from google.genai import types as genai_types from nanoid import generate as generate_nanoid -from openai import AsyncOpenAI -from .config import EmbeddingModelConfig, resolve_embedding_model_config, settings +from .config import ( + EmbeddingEncodingFormat, + EmbeddingModelConfig, + resolve_embedding_model_config, + settings, +) + +if TYPE_CHECKING: + from google import genai + from openai import AsyncOpenAI logger = logging.getLogger(__name__) @@ -173,19 +181,27 @@ class _EmbeddingClient: max_input_tokens: int, max_tokens_per_request: int, send_dimensions: bool, + encoding_format: EmbeddingEncodingFormat = "float", ): self.transport: str = config.transport self.model: str = config.model self.vector_dimensions: int = vector_dimensions self.send_dimensions: bool = send_dimensions + self.encoding_format: EmbeddingEncodingFormat = encoding_format if self.transport == "gemini": if not config.api_key: raise ValueError("Gemini API key is required") - http_options = ( - genai_types.HttpOptions(base_url=config.base_url) - if config.base_url - else None + from google import genai + from google.genai import types as genai_types + + # 10-minute HTTP timeout, in lockstep with the LLM registry's Gemini + # client (`src/llm/registry.py:_build_gemini_http_options`). Without + # this, a stalled Gemini embedding socket wedges the deriver worker + # exactly the way #785 describes for the LLM client. + http_options = genai_types.HttpOptions( + base_url=config.base_url, + timeout=600_000, ) self.client: genai.Client | AsyncOpenAI = genai.Client( api_key=config.api_key, @@ -194,16 +210,18 @@ class _EmbeddingClient: # Gemini has a 2048 token limit self.max_embedding_tokens: int = min(max_input_tokens, 2048) # Gemini batch size is not documented, using conservative estimate - self.max_batch_size: int = 100 + self.max_batch_size: int = config.max_batch_size or 100 else: # openai if not config.api_key: raise ValueError("OpenAI API key is required") + from openai import AsyncOpenAI + self.client = AsyncOpenAI( api_key=config.api_key, base_url=config.base_url, ) self.max_embedding_tokens = max_input_tokens - self.max_batch_size = 2048 # OpenAI batch limit + self.max_batch_size = config.max_batch_size or 2048 try: self.encoding: tiktoken.Encoding = tiktoken.encoding_for_model(self.model) @@ -223,6 +241,29 @@ class _EmbeddingClient: ) return embedding + def _apply_encoding_format(self, openai_kwargs: dict[str, Any]) -> None: + """Set the embedding wire format on an openai request. + + Base64 is requested by omission, not by name: the SDK injects + `encoding_format=base64` when the caller passes nothing and decodes the + response, but skips that decode for any format the caller names, handing + back the raw base64 string. + """ + if self.encoding_format != "base64": + openai_kwargs["encoding_format"] = self.encoding_format + + def _validate_embedding_count(self, expected: int, received: int) -> None: + """Guard against a 200 response whose embedding count differs from inputs. + + An explicit `encoding_format` disables the openai SDK's own empty-data + check, so this has to live here. + """ + if received != expected: + raise ValueError( + f"Embedding count mismatch for {self.transport}:{self.model}. " + + f"Expected {expected}, got {received}." + ) + async def embed(self, query: str) -> list[float]: token_count = len(self.encoding.encode(query)) @@ -231,11 +272,11 @@ class _EmbeddingClient: f"Query exceeds maximum token limit of {self.max_embedding_tokens} tokens (got {token_count} tokens)" ) - # Bind the typed client at the dispatch site so pyright can narrow it - # for the closures without needing `assert isinstance(...)` (bandit - # B101). The closures close over the narrowed local, not `self.client`. - if isinstance(self.client, genai.Client): - gemini_client = self.client + # Dispatch on transport rather than isinstance so this module never + # needs the SDK types at runtime; the cast gives the closures a typed + # local to close over. + if self.transport == "gemini": + gemini_client = cast("genai.Client", self.client) async def _call_gemini() -> list[float]: response = await gemini_client.aio.models.embed_content( @@ -257,13 +298,15 @@ class _EmbeddingClient: fn=_call_gemini, ) - openai_client = self.client + openai_client = cast("AsyncOpenAI", self.client) async def _call_openai() -> list[float]: openai_kwargs: dict[str, Any] = {"model": self.model, "input": [query]} + self._apply_encoding_format(openai_kwargs) if self.send_dimensions: openai_kwargs["dimensions"] = self.vector_dimensions response = await openai_client.embeddings.create(**openai_kwargs) + self._validate_embedding_count(1, len(response.data)) return self._validate_embedding_dimensions(response.data[0].embedding) return await _emit_embedding_call( @@ -447,10 +490,19 @@ class _EmbeddingClient: attempt is a distinct provider hit and shows up as its own line item in analytics.""" result: dict[str, dict[int, list[float]]] = defaultdict(dict) - if isinstance(self.client, genai.Client): - response = await self.client.aio.models.embed_content( + if self.transport == "gemini": + from google.genai import types as genai_types + + gemini_client = cast("genai.Client", self.client) + response = await gemini_client.aio.models.embed_content( model=self.model, - contents=[item.text for item in batch], + # One Content per item: a list of bare strings is folded + # into a single document by gemini-embedding-2*, which + # returns one embedding for the whole batch (#745). + contents=[ + genai_types.Content(parts=[genai_types.Part(text=item.text)]) + for item in batch + ], config={"output_dimensionality": self.vector_dimensions}, ) if response.embeddings: @@ -464,9 +516,12 @@ class _EmbeddingClient: "model": self.model, "input": [item.text for item in batch], } + self._apply_encoding_format(openai_kwargs) if self.send_dimensions: openai_kwargs["dimensions"] = self.vector_dimensions - response = await self.client.embeddings.create(**openai_kwargs) + openai_client = cast("AsyncOpenAI", self.client) + response = await openai_client.embeddings.create(**openai_kwargs) + self._validate_embedding_count(len(batch), len(response.data)) for item, embedding_data in zip(batch, response.data, strict=True): result[item.text_id][item.chunk_index] = ( self._validate_embedding_dimensions(embedding_data.embedding) @@ -574,10 +629,10 @@ class EmbeddingClient: and allowing the application to start even if API keys are not yet configured. """ - _instance: "_EmbeddingClient | None" = None + _instance: _EmbeddingClient | None = None _instance_signature: tuple[object, ...] | None = None _lock: threading.Lock = threading.Lock() - _wrapper_instance: "EmbeddingClient | None" = None + _wrapper_instance: EmbeddingClient | None = None def __new__(cls): """Ensure only one instance of EmbeddingClient exists.""" @@ -603,6 +658,7 @@ class EmbeddingClient: max_input_tokens=settings.EMBEDDING.MAX_INPUT_TOKENS, max_tokens_per_request=settings.EMBEDDING.MAX_TOKENS_PER_REQUEST, send_dimensions=settings.EMBEDDING.resolve_send_dimensions(), + encoding_format=settings.EMBEDDING.resolve_encoding_format(), ) self._instance_signature = signature logger.debug( @@ -623,10 +679,12 @@ class EmbeddingClient: runtime_config.model, runtime_config.api_key, runtime_config.base_url, + runtime_config.max_batch_size, settings.EMBEDDING.VECTOR_DIMENSIONS, settings.EMBEDDING.MAX_INPUT_TOKENS, settings.EMBEDDING.MAX_TOKENS_PER_REQUEST, settings.EMBEDDING.resolve_send_dimensions(), + settings.EMBEDDING.resolve_encoding_format(), ) async def embed(self, query: str) -> list[float]: @@ -674,8 +732,19 @@ class EmbeddingClient: @property def encoding(self) -> tiktoken.Encoding: - """Get the tiktoken encoding.""" - return self._get_client().encoding + """Get the tiktoken encoding. + + Resolved without constructing the underlying client: tiktoken needs no + API key, and token-counting callers (e.g. the document dedup tie-break) + must work in environments with no embedding credentials, such as CI for + pull requests from forks. + """ + if self._instance is not None: + return self._instance.encoding + try: + return tiktoken.encoding_for_model(self._resolve_runtime_config().model) + except KeyError: + return tiktoken.get_encoding("cl100k_base") # Shared singleton embedding client instance diff --git a/src/llm/__init__.py b/src/llm/__init__.py index ae47bc53..e0913685 100644 --- a/src/llm/__init__.py +++ b/src/llm/__init__.py @@ -15,6 +15,7 @@ from .registry import ( CLIENTS, backend_for_provider, client_for_model_config, + default_client, get_anthropic_client, get_anthropic_override_client, get_backend, @@ -51,6 +52,7 @@ __all__ = [ "VerbosityType", "backend_for_provider", "client_for_model_config", + "default_client", "default_transport_api_key", "get_anthropic_client", "get_anthropic_override_client", diff --git a/src/llm/backends/anthropic.py b/src/llm/backends/anthropic.py index 614a2e32..9f58af07 100644 --- a/src/llm/backends/anthropic.py +++ b/src/llm/backends/anthropic.py @@ -9,7 +9,10 @@ from anthropic.types import TextBlock, ThinkingBlock, ToolUseBlock from pydantic import BaseModel, ValidationError from src.llm.backend import CompletionResult, StreamChunk, ToolCallResult -from src.llm.request_builder import apply_sdk_passthroughs +from src.llm.request_builder import ( + apply_sdk_passthroughs, + request_timeout_from_extra_params, +) from src.llm.structured_output import repair_response_model_json, schema_instruction @@ -74,6 +77,10 @@ class AnthropicBackend: # from ModelConfig.provider_params. Shallow merge with operator-wins. apply_sdk_passthroughs(params, extra_params) + timeout = request_timeout_from_extra_params(extra_params) + if timeout is not None: + params["timeout"] = timeout + # The '{' prefill forces a JSON-first response, which suppresses # tool_use blocks — skip it when tools are available and rely on the # conditional instruction + repair fallback instead. @@ -157,6 +164,11 @@ class AnthropicBackend: # Operator escape hatch: forward Anthropic SDK passthrough kwargs # from ModelConfig.provider_params. Shallow merge with operator-wins. apply_sdk_passthroughs(params, extra_params) + + timeout = request_timeout_from_extra_params(extra_params) + if timeout is not None: + params["timeout"] = timeout + # See complete(): no '{' prefill when tools are available, so # tool_use blocks stay reachable on the streamed path too. use_json_prefill = ( diff --git a/src/llm/backends/gemini.py b/src/llm/backends/gemini.py index c114196e..3f6ade64 100644 --- a/src/llm/backends/gemini.py +++ b/src/llm/backends/gemini.py @@ -4,6 +4,7 @@ from collections.abc import AsyncIterator from datetime import datetime, timedelta, timezone from typing import Any, ClassVar, cast +from google.genai import types as genai_types from pydantic import BaseModel from src.exceptions import LLMError, ValidationException @@ -14,7 +15,10 @@ from src.llm.caching import ( build_cache_key, gemini_cache_store, ) -from src.llm.request_builder import coerce_passthrough_mapping +from src.llm.request_builder import ( + coerce_passthrough_mapping, + request_timeout_from_extra_params, +) from src.llm.structured_output import repair_response_model_json, schema_instruction GEMINI_BLOCKED_FINISH_REASONS = { @@ -289,19 +293,38 @@ class GeminiBackend: # extra_query has no SDK-level equivalent and is ignored. Shallow # merge with operator-wins. Operators are responsible for not setting # unknown fields that google-genai's validation will reject. + http_options: genai_types.HttpOptions | None = None if extra_params: operator_extra_body = extra_params.get("extra_body") if operator_extra_body: config.update( coerce_passthrough_mapping("extra_body", operator_extra_body) ) + raw_http_options = config.get("http_options") + if isinstance(raw_http_options, genai_types.HttpOptions): + http_options = raw_http_options + elif isinstance(raw_http_options, dict): + http_options = genai_types.HttpOptions.model_validate( + raw_http_options + ) operator_extra_headers = extra_params.get("extra_headers") if operator_extra_headers: - http_options = config.setdefault("http_options", {}) - existing_headers = http_options.setdefault("headers", {}) + if http_options is None: + http_options = genai_types.HttpOptions() + existing_headers = dict(http_options.headers or {}) existing_headers.update( coerce_passthrough_mapping("extra_headers", operator_extra_headers) ) + http_options.headers = existing_headers + + timeout = request_timeout_from_extra_params(extra_params) + if timeout is not None: + if http_options is None: + http_options = genai_types.HttpOptions() + # Gemini has no native timeout kwarg; set the httpx-level value in ms. + http_options.timeout = int(timeout * 1000) + if http_options is not None: + config["http_options"] = http_options return config def _normalize_response( diff --git a/src/llm/backends/openai.py b/src/llm/backends/openai.py index d5d0ed73..672709ff 100644 --- a/src/llm/backends/openai.py +++ b/src/llm/backends/openai.py @@ -11,7 +11,10 @@ from pydantic import BaseModel, ValidationError from src.exceptions import ValidationException from src.llm.backend import CompletionResult, StreamChunk, ToolCallResult -from src.llm.request_builder import apply_sdk_passthroughs +from src.llm.request_builder import ( + apply_sdk_passthroughs, + request_timeout_from_extra_params, +) from src.llm.structured_output import ( StructuredOutputError, empty_structured_output, @@ -397,6 +400,10 @@ class OpenAIBackend: # if the operator supplies `extra_body.reasoning`, it replaces any # value Honcho auto-injected above. apply_sdk_passthroughs(params, extra_params) + + timeout = request_timeout_from_extra_params(extra_params) + if timeout is not None: + params["timeout"] = timeout return params def _normalize_response( diff --git a/src/llm/executor.py b/src/llm/executor.py index 956bc673..eaa79c1d 100644 --- a/src/llm/executor.py +++ b/src/llm/executor.py @@ -26,7 +26,7 @@ from .backend import CompletionResult as BackendCompletionResult from .backend import StreamChunk as BackendStreamChunk from .backend import ToolCallResult from .capture import build_captured_call, dispatch_captured_call, has_exporters -from .registry import CLIENTS, backend_for_provider +from .registry import backend_for_provider, default_client from .request_builder import execute_completion, execute_stream from .runtime import ( AttemptPlan, @@ -439,7 +439,7 @@ async def honcho_llm_call_inner( post-stream at this layer; aggregate envelopes (DialecticCompletedEvent etc.) carry the accurate totals. """ - client = client_override or CLIENTS.get(provider) + client = client_override or default_client(provider) if client is None: raise ValueError(f"Missing client for {provider}") diff --git a/src/llm/registry.py b/src/llm/registry.py index 6dd5eecc..9b18bcb8 100644 --- a/src/llm/registry.py +++ b/src/llm/registry.py @@ -9,20 +9,12 @@ history adapter selection) lives here now. from __future__ import annotations from functools import lru_cache -from typing import assert_never - -from anthropic import AsyncAnthropic -from google import genai -from google.genai import types as genai_types -from openai import AsyncOpenAI +from typing import TYPE_CHECKING, assert_never from src.config import ModelConfig, ModelTransport, settings from src.exceptions import ValidationException from .backend import ProviderBackend -from .backends.anthropic import AnthropicBackend -from .backends.gemini import GeminiBackend -from .backends.openai import OpenAIBackend from .credentials import default_transport_api_key from .history_adapters import ( AnthropicHistoryAdapter, @@ -32,6 +24,23 @@ from .history_adapters import ( ) from .types import ProviderClient +if TYPE_CHECKING: + from anthropic import AsyncAnthropic + from google import genai + from google.genai import types as genai_types + from openai import AsyncOpenAI + +# Provider SDKs are imported lazily inside the client factories below so a +# process only pays the import-time memory cost of the providers it uses. + +# Default client-level HTTP timeouts. Anthropic accepts seconds (float); +# google-genai's HttpOptions.timeout is an int in milliseconds, so the Gemini +# value is kept separately. Both default to 10 minutes to match the existing +# Anthropic behavior — long enough for slow streamed responses, short enough +# that a stalled socket can no longer wedge the deriver worker (see #785). +_ANTHROPIC_TIMEOUT_S = 600.0 +_GEMINI_TIMEOUT_MS = 600_000 + # Client-level ``default_headers`` applied to OpenAI-compatible clients, keyed by # base-URL prefix. Currently only OpenRouter, which uses them for app attribution # (https://openrouter.ai/docs/app-attribution); add a prefix here to tag another @@ -54,19 +63,40 @@ def _default_headers_for(base_url: str | None) -> dict[str, str]: return {} +def _build_gemini_http_options(base_url: str | None) -> genai_types.HttpOptions: + """Build Gemini ``HttpOptions`` carrying a default HTTP timeout. + + google-genai's ``HttpOptions.timeout`` is an int in milliseconds. A stalled + Gemini socket without this value wedges the entire deriver process because + all deriver workers share one uvloop event loop (see #785). Keep the + timeout even when no ``base_url`` is configured — that's the path the + default ``get_gemini_client`` takes and it's the one that was hanging. + """ + from google.genai import types as genai_types + + return genai_types.HttpOptions( + base_url=base_url, + timeout=_GEMINI_TIMEOUT_MS, + ) + + @lru_cache(maxsize=1) def get_anthropic_client() -> AsyncAnthropic: """Default Anthropic client built from settings.LLM.ANTHROPIC_API_KEY.""" + from anthropic import AsyncAnthropic + return AsyncAnthropic( api_key=settings.LLM.ANTHROPIC_API_KEY, base_url=settings.LLM.ANTHROPIC_BASE_URL, - timeout=600.0, + timeout=_ANTHROPIC_TIMEOUT_S, ) @lru_cache(maxsize=1) def get_openai_client() -> AsyncOpenAI: """Default OpenAI client built from settings.LLM.OPENAI_API_KEY.""" + from openai import AsyncOpenAI + return AsyncOpenAI( api_key=settings.LLM.OPENAI_API_KEY, base_url=settings.LLM.OPENAI_BASE_URL, @@ -77,12 +107,12 @@ def get_openai_client() -> AsyncOpenAI: @lru_cache(maxsize=1) def get_gemini_client() -> genai.Client: """Default Gemini client built from settings.LLM.GEMINI_API_KEY.""" - http_options = ( - genai_types.HttpOptions(base_url=settings.LLM.GEMINI_BASE_URL) - if settings.LLM.GEMINI_BASE_URL - else None + from google import genai + + return genai.Client( + api_key=settings.LLM.GEMINI_API_KEY, + http_options=_build_gemini_http_options(settings.LLM.GEMINI_BASE_URL), ) - return genai.Client(api_key=settings.LLM.GEMINI_API_KEY, http_options=http_options) # Bounded cache — in practice the (base_url, api_key) key space is small @@ -92,6 +122,8 @@ def get_openai_override_client( base_url: str | None, api_key: str | None ) -> AsyncOpenAI: """OpenAI client for a specific (base_url, api_key) pair. Cached by key.""" + from openai import AsyncOpenAI + return AsyncOpenAI( api_key=api_key, base_url=base_url, @@ -105,7 +137,11 @@ def get_anthropic_override_client( api_key: str | None, ) -> AsyncAnthropic: """Anthropic client for a specific (base_url, api_key) pair. Cached by key.""" - return AsyncAnthropic(api_key=api_key, base_url=base_url, timeout=600.0) + from anthropic import AsyncAnthropic + + return AsyncAnthropic( + api_key=api_key, base_url=base_url, timeout=_ANTHROPIC_TIMEOUT_S + ) @lru_cache(maxsize=128) @@ -113,38 +149,48 @@ def get_gemini_override_client( base_url: str | None, api_key: str | None ) -> genai.Client: """Gemini client for a specific (base_url, api_key) pair. Cached by key.""" - http_options = genai_types.HttpOptions(base_url=base_url) if base_url else None - return genai.Client(api_key=api_key, http_options=http_options) + from google import genai + + return genai.Client( + api_key=api_key, + http_options=_build_gemini_http_options(base_url), + ) -# Module-level default-client registry, populated at import time. Tests patch -# this dict via `patch.dict(CLIENTS, {...})` to inject mock provider clients. +# Module-level default-client registry, populated lazily on first use so a +# provider's SDK is only imported when that provider is actually called. Tests +# patch this dict via `patch.dict(CLIENTS, {...})` to inject mock provider +# clients; a patched entry always wins because `default_client` checks the +# dict before constructing anything. CLIENTS: dict[ModelTransport, ProviderClient] = {} -if settings.LLM.ANTHROPIC_API_KEY: - CLIENTS["anthropic"] = AsyncAnthropic( - api_key=settings.LLM.ANTHROPIC_API_KEY, - base_url=settings.LLM.ANTHROPIC_BASE_URL, - timeout=600.0, - ) -if settings.LLM.OPENAI_API_KEY: - CLIENTS["openai"] = AsyncOpenAI( - api_key=settings.LLM.OPENAI_API_KEY, - base_url=settings.LLM.OPENAI_BASE_URL, - default_headers=_default_headers_for(settings.LLM.OPENAI_BASE_URL), - ) +def default_client(provider: ModelTransport) -> ProviderClient | None: + """Default client for ``provider``, built on first use. -if settings.LLM.GEMINI_API_KEY: - http_options = ( - genai_types.HttpOptions(base_url=settings.LLM.GEMINI_BASE_URL) - if settings.LLM.GEMINI_BASE_URL - else None - ) - CLIENTS["gemini"] = genai.Client( - api_key=settings.LLM.GEMINI_API_KEY, - http_options=http_options, - ) + Returns None when no API key is configured for the provider. + """ + existing = CLIENTS.get(provider) + if existing is not None: + return existing + + if provider == "anthropic": + if not settings.LLM.ANTHROPIC_API_KEY: + return None + client: ProviderClient = get_anthropic_client() + elif provider == "openai": + if not settings.LLM.OPENAI_API_KEY: + return None + client = get_openai_client() + elif provider == "gemini": + if not settings.LLM.GEMINI_API_KEY: + return None + client = get_gemini_client() + else: + assert_never(provider) + + CLIENTS[provider] = client + return client def client_for_model_config( @@ -158,7 +204,7 @@ def client_for_model_config( override factories. """ if model_config.api_key is None and model_config.base_url is None: - existing_client = CLIENTS.get(provider) + existing_client = default_client(provider) if existing_client is not None: return existing_client @@ -182,10 +228,16 @@ def backend_for_provider( ) -> ProviderBackend: """Wrap a raw provider SDK client in the matching ProviderBackend adapter.""" if provider == "anthropic": + from .backends.anthropic import AnthropicBackend + return AnthropicBackend(client) if provider == "openai": + from .backends.openai import OpenAIBackend + return OpenAIBackend(client) if provider == "gemini": + from .backends.gemini import GeminiBackend + return GeminiBackend(client) assert_never(provider) @@ -216,6 +268,7 @@ __all__ = [ "CLIENTS", "backend_for_provider", "client_for_model_config", + "default_client", "get_anthropic_client", "get_anthropic_override_client", "get_backend", diff --git a/src/llm/request_builder.py b/src/llm/request_builder.py index 3da437b5..7eadab71 100644 --- a/src/llm/request_builder.py +++ b/src/llm/request_builder.py @@ -11,10 +11,14 @@ from typing import Any, cast from pydantic import BaseModel -from src.config import ModelConfig, PromptCachePolicy +from src.config import ModelConfig, PromptCachePolicy, coerce_provider_timeout from src.exceptions import ValidationException -from .backend import CompletionResult, ProviderBackend, StreamChunk +from .backend import ( + CompletionResult, + ProviderBackend, + StreamChunk, +) # Operator escape-hatch keys recognized inside ModelConfig.provider_params. PASSTHROUGH_KEYS = ("extra_body", "extra_headers", "extra_query") @@ -99,6 +103,45 @@ def build_config_extra_params(config: ModelConfig) -> dict[str, Any]: return extra_params +def request_timeout_from_extra_params( + extra_params: dict[str, Any] | None, +) -> float | None: + """Return a validated per-request provider timeout from extra params. + + Config-sourced timeouts are already validated and normalized at config + load (`coerce_provider_timeout` in src.config); this guards extra_params + passed programmatically at call time. + """ + if not extra_params or "timeout" not in extra_params: + return None + + try: + return coerce_provider_timeout(extra_params["timeout"]) + except ValueError as exc: + raise ValidationException(str(exc)) from exc + + +def _strip_none_params( + params: dict[str, Any], + keys: tuple[str, ...], +) -> dict[str, Any]: + """Remove specified keys from extra params when their values are None.""" + return {k: v for k, v in params.items() if not (k in keys and v is None)} + + +def _normalize_extra_params(extra_params: dict[str, Any]) -> dict[str, Any]: + """Normalize and clean shared extra params before they reach backends. + + Centralizes per-key coercion and null-stripping so new keys are added + here rather than spawning one-off normalizers. + """ + result = dict(extra_params) + timeout = request_timeout_from_extra_params(result) + if timeout is not None: + result["timeout"] = timeout + return _strip_none_params(result, ("timeout",)) + + async def execute_completion( backend: ProviderBackend, config: ModelConfig, @@ -120,6 +163,7 @@ async def execute_completion( **build_config_extra_params(config), **(extra_params or {}), } + merged_extra_params = _normalize_extra_params(merged_extra_params) if cache_policy is not None: merged_extra_params["cache_policy"] = cache_policy @@ -158,6 +202,7 @@ async def execute_stream( **build_config_extra_params(config), **(extra_params or {}), } + merged_extra_params = _normalize_extra_params(merged_extra_params) if cache_policy is not None: merged_extra_params["cache_policy"] = cache_policy diff --git a/src/llm/types.py b/src/llm/types.py index 33058e09..b010f6a8 100644 --- a/src/llm/types.py +++ b/src/llm/types.py @@ -12,12 +12,13 @@ from collections.abc import AsyncIterator, Callable from dataclasses import dataclass, field from typing import TYPE_CHECKING, Any, Generic, Literal, TypeVar -from anthropic import AsyncAnthropic -from google import genai -from openai import AsyncOpenAI from pydantic import BaseModel, Field if TYPE_CHECKING: + from anthropic import AsyncAnthropic + from google import genai + from openai import AsyncOpenAI + from src.llm.capture import CapturedMessage logger = logging.getLogger(__name__) @@ -30,8 +31,13 @@ ReasoningEffortType = ( ) VerbosityType = Literal["low", "medium", "high"] | None -# Raw SDK client union used by the provider-selection layer. -ProviderClient = AsyncAnthropic | AsyncOpenAI | genai.Client +# Raw SDK client union used by the provider-selection layer. The SDK types are +# only imported for type checking; at runtime this stays Any so importing this +# module doesn't load any provider SDK. +if TYPE_CHECKING: + ProviderClient = AsyncAnthropic | AsyncOpenAI | genai.Client +else: + ProviderClient = Any @dataclass diff --git a/src/main.py b/src/main.py index 3c8bf3e8..75eac455 100644 --- a/src/main.py +++ b/src/main.py @@ -24,6 +24,7 @@ from src.routers import ( keys, messages, peers, + scopes, sessions, webhooks, workspaces, @@ -171,6 +172,7 @@ add_pagination(app) app.include_router(workspaces.router, prefix="/v3") app.include_router(peers.router, prefix="/v3") app.include_router(sessions.router, prefix="/v3") +app.include_router(scopes.router, prefix="/v3") app.include_router(messages.router, prefix="/v3") app.include_router(conclusions.router, prefix="/v3") app.include_router(keys.router, prefix="/v3") diff --git a/src/routers/conclusions.py b/src/routers/conclusions.py index 3a25a5d7..89d6207f 100644 --- a/src/routers/conclusions.py +++ b/src/routers/conclusions.py @@ -110,7 +110,10 @@ async def query_conclusions( if not observer or not observed: raise ValidationException( - "observer and observed must be specified for semantic search" + "observer and observed must be specified for semantic search. " + + "Pass them inside the 'filters' object, e.g. " + + '{"query": "...", "filters": {"observer": "alice", "observed": "bob"}}. ' + + "Both 'observer'/'observer_id' and 'observed'/'observed_id' are accepted." ) with embedding_call_purpose( diff --git a/src/routers/peers.py b/src/routers/peers.py index bccb2dd0..7843081e 100644 --- a/src/routers/peers.py +++ b/src/routers/peers.py @@ -5,6 +5,7 @@ import logging from collections.abc import AsyncIterator from contextlib import suppress from time import perf_counter +from typing import Any from fastapi import APIRouter, Body, Depends, Path, Query, Response from fastapi.responses import StreamingResponse @@ -28,8 +29,13 @@ from src.exceptions import ( from src.security import JWTParams, require_auth from src.telemetry import prometheus_metrics from src.telemetry.events import EmbeddingCallPurpose, GetContextEvent, emit -from src.utils.filter import extract_session_allowlist +from src.utils.filter import MAX_SESSION_ALLOWLIST_ENTRIES, extract_session_allowlist from src.utils.schema_conversion import json_response_schema_to_pydantic +from src.utils.scopes import ( + is_scope_peer, + is_scope_peer_name, + validate_no_scope_peer_names, +) from src.utils.search import search from src.utils.types import embedding_call_purpose @@ -41,6 +47,73 @@ router = APIRouter( ) +def _validate_scope_option( + *, + filters: dict[str, Any] | None, + session_id: str | None, + jwt_params: JWTParams, +) -> None: + """Enforce the v1 `scope` exclusions and auth rule (chat/representation). + + `scope` is mutually exclusive with `filters` and `session_id` (422), and a + scope's member sessions may exceed a peer's own membership, so scoped + reads require a workspace- or admin-level key. + + 401 rather than 403: every other scope surface refuses a narrow key with 401 + — the `/scopes` router via `require_auth`, and the `scopes` field on session + create — so a peer key would otherwise get two different codes for the same + feature depending on which side of it was touched. + """ + if filters is not None: + raise ValidationException("`scope` and `filters` are mutually exclusive") + if session_id: + raise ValidationException("`scope` and `session_id` are mutually exclusive") + if jwt_params.p is not None: + raise AuthenticationException( + "`scope` requires a workspace- or admin-level key" + ) + + +async def _resolve_scope_option( + workspace_id: str, + scope: str | list[str], + *, + db_action: str, +) -> tuple[str | None, list[str] | None]: + """Map a validated `scope` option to (observer_override, session_allowlist). + + A single scope swaps the observer to the scope peer: conclusion recall is + then confined to the (scope, observed) collection and message recall to + the scope's session membership by existing observer semantics. A list of + scopes keeps the path peer as observer and returns the union of the + scopes' member sessions as an explicit allowlist (fail-closed when empty). + """ + async with tracked_db(db_action, read_only=True) as scope_db: + if isinstance(scope, str): + [scope_peer] = await crud.resolve_scope_peers( + scope_db, workspace_id, [scope] + ) + return scope_peer, None + + scope_peers = await crud.resolve_scope_peers(scope_db, workspace_id, scope) + union: list[str] = [] + seen: set[str] = set() + for scope_peer in scope_peers: + for session_name in await get_peer_session_names( + scope_db, workspace_id, scope_peer + ): + if session_name not in seen: + seen.add(session_name) + union.append(session_name) + + if len(union) > MAX_SESSION_ALLOWLIST_ENTRIES: + raise ValidationException( + "The scopes' combined membership exceeds the maximum of " + + f"{MAX_SESSION_ALLOWLIST_ENTRIES} sessions per request" + ) + return None, union + + @router.post( "/list", response_model=Page[schemas.Peer], @@ -54,7 +127,11 @@ async def get_peers( reverse: bool = Query(False, description="Whether to reverse the order of results"), db: AsyncSession = read_db, ): - """Get all Peers for a Workspace, paginated with optional filters.""" + """Get all Peers for a Workspace, paginated with optional filters. + + Scope peers are excluded by default; set `kind` to "scope" for scope peers + only, or "all" for everything. + """ filter_param = None if options and hasattr(options, "filters"): filter_param = options.filters @@ -67,6 +144,7 @@ async def get_peers( workspace_name=workspace_id, filters=filter_param, reverse=reverse, + kind=options.kind if options else None, ), ) @@ -100,6 +178,14 @@ async def get_or_create_peer( if not jwt_params.p: raise AuthenticationException("Peer ID not found in query parameter or JWT") peer.name = jwt_params.p + + # The scope namespace is reserved: scope peers are only created through + # the scopes facade (POST /workspaces/{workspace_id}/scopes). + validate_no_scope_peer_names( + [peer.name], + action="Use the scopes routes to create scopes.", + ) + result = await crud.get_or_create_peers( db, workspace_name=workspace_id, peers=[peer] ) @@ -122,7 +208,19 @@ async def update_peer( peer: schemas.PeerUpdate = Body(..., description="Updated peer parameters"), db: AsyncSession = db, ): - """Update a Peer's metadata and/or configuration.""" + """Update a Peer's metadata and/or configuration. + + Returns 422 if the peer is a scope — use the scopes routes to manage scopes. + """ + # Three-way on the reserved namespace, all enforced inside ``crud.update_peer`` + # on the resolved row so there is no check-then-use window: a real scope is + # refused (this route replaces `configuration` wholesale, so it must never touch + # a facade-managed peer); an existing *unflagged* peer that merely occupies the + # namespace is a normal peer and updates fine; and a reserved-prefix name that + # does not exist is refused by create-path validation rather than being minted. + # + # Kept out of the docstring deliberately: FastAPI publishes that into the + # OpenAPI description, and callers need the contract, not the mechanism. updated_peer = await crud.update_peer( db, workspace_name=workspace_id, peer_name=peer_id, peer=peer ) @@ -189,6 +287,46 @@ async def chat( Query a Peer's representation using natural language. Performs agentic search and reasoning to comprehensively answer the query based on all latent knowledge gathered about the peer from their messages and conclusions. """ + # Scope peers are never observed, so no representation of them exists to + # query. Covers the path-level observer too: a scope `peer_id` no longer + # errors out downstream now that crud.get_peer takes a plain name, and + # querying from a scope's perspective is a read-side surface that does not + # exist yet. + scope_candidates = [ + n for n in (peer_id, options.target) if n is not None and is_scope_peer_name(n) + ] + if scope_candidates: + async with tracked_db("peers.chat.scope_check", read_only=True) as s_db: + # Strict variant, matching the representation route: `target` is an + # observed position and nothing here creates the peer, so a reserved + # name that does not exist yet must be refused rather than answered + # and then turned into a scope. + await crud.reject_scope_observed( + s_db, + workspace_id, + scope_candidates, + action=( + "No representation is formed of a scope, so a scope cannot " + "be a chat observer or target." + ), + ) + + # Scoped reads: a single scope swaps the observer to the scope + # peer; a list of scopes becomes a session allowlist over their union. + observer = peer_id + scope_session_union: list[str] | None = None + if options.scope is not None: + _validate_scope_option( + filters=options.filters, + session_id=options.session_id, + jwt_params=jwt_params, + ) + observer_override, scope_session_union = await _resolve_scope_option( + workspace_id, options.scope, db_action="peers.chat.resolve_scope" + ) + if observer_override is not None: + observer = observer_override + # The session id arrives in the body, so require_auth can't gate on it. A # peer-scoped key may only scope a chat to a session its peer belongs to; # without this check it could read any session's messages (the dialectic @@ -220,6 +358,12 @@ async def chat( if not set(session_allowlist) <= member_sessions: raise AuthenticationException("JWT not permissioned for this resource") + # A list of scopes resolves to a session allowlist over their union, which + # replaces any filters-derived allowlist (the two are mutually exclusive, so + # only one can be set). + if scope_session_union is not None: + session_allowlist = scope_session_union + # Convert the caller's JSON Schema so malformed schemas fail immediately with 422 response_model: type[BaseModel] | None = None if options.response_format is not None: @@ -233,8 +377,20 @@ async def chat( peers_result = await crud.get_or_create_peers( peer_db, workspace_name=workspace_id, - peers=[schemas.PeerCreate(name=peer_id)], + peers=[schemas.PeerSpec(name=peer_id)], ) + # Re-check on the resolved row: the name-level check above ran before the + # peer was resolved, so a scope created in between would be picked up here + # as existing and used as the chat observer. Deliberately NOT named + # `observer` — that holds the effective observer, which a single `scope` + # has already swapped to the scope peer, and rebinding it here would + # silently undo the swap. + path_peer = peers_result.resource[0] + if is_scope_peer(path_peer.name, path_peer.internal_metadata): + raise ValidationException( + "No representation is formed of a scope, so a scope cannot be a " + + "chat observer or target." + ) await peer_db.commit() await peers_result.post_commit() @@ -262,7 +418,7 @@ async def chat( workspace_name=workspace_id, session_name=options.session_id, query=options.query, - observer=peer_id, + observer=observer, observed=options.target if options.target is not None else peer_id, reasoning_level=options.reasoning_level, session_allowlist=session_allowlist, @@ -276,7 +432,8 @@ async def chat( workspace_name=workspace_id, session_name=options.session_id, query=options.query, - observer=peer_id, + # a single `scope` swaps the observer to the scope peer + observer=observer, # if target is given, that's the observed peer. otherwise, observer==observed # and it's answered from the omniscient Honcho perspective observed=options.target if options.target is not None else peer_id, @@ -298,9 +455,6 @@ async def chat( @router.post( "/{peer_id}/representation", response_model=schemas.RepresentationResponse, - dependencies=[ - Depends(require_auth(workspace_name="workspace_id", peer_name="peer_id")) - ], ) async def get_representation( workspace_id: str = Path(...), @@ -308,6 +462,9 @@ async def get_representation( options: schemas.PeerRepresentationGet = Body( ..., description="Options for getting the peer representation" ), + jwt_params: JWTParams = Depends( + require_auth(workspace_name="workspace_id", peer_name="peer_id") + ), ): """Get a curated subset of a Peer's Representation. A Representation is always a subset of the total knowledge about the Peer. The subset can be scoped and filtered in various ways. @@ -317,45 +474,124 @@ async def get_representation( If a target is provided, we get the Representation of the target from the perspective of the Peer. If no target is provided, we get the omniscient Honcho Representation of the Peer. """ + # Fast-fail before any embedding work. Same guard as the authoritative one + # below, so a reserved name is refused here rather than after paying for an + # embedding; the check is repeated at the read because this session closes and + # a scope could be created in between. + scope_candidates = [ + n for n in (peer_id, options.target) if n is not None and is_scope_peer_name(n) + ] + if scope_candidates: + async with tracked_db( + "peers.representation.scope_check", read_only=True + ) as s_db: + await crud.reject_scope_observed( + s_db, + workspace_id, + scope_candidates, + action=( + "No representation is formed of a scope, so a scope cannot " + "be a representation observer or target." + ), + ) + # Parse the session allowlist from filters (422 on unsupported keys/shapes, # and on a session_id the allowlist doesn't cover). session_allowlist = extract_session_allowlist( options.filters, must_include=options.session_id ) + # Scoped reads: a single scope swaps the observer to the scope + # peer; a list of scopes becomes a session allowlist over their union. + observer = peer_id + scope_session_union: list[str] | None = None + if options.scope is not None: + _validate_scope_option( + filters=options.filters, + session_id=options.session_id, + jwt_params=jwt_params, + ) + observer_override, scope_session_union = await _resolve_scope_option( + workspace_id, options.scope, db_action="peers.representation.resolve_scope" + ) + if observer_override is not None: + observer = observer_override + if scope_session_union is not None: + session_allowlist = scope_session_union + try: embedding: list[float] | None = None if options.search_query: - with ( - suppress(Exception), - embedding_call_purpose( + try: + with embedding_call_purpose( EmbeddingCallPurpose.SEARCH_MEMORY.value, workspace_name=workspace_id, parent_category="api", - ), - ): - embedding = await embedding_client.embed(options.search_query) + ): + embedding = await embedding_client.embed(options.search_query) + except Exception: + # Swallowed on purpose (see include_semantic_query below), but not + # silently: without this a provider outage degrades every search + # request to derived+recent retrieval with no signal anywhere. + logger.warning( + "Representation search embedding failed for workspace %s," + + " degrading to non-semantic retrieval", + workspace_id, + exc_info=True, + ) - # If no target specified, get global representation (omniscient Honcho perspective) - representation = await crud.get_working_representation( - workspace_id, - observer=peer_id, - observed=options.target if options.target is not None else peer_id, - session_allowlist=[options.session_id] - if options.session_id is not None - else session_allowlist, - include_semantic_query=options.search_query, - embedding=embedding, - semantic_search_top_k=options.search_top_k, - semantic_search_max_distance=options.search_max_distance, - include_most_derived=options.include_most_frequent - if options.include_most_frequent is not None - else False, - max_observations=options.max_conclusions - if options.max_conclusions is not None - else settings.DERIVER.WORKING_REPRESENTATION_MAX_OBSERVATIONS, - parent_category="api", - ) + observed = options.target if options.target is not None else peer_id + # Re-check and read in one short session, opened only now — after the + # embedding call above, so no connection is held across external work. + # The early check ran in a session that has since closed and, being + # name-based, also passed any reserved name that did not yet exist; a scope + # created in between would otherwise be used here. Sharing the session with + # the read means a scope committed after this check cannot have any + # conclusions in the collection the read then examines. + async with tracked_db( + "peers.representation.read", read_only=True + ) as read_session: + await crud.reject_scope_observed( + read_session, + workspace_id, + {peer_id, observed}, + action=( + "No representation is formed of a scope, so a scope cannot be" + " a representation observer or target." + ), + ) + # If no target specified, this is the global (omniscient) representation + representation = await crud.get_working_representation( + workspace_id, + db=read_session, + # a single `scope` swaps the observer to the scope peer + observer=observer, + observed=observed, + session_allowlist=[options.session_id] + if options.session_id is not None + else session_allowlist, + # Only ask for the semantic branch when we actually have an + # embedding. The precompute above is suppressed, and both + # `RepresentationManager.get_working_representation` and + # `crud.query_documents` fall back to embedding internally when a + # query arrives without one — which would run an external call + # inside this session, and the innermost fallback is unsuppressed + # (a provider outage would surface as a 500). Degrading to + # derived+recent retrieval keeps the session DB-only. + include_semantic_query=options.search_query + if embedding is not None + else None, + embedding=embedding, + semantic_search_top_k=options.search_top_k, + semantic_search_max_distance=options.search_max_distance, + include_most_derived=options.include_most_frequent + if options.include_most_frequent is not None + else False, + max_observations=options.max_conclusions + if options.max_conclusions is not None + else settings.DERIVER.WORKING_REPRESENTATION_MAX_OBSERVATIONS, + parent_category="api", + ) return schemas.RepresentationResponse( representation=representation.format_as_markdown() ) @@ -421,6 +657,10 @@ async def set_peer_card( # If no target specified, set the observer's own card observed = target if target is not None else peer_id + # The scope guard lives in crud.set_peer_card, in the same transaction as the + # JSONB write, so the Dreamer and agent-tool paths are covered too. Nothing + # expensive happens between here and there, so a duplicate early check would + # only cost an extra query. await crud.set_peer_card( db, workspace_id, @@ -484,6 +724,25 @@ async def get_peer_context( This is useful for getting all the context needed about a peer without making multiple API calls. """ + # Scope peers may not appear on the generic peer-context surface: no + # representation is formed of a scope, and scoped reads go through the + # `scope` option on chat/representation/session-context instead. Flag-based + # rather than prefix-based, so a legacy peer merely occupying the reserved + # name keeps working; strict on a reserved name that does not exist yet, + # since nothing here creates it. Costs no query when no reserved name is + # present, and runs before any embedding work. + scope_candidates = [ + n for n in (peer_id, target) if n is not None and is_scope_peer_name(n) + ] + if scope_candidates: + async with tracked_db("peers.context.scope_check", read_only=True) as s_db: + await crud.reject_scope_observed( + s_db, + workspace_id, + scope_candidates, + action="Use the `scope` option on the read routes instead.", + ) + # If no target specified, get the peer's own context (self-observation) observed = target if target is not None else peer_id context_started = perf_counter() diff --git a/src/routers/scopes.py b/src/routers/scopes.py new file mode 100644 index 00000000..9d9087c0 --- /dev/null +++ b/src/routers/scopes.py @@ -0,0 +1,173 @@ +"""FastAPI routes for scope resources. + +A scope is a named grouping of sessions that provides a visibility boundary +within a peer. Internally a scope is a peer named ``scope.`` that +observes its member sessions and never speaks; these routes are the facade +that keeps the observer/observed mechanics hidden. + +All scopes routes require a workspace-level (or admin) key: scopes are an +app-level admin surface, so peer- and session-scoped keys are rejected. + +Note: scope membership only affects messages ingested *after* the membership +change. Conclusions already derived are neither backfilled on add nor +reconciled on removal. +""" + +import logging + +from fastapi import APIRouter, Body, Depends, Path, Query, Response +from fastapi_pagination import Page +from fastapi_pagination.ext.sqlalchemy import apaginate +from sqlalchemy.ext.asyncio import AsyncSession + +from src import crud, schemas +from src.dependencies import db, read_db +from src.security import require_auth + +logger = logging.getLogger(__name__) + +router = APIRouter( + prefix="/workspaces/{workspace_id}/scopes", + tags=["scopes"], +) + + +@router.post( + "", + response_model=schemas.Scope, + dependencies=[Depends(require_auth(workspace_name="workspace_id"))], +) +async def get_or_create_scope( + response: Response, + workspace_id: str = Path(...), + scope: schemas.ScopeCreate = Body(..., description="Scope creation parameters"), + db: AsyncSession = db, +): + """ + Get a Scope by ID or create a new Scope with the given ID. + + Returns 201 when the scope is created and 200 when it already exists. + A pre-existing peer occupying the scope's reserved internal name is never + adopted; that conflict returns 409. + """ + result = await crud.get_or_create_scopes(db, workspace_id, [scope]) + await db.commit() + await result.post_commit() + response.status_code = 201 if result.created else 200 + return result.resource[0] + + +@router.post( + "/list", + response_model=Page[schemas.Scope], + dependencies=[Depends(require_auth(workspace_name="workspace_id"))], +) +async def get_scopes( + workspace_id: str = Path(...), + reverse: bool = Query(False, description="Whether to reverse the order of results"), + db: AsyncSession = read_db, +): + """Get all Scopes for a Workspace. Results are paginated.""" + return await apaginate( + db, + await crud.get_scopes(workspace_name=workspace_id, reverse=reverse), + ) + + +@router.get( + "/{scope_id}", + response_model=schemas.Scope, + dependencies=[Depends(require_auth(workspace_name="workspace_id"))], +) +async def get_scope( + workspace_id: str = Path(...), + scope_id: str = Path(...), + db: AsyncSession = read_db, +): + """Get a single Scope by ID.""" + return await crud.get_scope_or_raise(db, workspace_id, scope_id) + + +@router.post( + "/{scope_id}/sessions", + status_code=204, + response_model=None, + dependencies=[Depends(require_auth(workspace_name="workspace_id"))], +) +async def add_sessions_to_scope( + workspace_id: str = Path(...), + scope_id: str = Path(...), + body: schemas.ScopeSessionsAdd = Body( + ..., description="IDs of the sessions to add to the scope" + ), + db: AsyncSession = db, +): + """ + Add Sessions to a Scope. + + All named sessions must already exist (404 otherwise). Adding a session that + is already a member is a no-op. List the resulting membership with + `POST /scopes/{scope_id}/sessions/list`. + + Note: membership applies only to messages ingested after this call; + conclusions already derived are not backfilled. + """ + await crud.add_sessions_to_scope( + db, + workspace_name=workspace_id, + scope_name=scope_id, + session_names=body.session_ids, + ) + + +@router.delete( + "/{scope_id}/sessions/{session_id}", + status_code=204, + response_model=None, + dependencies=[Depends(require_auth(workspace_name="workspace_id"))], +) +async def remove_session_from_scope( + workspace_id: str = Path(...), + scope_id: str = Path(...), + session_id: str = Path(...), + db: AsyncSession = db, +): + """ + Remove a Session from a Scope. + + Note: conclusions already derived while the session was a member are left in + place. + """ + await crud.remove_session_from_scope( + db, + workspace_name=workspace_id, + scope_name=scope_id, + session_name=session_id, + ) + + +@router.post( + "/{scope_id}/sessions/list", + response_model=Page[schemas.Session], + dependencies=[Depends(require_auth(workspace_name="workspace_id"))], +) +async def get_scope_sessions( + workspace_id: str = Path(...), + scope_id: str = Path(...), + reverse: bool = Query(False, description="Whether to reverse the order of results"), + db: AsyncSession = read_db, +): + """Get the Sessions that are members of a Scope, paginated. + + Ordered by how long each session has been a member: longest-standing member + first, or most recently added first when `reverse` is true. + """ + # Distinguishes an empty scope from one that does not exist; the query itself + # returns an empty page either way. + await crud.get_scope_or_raise(db, workspace_id, scope_id) + return await apaginate( + db, + await crud.get_scope_sessions( + workspace_name=workspace_id, scope_name=scope_id, reverse=reverse + ), + ) diff --git a/src/routers/sessions.py b/src/routers/sessions.py index 191748d4..10b769c4 100644 --- a/src/routers/sessions.py +++ b/src/routers/sessions.py @@ -35,6 +35,15 @@ router = APIRouter( tags=["sessions"], ) +# Guidance appended to guardrail errors when a scope peer is passed to the +# generic session-peer surface. Scope membership is managed only through the +# scopes facade so the observer mechanics stay internal. +_SCOPES_ROUTE_GUIDANCE = ( + "Scope membership is managed via the scopes routes " + "(/v3/workspaces/{workspace_id}/scopes/{scope_id}/sessions) or the `scopes` " + "field at session creation." +) + async def _get_working_representation_task( db: AsyncSession, @@ -309,6 +318,23 @@ async def get_or_create_session( ) session.name = jwt_params.s + # The `scopes` field does what the scopes routes do — create scope peers and + # attach memberships — so it needs their authorization: workspace-level or + # admin only. Checked here rather than through `require_auth(...)` because + # that closure only resolves path and query params, never the body, so a + # declarative gate cannot see this field. + if session.scopes and not ( + jwt_params.ad or (jwt_params.p is None and jwt_params.s is None) + ): + raise AuthenticationException("Scope membership requires a workspace-level key") + + # Scope peers may not be added through the generic peers mapping; use the + # `scopes` field (which handles scope-peer creation and observer config). + if session.peer_names: + await crud.reject_scope_peers( + db, workspace_id, session.peer_names.keys(), action=_SCOPES_ROUTE_GUIDANCE + ) + # Handle session creation with proper error handling try: result = await crud.get_or_create_session( @@ -440,7 +466,13 @@ async def add_peers_to_session( ), db: AsyncSession = db, ): - """Add Peers to a Session. If a Peer does not yet exist, it will be created automatically.""" + """Add Peers to a Session. If a Peer does not yet exist, it will be created automatically. + + Scope peers cannot be added here; scope membership is managed via the scopes routes. + """ + await crud.reject_scope_peers( + db, workspace_id, peers.keys(), action=_SCOPES_ROUTE_GUIDANCE + ) try: result = await crud.get_or_create_session( db, @@ -476,7 +508,12 @@ async def set_session_peers( Set the Peers in a Session. If a Peer does not yet exist, it will be created automatically. This will fully replace the current set of Peers in the Session. + + Scope peers cannot be set here; scope membership is managed via the scopes routes. """ + await crud.reject_scope_peers( + db, workspace_id, peers.keys(), action=_SCOPES_ROUTE_GUIDANCE + ) try: await crud.set_peers_for_session( db, @@ -512,7 +549,13 @@ async def remove_peers_from_session( ), db: AsyncSession = db, ): - """Remove Peers by ID from a Session.""" + """Remove Peers by ID from a Session. + + Scope peers cannot be removed here; scope membership is managed via the scopes routes. + """ + await crud.reject_scope_peers( + db, workspace_id, peers, action=_SCOPES_ROUTE_GUIDANCE + ) try: await crud.remove_peers_from_session( db, @@ -632,19 +675,17 @@ async def get_session_peers( @router.get( "/{session_id}/context", response_model=schemas.SessionContext, - dependencies=[ - Depends( - require_auth( - workspace_name="workspace_id", - session_name="session_id", - allow_member_read=True, - ) - ) - ], ) async def get_session_context( workspace_id: str = Path(...), session_id: str = Path(...), + jwt_params: JWTParams = Depends( + require_auth( + workspace_name="workspace_id", + session_name="session_id", + allow_member_read=True, + ) + ), db: AsyncSession = read_db, tokens: int | None = Query( None, @@ -669,6 +710,10 @@ async def get_session_context( None, description="A peer to get context for. If given, response will attempt to include representation and card from the perspective of that peer. Must be provided with `peer_target`.", ), + scope: str | None = Query( + None, + description="An (unprefixed) scope name to use as the perspective source: the representation and peer card of `peer_target` are read from the scope's observations instead of the global (or `peer_perspective`) view. Must be provided with `peer_target`; mutually exclusive with `peer_perspective`. Requires a workspace- or admin-level key.", + ), limit_to_session: bool = Query( default=False, description="Only used if `search_query` is provided. Whether to limit the representation to the session (as opposed to everything known about the target peer)", @@ -712,6 +757,50 @@ async def get_session_context( "peer_target must be provided if peer_perspective is provided" ) + # peer_target is the *observed* peer, and no representation or card is ever + # formed of a scope. Strict variant: an observed position that creates + # nothing, so a reserved name which does not exist yet must be refused too. + if peer_target is not None: + await crud.reject_scope_observed( + db, + workspace_id, + [peer_target], + action=( + "No representation is formed of a scope, so a scope cannot be a" + " context target." + ), + ) + + # peer_perspective is an observer position, where a scope is mechanically + # legitimate — but `scope` below is the supported way to ask for a scope's + # perspective, and routing through it is what keeps the observer mechanics + # hidden. Flag-based (not prefix-based) so a legacy peer merely occupying the + # reserved name keeps working, same as everywhere else. + if peer_perspective is not None: + await crud.reject_scope_peers( + db, + workspace_id, + [peer_perspective], + action="Use the `scope` parameter instead.", + ) + + if scope is not None: + if peer_perspective: + raise ValidationException( + "`scope` and `peer_perspective` are mutually exclusive" + ) + if not peer_target: + raise ValidationException( + "peer_target must be provided if scope is provided" + ) + # A scope's perspective spans sessions beyond this one, so scoped reads + # require a workspace- or admin-level key. 401, matching every other + # scope surface (see _validate_scope_option in routers/peers.py). + if jwt_params.p is not None or jwt_params.s is not None: + raise AuthenticationException( + "`scope` requires a workspace- or admin-level key" + ) + if not peer_target: # No representation or card needed summary, messages = await _get_session_context_task( @@ -745,6 +834,24 @@ async def get_session_context( observer = peer_perspective or peer_target observed = peer_target + # Member-read lets a peer-scoped key reach this route, but membership grants + # access to the *session*, not to a co-member's representation or peer card. + # The observer is whose knowledge is being read, so a peer-scoped key may only + # read from its own perspective — mirroring + # `POST /peers/{peer_id}/representation`, where require_auth pins the observer + # to the path peer and any `target` is that observer's own view. A bare + # `peer_target` naming another peer is the omniscient view of them, which is + # nobody's own perspective, so it is refused too. Workspace/admin and + # session-scoped tokens are unaffected. + if jwt_params.p is not None and jwt_params.p != observer: + raise AuthenticationException("JWT not permissioned for this resource") + + # A scope swaps the perspective source: the scope peer becomes the + # observer for both the working representation and the peer card, so the + # scoped collection and scoped card are read instead of the global ones. + if scope is not None: + [observer] = await crud.resolve_scope_peers(db, workspace_id, [scope]) + # Pre-compute embedding outside the DB session (best-effort) embedding: list[float] | None = None if search_query: diff --git a/src/routers/workspaces.py b/src/routers/workspaces.py index b7e15a71..42c111ad 100644 --- a/src/routers/workspaces.py +++ b/src/routers/workspaces.py @@ -7,11 +7,12 @@ from fastapi_pagination import Page from fastapi_pagination.ext.sqlalchemy import apaginate from sqlalchemy.ext.asyncio import AsyncSession -from src import crud, schemas +from src import crud, models, schemas from src.config import settings -from src.dependencies import db, read_db +from src.crud.message import get_peer_session_names +from src.dependencies import db, read_db, tracked_db from src.deriver.enqueue import enqueue_deletion, enqueue_dream -from src.exceptions import AuthenticationException +from src.exceptions import AuthenticationException, ValidationException from src.security import JWTParams, require_auth from src.utils.search import search @@ -141,16 +142,38 @@ async def delete_workspace( ) async def search_workspace( workspace_id: str = Path(...), - body: schemas.MessageSearchOptions = Body( + body: schemas.WorkspaceMessageSearchOptions = Body( ..., description="Message search parameters" ), ): """ Search messages in a Workspace using optional filters. Use `limit` to control the number of results returned. + + Pass `scope` to restrict the search to a scope's member sessions. A scope + with no member sessions returns no results (fail-closed). """ # take user-provided filter and add workspace_id to it filters = body.filters or {} + if body.scope is not None: + if "session_id" in filters: + raise ValidationException( + "`scope` and a 'session_id' filter are mutually exclusive" + ) + async with tracked_db( + "workspaces.search.resolve_scope", read_only=True + ) as scope_db: + [scope_peer] = await crud.resolve_scope_peers( + scope_db, workspace_id, [body.scope] + ) + scope_sessions = await get_peer_session_names( + scope_db, workspace_id, scope_peer + ) + if not scope_sessions: + # A scope with no member sessions matches nothing, not everything. + no_results: list[models.Message] = [] + return no_results + filters["session_id"] = {"in": scope_sessions} filters["workspace_id"] = workspace_id return await search(body.query, filters=filters, limit=body.limit) @@ -225,6 +248,10 @@ async def schedule_dream( observed = request.observed if request.observed is not None else request.observer dream_type = request.dream_type + # The authoritative observed-position check lives in enqueue_dream, in the same + # transaction as the queue insert. Nothing expensive happens before it here, so + # no early duplicate is needed. + await enqueue_dream( workspace_id, observer=observer, diff --git a/src/schemas/__init__.py b/src/schemas/__init__.py index 85d5da44..32e144b3 100644 --- a/src/schemas/__init__.py +++ b/src/schemas/__init__.py @@ -31,10 +31,14 @@ from src.schemas.api import ( PeerCreate, PeerGet, PeerRepresentationGet, + PeerSpec, PeerUpdate, QueueStatus, RepresentationResponse, ScheduleDreamRequest, + Scope, + ScopeCreate, + ScopeSessionsAdd, Session, SessionBase, SessionContext, @@ -51,6 +55,7 @@ from src.schemas.api import ( WorkspaceBase, WorkspaceCreate, WorkspaceGet, + WorkspaceMessageSearchOptions, WorkspaceUpdate, ) from src.schemas.configuration import ( @@ -125,12 +130,16 @@ __all__ = [ "PeerContext", "PeerCreate", "PeerGet", + "PeerSpec", "PeerRepresentationGet", "PeerUpdate", "QueueStatus", "RESOURCE_NAME_PATTERN", "RepresentationResponse", "ScheduleDreamRequest", + "Scope", + "ScopeCreate", + "ScopeSessionsAdd", "Session", "SessionBase", "SessionContext", @@ -147,6 +156,7 @@ __all__ = [ "WorkspaceBase", "WorkspaceCreate", "WorkspaceGet", + "WorkspaceMessageSearchOptions", "WorkspaceUpdate", # internal "DocumentBase", diff --git a/src/schemas/api.py b/src/schemas/api.py index 78e5a125..411f2d14 100644 --- a/src/schemas/api.py +++ b/src/schemas/api.py @@ -6,11 +6,13 @@ API contract. import datetime import ipaddress -from typing import Annotated, Any, Self, cast +import re +from typing import Annotated, Any, Literal, Self, cast from urllib.parse import urlparse import tiktoken from pydantic import ( + AfterValidator, AliasChoices, BaseModel, BeforeValidator, @@ -29,6 +31,11 @@ from src.schemas.configuration import ( SessionPeerConfig, WorkspaceConfiguration, ) +from src.utils.scopes import ( + SCOPE_PEER_PREFIX, + is_scope_peer_name, + scope_name_from_peer, +) from src.utils.types import DocumentLevel # --------------------------------------------------------------------------- @@ -86,6 +93,44 @@ def _validate_metadata(v: Any) -> Any: _SanitizedMetadata = Annotated[dict[str, Any], BeforeValidator(_validate_metadata)] +# Scope names are stored as peer names with the reserved prefix prepended, so +# they must leave room for the prefix within the 512-character peer name limit. +_SCOPE_NAME_MAX_LENGTH = 512 - len(SCOPE_PEER_PREFIX) + + +def _validate_scope_name(name: str) -> str: + """Validate an unprefixed scope name.""" + if not 1 <= len(name) <= _SCOPE_NAME_MAX_LENGTH: + raise ValueError( + f"Scope name must be between 1 and {_SCOPE_NAME_MAX_LENGTH} characters" + ) + # Checked before the charset pattern: the reserved prefix is itself outside + # RESOURCE_NAME_PATTERN, so the pattern would otherwise reject a + # double-prefixed name first and report the charset instead of the real + # mistake. + if name.startswith(SCOPE_PEER_PREFIX): + raise ValueError( + "Scope name must not start with the reserved prefix " + + f"'{SCOPE_PEER_PREFIX}' (scope names are unprefixed)" + ) + if not re.fullmatch(RESOURCE_NAME_PATTERN, name): + raise ValueError(f"Scope name must match pattern {RESOURCE_NAME_PATTERN}") + return name + + +_ScopeName = Annotated[str, AfterValidator(_validate_scope_name)] + +# The `scope` read option (chat / representation): one scope name, or a bounded +# list of them. The length cap sits on the list member so it bounds the *list* — +# a single name is already bounded by `_validate_scope_name`, and a union-level +# `max_length` would cap that name's characters instead. The upper bound matches +# `SessionCreate.scopes`; the lower one rejects `[]`, which would otherwise +# resolve to an empty allowlist and silently recall nothing. +_ScopeOption = ( + _ScopeName | Annotated[list[_ScopeName], Field(min_length=1, max_length=100)] +) + + # --------------------------------------------------------------------------- # Workspace schemas # --------------------------------------------------------------------------- @@ -139,19 +184,47 @@ class PeerBase(BaseModel): pass -class PeerCreate(PeerBase): +class PeerSpec(PeerBase): + """Peer identity plus optional updates, for callers that already have a name. + + ``PeerCreate`` narrows ``name`` with ``pattern=RESOURCE_NAME_PATTERN`` because it + validates a *new, user-supplied* peer id at the API boundary. crud paths reach + ``get_or_create_peers`` with names that already exist — a path param, a message + author, an existing row — including pre-``d429de0e5338`` legacy names containing + '.' and every ``scope.``-prefixed peer name. Re-validating those turns a lookup + into a raw pydantic ValidationError, i.e. an HTTP 500. + + Carries **no** constraints at all, deliberately. Length limits here were the + same trap as the charset pattern: request-bound peer names (message authors, + session peer-map keys) have no length bound of their own, so an empty or + over-long name reached ``PeerSpec(...)`` and raised internally — again a 500. + Every rule for a *new* name lives in ``crud.peer._validate_new_peer_names``, + which runs on the insert path only. + """ + + name: str + metadata: _SanitizedMetadata | None = None + configuration: dict[str, Any] | None = None + + +class PeerCreate(PeerSpec): name: Annotated[ str, Field(alias="id", min_length=1, max_length=512, pattern=RESOURCE_NAME_PATTERN), ] - metadata: _SanitizedMetadata | None = None - configuration: dict[str, Any] | None = None model_config = ConfigDict(populate_by_name=True) # pyright: ignore class PeerGet(PeerBase): filters: dict[str, Any] | None = None + kind: Literal["scope", "all"] | None = Field( + default=None, + description=( + "Which kinds of peers to list. Omitted (default): regular peers only " + "(scope peers are excluded). 'scope': scope peers only. 'all': every peer." + ), + ) class PeerUpdate(PeerBase): @@ -186,6 +259,19 @@ class PeerRepresentationGet(BaseModel): "must be included in the allowlist." ), ) + scope: _ScopeOption | None = Field( + None, + description=( + "Optional (unprefixed) scope name(s) to confine the representation. " + "A single scope reads the scope's own representation of the target " + "peer, formed only from the scope's member sessions. A list of " + "scopes restricts the representation to conclusions from the union " + "of the scopes' member sessions (explicit allowlist, fail-closed: " + "an empty union yields an empty representation). Mutually " + "exclusive with `filters` and `session_id`. Requires a workspace- " + "or admin-level key." + ), + ) target: str | None = Field( None, description="Optional peer ID to get the representation for, from the perspective of this peer", @@ -337,6 +423,25 @@ class SessionCreate(SessionBase): metadata: _SanitizedMetadata | None = None peer_names: dict[str, SessionPeerConfig] | None = Field(default=None, alias="peers") configuration: SessionConfiguration | None = None + scopes: list[str] | None = Field( + default=None, + max_length=100, + description=( + "Optional list of (unprefixed) scope names to add this session to. " + "Each scope is created if it does not exist yet. Membership applies " + "only to messages ingested after the session is added to the scope; " + "conclusions already derived are not backfilled." + ), + ) + + @field_validator("scopes") + @classmethod + def validate_scopes(cls, v: list[str] | None) -> list[str] | None: + if v is None: + return v + for scope_name in v: + _validate_scope_name(scope_name) + return v model_config = ConfigDict(populate_by_name=True) # pyright: ignore @@ -431,6 +536,61 @@ class SessionSummaries(SessionBase): ) +# --------------------------------------------------------------------------- +# Scope schemas +# --------------------------------------------------------------------------- + + +class ScopeCreate(BaseModel): + """Schema for creating (or getting) a scope by its unprefixed name.""" + + name: Annotated[str, Field(alias="id", min_length=1)] + metadata: _SanitizedMetadata | None = None + + @field_validator("name") + @classmethod + def validate_name(cls, v: str) -> str: + return _validate_scope_name(v) + + model_config = ConfigDict(populate_by_name=True) # pyright: ignore + + +class Scope(BaseModel): + """Scope response — external view of the peer backing a scope. + + The ``id`` is the unprefixed scope name; the reserved peer-name prefix is + an internal implementation detail and never surfaces here. + """ + + name: str = Field(serialization_alias="id") + h_metadata: dict[str, Any] = Field( + default_factory=dict, serialization_alias="metadata" + ) + created_at: datetime.datetime + + @field_validator("name", mode="after") + @classmethod + def strip_scope_prefix(cls, v: str) -> str: + # Constructed from Peer ORM rows whose names carry the prefix; accept + # already-unprefixed names too so manual construction works. + return scope_name_from_peer(v) if is_scope_peer_name(v) else v + + model_config = ConfigDict( # pyright: ignore + from_attributes=True, populate_by_name=True + ) + + +class ScopeSessionsAdd(BaseModel): + """Schema for adding sessions to a scope.""" + + session_ids: list[str] = Field( + ..., + min_length=1, + max_length=100, + description="IDs of existing sessions to add to the scope", + ) + + # --------------------------------------------------------------------------- # Conclusion schemas # --------------------------------------------------------------------------- @@ -562,6 +722,20 @@ class MessageSearchOptions(BaseModel): return v.replace("\x00", "") +class WorkspaceMessageSearchOptions(MessageSearchOptions): + """Workspace-level message search options, extended with `scope`.""" + + scope: str | None = Field( + default=None, + description=( + "Optional (unprefixed) scope name restricting search to the " + "scope's member sessions. A scope with no member sessions returns " + "no results. Mutually exclusive with a 'session_id' key in " + "`filters`." + ), + ) + + # --------------------------------------------------------------------------- # Dialectic schemas # --------------------------------------------------------------------------- @@ -581,6 +755,19 @@ class DialecticOptions(BaseModel): "also set, it must be included in the allowlist." ), ) + scope: _ScopeOption | None = Field( + None, + description=( + "Optional (unprefixed) scope name(s) to confine recall. A single " + "scope answers from the scope's own representation of the target " + "peer: conclusion recall is confined to what the scope observed " + "and message recall to the scope's member sessions. A list of " + "scopes restricts recall to the union of the scopes' member " + "sessions (explicit allowlist, fail-closed: an empty union " + "recalls nothing). Mutually exclusive with `filters` and " + "`session_id`. Requires a workspace- or admin-level key." + ), + ) target: str | None = Field( None, description="Optional peer to get the representation for, from the perspective of this peer", diff --git a/src/telemetry/events/trace.py b/src/telemetry/events/trace.py index 37383bc0..5218c302 100644 --- a/src/telemetry/events/trace.py +++ b/src/telemetry/events/trace.py @@ -143,8 +143,8 @@ class TraceContentEvent(BaseEvent): # Tool calls in a unified {id, name, input} shape (provider-agnostic). tool_calls: list[dict[str, Any]] = Field(default_factory=list) # Tags Honcho-authored content (system prompts, scaffold) so tenant-facing - # views can withhold globally-shared content (the §6.3 access invariant — - # dedup is global, the content store has no tenant column). + # views can withhold globally-shared content: dedup is global, and the + # content store has no tenant column. honcho_authored: bool = False def get_resource_id(self) -> str: diff --git a/src/utils/scopes.py b/src/utils/scopes.py new file mode 100644 index 00000000..c7b89192 --- /dev/null +++ b/src/utils/scopes.py @@ -0,0 +1,89 @@ +"""Scope namespace helpers. + +A *scope* is a named grouping of sessions that provides a visibility boundary +within a peer. Under the hood a scope named ``therapy`` is a peer named +``scope.therapy`` that observes its member sessions and never speaks. +Developers manage scopes exclusively through the ``/scopes`` routes (and the +``scopes`` field on session creation) and never see the observer/observed +mechanics. + +This module is the single source of truth for the reserved name prefix and the +``kind`` flag. Being a scope requires **both**: the reserved name prefix (the +namespace) and ``{"kind": "scope"}`` in the peer's ``internal_metadata`` JSONB +(the authoritative marker). Neither half is forgeable — the prefix sits outside +``RESOURCE_NAME_PATTERN``, and ``internal_metadata`` appears in no API schema — +so requiring both means a peer that merely occupies the namespace, or merely +carries a look-alike ``configuration``, is not a scope. +""" + +from collections.abc import Iterable +from typing import Any + +from src.exceptions import ValidationException + +# Reserved peer-name prefix for scope peers. User-created peers may not use it. +# +# The '.' is load-bearing: it is outside RESOURCE_NAME_PATTERN +# (^[a-zA-Z0-9_-]+$), the charset every peer name created through the API must +# match. No peer created through the validated API can therefore occupy this +# namespace. (Peers carried over by the users->peers rename in +# d429de0e5338 predate that pattern and were never charset-validated, so the +# legacy-collision path in crud/scope.py stays as a backstop.) +SCOPE_PEER_PREFIX = "scope." + +# Value of the `kind` configuration flag carried by scope peers. +SCOPE_KIND = "scope" + + +def scope_peer_name(scope_name: str) -> str: + """Return the peer name backing the given (unprefixed) scope name.""" + return f"{SCOPE_PEER_PREFIX}{scope_name}" + + +def is_scope_peer_name(name: str) -> bool: + """Return whether a peer name lives in the reserved scope namespace.""" + return name.startswith(SCOPE_PEER_PREFIX) + + +def scope_name_from_peer(peer_name: str) -> str: + """Return the unprefixed scope name for a scope peer name. + + Raises: + ValueError: If the peer name is not in the scope namespace. + """ + if not is_scope_peer_name(peer_name): + raise ValueError(f"{peer_name} is not a scope peer name") + return peer_name[len(SCOPE_PEER_PREFIX) :] + + +def is_scope_peer(name: str, internal_metadata: dict[str, Any] | None) -> bool: + """Authoritative scope test: reserved name AND the internal kind flag. + + Takes ``(name, internal_metadata)`` rather than a ``Peer`` so it is callable + from an ORM instance, the cached plain dict built by ``crud.peer._fetch_peer``, + or a raw row. + """ + return ( + is_scope_peer_name(name) + and bool(internal_metadata) + and internal_metadata.get("kind") == SCOPE_KIND + ) + + +def validate_no_scope_peer_names(names: Iterable[str], *, action: str) -> None: + """Reject any peer name that uses the reserved scope namespace. + + Args: + names: Peer names to check. + action: Human-readable guidance appended to the error, directing the + caller to the supported path (e.g. the ``/scopes`` routes). + + Raises: + ValidationException: If any name starts with the reserved prefix. + """ + offenders = sorted({name for name in names if is_scope_peer_name(name)}) + if offenders: + raise ValidationException( + f"Peer name(s) {offenders} use the reserved scope prefix " + + f"'{SCOPE_PEER_PREFIX}'. {action}" + ) diff --git a/src/vector_store/__init__.py b/src/vector_store/__init__.py index 96ae172d..6f9eb413 100644 --- a/src/vector_store/__init__.py +++ b/src/vector_store/__init__.py @@ -207,9 +207,9 @@ def _create_store_by_type(store_type: str) -> VectorStore: except ImportError as exc: raise RuntimeError( "VECTOR_STORE.TYPE is set to 'lancedb', but the 'lancedb' package " - + "is not installed (for example on macOS Intel, where it is omitted " - + "from dependencies because PyPI has no wheel). " - + "Use TYPE 'pgvector' or 'turbopuffer', or install lancedb manually. " + + "could not be imported. Install Honcho's 'lancedb' extra " + + "(for example, `uv sync --extra lancedb`; unavailable on Intel " + + "macOS), or use TYPE 'pgvector' or 'turbopuffer'. " + f"Original import error: {exc}" ) from exc diff --git a/tests/conftest.py b/tests/conftest.py index 1ec64055..b57e3786 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -10,9 +10,7 @@ from unittest.mock import AsyncMock, MagicMock, patch import jwt import pytest import pytest_asyncio -from cashews.backends.interface import ControlMixin from cashews.picklers import PicklerType -from fakeredis import FakeAsyncRedis from fastapi import Request from fastapi.responses import JSONResponse from fastapi.testclient import TestClient @@ -385,54 +383,32 @@ async def db_session(db_engine: AsyncEngine): @pytest_asyncio.fixture(scope="session") async def fake_cache_session(): - """Set up fakeredis for caching once per test session.""" + """Set up a taskless in-memory cache once per test session. + + Cashews' normal memory backend starts a periodic expiry task on whichever + event loop first uses it. Tests use both pytest-asyncio loops and TestClient + portal loops, so that task can be cancelled when its originating loop closes + and then leak a CancelledError into the next app startup. Disabling the + periodic sweep keeps the backend loop-agnostic; expired entries are still + discarded lazily when read. + """ # Store original settings original_enabled = settings.CACHE.ENABLED original_url = settings.CACHE.URL - # Create a fake redis instance that persists for the session - fake_redis = FakeAsyncRedis(decode_responses=True) - - # Patch redis creation to use fakeredis - # Cashews uses redis.asyncio.from_url to create connections - def fake_redis_from_url(*_args: Any, **_kwargs: Any): - return fake_redis - - # Patch the cashews backend's _disable property to avoid ContextVar issues - # This works around cashews' ContextVar not being properly initialized in TestClient context - - original_disable_property = ControlMixin._disable # pyright: ignore[reportPrivateUsage] - - @property # type: ignore - def patched_disable_property(self): # pyright: ignore - try: - return original_disable_property.fget(self) # pyright: ignore[reportOptionalCall] - except LookupError: - # Return empty set as default if ContextVar not set in current context - return set() # pyright: ignore - - # Start patching - redis_patch = patch("redis.asyncio.from_url", fake_redis_from_url) - redis_patch.start() - ControlMixin._disable = patched_disable_property # pyright: ignore[reportPrivateUsage, reportAttributeAccessIssue] - try: - # Enable caching and set URL for tests + # Use the same backend from pytest-asyncio and TestClient event loops. settings.CACHE.ENABLED = True - settings.CACHE.URL = "redis://fake-redis:6379/0" - - # Setup cache for tests that don't use TestClient (direct CRUD tests) - # For TestClient tests, the app's lifespan handler will also call cache.setup() - # The ContextVar patch above handles any context issues + settings.CACHE.URL = "mem://?check_interval=0" cache.setup( - "redis://fake-redis:6379/0", pickle_type=PicklerType.SQLALCHEMY, enable=True + settings.CACHE.URL, + pickle_type=PicklerType.SQLALCHEMY, + enable=True, ) - yield fake_redis + yield cache finally: - # Stop the patches - redis_patch.stop() - ControlMixin._disable = original_disable_property # pyright: ignore[reportPrivateUsage, reportAttributeAccessIssue] + await cache.close() # Restore original settings settings.CACHE.ENABLED = original_enabled @@ -440,21 +416,21 @@ async def fake_cache_session(): @pytest_asyncio.fixture(scope="function", autouse=True) -async def fake_cache(fake_cache_session: FakeAsyncRedis): +async def fake_cache(fake_cache_session: Any): # pyright: ignore[reportUnusedParameter] """Clear cache between tests.""" # Clear cache before each test - await fake_cache_session.flushall() # pyright: ignore[reportUnknownMemberType] + await cache.clear() yield cache # Clear cache after each test - await fake_cache_session.flushall() # pyright: ignore[reportUnknownMemberType] + await cache.clear() @pytest.fixture(scope="function") async def client( db_session: AsyncSession, - fake_cache_session: FakeAsyncRedis, # pyright: ignore[reportUnusedParameter] + fake_cache_session: Any, # pyright: ignore[reportUnusedParameter] monkeypatch: pytest.MonkeyPatch, ) -> AsyncGenerator[TestClient, Any]: """Create a FastAPI TestClient for the scope of a single test function""" @@ -964,6 +940,7 @@ def mock_tracked_db(request: pytest.FixtureRequest): "src.deriver.consumer.tracked_db", "src.deriver.enqueue.tracked_db", "src.routers.peers.tracked_db", + "src.routers.workspaces.tracked_db", "src.crud.representation.tracked_db", "src.dreamer.orchestrator.tracked_db", "src.dreamer.dream_scheduler.tracked_db", diff --git a/tests/crud/test_get_or_create_retry_invalidation.py b/tests/crud/test_get_or_create_retry_invalidation.py new file mode 100644 index 00000000..fc5e93b9 --- /dev/null +++ b/tests/crud/test_get_or_create_retry_invalidation.py @@ -0,0 +1,197 @@ +"""Regression tests for cache invalidation across the get_or_create retry path. + +`get_or_create_peers` / `get_or_create_scopes` mutate existing rows, then insert +new ones inside `db.begin_nested()`. A concurrent writer that creates one of those +rows first makes the insert raise `IntegrityError`, and the function retries. + +The subtlety: `begin_nested()` autoflushes the pending mutations *before* opening +the savepoint, so the rollback neither undoes them nor expires the ORM state. A +retry that recomputed "what changed" from that state would see no change and skip +the cache purge — while the row change still commits anyway, leaving the cache +stale until TTL. These tests pin the purge. + +The race is real (a second session committing a real row, producing a real +IntegrityError from the database); only its *timing* is made deterministic, by +hooking the one point that sits between the SELECT and the INSERT. +""" + +from unittest.mock import AsyncMock, patch + +import pytest +from nanoid import generate as generate_nanoid +from sqlalchemy.ext.asyncio import ( + AsyncEngine, + AsyncSession, + AsyncSessionTransaction, + async_sessionmaker, +) + +from src import crud, models, schemas +from src.crud.peer import peer_cache_key +from src.crud.scope import SCOPE_PEER_CONFIGURATION, SCOPE_PEER_INTERNAL_METADATA +from src.utils.scopes import scope_peer_name + + +class _RaceOnBeginNested: + """Commit a racing row on entry to `begin_nested()`, then delegate. + + That entry point is after the function's SELECT and metadata mutation but + before its INSERT flushes — precisely the window a real concurrent writer + has to slip through to trigger the IntegrityError retry. + """ + + _db: AsyncSession + _engine: AsyncEngine + _rows: list[models.Peer] + _real: AsyncSessionTransaction | None + fired: bool + + def __init__(self, db: AsyncSession, engine: AsyncEngine, rows: list[models.Peer]): + self._db = db + self._engine = engine + self._rows = rows + self._real = None + self.fired = False + + def __call__(self): + return self + + async def __aenter__(self): + if self._rows: + Session = async_sessionmaker(bind=self._engine, expire_on_commit=False) + async with Session() as other: + other.add_all(self._rows) + await other.commit() + self._rows = [] # race only once; the retry must succeed + self.fired = True + self._real = AsyncSession.begin_nested(self._db) + return await self._real.__aenter__() + + async def __aexit__(self, *exc_info: object): + assert self._real is not None + return await self._real.__aexit__(*exc_info) + + +@pytest.mark.asyncio +async def test_peer_retry_still_invalidates_mutated_peer( + db_session: AsyncSession, + db_engine: AsyncEngine, + sample_data: tuple[models.Workspace, models.Peer], +): + """A peer mutated before a losing race still gets its cache key purged.""" + test_workspace, existing_peer = sample_data + racer_name = str(generate_nanoid()) + + # Give the existing peer metadata we will then change, so it is a real update. + existing_peer.h_metadata = {"v": "old"} + await db_session.commit() + + race = _RaceOnBeginNested( + db_session, + db_engine, + [models.Peer(name=racer_name, workspace_name=test_workspace.name)], + ) + + with ( + patch("src.crud.peer.safe_cache_delete", new=AsyncMock()) as mock_delete, + patch.object(db_session, "begin_nested", race), + ): + result = await crud.get_or_create_peers( + db_session, + test_workspace.name, + [ + schemas.PeerCreate(name=existing_peer.name, metadata={"v": "new"}), + schemas.PeerCreate(name=racer_name), + ], + ) + await db_session.commit() + await result.post_commit() + + assert race.fired, "the race must actually have fired" + + purged = {call.args[0] for call in mock_delete.await_args_list} + assert ( + peer_cache_key(test_workspace.name, existing_peer.name) in purged + ), "the mutated peer's cache key must still be purged after the retry" + + # The mutation really did land — which is what makes a missed purge stale. + await db_session.refresh(existing_peer) + assert existing_peer.h_metadata == {"v": "new"} + + +@pytest.mark.asyncio +async def test_scope_retry_still_invalidates_mutated_scope( + db_session: AsyncSession, + db_engine: AsyncEngine, + sample_data: tuple[models.Workspace, models.Peer], +): + """Same guarantee for the scopes facade, which mirrors get_or_create_peers.""" + test_workspace, _ = sample_data + kept_scope, racing_scope = str(generate_nanoid()), str(generate_nanoid()) + + seeded = await crud.get_or_create_scopes( + db_session, + test_workspace.name, + [schemas.ScopeCreate(name=kept_scope, metadata={"v": "old"})], + ) + await db_session.commit() + await seeded.post_commit() + + # The racer creates the second scope's backing peer — as a *valid* scope peer, + # so the flow reaches the insert rather than tripping the legacy-collision 409. + race = _RaceOnBeginNested( + db_session, + db_engine, + [ + models.Peer( + name=scope_peer_name(racing_scope), + workspace_name=test_workspace.name, + internal_metadata=dict(SCOPE_PEER_INTERNAL_METADATA), + configuration=dict(SCOPE_PEER_CONFIGURATION), + ) + ], + ) + + with ( + patch("src.crud.scope.safe_cache_delete", new=AsyncMock()) as mock_delete, + patch.object(db_session, "begin_nested", race), + ): + result = await crud.get_or_create_scopes( + db_session, + test_workspace.name, + [ + schemas.ScopeCreate(name=kept_scope, metadata={"v": "new"}), + schemas.ScopeCreate(name=racing_scope), + ], + ) + await db_session.commit() + await result.post_commit() + + assert race.fired, "the race must actually have fired" + + purged = {call.args[0] for call in mock_delete.await_args_list} + assert ( + peer_cache_key(test_workspace.name, scope_peer_name(kept_scope)) in purged + ), "the mutated scope peer's cache key must still be purged after the retry" + + +@pytest.mark.asyncio +async def test_peer_no_race_does_not_invalidate_unchanged_peer( + db_session: AsyncSession, + sample_data: tuple[models.Workspace, models.Peer], +): + """Baseline: with no race, an unchanged peer is not purged.""" + test_workspace, existing_peer = sample_data + existing_peer.h_metadata = {"v": "same"} + await db_session.commit() + + with patch("src.crud.peer.safe_cache_delete", new=AsyncMock()) as mock_delete: + result = await crud.get_or_create_peers( + db_session, + test_workspace.name, + [schemas.PeerCreate(name=existing_peer.name, metadata={"v": "same"})], + ) + await db_session.commit() + await result.post_commit() + + assert mock_delete.await_count == 0, "an unchanged peer must not be purged" diff --git a/tests/crud/test_representation_manager.py b/tests/crud/test_representation_manager.py index f551de76..6c5e094c 100644 --- a/tests/crud/test_representation_manager.py +++ b/tests/crud/test_representation_manager.py @@ -236,7 +236,7 @@ class TestRepresentationManagerSoftDelete: class TestRepresentationManagerSessionScoping: """Tests that the session allowlist is applied uniformly to every query path. - Regression for DEV-1994: session_name used to be applied only to the + Regression: session_name used to be applied only to the recent-documents query; the semantic and most-derived paths ignored it, so limit_to_session leaked cross-session conclusions. """ @@ -368,7 +368,7 @@ class TestRepresentationManagerSessionScoping: assert mock_query.await_args.kwargs["filters"] == { "session_name": {"in": [session_a.name]}, # Scoped recall serves only levels with a trustworthy session - # stamp (ALLOWLIST_SAFE_LEVELS / DEV-2201). + # stamp (ALLOWLIST_SAFE_LEVELS). "level": {"in": ["explicit"]}, } @@ -445,7 +445,7 @@ class TestRepresentationManagerSessionScoping: ) # Scoping also narrows to levels whose session stamp is trustworthy - # (see ALLOWLIST_SAFE_LEVELS / DEV-2201). + # (see ALLOWLIST_SAFE_LEVELS). assert manager._build_filter_conditions(session_allowlist=[]) == { # pyright: ignore[reportPrivateUsage] "session_name": {"in": []}, "level": {"in": ["explicit"]}, @@ -629,3 +629,72 @@ class TestRepresentationManagerSave: assert len(saved.created_documents) == 0 mock_embed.assert_not_awaited() mock_save.assert_not_awaited() + + +class TestVectorQueryTopKFloor: + """Regression for HONCHO-19Q / HONCHO-4Q4. + + A top_k of 0 reached Turbopuffer, which rejects it with a 400 + ('top_k must be between 1 and 10000'). Two independent paths produced it: + the working-representation budget split (``total // 3`` rounds to 0 for + max_conclusions < 3) and the dialectic ``search_memory`` tool, whose + LLM-supplied top_k has an upper clamp but no floor. + """ + + @pytest.mark.asyncio + async def test_query_documents_returns_empty_without_querying_on_zero_top_k(self): + """The choke point every semantic document query routes through.""" + from src.crud.document import query_documents + + with ( + patch( + "src.crud.document.embedding_client.embed", new=AsyncMock() + ) as mock_embed, + patch( + "src.crud.document.query_external_vector_document_ids", + new=AsyncMock(), + ) as mock_vector, + ): + for top_k in (0, -1): + assert ( + await query_documents( + None, + "workspace", + "query", + observer="observer", + observed="observed", + top_k=top_k, + ) + == [] + ) + + mock_embed.assert_not_awaited() + mock_vector.assert_not_awaited() + + @pytest.mark.asyncio + async def test_requested_semantic_search_always_gets_budget( + self, + db_session: AsyncSession, + sample_data: tuple[models.Workspace, models.Peer], + ): + """max_conclusions < 3 must not allocate 0 to an explicitly requested search.""" + test_workspace, test_peer = sample_data + manager = RepresentationManager( + test_workspace.name, observer=test_peer.name, observed=test_peer.name + ) + + for max_observations in (1, 2, 100): + with patch( + "src.crud.query_documents", new=AsyncMock(return_value=[]) + ) as mock_query: + await manager._get_working_representation_internal( # pyright: ignore[reportPrivateUsage] + db_session, + include_semantic_query="what do they like?", + embedding=[0.1], + max_observations=max_observations, + ) + + assert mock_query.await_args is not None + top_k = mock_query.await_args.kwargs["top_k"] + assert top_k >= 1, f"max_observations={max_observations} gave top_k={top_k}" + assert top_k <= max_observations diff --git a/tests/dialectic/test_scope_preflight.py b/tests/dialectic/test_scope_preflight.py new file mode 100644 index 00000000..d3bacb47 --- /dev/null +++ b/tests/dialectic/test_scope_preflight.py @@ -0,0 +1,82 @@ +"""The scope guard inside the dialectic entry points. + +The route tests mock `agentic_chat` wholesale (`mock_llm_call_functions` in +tests/conftest.py), so the preflight *inside* it has no coverage there — which is +how a guard that rejected every scoped chat went unnoticed. These call it +directly with the agent stubbed, so no LLM work happens. +""" + +from unittest.mock import AsyncMock, patch + +import pytest +from fastapi.testclient import TestClient +from nanoid import generate as generate_nanoid +from sqlalchemy.ext.asyncio import AsyncSession + +from src.dialectic.chat import agentic_chat +from src.exceptions import ValidationException +from src.models import Peer, Workspace +from src.utils.scopes import scope_peer_name + + +async def _create_scope( + client: TestClient, db_session: AsyncSession, workspace_name: str +) -> str: + """Create a scope and commit it — the preflight opens its own connection.""" + scope_name = str(generate_nanoid()) + response = client.post( + f"/v3/workspaces/{workspace_name}/scopes", json={"id": scope_name} + ) + assert response.status_code in [200, 201] + await db_session.commit() + return scope_name + + +@pytest.mark.asyncio +async def test_scope_observer_reaches_the_agent( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], +): + """A single `scope` swaps the observer to the scope peer, so the preflight must + let a scope through in the observer position — otherwise every scoped chat 422s.""" + workspace, peer = sample_data + scope_name = await _create_scope(client, db_session, workspace.name) + + with patch("src.dialectic.chat.DialecticAgent") as agent_cls: + agent_cls.return_value.answer = AsyncMock(return_value="answered") + answer = await agentic_chat( + workspace_name=workspace.name, + session_name=None, + query="what do you know?", + observer=scope_peer_name(scope_name), + observed=peer.name, + ) + + assert answer == "answered" + assert agent_cls.call_args.kwargs["observer"] == scope_peer_name(scope_name) + + +@pytest.mark.asyncio +async def test_scope_observed_still_rejected( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], +): + """The invariant the guard exists for: no representation is formed of a scope, + so it can never be the subject — even if the route's name check was raced.""" + workspace, peer = sample_data + scope_name = await _create_scope(client, db_session, workspace.name) + + with ( + patch("src.dialectic.chat.DialecticAgent") as agent_cls, + pytest.raises(ValidationException, match=scope_peer_name(scope_name)), + ): + await agentic_chat( + workspace_name=workspace.name, + session_name=None, + query="what do you know?", + observer=peer.name, + observed=scope_peer_name(scope_name), + ) + agent_cls.assert_not_called() diff --git a/tests/dreamer/test_card_refresh.py b/tests/dreamer/test_card_refresh.py index 43d1edea..c039ff0c 100644 --- a/tests/dreamer/test_card_refresh.py +++ b/tests/dreamer/test_card_refresh.py @@ -1,4 +1,4 @@ -"""Tests for the card_refresh dream type (DEV-2000, Scopes RFC prerequisite). +"""Tests for the card_refresh dream type. Covers: - queue plumbing: payload roundtrip, work-unit key isolation from omni, diff --git a/tests/live_llm/README.md b/tests/live_llm/README.md index cef10802..6da7dfb4 100644 --- a/tests/live_llm/README.md +++ b/tests/live_llm/README.md @@ -24,6 +24,17 @@ Model-family env vars: - `LIVE_LLM_GEMINI_30_MODELS` - `LIVE_LLM_GEMINI_31_MODELS` +Embedding-model env vars: + +- `LIVE_EMBEDDING_GEMINI_MODELS` (default: `gemini-embedding-001,gemini-embedding-2`; add `gemini-embedding-2-preview` to cover the preview twin) +- `LIVE_EMBEDDING_OPENAI_MODELS` (default: `text-embedding-3-small`) +- `LIVE_EMBEDDING_OPENAI_COMPATIBLE_MODELS` (no default → skipped) — OpenAI transport pointed at a third-party OpenAI-compatible provider. Also reads `OPENROUTER_API_KEY`, `LIVE_EMBEDDING_OPENAI_COMPATIBLE_BASE_URL` (default `https://openrouter.ai/api/v1`), `LIVE_EMBEDDING_OPENAI_COMPATIBLE_DIMENSIONS` (default `3072`) and `LIVE_EMBEDDING_OPENAI_COMPATIBLE_SEND_DIMENSIONS` (default on; set to `0` for a provider that rejects OpenAI's `dimensions` param) + +```bash +export OPENROUTER_API_KEY="sk-or-v1-..." +export LIVE_EMBEDDING_OPENAI_COMPATIBLE_MODELS="google/gemini-embedding-001" +``` + Each model env var accepts a comma-separated list of bare model ids or provider-qualified ids. Examples: @@ -57,3 +68,5 @@ Coverage by provider: - OpenAI transport → OpenRouter non-reasoning models (e.g. `inception/mercury-2`): non-chat / diffusion architectures must stay on `max_tokens`, no `reasoning_effort`, tool-calling parameter-schema compatibility is the canary for exotic OR-served providers - Gemini 2.5/3.0 classes: structured outputs, cached-content reuse, thought signatures, multi-turn tool replay - Gemini 3.1 class: thinking and tool replay coverage by default; structured-output/caching coverage should only be added once Google documents support for that path +- Embeddings (`test_live_embeddings.py`): single embed, batched embed, batch-vs-single alignment, and chunk-to-id mapping for every configured embedding model. `gemini-embedding-2*` is the reason this exists — those models collapse a list of bare strings into one document (#745), and only a live call catches it +- OpenAI-compatible embedding providers (e.g. OpenRouter's `google/gemini-embedding-001`): the #932 surface. Those providers reject a base64 embedding request outright (HTTP 400) or answer HTTP 200 with empty data, so the whole matrix fails without `encoding_format="float"`. Real OpenAI accepts base64 happily, so only a third-party provider catches it. Note that OpenRouter load-balances across upstreams, so the base64 failure is per-attempt rather than guaranteed: a retry can land on an endpoint that accepts it. `test_live_openai_float_encoding_matches_base64` covers the other side, that the float switch must not move vectors on real OpenAI diff --git a/tests/live_llm/conftest.py b/tests/live_llm/conftest.py index d9646383..eb7b8dbb 100644 --- a/tests/live_llm/conftest.py +++ b/tests/live_llm/conftest.py @@ -1,15 +1,18 @@ from __future__ import annotations +import os from collections.abc import Iterator from typing import Any import pytest from pydantic import BaseModel -from src.config import ModelConfig, settings +from src.config import EmbeddingModelConfig, ModelConfig, settings +from src.embedding_client import _EmbeddingClient # pyright: ignore[reportPrivateUsage] from src.llm import get_backend from src.llm.caching import gemini_cache_store +from .embedding_matrix import LiveEmbeddingSpec, selected_embedding_summary_lines from .model_matrix import LiveModelSpec, selected_model_summary_lines @@ -22,9 +25,12 @@ class StructuredLiveResponse(BaseModel): def pytest_report_header(config: pytest.Config) -> list[str] | None: if not config.getoption("--live-llm"): return None - return ["live llm model matrix:"] + [ - f" {line}" for line in selected_model_summary_lines() - ] + return ( + ["live llm model matrix:"] + + [f" {line}" for line in selected_model_summary_lines()] + + ["live embedding model matrix:"] + + [f" {line}" for line in selected_embedding_summary_lines()] + ) @pytest.fixture(autouse=True) @@ -45,6 +51,57 @@ def require_provider_key(model_spec: LiveModelSpec) -> None: pytest.skip(f"Missing API key for live provider {model_spec.provider}") +def require_embedding_key(spec: LiveEmbeddingSpec) -> str: + if spec.api_key_env: + key = os.getenv(spec.api_key_env) + if not key: + pytest.skip(f"Missing {spec.api_key_env} for live embedding {spec.id}") + return key + key = { + "openai": settings.LLM.OPENAI_API_KEY, + "gemini": settings.LLM.GEMINI_API_KEY, + }[spec.transport] + if not key: + pytest.skip(f"Missing API key for live embedding transport {spec.transport}") + return key + + +def make_embedding_client( + spec: LiveEmbeddingSpec, **overrides: Any +) -> _EmbeddingClient: + """Build a live embedding client for one matrix entry. + + Bypasses the `EmbeddingClient` singleton so each spec gets its own client + without mutating global settings. + """ + kwargs: dict[str, Any] = { + "vector_dimensions": spec.dimensions, + "max_input_tokens": 2048, + "max_tokens_per_request": 300_000, + "send_dimensions": spec.send_dimensions, + # Pinned rather than resolved from settings: the matrix exists to exercise + # the float path that `auto` only picks for third-party providers. + "encoding_format": "float", + } + kwargs.update(overrides) + return _EmbeddingClient( + EmbeddingModelConfig( + transport=spec.transport, + model=spec.model, + api_key=require_embedding_key(spec), + base_url=spec.base_url, + ), + **kwargs, + ) + + +def cosine_similarity(a: list[float], b: list[float]) -> float: + dot = sum(x * y for x, y in zip(a, b, strict=True)) + norm_a = sum(x * x for x in a) ** 0.5 + norm_b = sum(y * y for y in b) ** 0.5 + return dot / (norm_a * norm_b) + + def make_model_config(model_spec: LiveModelSpec, **overrides: Any) -> ModelConfig: return ModelConfig( model=model_spec.model, diff --git a/tests/live_llm/embedding_matrix.py b/tests/live_llm/embedding_matrix.py new file mode 100644 index 00000000..198d1d09 --- /dev/null +++ b/tests/live_llm/embedding_matrix.py @@ -0,0 +1,138 @@ +from __future__ import annotations + +import os +from dataclasses import dataclass + +from src.config import EmbeddingTransport + + +@dataclass(frozen=True) +class LiveEmbeddingFamily: + transport: EmbeddingTransport + family: str + env_var: str + dimensions: int + default_models: tuple[str, ...] = () + docs_url: str | None = None + base_url: str | None = None + # Falls back to the transport's own key when unset. + api_key_env: str | None = None + dimensions_env: str | None = None + base_url_env: str | None = None + send_dimensions: bool = True + send_dimensions_env: str | None = None + + +@dataclass(frozen=True) +class LiveEmbeddingSpec: + transport: EmbeddingTransport + family: str + model: str + env_var: str + dimensions: int + docs_url: str | None = None + base_url: str | None = None + api_key_env: str | None = None + send_dimensions: bool = True + + @property + def id(self) -> str: + return f"{self.transport}:{self.family}:{self.model}" + + +EMBEDDING_FAMILIES: tuple[LiveEmbeddingFamily, ...] = ( + # gemini-embedding-2 is the regression surface for #745: the SDK folds a + # list of bare strings into a single document and returns one embedding for + # the whole batch. Its preview twin behaves identically and is reachable + # through the env var when it needs checking. + LiveEmbeddingFamily( + transport="gemini", + family="gemini_embedding", + env_var="LIVE_EMBEDDING_GEMINI_MODELS", + # Matryoshka dimension supported across the family; keeps vectors small. + dimensions=768, + default_models=( + "gemini-embedding-001", + "gemini-embedding-2", + ), + docs_url="https://ai.google.dev/gemini-api/docs/embeddings", + ), + LiveEmbeddingFamily( + transport="openai", + family="openai_embedding", + env_var="LIVE_EMBEDDING_OPENAI_MODELS", + dimensions=1536, + default_models=("text-embedding-3-small",), + docs_url="https://platform.openai.com/docs/guides/embeddings", + ), + # OpenAI transport pointed at an OpenAI-compatible provider. This is the + # regression surface for #932: the openai SDK asks for base64 embeddings + # unless told otherwise, and third-party providers reject or empty out that + # request. Empty default_models → skipped unless set. + LiveEmbeddingFamily( + transport="openai", + family="openai_compatible_embedding", + env_var="LIVE_EMBEDDING_OPENAI_COMPATIBLE_MODELS", + dimensions=3072, + dimensions_env="LIVE_EMBEDDING_OPENAI_COMPATIBLE_DIMENSIONS", + base_url="https://openrouter.ai/api/v1", + base_url_env="LIVE_EMBEDDING_OPENAI_COMPATIBLE_BASE_URL", + api_key_env="OPENROUTER_API_KEY", + # Mirrors honcho's own behaviour once VECTOR_DIMENSIONS is set; turn off + # for a provider that rejects the param. + send_dimensions=True, + send_dimensions_env="LIVE_EMBEDDING_OPENAI_COMPATIBLE_SEND_DIMENSIONS", + docs_url="https://openrouter.ai/docs/api-reference/embeddings", + ), +) + + +def _parse_env_models(value: str | None) -> tuple[str, ...]: + if value is None: + return () + return tuple(model.strip() for model in value.split(",") if model.strip()) + + +def get_live_embedding_specs( + *, transport: EmbeddingTransport | None = None +) -> tuple[LiveEmbeddingSpec, ...]: + specs: list[LiveEmbeddingSpec] = [] + for family in EMBEDDING_FAMILIES: + if transport is not None and family.transport != transport: + continue + models = _parse_env_models(os.getenv(family.env_var)) or family.default_models + dimensions = family.dimensions + if family.dimensions_env: + dimensions = int(os.getenv(family.dimensions_env) or family.dimensions) + base_url = family.base_url + if family.base_url_env: + base_url = os.getenv(family.base_url_env) or family.base_url + send_dimensions = family.send_dimensions + if family.send_dimensions_env: + raw = os.getenv(family.send_dimensions_env) + if raw is not None: + send_dimensions = raw.strip().lower() in {"1", "true", "yes"} + for model in models: + specs.append( + LiveEmbeddingSpec( + transport=family.transport, + family=family.family, + model=model, + env_var=family.env_var, + dimensions=dimensions, + docs_url=family.docs_url, + base_url=base_url, + api_key_env=family.api_key_env, + send_dimensions=send_dimensions, + ) + ) + return tuple(specs) + + +def selected_embedding_summary_lines() -> list[str]: + lines: list[str] = [] + for family in EMBEDDING_FAMILIES: + models = _parse_env_models(os.getenv(family.env_var)) or family.default_models + joined = ", ".join(models) if models else "(none configured)" + lines.append(f"{family.env_var} [{family.transport}/{family.family}]: {joined}") + return lines diff --git a/tests/live_llm/test_live_embeddings.py b/tests/live_llm/test_live_embeddings.py new file mode 100644 index 00000000..c8af34b8 --- /dev/null +++ b/tests/live_llm/test_live_embeddings.py @@ -0,0 +1,162 @@ +from __future__ import annotations + +from typing import Any, cast + +import pytest +from openai import AsyncOpenAI + +from .conftest import cosine_similarity, make_embedding_client +from .embedding_matrix import LiveEmbeddingSpec, get_live_embedding_specs + +pytestmark = pytest.mark.live_llm + +# Deliberately unrelated topics so a mix-up between them is visible in cosine +# similarity rather than lost in noise. +BATCH_TEXTS: list[str] = [ + "The mitochondria generates ATP through oxidative phosphorylation.", + "Barcelona won the treble in the 2014-15 football season.", + "Sourdough starter needs regular feeding with flour and water.", + "Rust's borrow checker enforces ownership rules at compile time.", +] + +ALL_SPECS = get_live_embedding_specs() +GEMINI_SPECS = get_live_embedding_specs(transport="gemini") +OPENAI_NATIVE_SPECS = tuple( + spec for spec in ALL_SPECS if spec.family == "openai_embedding" +) + + +@pytest.mark.asyncio +@pytest.mark.parametrize("spec", ALL_SPECS, ids=lambda spec: spec.id) +async def test_live_embed_single_returns_configured_dimensions( + spec: LiveEmbeddingSpec, +) -> None: + client = make_embedding_client(spec) + + embedding = await client.embed(BATCH_TEXTS[0]) + + assert len(embedding) == spec.dimensions + assert any(value != 0.0 for value in embedding) + + +@pytest.mark.asyncio +@pytest.mark.parametrize("spec", ALL_SPECS, ids=lambda spec: spec.id) +async def test_live_batch_embed_returns_one_vector_per_input( + spec: LiveEmbeddingSpec, +) -> None: + """Regression guard for #745. + + `gemini-embedding-2*` treats a list of bare strings as parts of one + document and returns a single embedding, which trips the strict zip in + `_process_batch`. Each input must come back with its own distinct vector. + """ + client = make_embedding_client(spec) + + embeddings = await client.simple_batch_embed(BATCH_TEXTS) + + assert len(embeddings) == len(BATCH_TEXTS) + assert all(len(embedding) == spec.dimensions for embedding in embeddings) + # A collapsed batch would hand the same vector back for every input. + assert len({tuple(embedding) for embedding in embeddings}) == len(BATCH_TEXTS) + + +@pytest.mark.asyncio +@pytest.mark.parametrize("spec", ALL_SPECS, ids=lambda spec: spec.id) +async def test_live_batch_embed_aligns_with_single_embed( + spec: LiveEmbeddingSpec, +) -> None: + """Batched vectors must match the one-at-a-time vectors, position for + position. Catches both a collapsed batch and a silently reordered one.""" + client = make_embedding_client(spec) + + batched = await client.simple_batch_embed(BATCH_TEXTS) + singles = [await client.embed(text) for text in BATCH_TEXTS] + + for index, (batched_vector, single_vector) in enumerate( + zip(batched, singles, strict=True) + ): + self_similarity = cosine_similarity(batched_vector, single_vector) + assert self_similarity > 0.95, ( + f"{spec.id}: batched vector {index} does not match its own " + f"single embedding (cosine={self_similarity:.3f})" + ) + for other_index, other_single in enumerate(singles): + if other_index == index: + continue + assert self_similarity > cosine_similarity(batched_vector, other_single), ( + f"{spec.id}: batched vector {index} is closer to text " + f"{other_index} than to its own" + ) + + +@pytest.mark.asyncio +@pytest.mark.parametrize("spec", ALL_SPECS, ids=lambda spec: spec.id) +async def test_live_batch_embed_maps_chunks_to_their_ids( + spec: LiveEmbeddingSpec, +) -> None: + """`batch_embed` splits oversized inputs, so one request carries chunks + belonging to several ids. Every id must get back exactly its own chunks.""" + client = make_embedding_client(spec) + long_text = " ".join( + f"paragraph {index} about photosynthesis" for index in range(900) + ) + expected_chunks = { + text_id: len(chunks) + for text_id, chunks in client.prepare_chunks( + {"long": long_text, "short": BATCH_TEXTS[1]} + ).items() + } + assert expected_chunks["long"] > 1, "test input must exceed the token limit" + + result = await client.batch_embed({"long": long_text, "short": BATCH_TEXTS[1]}) + + assert {text_id: len(vectors) for text_id, vectors in result.items()} == ( + expected_chunks + ) + assert all( + len(vector) == spec.dimensions + for vectors in result.values() + for vector in vectors + ) + + +@pytest.mark.asyncio +@pytest.mark.parametrize("spec", OPENAI_NATIVE_SPECS, ids=lambda spec: spec.id) +async def test_live_openai_float_encoding_matches_base64( + spec: LiveEmbeddingSpec, +) -> None: + """Guard for #938, which switched the openai paths to `encoding_format="float"`. + + Requesting floats must return the same vectors the SDK's base64 default + decoded to, so existing stored embeddings stay comparable. + """ + client = make_embedding_client(spec) + openai_client = cast(AsyncOpenAI, client.client) + base64_kwargs: dict[str, Any] = {"model": spec.model, "input": [BATCH_TEXTS[0]]} + if spec.send_dimensions: + base64_kwargs["dimensions"] = spec.dimensions + + float_vector = await client.embed(BATCH_TEXTS[0]) + # No encoding_format → SDK sends base64 and decodes it, the pre-#938 path. + base64_response = await openai_client.embeddings.create(**base64_kwargs) + + similarity = cosine_similarity(float_vector, base64_response.data[0].embedding) + assert ( + similarity > 0.99999 + ), f"{spec.id}: float encoding diverges from base64 (cosine={similarity:.8f})" + + +@pytest.mark.asyncio +@pytest.mark.parametrize("spec", GEMINI_SPECS, ids=lambda spec: spec.id) +async def test_live_gemini_batch_embed_survives_batch_split( + spec: LiveEmbeddingSpec, +) -> None: + """Same fix across the batch boundary: with max_batch_size=2 the four + inputs go out as two separate Gemini requests.""" + client = make_embedding_client(spec) + client.max_batch_size = 2 + + embeddings = await client.simple_batch_embed(BATCH_TEXTS) + + assert len(embeddings) == len(BATCH_TEXTS) + assert len({tuple(embedding) for embedding in embeddings}) == len(BATCH_TEXTS) diff --git a/tests/live_llm/test_live_timeouts.py b/tests/live_llm/test_live_timeouts.py new file mode 100644 index 00000000..28bf6143 --- /dev/null +++ b/tests/live_llm/test_live_timeouts.py @@ -0,0 +1,117 @@ +from __future__ import annotations + +import time +from typing import Any + +import anthropic +import httpx +import openai +import pytest + +from src.llm.request_builder import execute_completion + +from .conftest import make_backend, require_provider_key, wrap_async_method +from .model_matrix import LiveModelSpec, ProviderName, get_live_model_specs + +pytestmark = [pytest.mark.live_llm] + +GENEROUS_TIMEOUT_SECONDS = 120 +TIGHT_TIMEOUT_SECONDS = 0.01 +# Well under the 600s client default; generous enough to absorb SDK retries. +TIGHT_TIMEOUT_WALL_CLOCK_LIMIT_SECONDS = 30 + +TIMEOUT_EXCEPTIONS: dict[ProviderName, tuple[type[BaseException], ...]] = { + "anthropic": (anthropic.APITimeoutError,), + "openai": (openai.APITimeoutError,), + # google-genai raises httpx or aiohttp timeouts depending on its transport; + # aiohttp surfaces as asyncio.TimeoutError (== builtins.TimeoutError). + "gemini": (httpx.TimeoutException, TimeoutError), +} + +PROVIDER_MARKS = { + "anthropic": pytest.mark.requires_anthropic, + "openai": pytest.mark.requires_openai, + "gemini": pytest.mark.requires_gemini, +} + + +def representative_specs() -> list[Any]: + """One spec per provider — timeout plumbing is transport-level, not model-level.""" + params: list[Any] = [] + for provider in ("anthropic", "openai", "gemini"): + specs = get_live_model_specs(provider=provider) + if not specs: + continue + params.append( + pytest.param(specs[0], marks=PROVIDER_MARKS[provider], id=specs[0].id) + ) + return params + + +def assert_timeout_reached_sdk( + model_spec: LiveModelSpec, call_kwargs: dict[str, Any], timeout_seconds: float +) -> None: + if model_spec.provider == "gemini": + http_options = call_kwargs["config"]["http_options"] + assert http_options.timeout == int(timeout_seconds * 1000) + else: + assert call_kwargs["timeout"] == timeout_seconds + + +def sdk_call_target(backend: Any, model_spec: LiveModelSpec) -> tuple[Any, str]: + if model_spec.provider == "gemini": + return backend._client.aio.models, "generate_content" + if model_spec.provider == "anthropic": + return backend._client.messages, "create" + return backend._client.chat.completions, "create" + + +@pytest.mark.asyncio +@pytest.mark.parametrize("model_spec", representative_specs()) +async def test_live_provider_timeout_reaches_the_wire( + model_spec: LiveModelSpec, + monkeypatch: pytest.MonkeyPatch, +) -> None: + require_provider_key(model_spec) + backend, config = make_backend( + model_spec, provider_params={"timeout": GENEROUS_TIMEOUT_SECONDS} + ) + target, attribute = sdk_call_target(backend, model_spec) + calls = wrap_async_method(monkeypatch, target, attribute) + + result = await execute_completion( + backend, + config, + messages=[{"role": "user", "content": "Reply with the single word: ok"}], + max_tokens=256, + ) + + assert isinstance(result.content, str) + assert result.content.strip() + assert len(calls) == 1 + assert_timeout_reached_sdk(model_spec, calls[0]["kwargs"], GENEROUS_TIMEOUT_SECONDS) + + +@pytest.mark.asyncio +@pytest.mark.parametrize("model_spec", representative_specs()) +async def test_live_tight_provider_timeout_aborts_request( + model_spec: LiveModelSpec, +) -> None: + require_provider_key(model_spec) + backend, config = make_backend( + model_spec, provider_params={"timeout": TIGHT_TIMEOUT_SECONDS} + ) + + started = time.monotonic() + with pytest.raises(TIMEOUT_EXCEPTIONS[model_spec.provider]): + await execute_completion( + backend, + config, + messages=[{"role": "user", "content": "Reply with the single word: ok"}], + max_tokens=256, + ) + elapsed = time.monotonic() - started + + assert ( + elapsed < TIGHT_TIMEOUT_WALL_CLOCK_LIMIT_SECONDS + ), f"tight timeout took {elapsed:.1f}s — per-request timeout likely not applied" diff --git a/tests/llm/test_backends/test_anthropic.py b/tests/llm/test_backends/test_anthropic.py index 13255ec7..a226f626 100644 --- a/tests/llm/test_backends/test_anthropic.py +++ b/tests/llm/test_backends/test_anthropic.py @@ -449,3 +449,82 @@ async def test_anthropic_backend_stream_no_prefill_when_tools_present() -> None: "If not responding with a tool call, respond with valid JSON" in call["messages"][0]["content"] ) + + +@pytest.mark.asyncio +async def test_anthropic_backend_passes_timeout_to_completion_request() -> None: + """Anthropic completion requests receive per-request provider timeout.""" + client = Mock() + client.messages.create = AsyncMock( + return_value=SimpleNamespace( + content=[TextBlock(type="text", text="ok")], + usage=SimpleNamespace( + input_tokens=10, + output_tokens=5, + cache_creation_input_tokens=0, + cache_read_input_tokens=0, + ), + stop_reason="end_turn", + ) + ) + + backend = AnthropicBackend(client) + await backend.complete( + model="claude-haiku-4-5", + messages=[{"role": "user", "content": "Hello"}], + max_tokens=100, + extra_params={"timeout": 45}, + ) + + await_args = client.messages.create.await_args + if await_args is None: + raise AssertionError("Expected Anthropic create call") + assert await_args.kwargs["timeout"] == 45.0 + + +@pytest.mark.asyncio +async def test_anthropic_backend_passes_timeout_to_stream_request() -> None: + """Anthropic stream requests receive per-request provider timeout.""" + + class FakeStream: + """Minimal async stream manager for Anthropic streaming tests.""" + + async def __aenter__(self): + """Return the stream object used by the backend.""" + return self + + async def __aexit__(self, *_args: object) -> bool: + """Do not suppress stream errors.""" + return False + + def __aiter__(self): + """Return the async iterator used by the backend.""" + return self + + async def __anext__(self): + """End the fake stream immediately.""" + raise StopAsyncIteration + + async def get_final_message(self): + """Return the final message required by the backend.""" + return SimpleNamespace( + usage=SimpleNamespace(output_tokens=1), + stop_reason="end_turn", + ) + + client = Mock() + client.messages.stream = Mock(return_value=FakeStream()) + + backend = AnthropicBackend(client) + chunks = [ + chunk + async for chunk in backend.stream( + model="claude-haiku-4-5", + messages=[{"role": "user", "content": "Hello"}], + max_tokens=100, + extra_params={"timeout": "60"}, + ) + ] + + assert chunks[-1].is_done is True + assert client.messages.stream.call_args.kwargs["timeout"] == 60.0 diff --git a/tests/llm/test_backends/test_gemini.py b/tests/llm/test_backends/test_gemini.py index 1e3933b4..8295ab26 100644 --- a/tests/llm/test_backends/test_gemini.py +++ b/tests/llm/test_backends/test_gemini.py @@ -98,6 +98,40 @@ async def test_gemini_backend_maps_thinking_effort_to_thinking_level() -> None: assert call["config"]["thinking_config"] == {"thinking_level": "low"} +@pytest.mark.asyncio +async def test_gemini_backend_maps_timeout_to_http_options() -> None: + """Gemini requests receive provider timeout through config http_options.""" + client = Mock() + client.aio.models.generate_content = AsyncMock( + return_value=SimpleNamespace( + candidates=[ + SimpleNamespace( + finish_reason=SimpleNamespace(name="STOP"), + content=SimpleNamespace(parts=[SimpleNamespace(text="ok")]), + ) + ], + usage_metadata=SimpleNamespace( + prompt_token_count=12, + candidates_token_count=6, + ), + parsed=None, + ) + ) + + backend = GeminiBackend(client) + await backend.complete( + model="gemini-2.5-flash", + messages=[{"role": "user", "content": "Hello"}], + max_tokens=100, + extra_params={"timeout": "90"}, + ) + + await_args = client.aio.models.generate_content.await_args + if await_args is None: + raise AssertionError("Expected Gemini generate_content call") + assert await_args.kwargs["config"]["http_options"].timeout == 90_000 + + @pytest.mark.asyncio async def test_gemini_backend_rejects_budget_and_effort_together() -> None: backend = GeminiBackend(Mock()) @@ -385,7 +419,7 @@ async def test_gemini_backend_forwards_provider_params_extra_headers() -> None: if await_args is None: raise AssertionError("Expected Gemini generate_content call") call = await_args.kwargs - assert call["config"]["http_options"]["headers"] == {"X-Trace-Id": "abc123"} + assert call["config"]["http_options"].headers == {"X-Trace-Id": "abc123"} @pytest.mark.asyncio diff --git a/tests/llm/test_backends/test_openai.py b/tests/llm/test_backends/test_openai.py index 8567e034..fbd8e719 100644 --- a/tests/llm/test_backends/test_openai.py +++ b/tests/llm/test_backends/test_openai.py @@ -510,6 +510,7 @@ async def test_openai_backend_converts_anthropic_style_tools() -> None: assert call["tool_choice"] == "required" +@pytest.mark.asyncio async def test_openai_backend_translates_canonical_any_tool_choice_to_required() -> ( None ): @@ -571,6 +572,78 @@ def test_openai_convert_tool_choice(canonical: Any, expected: Any) -> None: assert OpenAIBackend._convert_tool_choice(canonical) == expected # pyright: ignore[reportPrivateUsage] +@pytest.mark.asyncio +async def test_openai_backend_passes_timeout_to_completion_request() -> None: + """OpenAI completion requests receive per-request provider timeout.""" + client = Mock() + client.chat.completions.create = AsyncMock( + return_value=SimpleNamespace( + choices=[ + SimpleNamespace( + finish_reason="stop", + message=SimpleNamespace( + content="ok", + tool_calls=[], + reasoning_details=[], + ), + ) + ], + usage=SimpleNamespace( + prompt_tokens=10, + completion_tokens=5, + prompt_tokens_details=None, + ), + ) + ) + + backend = OpenAIBackend(client) + await backend.complete( + model="gpt-4.1", + messages=[{"role": "user", "content": "Hello"}], + max_tokens=100, + extra_params={"timeout": 12.5}, + ) + + assert _await_kwargs(client.chat.completions.create)["timeout"] == 12.5 + + +@pytest.mark.asyncio +async def test_openai_backend_passes_timeout_to_structured_parse_request() -> None: + """OpenAI structured parse requests receive per-request provider timeout.""" + client = Mock() + client.chat.completions.parse = AsyncMock( + return_value=SimpleNamespace( + choices=[ + SimpleNamespace( + finish_reason="stop", + message=SimpleNamespace( + parsed=_StructuredResponse(answer="ok"), + content='{"answer":"ok"}', + tool_calls=[], + refusal=None, + ), + ) + ], + usage=SimpleNamespace( + prompt_tokens=10, + completion_tokens=5, + prompt_tokens_details=None, + ), + ) + ) + + backend = OpenAIBackend(client) + await backend.complete( + model="gpt-4.1", + messages=[{"role": "user", "content": "Hello"}], + max_tokens=100, + response_format=_StructuredResponse, + extra_params={"timeout": "30"}, + ) + + assert _await_kwargs(client.chat.completions.parse)["timeout"] == 30.0 + + @pytest.mark.parametrize( "model", [ diff --git a/tests/llm/test_embedding_client.py b/tests/llm/test_embedding_client.py index a4642159..b50a66f0 100644 --- a/tests/llm/test_embedding_client.py +++ b/tests/llm/test_embedding_client.py @@ -1,16 +1,34 @@ +import array +import base64 from types import SimpleNamespace -from typing import Any +from typing import Any, cast import pytest +from google.genai import types as genai_types -from src.config import EmbeddingModelConfig -from src.embedding_client import _EmbeddingClient # pyright: ignore[reportPrivateUsage] +from src.config import ( + EmbeddingEncodingFormat, + EmbeddingModelConfig, + resolve_embedding_model_config, +) +from src.embedding_client import ( + BatchItem, + _EmbeddingClient, # pyright: ignore[reportPrivateUsage] +) + + +def gemini_call_texts(contents: Any) -> list[str]: + """Unwrap a recorded Gemini `contents` argument back to plain texts.""" + return [content.parts[0].text for content in contents] class FakeOpenAIEmbeddingsAPI: def __init__(self, embedding: list[float]) -> None: self.embedding: list[float] = embedding self.calls: list[dict[str, Any]] = [] + # Simulate a provider answering 200 with missing embeddings. + self.returns_no_data: bool = False + self.truncate_data_to: int | None = None async def create( self, @@ -22,10 +40,21 @@ class FakeOpenAIEmbeddingsAPI: call: dict[str, Any] = {"model": model, "input": input} call.update(kwargs) self.calls.append(call) + # Mirror the SDK: a named encoding_format skips its base64 decode, so the + # response carries the raw string instead of floats. + payload: Any = self.embedding + if kwargs.get("encoding_format") == "base64": + payload = base64.b64encode( + array.array("f", self.embedding).tobytes() + ).decode() if isinstance(input, list): - data = [SimpleNamespace(embedding=self.embedding) for _ in input] + data = [SimpleNamespace(embedding=payload) for _ in input] else: - data = [SimpleNamespace(embedding=self.embedding)] + data = [SimpleNamespace(embedding=payload)] + if self.returns_no_data: + data = [] + elif self.truncate_data_to is not None: + data = data[: self.truncate_data_to] return SimpleNamespace(data=data) @@ -41,7 +70,7 @@ async def test_openai_embedding_client_uses_configured_model_and_dimensions( self.base_url: str | None = base_url self.embeddings: FakeOpenAIEmbeddingsAPI = fake_embeddings - monkeypatch.setattr("src.embedding_client.AsyncOpenAI", FakeOpenAIClient) + monkeypatch.setattr("openai.AsyncOpenAI", FakeOpenAIClient) client = _EmbeddingClient( EmbeddingModelConfig( @@ -60,7 +89,11 @@ async def test_openai_embedding_client_uses_configured_model_and_dimensions( assert embedding == [0.1] * 8 assert fake_embeddings.calls == [ - {"model": "text-embedding-3-small", "input": ["hello world"]} + { + "model": "text-embedding-3-small", + "input": ["hello world"], + "encoding_format": "float", + } ] @@ -74,7 +107,7 @@ async def test_openai_embedding_client_rejects_dimension_mismatch( def __init__(self, *, api_key: str | None, base_url: str | None) -> None: self.embeddings: FakeOpenAIEmbeddingsAPI = fake_embeddings - monkeypatch.setattr("src.embedding_client.AsyncOpenAI", FakeOpenAIClient) + monkeypatch.setattr("openai.AsyncOpenAI", FakeOpenAIClient) client = _EmbeddingClient( EmbeddingModelConfig( @@ -103,7 +136,7 @@ async def test_gemini_embedding_client_uses_output_dimensionality( self, *, model: str, - contents: str | list[str], + contents: Any, config: dict[str, Any], ) -> SimpleNamespace: calls.append( @@ -123,7 +156,7 @@ async def test_gemini_embedding_client_uses_output_dimensionality( self.http_options: Any = http_options self.aio: Any = SimpleNamespace(models=FakeGeminiModels()) - monkeypatch.setattr("src.embedding_client.genai.Client", FakeGeminiClient) + monkeypatch.setattr("google.genai.Client", FakeGeminiClient) client = _EmbeddingClient( EmbeddingModelConfig( @@ -141,6 +174,12 @@ async def test_gemini_embedding_client_uses_output_dimensionality( embedding = await client.embed("hello world") assert embedding == [0.2] * 12 + # 10-minute HTTP timeout, in lockstep with the LLM registry's Gemini client + # (see #785). Without this, a stalled Gemini embedding socket wedges the + # deriver worker — the same failure mode the LLM fix addresses. + gemini_client = cast(Any, client.client) + assert gemini_client.http_options.base_url == "https://gemini-proxy.example/v1beta" + assert gemini_client.http_options.timeout == 600_000 assert calls == [ { "model": "gemini-embedding-001", @@ -150,6 +189,37 @@ async def test_gemini_embedding_client_uses_output_dimensionality( ] +@pytest.mark.asyncio +async def test_gemini_embedding_client_keeps_timeout_without_base_url( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """No-base-url Gemini embedding client must still carry an HTTP timeout.""" + + class FakeGeminiClient: + def __init__(self, *, api_key: str | None, http_options: Any) -> None: + self.api_key: str | None = api_key + self.http_options: Any = http_options + self.aio: Any = SimpleNamespace(models=SimpleNamespace()) + + monkeypatch.setattr("google.genai.Client", FakeGeminiClient) + + client = _EmbeddingClient( + EmbeddingModelConfig( + transport="gemini", + model="gemini-embedding-001", + api_key="gemini-key", + ), + vector_dimensions=8, + max_input_tokens=4096, + max_tokens_per_request=300_000, + send_dimensions=False, + ) + + gemini_client = cast(Any, client.client) + assert gemini_client.http_options.base_url is None + assert gemini_client.http_options.timeout == 600_000 + + def _build_openai_client( monkeypatch: pytest.MonkeyPatch, *, @@ -157,6 +227,8 @@ def _build_openai_client( model: str, send_dimensions: bool, vector_dimensions: int, + max_batch_size: int | None = None, + encoding_format: EmbeddingEncodingFormat = "float", ) -> tuple[_EmbeddingClient, FakeOpenAIEmbeddingsAPI]: fake_embeddings = FakeOpenAIEmbeddingsAPI(embedding) @@ -166,18 +238,20 @@ def _build_openai_client( self.base_url: str | None = base_url self.embeddings: FakeOpenAIEmbeddingsAPI = fake_embeddings - monkeypatch.setattr("src.embedding_client.AsyncOpenAI", FakeOpenAIClient) + monkeypatch.setattr("openai.AsyncOpenAI", FakeOpenAIClient) client = _EmbeddingClient( EmbeddingModelConfig( transport="openai", model=model, api_key="test-key", + max_batch_size=max_batch_size, ), vector_dimensions=vector_dimensions, max_input_tokens=8192, max_tokens_per_request=300_000, send_dimensions=send_dimensions, + encoding_format=encoding_format, ) return client, fake_embeddings @@ -200,6 +274,7 @@ async def test_openai_embed_forwards_dimensions_when_send_dimensions_true( { "model": "text-embedding-3-small", "input": ["hello"], + "encoding_format": "float", "dimensions": 768, } ] @@ -219,7 +294,13 @@ async def test_openai_embed_omits_dimensions_when_send_dimensions_false( await client.embed("hello") - assert fake.calls == [{"model": "text-embedding-3-small", "input": ["hello"]}] + assert fake.calls == [ + { + "model": "text-embedding-3-small", + "input": ["hello"], + "encoding_format": "float", + } + ] @pytest.mark.asyncio @@ -241,6 +322,136 @@ async def test_openai_simple_batch_embed_forwards_dimensions( assert fake.calls[0]["input"] == ["a", "b"] +@pytest.mark.asyncio +async def test_openai_simple_batch_embed_respects_configured_max_batch_size( + monkeypatch: pytest.MonkeyPatch, +) -> None: + client, fake = _build_openai_client( + monkeypatch, + embedding=[0.1] * 1536, + model="text-embedding-3-small", + send_dimensions=False, + vector_dimensions=1536, + max_batch_size=2, + ) + + await client.simple_batch_embed(["a", "b", "c"]) + + assert [call["input"] for call in fake.calls] == [["a", "b"], ["c"]] + + +@pytest.mark.asyncio +async def test_openai_simple_batch_embed_defaults_to_2048_when_unset( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Unset max_batch_size must keep the OpenAI default: one request.""" + client, fake = _build_openai_client( + monkeypatch, + embedding=[0.1] * 1536, + model="text-embedding-3-small", + send_dimensions=False, + vector_dimensions=1536, + ) + assert client.max_batch_size == 2048 + + await client.simple_batch_embed(["a", "b", "c"]) + + assert [call["input"] for call in fake.calls] == [["a", "b", "c"]] + + +@pytest.mark.asyncio +async def test_gemini_simple_batch_embed_respects_configured_max_batch_size( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Gemini transport must split batches at the configured limit too.""" + calls: list[dict[str, Any]] = [] + + class FakeGeminiModels: + async def embed_content( + self, + *, + model: str, + contents: Any, + config: dict[str, Any], + ) -> SimpleNamespace: + calls.append({"model": model, "contents": contents, "config": config}) + n = len(contents) + return SimpleNamespace( + embeddings=[SimpleNamespace(values=[0.2] * 12) for _ in range(n)] + ) + + class FakeGeminiClient: + def __init__(self, *, api_key: str | None, http_options: Any) -> None: + self.aio: Any = SimpleNamespace(models=FakeGeminiModels()) + + monkeypatch.setattr("google.genai.Client", FakeGeminiClient) + + client = _EmbeddingClient( + EmbeddingModelConfig( + transport="gemini", + model="gemini-embedding-001", + api_key="gemini-key", + max_batch_size=2, + ), + vector_dimensions=12, + max_input_tokens=4096, + max_tokens_per_request=300_000, + send_dimensions=False, + ) + + await client.simple_batch_embed(["a", "b", "c"]) + + assert [gemini_call_texts(call["contents"]) for call in calls] == [ + ["a", "b"], + ["c"], + ] + + +@pytest.mark.asyncio +async def test_gemini_simple_batch_embed_defaults_to_100_when_unset( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Unset max_batch_size must keep the Gemini conservative default.""" + calls: list[dict[str, Any]] = [] + + class FakeGeminiModels: + async def embed_content( + self, + *, + model: str, + contents: Any, + config: dict[str, Any], + ) -> SimpleNamespace: + calls.append({"model": model, "contents": contents, "config": config}) + n = len(contents) + return SimpleNamespace( + embeddings=[SimpleNamespace(values=[0.2] * 12) for _ in range(n)] + ) + + class FakeGeminiClient: + def __init__(self, *, api_key: str | None, http_options: Any) -> None: + self.aio: Any = SimpleNamespace(models=FakeGeminiModels()) + + monkeypatch.setattr("google.genai.Client", FakeGeminiClient) + + client = _EmbeddingClient( + EmbeddingModelConfig( + transport="gemini", + model="gemini-embedding-001", + api_key="gemini-key", + ), + vector_dimensions=12, + max_input_tokens=4096, + max_tokens_per_request=300_000, + send_dimensions=False, + ) + assert client.max_batch_size == 100 + + await client.simple_batch_embed(["a", "b", "c"]) + + assert [gemini_call_texts(call["contents"]) for call in calls] == [["a", "b", "c"]] + + @pytest.mark.asyncio async def test_openai_batch_embed_forwards_dimensions( monkeypatch: pytest.MonkeyPatch, @@ -259,6 +470,86 @@ async def test_openai_batch_embed_forwards_dimensions( assert fake.calls[0]["dimensions"] == 768 +@pytest.mark.asyncio +async def test_openai_embed_requests_float_encoding_format( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """The single-query path must request float embeddings explicitly. + + Without an explicit encoding_format, the openai SDK defaults to base64, + which OpenAI-compatible providers such as OpenRouter answer with empty + embedding data for models that don't support base64 encoding. + """ + client, fake = _build_openai_client( + monkeypatch, + embedding=[0.1] * 8, + model="text-embedding-3-small", + send_dimensions=False, + vector_dimensions=8, + ) + + await client.embed("hello") + + assert fake.calls[0]["encoding_format"] == "float" + + +@pytest.mark.asyncio +async def test_openai_batch_embed_requests_float_encoding_format( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """The batch path must request float embeddings explicitly, like embed().""" + client, fake = _build_openai_client( + monkeypatch, + embedding=[0.1] * 8, + model="text-embedding-3-small", + send_dimensions=False, + vector_dimensions=8, + ) + + await client.batch_embed({"a": "hello", "b": "world"}) + + assert len(fake.calls) == 1 + assert fake.calls[0]["encoding_format"] == "float" + + +@pytest.mark.asyncio +async def test_openai_embed_reports_missing_embedding_data( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """An explicit encoding_format turns off the SDK's own empty-data check, so + a provider answering 200 with no embeddings must still fail legibly.""" + client, fake = _build_openai_client( + monkeypatch, + embedding=[0.1] * 8, + model="text-embedding-3-small", + send_dimensions=False, + vector_dimensions=8, + ) + fake.returns_no_data = True + + with pytest.raises(ValueError, match="Embedding count mismatch"): + await client.embed("hello") + + +@pytest.mark.asyncio +async def test_openai_batch_embed_reports_short_embedding_data( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """A batch answered with fewer embeddings than inputs must name the counts + rather than surface a bare zip() error.""" + client, fake = _build_openai_client( + monkeypatch, + embedding=[0.1] * 8, + model="text-embedding-3-small", + send_dimensions=False, + vector_dimensions=8, + ) + fake.truncate_data_to = 1 + + with pytest.raises(ValueError, match="Expected 2, got 1"): + await client.batch_embed({"a": "hello", "b": "world"}) + + def _build_embedding_settings( env: dict[str, str], monkeypatch: pytest.MonkeyPatch, @@ -271,6 +562,9 @@ def _build_embedding_settings( "EMBEDDING_MODEL_CONFIG__MODEL", "EMBEDDING_MODEL_CONFIG__TRANSPORT", "EMBEDDING_MODEL_CONFIG__DIMENSIONS_MODE", + "EMBEDDING_MODEL_CONFIG__ENCODING_FORMAT_MODE", + "EMBEDDING_MODEL_CONFIG__OVERRIDES__BASE_URL", + "EMBEDDING_MODEL_CONFIG__MAX_BATCH_SIZE", ): monkeypatch.delenv(key, raising=False) for key, value in env.items(): @@ -278,6 +572,70 @@ def _build_embedding_settings( return EmbeddingSettings() +@pytest.mark.parametrize( + ("env", "expected"), + [ + # No base_url means real OpenAI, which serves base64 at ~1/3.6 the bytes. + ({}, "base64"), + ( + { + "EMBEDDING_MODEL_CONFIG__OVERRIDES__BASE_URL": "https://api.openai.com/v1" + }, + "base64", + ), + ( + { + "EMBEDDING_MODEL_CONFIG__OVERRIDES__BASE_URL": "https://openrouter.ai/api/v1" + }, + "float", + ), + ( + {"EMBEDDING_MODEL_CONFIG__OVERRIDES__BASE_URL": "http://localhost:8000/v1"}, + "float", + ), + ({"EMBEDDING_MODEL_CONFIG__ENCODING_FORMAT_MODE": "float"}, "float"), + ( + { + "EMBEDDING_MODEL_CONFIG__ENCODING_FORMAT_MODE": "base64", + "EMBEDDING_MODEL_CONFIG__OVERRIDES__BASE_URL": "https://openrouter.ai/api/v1", + }, + "base64", + ), + ], +) +def test_resolve_encoding_format( + env: dict[str, str], expected: str, monkeypatch: pytest.MonkeyPatch +) -> None: + s = _build_embedding_settings(env, monkeypatch) + assert s.resolve_encoding_format() == expected + + +@pytest.mark.asyncio +async def test_openai_base64_mode_omits_encoding_format_and_returns_floats( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """base64 mode must request by omission on both paths. + + Naming `base64` explicitly makes the SDK skip its own decode and hand back + the raw string, which then fails the dimension check. + """ + client, fake = _build_openai_client( + monkeypatch, + embedding=[0.1] * 8, + model="text-embedding-3-small", + send_dimensions=False, + vector_dimensions=8, + encoding_format="base64", + ) + + embedding = await client.embed("hello") + batched = await client.batch_embed({"a": "hello", "b": "world"}) + + assert all("encoding_format" not in call for call in fake.calls) + assert len(embedding) == 8 + assert [len(vectors[0]) for vectors in batched.values()] == [8, 8] + + def test_resolve_send_dimensions_auto_default_dim_returns_false( monkeypatch: pytest.MonkeyPatch, ) -> None: @@ -352,7 +710,7 @@ async def test_simple_batch_embed_respects_token_budget_per_request( def __init__(self, *, api_key: str | None, base_url: str | None) -> None: self.embeddings: FakeOpenAIEmbeddingsAPI = fake_embeddings - monkeypatch.setattr("src.embedding_client.AsyncOpenAI", FakeOpenAIClient) + monkeypatch.setattr("openai.AsyncOpenAI", FakeOpenAIClient) # max_input_tokens=100 per single input; max_tokens_per_request=120 total, # so two ~80-token inputs must end up in *separate* requests. @@ -390,7 +748,7 @@ async def test_simple_batch_embed_rejects_oversized_input( def __init__(self, *, api_key: str | None, base_url: str | None) -> None: self.embeddings: FakeOpenAIEmbeddingsAPI = fake_embeddings - monkeypatch.setattr("src.embedding_client.AsyncOpenAI", FakeOpenAIClient) + monkeypatch.setattr("openai.AsyncOpenAI", FakeOpenAIClient) client = _EmbeddingClient( EmbeddingModelConfig( @@ -420,7 +778,7 @@ def test_prepare_chunks_returns_ordered_chunks( def __init__(self, *, api_key: str | None, base_url: str | None) -> None: self.embeddings: FakeOpenAIEmbeddingsAPI = fake_embeddings - monkeypatch.setattr("src.embedding_client.AsyncOpenAI", FakeOpenAIClient) + monkeypatch.setattr("openai.AsyncOpenAI", FakeOpenAIClient) client = _EmbeddingClient( EmbeddingModelConfig( @@ -444,3 +802,75 @@ def test_prepare_chunks_returns_ordered_chunks( assert len(out["long"]) > 1 # Order preserved assert isinstance(out["long"][0], str) + + +def test_embedding_model_config_parses_max_batch_size_from_env( + monkeypatch: pytest.MonkeyPatch, +) -> None: + s = _build_embedding_settings( + {"EMBEDDING_MODEL_CONFIG__MAX_BATCH_SIZE": "10"}, + monkeypatch, + ) + + assert s.MODEL_CONFIG.max_batch_size == 10 + + resolved = resolve_embedding_model_config(s.MODEL_CONFIG) + assert resolved.max_batch_size == 10 + + +@pytest.mark.asyncio +async def test_gemini_process_batch_wraps_contents_as_content_part( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Each batch item must be its own Content so gemini-embedding-2* returns + one embedding per item instead of merging them into one document.""" + calls: list[dict[str, Any]] = [] + + class FakeGeminiModels: + async def embed_content( + self, + *, + model: str, + contents: Any, + config: dict[str, Any], + ) -> SimpleNamespace: + calls.append({"model": model, "contents": contents, "config": config}) + embeddings = [SimpleNamespace(values=[0.3] * 8) for _ in contents] + return SimpleNamespace(embeddings=embeddings) + + class FakeGeminiClient: + def __init__(self, *, api_key: str | None, http_options: Any) -> None: + self.api_key: str | None = api_key + self.http_options: Any = http_options + self.aio: Any = SimpleNamespace(models=FakeGeminiModels()) + + monkeypatch.setattr("google.genai.Client", FakeGeminiClient) + + client = _EmbeddingClient( + EmbeddingModelConfig( + transport="gemini", + model="gemini-embedding-2", + api_key="gemini-key", + base_url=None, + ), + vector_dimensions=8, + max_input_tokens=4096, + max_tokens_per_request=300_000, + send_dimensions=False, + ) + + batch = [ + BatchItem("hello", "id1", 0, 1), + BatchItem("world", "id2", 0, 1), + ] + result = await client._process_batch(batch) # pyright: ignore[reportPrivateUsage] + + assert result["id1"][0] == [0.3] * 8 + assert result["id2"][0] == [0.3] * 8 + + assert len(calls) == 1 + contents = calls[0]["contents"] + assert len(contents) == 2 + assert all(isinstance(c, genai_types.Content) for c in contents) + assert contents[0].parts[0].text == "hello" + assert contents[1].parts[0].text == "world" diff --git a/tests/llm/test_registry.py b/tests/llm/test_registry.py index 71f3302e..0e4b304f 100644 --- a/tests/llm/test_registry.py +++ b/tests/llm/test_registry.py @@ -1,22 +1,137 @@ -"""Tests for src.llm.registry helpers.""" +"""Tests for the provider-client registry in src/llm/registry.py. + +Locks the HTTP-timeout behavior added for the Gemini transport (#785) and +pins the existing 600s timeout on the Anthropic clients so regressions on +either side are caught at unit-test time. +""" from __future__ import annotations -from src.llm.registry import _default_headers_for # pyright: ignore[reportPrivateUsage] +from collections.abc import Iterator +from unittest.mock import patch + +import pytest +from google.genai import types as genai_types + +from src import config as app_config +from src.llm import registry as registry_module + +# Gemini's HttpOptions.timeout is an int in milliseconds; keep it in lockstep +# with the Anthropic client's 600s timeout to match the rest of the registry. +_GEMINI_TIMEOUT_MS = 600_000 +_ANTHROPIC_TIMEOUT_S = 600.0 -def test_default_headers_for_openrouter_base_url() -> None: - """OpenRouter base URLs get the app-attribution headers.""" - headers = _default_headers_for("https://openrouter.ai/api/v1") - assert headers["HTTP-Referer"] == "https://honcho.dev" - assert headers["X-Openrouter-Title"] == "Honcho" +@pytest.fixture(autouse=True) +def patch_settings(monkeypatch: pytest.MonkeyPatch) -> Iterator[None]: + """Default the LLM settings so the registry reads valid values.""" + monkeypatch.setenv("PYTHON_DOTENV_DISABLED", "1") + monkeypatch.setenv("ANTHROPIC_API_KEY", "test-anthropic-key") + monkeypatch.setenv("GEMINI_API_KEY", "test-gemini-key") + yield -def test_default_headers_for_non_openrouter_base_url() -> None: - """Other OpenAI-compatible providers get no extra headers.""" - assert _default_headers_for("https://api.openai.com/v1") == {} +@pytest.fixture +def fresh_lru_caches() -> Iterator[None]: + """Drop lru_cache state so each test exercises a fresh client build.""" + registry_module.get_anthropic_client.cache_clear() + registry_module.get_gemini_client.cache_clear() + registry_module.get_anthropic_override_client.cache_clear() + registry_module.get_gemini_override_client.cache_clear() + yield + registry_module.get_anthropic_client.cache_clear() + registry_module.get_gemini_client.cache_clear() + registry_module.get_anthropic_override_client.cache_clear() + registry_module.get_gemini_override_client.cache_clear() -def test_default_headers_for_none_base_url() -> None: - """A missing base URL (default OpenAI) gets no extra headers.""" - assert _default_headers_for(None) == {} +@pytest.mark.usefixtures("fresh_lru_caches") +def test_get_gemini_client_sets_http_timeout(monkeypatch: pytest.MonkeyPatch) -> None: + """Default Gemini client must carry an HttpOptions timeout, not None.""" + monkeypatch.setattr(app_config.settings.LLM, "GEMINI_BASE_URL", None) + + with patch("google.genai.Client") as mock_client: + registry_module.get_gemini_client() + + assert mock_client.call_count == 1 + http_options = mock_client.call_args.kwargs["http_options"] + assert isinstance(http_options, genai_types.HttpOptions) + assert http_options.timeout == _GEMINI_TIMEOUT_MS + + +@pytest.mark.usefixtures("fresh_lru_caches") +def test_get_gemini_client_preserves_custom_base_url( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Base URL and timeout must coexist on the default Gemini client.""" + monkeypatch.setattr( + app_config.settings.LLM, "GEMINI_BASE_URL", "https://gemini-proxy.example.com" + ) + + with patch("google.genai.Client") as mock_client: + registry_module.get_gemini_client() + + http_options = mock_client.call_args.kwargs["http_options"] + assert isinstance(http_options, genai_types.HttpOptions) + assert http_options.base_url == "https://gemini-proxy.example.com" + assert http_options.timeout == _GEMINI_TIMEOUT_MS + + +@pytest.mark.usefixtures("fresh_lru_caches") +def test_get_gemini_override_client_sets_http_timeout() -> None: + """Override Gemini client must also carry a timeout.""" + with patch("google.genai.Client") as mock_client: + registry_module.get_gemini_override_client( + "https://gemini-proxy.example.com", "sk-override" + ) + + http_options = mock_client.call_args.kwargs["http_options"] + assert isinstance(http_options, genai_types.HttpOptions) + assert http_options.base_url == "https://gemini-proxy.example.com" + assert http_options.timeout == _GEMINI_TIMEOUT_MS + + +@pytest.mark.usefixtures("fresh_lru_caches") +def test_get_gemini_override_client_handles_missing_base_url() -> None: + """Override Gemini client with no base URL still carries a timeout.""" + with patch("google.genai.Client") as mock_client: + registry_module.get_gemini_override_client(None, "sk-override") + + http_options = mock_client.call_args.kwargs["http_options"] + assert isinstance(http_options, genai_types.HttpOptions) + assert http_options.timeout == _GEMINI_TIMEOUT_MS + + +@pytest.mark.usefixtures("fresh_lru_caches") +def test_get_anthropic_client_keeps_600s_timeout( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Anthropic timeout is the established behavior — lock it.""" + monkeypatch.setattr(app_config.settings.LLM, "ANTHROPIC_BASE_URL", None) + + with patch("anthropic.AsyncAnthropic") as mock_anthropic: + registry_module.get_anthropic_client() + + assert mock_anthropic.call_args.kwargs["timeout"] == _ANTHROPIC_TIMEOUT_S + + +@pytest.mark.usefixtures("fresh_lru_caches") +def test_get_anthropic_override_client_keeps_600s_timeout() -> None: + """Override Anthropic client also keeps the 600s timeout.""" + with patch("anthropic.AsyncAnthropic") as mock_anthropic: + registry_module.get_anthropic_override_client(None, "sk-override") + + assert mock_anthropic.call_args.kwargs["timeout"] == _ANTHROPIC_TIMEOUT_S + + +def test_gemini_http_options_builder_applies_timeout() -> None: + """The shared helper must always set a timeout, even with no base_url.""" + options = registry_module._build_gemini_http_options(None) # pyright: ignore[reportPrivateUsage] + assert isinstance(options, genai_types.HttpOptions) + assert options.timeout == _GEMINI_TIMEOUT_MS + assert options.base_url is None + + options = registry_module._build_gemini_http_options("https://example.com") # pyright: ignore[reportPrivateUsage] + assert isinstance(options, genai_types.HttpOptions) + assert options.timeout == _GEMINI_TIMEOUT_MS + assert options.base_url == "https://example.com" diff --git a/tests/llm/test_request_builder.py b/tests/llm/test_request_builder.py index c8ed7dfd..a8355a51 100644 --- a/tests/llm/test_request_builder.py +++ b/tests/llm/test_request_builder.py @@ -1,6 +1,8 @@ +import pytest from pydantic import BaseModel from src.config import ModelConfig +from src.exceptions import ValidationException from src.llm.caching import PromptCachePolicy from src.llm.request_builder import execute_completion from tests.llm.conftest import FakeBackend @@ -95,3 +97,51 @@ async def test_provider_params_are_merged_into_extra_params( call = fake_backend.calls[0] assert call["extra_params"]["top_p"] == 0.9 assert call["extra_params"]["custom_flag"] is True + + +async def test_provider_timeout_is_normalized_into_extra_params( + fake_backend: FakeBackend, +) -> None: + """Numeric-string provider timeout values are normalized before backends.""" + config = ModelConfig( + model="gpt-4.1-mini", + transport="openai", + provider_params={"timeout": "42.5"}, + ) + + await execute_completion( + fake_backend, + config, + messages=[{"role": "user", "content": "Hello"}], + max_tokens=100, + ) + + call = fake_backend.calls[0] + assert call["extra_params"]["timeout"] == 42.5 + + +@pytest.mark.parametrize( + "timeout", + ["slow", "", 0, -1, True, float("nan"), float("inf"), "nan", "inf"], +) +async def test_provider_timeout_rejects_invalid_values( + fake_backend: FakeBackend, + timeout: object, +) -> None: + """Invalid provider timeout values fail before provider SDK calls.""" + config = ModelConfig( + model="gpt-4.1-mini", + transport="openai", + provider_params={"timeout": timeout}, + ) + + with pytest.raises( + ValidationException, + match=r"provider_params\.timeout must be a positive number of seconds", + ): + await execute_completion( + fake_backend, + config, + messages=[{"role": "user", "content": "Hello"}], + max_tokens=100, + ) diff --git a/tests/llm/test_telemetry_llm_call.py b/tests/llm/test_telemetry_llm_call.py index ef3cc21c..56a36191 100644 --- a/tests/llm/test_telemetry_llm_call.py +++ b/tests/llm/test_telemetry_llm_call.py @@ -277,7 +277,7 @@ class TestExecutorEndToEnd: @pytest.mark.asyncio async def test_success_path_emits_one_event(self): - from src.llm import executor + from src.llm import executor, registry emitted: list[BaseEvent] = [] result = BackendCompletionResult( @@ -285,7 +285,7 @@ class TestExecutorEndToEnd: ) with ( - patch.object(executor, "CLIENTS", {"anthropic": object()}), + patch.object(registry, "CLIENTS", {"anthropic": object()}), patch.object( executor, "backend_for_provider", @@ -326,7 +326,7 @@ class TestExecutorEndToEnd: 'error' — client disconnects / shutdowns must not pollute error rates.""" import asyncio - from src.llm import executor + from src.llm import executor, registry emitted: list[BaseEvent] = [] @@ -334,7 +334,7 @@ class TestExecutorEndToEnd: raise asyncio.CancelledError() with ( - patch.object(executor, "CLIENTS", {"anthropic": object()}), + patch.object(registry, "CLIENTS", {"anthropic": object()}), patch.object(executor, "backend_for_provider", return_value=object()), patch.object(executor, "execute_completion", new=_cancel), patch( @@ -364,7 +364,7 @@ class TestExecutorEndToEnd: import asyncio from collections.abc import AsyncIterator - from src.llm import executor + from src.llm import executor, registry emitted: list[BaseEvent] = [] @@ -377,7 +377,7 @@ class TestExecutorEndToEnd: return _cancelling_stream() with ( - patch.object(executor, "CLIENTS", {"anthropic": object()}), + patch.object(registry, "CLIENTS", {"anthropic": object()}), patch.object(executor, "backend_for_provider", return_value=object()), patch.object(executor, "execute_stream", new=_setup_stream), patch.object( @@ -418,7 +418,7 @@ class TestExecutorEndToEnd: generator without awaiting `execute_stream`, hiding setup failures from tenacity. """ - from src.llm import executor + from src.llm import executor, registry emitted: list[BaseEvent] = [] @@ -426,7 +426,7 @@ class TestExecutorEndToEnd: raise RuntimeError("rate limited") with ( - patch.object(executor, "CLIENTS", {"anthropic": object()}), + patch.object(registry, "CLIENTS", {"anthropic": object()}), patch.object(executor, "backend_for_provider", return_value=object()), patch.object(executor, "execute_stream", new=_setup_explodes), patch( @@ -455,7 +455,7 @@ class TestExecutorEndToEnd: @pytest.mark.asyncio async def test_error_path_still_emits_via_finally(self): - from src.llm import executor + from src.llm import executor, registry emitted: list[BaseEvent] = [] @@ -463,7 +463,7 @@ class TestExecutorEndToEnd: raise RuntimeError("backend exploded") with ( - patch.object(executor, "CLIENTS", {"anthropic": object()}), + patch.object(registry, "CLIENTS", {"anthropic": object()}), patch.object( executor, "backend_for_provider", @@ -570,7 +570,7 @@ class TestStreamFinalResponseRetryAttempt: async def test_attempt_index_bumps_across_retries(self): from collections.abc import AsyncIterator - from src.llm import executor, tool_loop + from src.llm import executor, registry, tool_loop emitted: list[BaseEvent] = [] @@ -607,7 +607,7 @@ class TestStreamFinalResponseRetryAttempt: ) with ( - patch.object(executor, "CLIENTS", {"anthropic": object()}), + patch.object(registry, "CLIENTS", {"anthropic": object()}), patch.object(executor, "backend_for_provider", return_value=object()), patch.object(executor, "execute_stream", new=_flaky_setup), patch( diff --git a/tests/routes/test_scope_reads.py b/tests/routes/test_scope_reads.py new file mode 100644 index 00000000..eafdecaf --- /dev/null +++ b/tests/routes/test_scope_reads.py @@ -0,0 +1,750 @@ +"""Tests for the `scope` option on the read routes. + +A single scope swaps the observer to the scope peer, so recall is confined to +the (scope, observed) collection and the scope's member sessions by existing +observer semantics. A list of scopes keeps the path peer as observer and +restricts recall to the union of the scopes' member sessions (the +session-allowlist arm). +""" + +from typing import Any + +import pytest +from fastapi.testclient import TestClient +from nanoid import generate as generate_nanoid +from sqlalchemy.ext.asyncio import AsyncSession + +from src import crud, models +from src.config import settings +from src.models import Peer, Workspace +from src.security import JWTParams, create_jwt +from src.utils.scopes import scope_peer_name + + +def _create_scope(client: TestClient, workspace_name: str, scope_name: str): + response = client.post( + f"/v3/workspaces/{workspace_name}/scopes", json={"id": scope_name} + ) + assert response.status_code in [200, 201] + return response + + +def _create_session( + client: TestClient, + workspace_name: str, + session_name: str | None = None, + **extra: Any, +) -> str: + session_name = session_name or str(generate_nanoid()) + response = client.post( + f"/v3/workspaces/{workspace_name}/sessions", + json={"id": session_name, **extra}, + ) + assert response.status_code in [200, 201] + return session_name + + +def _add_sessions_to_scope( + client: TestClient, workspace_name: str, scope_name: str, session_names: list[str] +) -> None: + response = client.post( + f"/v3/workspaces/{workspace_name}/scopes/{scope_name}/sessions", + json={"session_ids": session_names}, + ) + assert response.status_code == 204, response.text + + +async def _seed_documents( + db_session: AsyncSession, + workspace_name: str, + *, + observer: str, + observed: str, + contents: list[tuple[str, str | None]], +) -> None: + """Seed a collection plus documents for an (observer, observed) pair. + + ``contents`` is a list of (content, session_name) tuples. + """ + collection = models.Collection( + workspace_name=workspace_name, + observer=observer, + observed=observed, + ) + db_session.add(collection) + await db_session.flush() + db_session.add_all( + [ + models.Document( + workspace_name=workspace_name, + observer=observer, + observed=observed, + content=content, + session_name=session_name, + ) + for content, session_name in contents + ] + ) + await db_session.commit() + + +async def _seed_legacy_collision_peer( + db_session: AsyncSession, workspace_name: str, scope_name: str +) -> None: + """Create a plain peer squatting on a scope's reserved internal name.""" + db_session.add( + models.Peer( + workspace_name=workspace_name, + name=scope_peer_name(scope_name), + ) + ) + await db_session.commit() + + +class TestScopeReadValidation: + """4xx paths shared by chat and representation. + + Chat validation happens before any LLM work, so these are safe to exercise. + """ + + def _chat( + self, client: TestClient, workspace: Workspace, peer: Peer, body: dict[str, Any] + ): + return client.post( + f"/v3/workspaces/{workspace.name}/peers/{peer.name}/chat", + json={"query": "what do you know?", **body}, + ) + + def _representation( + self, client: TestClient, workspace: Workspace, peer: Peer, body: dict[str, Any] + ): + return client.post( + f"/v3/workspaces/{workspace.name}/peers/{peer.name}/representation", + json=body, + ) + + def test_unknown_scope_404( + self, client: TestClient, sample_data: tuple[Workspace, Peer] + ): + workspace, peer = sample_data + unknown = str(generate_nanoid()) + assert ( + self._chat(client, workspace, peer, {"scope": unknown}).status_code == 404 + ) + assert ( + self._representation( + client, workspace, peer, {"scope": unknown} + ).status_code + == 404 + ) + + def test_unknown_scope_in_list_404( + self, client: TestClient, sample_data: tuple[Workspace, Peer] + ): + workspace, peer = sample_data + scope_name = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_name) + resp = self._representation( + client, workspace, peer, {"scope": [scope_name, str(generate_nanoid())]} + ) + assert resp.status_code == 404 + + async def test_non_scope_peer_as_scope_422( + self, + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], + ): + """A peer squatting on the reserved name without the kind flag is not a scope.""" + workspace, peer = sample_data + scope_name = str(generate_nanoid()) + await _seed_legacy_collision_peer(db_session, workspace.name, scope_name) + + assert ( + self._chat(client, workspace, peer, {"scope": scope_name}).status_code + == 422 + ) + assert ( + self._representation( + client, workspace, peer, {"scope": scope_name} + ).status_code + == 422 + ) + + def test_scope_plus_filters_422( + self, client: TestClient, sample_data: tuple[Workspace, Peer] + ): + workspace, peer = sample_data + scope_name = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_name) + body = {"scope": scope_name, "filters": {"session_id": ["s1"]}} + assert self._chat(client, workspace, peer, body).status_code == 422 + assert self._representation(client, workspace, peer, body).status_code == 422 + + def test_scope_plus_session_id_422( + self, client: TestClient, sample_data: tuple[Workspace, Peer] + ): + workspace, peer = sample_data + scope_name = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_name) + body = {"scope": scope_name, "session_id": "s1"} + assert self._chat(client, workspace, peer, body).status_code == 422 + assert self._representation(client, workspace, peer, body).status_code == 422 + + def test_peer_scoped_jwt_401( + self, + client: TestClient, + sample_data: tuple[Workspace, Peer], + monkeypatch: pytest.MonkeyPatch, + ): + """A scope's sessions may exceed the peer's own membership: workspace/admin only. + + 401, matching every other scope surface — see _validate_scope_option. + """ + workspace, peer = sample_data + scope_name = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_name) + + monkeypatch.setattr(settings.AUTH, "USE_AUTH", True) + monkeypatch.setattr(settings.AUTH, "JWT_SECRET", "test-secret") + client.headers["Authorization"] = ( + f"Bearer {create_jwt(JWTParams(w=workspace.name, p=peer.name))}" + ) + + assert ( + self._chat(client, workspace, peer, {"scope": scope_name}).status_code + == 401 + ) + assert ( + self._representation( + client, workspace, peer, {"scope": scope_name} + ).status_code + == 401 + ) + + # A session-scoped key gets the same answer, but from `require_auth` + # rather than from `_validate_scope_option`: these routes declare + # `peer_name` and no `session_name`, so an `s` token never reaches the + # handler at all. Asserted here so the handler's peer-only check stays + # sufficient — if either route ever starts declaring a session, this + # fails and the check needs the `s` arm the session-context route has. + client.headers["Authorization"] = ( + f"Bearer {create_jwt(JWTParams(w=workspace.name, s='any-session'))}" + ) + assert ( + self._chat(client, workspace, peer, {"scope": scope_name}).status_code + == 401 + ) + assert ( + self._representation( + client, workspace, peer, {"scope": scope_name} + ).status_code + == 401 + ) + + # A workspace-level key is allowed through validation (404 here only + # if the scope were unknown; representation of an empty scope is 200). + client.headers["Authorization"] = ( + f"Bearer {create_jwt(JWTParams(w=workspace.name))}" + ) + assert ( + self._representation( + client, workspace, peer, {"scope": scope_name} + ).status_code + == 200 + ) + + def test_scope_union_cap_422( + self, + client: TestClient, + sample_data: tuple[Workspace, Peer], + monkeypatch: pytest.MonkeyPatch, + ): + workspace, peer = sample_data + scope_name = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_name) + session_a = _create_session(client, workspace.name) + session_b = _create_session(client, workspace.name) + _add_sessions_to_scope( + client, workspace.name, scope_name, [session_a, session_b] + ) + + monkeypatch.setattr("src.routers.peers.MAX_SESSION_ALLOWLIST_ENTRIES", 1) + resp = self._representation(client, workspace, peer, {"scope": [scope_name]}) + assert resp.status_code == 422 + assert "maximum" in resp.json()["detail"] + + @pytest.mark.parametrize( + "scope", + [ + pytest.param([], id="empty-list"), + pytest.param(["s"] * 101, id="over-list-cap"), + pytest.param([scope_peer_name("already-prefixed")], id="double-prefixed"), + pytest.param(["ok", "not a name!"], id="bad-charset-element"), + pytest.param(scope_peer_name("already-prefixed"), id="single-prefixed"), + ], + ) + def test_scope_option_bounds_422( + self, + client: TestClient, + sample_data: tuple[Workspace, Peer], + scope: str | list[str], + ): + """Schema-level bounds on `scope`, before any scope is resolved: the list is + bounded at both ends, and every element is validated as an unprefixed scope + name (so a double-prefixed one is a 422, not a 404 for `scope.scope.x`).""" + workspace, peer = sample_data + assert self._chat(client, workspace, peer, {"scope": scope}).status_code == 422 + assert ( + self._representation(client, workspace, peer, {"scope": scope}).status_code + == 422 + ) + + +class TestRepresentationWithScope: + async def test_single_scope_reads_scope_collection( + self, + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], + ): + """A single scope swaps the observer: only the (scope, peer) collection is read.""" + workspace, peer = sample_data + scope_name = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_name) + session_a = _create_session(client, workspace.name) + session_b = _create_session(client, workspace.name) + _add_sessions_to_scope(client, workspace.name, scope_name, [session_a]) + + # Conclusions the scope observed (session A) ... + await _seed_documents( + db_session, + workspace.name, + observer=scope_peer_name(scope_name), + observed=peer.name, + contents=[("scoped fact about hiking", session_a)], + ) + # ... and global self-observations from another session + await _seed_documents( + db_session, + workspace.name, + observer=peer.name, + observed=peer.name, + contents=[("global fact about cooking", session_b)], + ) + + resp = client.post( + f"/v3/workspaces/{workspace.name}/peers/{peer.name}/representation", + json={"scope": scope_name}, + ) + assert resp.status_code == 200 + representation = resp.json()["representation"] + assert "scoped fact about hiking" in representation + assert "global fact about cooking" not in representation + + async def test_scope_list_unions_member_sessions( + self, + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], + ): + """A scope list keeps the global observer and applies the union allowlist.""" + workspace, peer = sample_data + scope_a = str(generate_nanoid()) + scope_b = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_a) + _create_scope(client, workspace.name, scope_b) + session_a = _create_session(client, workspace.name) + session_b = _create_session(client, workspace.name) + session_c = _create_session(client, workspace.name) + _add_sessions_to_scope(client, workspace.name, scope_a, [session_a]) + _add_sessions_to_scope(client, workspace.name, scope_b, [session_b]) + + # All conclusions live in the GLOBAL (peer, peer) collection: only the + # union session-allowlist can explain the filtering below (this is the + # dynamic session-allowlist arm, not the observer swap). + await _seed_documents( + db_session, + workspace.name, + observer=peer.name, + observed=peer.name, + contents=[ + ("fact from session a", session_a), + ("fact from session b", session_b), + ("fact from session c", session_c), + ("sessionless dream fact", None), + ], + ) + + resp = client.post( + f"/v3/workspaces/{workspace.name}/peers/{peer.name}/representation", + json={"scope": [scope_a, scope_b]}, + ) + assert resp.status_code == 200 + representation = resp.json()["representation"] + assert "fact from session a" in representation + assert "fact from session b" in representation + assert "fact from session c" not in representation + assert "sessionless dream fact" not in representation + + def test_empty_scope_list_fails_closed( + self, client: TestClient, sample_data: tuple[Workspace, Peer] + ): + """A scope with no member sessions yields an empty representation.""" + workspace, peer = sample_data + scope_name = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_name) + + resp = client.post( + f"/v3/workspaces/{workspace.name}/peers/{peer.name}/representation", + json={"scope": [scope_name]}, + ) + assert resp.status_code == 200 + assert "fact" not in resp.json()["representation"] + + +class TestChatWithScope: + """Verify what the chat route hands the dialectic, without real LLM work. + + ``agentic_chat`` is mocked in conftest (``mock_llm_call_functions``); the + scoped peer-card fetch happens inside it and is covered end-to-end by the + session-context test. Here we assert the route passes the right observer / + observed / session_names — the wiring that keys the card fetch. + """ + + def test_single_scope_swaps_observer( + self, + client: TestClient, + sample_data: tuple[Workspace, Peer], + mock_llm_call_functions: dict[str, Any], + ): + workspace, peer = sample_data + scope_name = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_name) + + resp = client.post( + f"/v3/workspaces/{workspace.name}/peers/{peer.name}/chat", + json={"query": "what do you know?", "scope": scope_name}, + ) + assert resp.status_code == 200 + + kwargs = mock_llm_call_functions["agentic_chat"].await_args.kwargs + # The scope peer is the observer; the path peer stays the observed + assert kwargs["observer"] == scope_peer_name(scope_name) + assert kwargs["observed"] == peer.name + # Single-scope confinement rides on observer semantics, not an allowlist + assert kwargs["session_allowlist"] is None + + def test_scope_list_passes_union_allowlist( + self, + client: TestClient, + sample_data: tuple[Workspace, Peer], + mock_llm_call_functions: dict[str, Any], + ): + workspace, peer = sample_data + scope_a = str(generate_nanoid()) + scope_b = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_a) + _create_scope(client, workspace.name, scope_b) + session_a = _create_session(client, workspace.name) + session_b = _create_session(client, workspace.name) + _add_sessions_to_scope(client, workspace.name, scope_a, [session_a]) + _add_sessions_to_scope(client, workspace.name, scope_b, [session_b]) + + resp = client.post( + f"/v3/workspaces/{workspace.name}/peers/{peer.name}/chat", + json={"query": "what do you know?", "scope": [scope_a, scope_b]}, + ) + assert resp.status_code == 200 + + kwargs = mock_llm_call_functions["agentic_chat"].await_args.kwargs + # Union path: the path peer stays the observer, the allowlist is the union + assert kwargs["observer"] == peer.name + assert kwargs["observed"] == peer.name + assert set(kwargs["session_allowlist"]) == {session_a, session_b} + + +class TestWorkspaceSearchWithScope: + def _seed_message( + self, client: TestClient, workspace_name: str, session_name: str, peer: Peer + ) -> None: + resp = client.post( + f"/v3/workspaces/{workspace_name}/sessions/{session_name}/messages", + json={ + "messages": [{"peer_id": peer.name, "content": "needle in haystack"}] + }, + ) + assert resp.status_code == 201 + + def test_search_restricted_to_scope_sessions( + self, client: TestClient, sample_data: tuple[Workspace, Peer] + ): + workspace, peer = sample_data + scope_name = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_name) + session_a = _create_session(client, workspace.name, peers={peer.name: {}}) + session_b = _create_session(client, workspace.name, peers={peer.name: {}}) + _add_sessions_to_scope(client, workspace.name, scope_name, [session_a]) + self._seed_message(client, workspace.name, session_a, peer) + self._seed_message(client, workspace.name, session_b, peer) + + # Unscoped: both sessions' messages match + resp = client.post( + f"/v3/workspaces/{workspace.name}/search", + json={"query": "needle"}, + ) + assert resp.status_code == 200 + assert {m["session_id"] for m in resp.json()} == {session_a, session_b} + + # Scoped: only the scope's member session + resp = client.post( + f"/v3/workspaces/{workspace.name}/search", + json={"query": "needle", "scope": scope_name}, + ) + assert resp.status_code == 200 + results = resp.json() + assert results + assert {m["session_id"] for m in results} == {session_a} + + def test_empty_scope_returns_no_results( + self, client: TestClient, sample_data: tuple[Workspace, Peer] + ): + workspace, peer = sample_data + scope_name = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_name) + session_a = _create_session(client, workspace.name, peers={peer.name: {}}) + self._seed_message(client, workspace.name, session_a, peer) + + resp = client.post( + f"/v3/workspaces/{workspace.name}/search", + json={"query": "needle", "scope": scope_name}, + ) + assert resp.status_code == 200 + assert resp.json() == [] + + def test_unknown_scope_404( + self, client: TestClient, sample_data: tuple[Workspace, Peer] + ): + workspace, _ = sample_data + resp = client.post( + f"/v3/workspaces/{workspace.name}/search", + json={"query": "needle", "scope": str(generate_nanoid())}, + ) + assert resp.status_code == 404 + + def test_scope_plus_session_id_filter_422( + self, client: TestClient, sample_data: tuple[Workspace, Peer] + ): + workspace, _ = sample_data + scope_name = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_name) + resp = client.post( + f"/v3/workspaces/{workspace.name}/search", + json={ + "query": "needle", + "scope": scope_name, + "filters": {"session_id": "s1"}, + }, + ) + assert resp.status_code == 422 + + +class TestSessionContextWithScope: + async def test_scope_swaps_perspective_source( + self, + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], + ): + """`scope` reads the scope's collection and the scoped peer card.""" + workspace, peer = sample_data + scope_name = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_name) + session_name = _create_session(client, workspace.name, peers={peer.name: {}}) + _add_sessions_to_scope(client, workspace.name, scope_name, [session_name]) + + await _seed_documents( + db_session, + workspace.name, + observer=scope_peer_name(scope_name), + observed=peer.name, + contents=[("scoped fact about hiking", session_name)], + ) + await _seed_documents( + db_session, + workspace.name, + observer=peer.name, + observed=peer.name, + contents=[("global fact about cooking", session_name)], + ) + await crud.set_peer_card( + db_session, + workspace.name, + peer_card=["SCOPED CARD"], + observer=scope_peer_name(scope_name), + observed=peer.name, + ) + await crud.set_peer_card( + db_session, + workspace.name, + peer_card=["GLOBAL CARD"], + observer=peer.name, + observed=peer.name, + ) + await db_session.commit() + + # Without scope: the global (self) perspective + resp = client.get( + f"/v3/workspaces/{workspace.name}/sessions/{session_name}/context", + params={"peer_target": peer.name}, + ) + assert resp.status_code == 200 + data = resp.json() + assert "global fact about cooking" in data["peer_representation"] + assert data["peer_card"] == ["GLOBAL CARD"] + + # With scope: the scope's perspective + resp = client.get( + f"/v3/workspaces/{workspace.name}/sessions/{session_name}/context", + params={"peer_target": peer.name, "scope": scope_name}, + ) + assert resp.status_code == 200 + data = resp.json() + assert "scoped fact about hiking" in data["peer_representation"] + assert "global fact about cooking" not in data["peer_representation"] + assert data["peer_card"] == ["SCOPED CARD"] + + def test_scope_requires_peer_target( + self, client: TestClient, sample_data: tuple[Workspace, Peer] + ): + workspace, peer = sample_data + scope_name = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_name) + session_name = _create_session(client, workspace.name, peers={peer.name: {}}) + + resp = client.get( + f"/v3/workspaces/{workspace.name}/sessions/{session_name}/context", + params={"scope": scope_name}, + ) + assert resp.status_code == 422 + + def test_scope_and_peer_perspective_mutually_exclusive( + self, client: TestClient, sample_data: tuple[Workspace, Peer] + ): + workspace, peer = sample_data + scope_name = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_name) + session_name = _create_session(client, workspace.name, peers={peer.name: {}}) + + resp = client.get( + f"/v3/workspaces/{workspace.name}/sessions/{session_name}/context", + params={ + "peer_target": peer.name, + "peer_perspective": peer.name, + "scope": scope_name, + }, + ) + assert resp.status_code == 422 + + def test_unknown_scope_404( + self, client: TestClient, sample_data: tuple[Workspace, Peer] + ): + workspace, peer = sample_data + session_name = _create_session(client, workspace.name, peers={peer.name: {}}) + + resp = client.get( + f"/v3/workspaces/{workspace.name}/sessions/{session_name}/context", + params={"peer_target": peer.name, "scope": str(generate_nanoid())}, + ) + assert resp.status_code == 404 + + def test_narrow_keys_rejected_401( + self, + client: TestClient, + sample_data: tuple[Workspace, Peer], + monkeypatch: pytest.MonkeyPatch, + ): + """Peer- and session-scoped keys may not widen reads through a scope.""" + workspace, peer = sample_data + scope_name = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_name) + session_name = _create_session(client, workspace.name, peers={peer.name: {}}) + _add_sessions_to_scope(client, workspace.name, scope_name, [session_name]) + + monkeypatch.setattr(settings.AUTH, "USE_AUTH", True) + monkeypatch.setattr(settings.AUTH, "JWT_SECRET", "test-secret") + url = f"/v3/workspaces/{workspace.name}/sessions/{session_name}/context" + params = {"peer_target": peer.name, "scope": scope_name} + + # Peer-scoped key (member read grants access to the route, not to scope) + client.headers["Authorization"] = ( + f"Bearer {create_jwt(JWTParams(w=workspace.name, p=peer.name))}" + ) + assert client.get(url, params=params).status_code == 401 + + # Session-scoped key + client.headers["Authorization"] = ( + f"Bearer {create_jwt(JWTParams(w=workspace.name, s=session_name))}" + ) + assert client.get(url, params=params).status_code == 401 + + # Workspace-scoped key is allowed + client.headers["Authorization"] = ( + f"Bearer {create_jwt(JWTParams(w=workspace.name))}" + ) + assert client.get(url, params=params).status_code == 200 + + +class TestScopePeerGuardrailClosure: + """Scope peers are rejected on the generic perspective/context surfaces.""" + + def test_session_context_rejects_scope_peer_target( + self, client: TestClient, sample_data: tuple[Workspace, Peer] + ): + workspace, peer = sample_data + scope_name = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_name) + session_name = _create_session(client, workspace.name, peers={peer.name: {}}) + + resp = client.get( + f"/v3/workspaces/{workspace.name}/sessions/{session_name}/context", + params={"peer_target": scope_peer_name(scope_name)}, + ) + assert resp.status_code == 422 + + def test_session_context_rejects_scope_peer_perspective( + self, client: TestClient, sample_data: tuple[Workspace, Peer] + ): + workspace, peer = sample_data + scope_name = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_name) + session_name = _create_session(client, workspace.name, peers={peer.name: {}}) + + resp = client.get( + f"/v3/workspaces/{workspace.name}/sessions/{session_name}/context", + params={ + "peer_target": peer.name, + "peer_perspective": scope_peer_name(scope_name), + }, + ) + assert resp.status_code == 422 + + def test_peer_context_rejects_scope_peer( + self, client: TestClient, sample_data: tuple[Workspace, Peer] + ): + workspace, peer = sample_data + scope_name = str(generate_nanoid()) + _create_scope(client, workspace.name, scope_name) + + # As the path-level peer + resp = client.get( + f"/v3/workspaces/{workspace.name}/peers/{scope_peer_name(scope_name)}/context" + ) + assert resp.status_code == 422 + + # As the target + resp = client.get( + f"/v3/workspaces/{workspace.name}/peers/{peer.name}/context", + params={"target": scope_peer_name(scope_name)}, + ) + assert resp.status_code == 422 diff --git a/tests/routes/test_scope_route_policy.py b/tests/routes/test_scope_route_policy.py new file mode 100644 index 00000000..9332ca46 --- /dev/null +++ b/tests/routes/test_scope_route_policy.py @@ -0,0 +1,960 @@ +"""Route-policy enumeration for the scopes facade, per peer *position*. + +Four review passes over the scopes work each found the same class of defect: a +place nobody had checked, rather than logic that was subtly wrong. The first +version of this module enumerated routes and classified each one guarded or +exempt — and that model was itself the fifth defect. A binary per-route verdict +cannot express the actual invariant, which is positional: + + A scope may be an OBSERVER. A scope may never be OBSERVED. + +`POST /conclusions` is the case that proves it: a scope as `observer_id` is how +scoped conclusions are stored and must work, while a scope as `observed_id` +persisted a conclusion about something that carries ``observe_me=false``. One +route, two positions, opposite verdicts. The same split applies to +`schedule_dream`, the peer-card routes, and session context. + +One refinement, added with the `scope` read option: on the *read* routes an +observer position is refused too, even though a scope there is mechanically +legitimate. Asking for a scope's perspective is what `scope` is for, and routing +through it is what keeps the observer mechanics hidden — so `peer_perspective`, +`GET /peers/{peer_id}/context`, chat and representation all refuse a raw scope +peer name and point at `scope` instead. The invariant above still governs the +storage side, where `observer_id` / `observer` remain ALLOW: a scope observing is +the entire mechanism. Read "OBSERVER" as "may observe", not "may be named as one +on any route". + +So classification here is keyed by ``(method, path, position)``, where position is +the request parameter carrying the peer name. Every derived triple must appear in +`POLICY` as either REFUSE or ALLOW-with-a-reason; a new one fails +`test_every_peer_position_is_classified` until someone classifies it. + +Each REFUSE case is then asserted behaviorally — by calling the route, because the +guards deliberately live in crud (which is what makes `messages/upload` guarded +for free via `crud.create_messages`) — in both directions: + +1. a real scope is refused, and the rejection must actually name it, so an + unrelated 422 cannot pass the assertion; +2. an *unflagged* peer merely occupying the reserved namespace is NOT a scope and + is unaffected. That half regressed once already when `update_peer` keyed off + the name prefix. + +Known limitation: this covers the HTTP surface only. Peer names also reach the +system through the deriver, dreamer, and queue, which have no route table to +enumerate; a gap there would not be caught here. +""" + +from collections.abc import Callable, Iterator +from dataclasses import dataclass + +import pytest +from fastapi.routing import APIRoute +from fastapi.testclient import TestClient +from httpx import Response +from nanoid import generate as generate_nanoid +from pydantic import BaseModel +from pydantic.fields import FieldInfo +from sqlalchemy import select +from sqlalchemy.ext.asyncio import AsyncSession + +from src import models +from src.main import app +from src.models import Peer, Workspace +from src.utils.scopes import scope_peer_name + +# Request parameters and model fields that carry a peer name, in any position. +_PEER_PARAM_NAMES = { + "peer_id", + "peer_name", + "peer_names", + "observer", + "observer_id", + "observed", + "observed_id", + "sender_id", + "target", + "peer_target", + "peer_perspective", +} + +# Peer names arriving as dict keys or an aliased body field are invisible to +# parameter-name detection, so these paths are matched by shape instead. The +# position recorded for them is the body field or key role. +_KEY_POSITION = "body_peer_keys" + +# The scopes router *is* the facade; scope peers are its whole subject. +_SCOPES_PREFIX = "/v3/workspaces/{workspace_id}/scopes" + +# A builder places `peer` into one position of one route and returns the response. +Builder = Callable[[TestClient, str, str, str], Response] + + +@dataclass(frozen=True) +class Case: + """Policy for one peer position on one route.""" + + method: str + path: str + position: str + refuse: bool + reason: str = "" + build: Builder | None = None + # REFUSE cases only. Whether a reserved name that does NOT YET EXIST is also + # refused — the third axis, and the one that is not derivable from `refuse`. + # It follows from the guard the call site picked: + # + # validate_no_scope_peer_names name-only, no DB refuses missing + # reject_scope_observed strict on the observed refuses missing + # reject_scope_peers flag-based, permissive allows missing + # + # Permissive is correct where something downstream still stops it (the create + # path validates new names) or where the name simply resolves to nothing (404 + # before any guard runs). Each False therefore needs `missing_reason`. + refuse_missing: bool | None = None + missing_reason: str = "" + # Required when refuse_missing is False: the exact status(es) a missing + # reserved name may receive. Deliberately not a bare `!= 422` — that passes on + # a 5xx too, which is the same hole the squatter assertion below had. + missing_status: tuple[int, ...] = () + # Set when the 422 legitimately comes from request-schema validation rather + # than a scope guard, so the detail is pydantic's rather than ours. + schema_level: bool = False + # Set when the squatter direction cannot be asserted here, with why. + skip_squatter: str = "" + # ALLOW cases only: builder plus the status a real scope must receive, so the + # suite proves legitimate observer positions keep working. + allow_status: tuple[int, ...] = () + + @property + def key(self) -> tuple[str, str, str]: + return (self.method, self.path, self.position) + + +_W = "/v3/workspaces/{workspace_id}" + + +def _b_create_peer(c: TestClient, ws: str, _s: str, p: str): + return c.post(f"/v3/workspaces/{ws}/peers", json={"id": p}) + + +def _b_update_peer(c: TestClient, ws: str, _s: str, p: str): + return c.put(f"/v3/workspaces/{ws}/peers/{p}", json={"metadata": {"k": "v"}}) + + +def _b_chat_observer(c: TestClient, ws: str, _s: str, p: str): + return c.post(f"/v3/workspaces/{ws}/peers/{p}/chat", json={"query": "hi"}) + + +def _b_chat_target(c: TestClient, ws: str, _s: str, p: str): + return c.post( + f"/v3/workspaces/{ws}/peers/{_OTHER}/chat", json={"query": "hi", "target": p} + ) + + +def _b_repr_observer(c: TestClient, ws: str, _s: str, p: str): + return c.post(f"/v3/workspaces/{ws}/peers/{p}/representation", json={}) + + +def _b_repr_target(c: TestClient, ws: str, _s: str, p: str): + return c.post( + f"/v3/workspaces/{ws}/peers/{_OTHER}/representation", json={"target": p} + ) + + +def _b_card_target(c: TestClient, ws: str, _s: str, p: str): + return c.put( + f"/v3/workspaces/{ws}/peers/{_OTHER}/card?target={p}", + json={"peer_card": ["note"]}, + ) + + +def _b_conclusion_observed(c: TestClient, ws: str, _s: str, p: str): + return c.post( + f"/v3/workspaces/{ws}/conclusions", + json={ + "conclusions": [ + { + "observer_id": _OTHER, + "observed_id": p, + "content": "something", + "level": "explicit", + } + ] + }, + ) + + +def _b_dream_observed(c: TestClient, ws: str, _s: str, p: str): + return c.post( + f"/v3/workspaces/{ws}/schedule_dream", + json={"observer": _OTHER, "observed": p, "dream_type": "omni"}, + ) + + +def _b_session_create(c: TestClient, ws: str, _s: str, p: str): + return c.post( + f"/v3/workspaces/{ws}/sessions", + json={"id": str(generate_nanoid()), "peers": {p: {}}}, + ) + + +def _b_session_context_target(c: TestClient, ws: str, s: str, p: str): + return c.get(f"/v3/workspaces/{ws}/sessions/{s}/context?peer_target={p}") + + +def _b_message(c: TestClient, ws: str, s: str, p: str): + return c.post( + f"/v3/workspaces/{ws}/sessions/{s}/messages", + json={"messages": [{"peer_id": p, "content": "hello"}]}, + ) + + +def _b_upload(c: TestClient, ws: str, s: str, p: str): + return c.post( + f"/v3/workspaces/{ws}/sessions/{s}/messages/upload", + data={"peer_id": p}, + files={"file": ("note.txt", b"hello there", "text/plain")}, + ) + + +def _b_add_peers(c: TestClient, ws: str, s: str, p: str): + return c.post(f"/v3/workspaces/{ws}/sessions/{s}/peers", json={p: {}}) + + +def _b_set_peers(c: TestClient, ws: str, s: str, p: str): + return c.put(f"/v3/workspaces/{ws}/sessions/{s}/peers", json={p: {}}) + + +def _b_remove_peers(c: TestClient, ws: str, s: str, p: str): + return c.request("DELETE", f"/v3/workspaces/{ws}/sessions/{s}/peers", json=[p]) + + +def _b_conclusion_observer(c: TestClient, ws: str, _s: str, p: str): + return c.post( + f"/v3/workspaces/{ws}/conclusions", + json={ + "conclusions": [ + { + "observer_id": p, + "observed_id": _OTHER, + "content": "something", + "level": "explicit", + } + ] + }, + ) + + +def _b_dream_observer(c: TestClient, ws: str, _s: str, p: str): + return c.post( + f"/v3/workspaces/{ws}/schedule_dream", + json={"observer": p, "observed": _OTHER, "dream_type": "omni"}, + ) + + +def _b_card_observer_put(c: TestClient, ws: str, _s: str, p: str): + return c.put( + f"/v3/workspaces/{ws}/peers/{p}/card?target={_OTHER}", + json={"peer_card": ["note"]}, + ) + + +def _b_card_observer_get(c: TestClient, ws: str, _s: str, p: str): + return c.get(f"/v3/workspaces/{ws}/peers/{p}/card?target={_OTHER}") + + +def _b_peer_context_observer(c: TestClient, ws: str, _s: str, p: str): + return c.get(f"/v3/workspaces/{ws}/peers/{p}/context") + + +def _b_peer_context_target(c: TestClient, ws: str, _s: str, p: str): + return c.get(f"/v3/workspaces/{ws}/peers/{_OTHER}/context?target={p}") + + +def _b_context_perspective(c: TestClient, ws: str, s: str, p: str): + query = f"?peer_perspective={p}&peer_target={_OTHER}" + return c.get(f"/v3/workspaces/{ws}/sessions/{s}/context{query}") + + +def _b_queue_status_observer(c: TestClient, ws: str, _s: str, p: str): + return c.get(f"/v3/workspaces/{ws}/queue/status?observer_id={p}") + + +def _b_queue_status_sender(c: TestClient, ws: str, _s: str, p: str): + return c.get(f"/v3/workspaces/{ws}/queue/status?sender_id={p}") + + +def _b_peer_config(c: TestClient, ws: str, s: str, p: str): + return c.put( + f"/v3/workspaces/{ws}/sessions/{s}/peers/{p}/config", + json={"observe_others": False, "observe_me": True}, + ) + + +def _b_peer_config_get(c: TestClient, ws: str, s: str, p: str): + return c.get(f"/v3/workspaces/{ws}/sessions/{s}/peers/{p}/config") + + +# A plain peer used for the *other* side of two-position routes, so the position +# under test is the only scope in the request. Created by the fixtures below. +_OTHER = "policy-counterparty" + +_OBSERVER_OK = ( + "Observer position. A scope observing others is the entire mechanism scopes " + "are built on, so this must keep working." +) +_READ_ONLY_OK = ( + "Read-only. Returns nothing meaningful for a scope rather than creating or " + "mutating knowledge about one." +) + +POLICY: tuple[Case, ...] = ( + # ---- observed position: a scope must never be the subject ---- + Case( + "POST", + f"{_W}/conclusions", + "observed_id", + True, + refuse_missing=False, + missing_reason=( + "Every observer and observed peer is resolved before the scope check, " + "so a name that does not exist is a 404 and no conclusion is written. " + "The guard is still the strict variant, for if that ever changes." + ), + missing_status=(404,), + build=_b_conclusion_observed, + ), + Case( + "POST", + f"{_W}/schedule_dream", + "observed", + True, + refuse_missing=True, + build=_b_dream_observed, + ), + Case( + "PUT", + f"{_W}/peers/{{peer_id}}/card", + "target", + True, + refuse_missing=True, + build=_b_card_target, + ), + Case( + "POST", + f"{_W}/peers/{{peer_id}}/chat", + "target", + True, + refuse_missing=True, + build=_b_chat_target, + ), + Case( + "POST", + f"{_W}/peers/{{peer_id}}/representation", + "target", + True, + refuse_missing=True, + build=_b_repr_target, + ), + Case( + "GET", + f"{_W}/sessions/{{session_id}}/context", + "peer_target", + True, + refuse_missing=True, + build=_b_session_context_target, + ), + # ---- observer position: legitimately a scope ---- + Case( + "POST", + f"{_W}/conclusions", + "observer_id", + False, + reason=_OBSERVER_OK, + build=_b_conclusion_observer, + allow_status=(200, 201), + ), + Case( + "POST", + f"{_W}/schedule_dream", + "observer", + False, + reason=_OBSERVER_OK, + build=_b_dream_observer, + allow_status=(204,), + ), + Case( + "PUT", + f"{_W}/peers/{{peer_id}}/card", + "peer_id", + False, + reason=_OBSERVER_OK, + build=_b_card_observer_put, + allow_status=(200,), + ), + Case( + "GET", + f"{_W}/peers/{{peer_id}}/card", + "peer_id", + False, + reason=_OBSERVER_OK, + build=_b_card_observer_get, + allow_status=(200,), + ), + Case( + "GET", + f"{_W}/sessions/{{session_id}}/context", + "peer_perspective", + True, + refuse_missing=False, + missing_reason=( + "The perspective peer is resolved before the flag-based guard runs, so a " + "reserved name that does not exist yet is a 404 — the same answer any " + "absent peer gets here — and nothing on this path creates it." + ), + missing_status=(404,), + build=_b_context_perspective, + ), + Case( + "GET", + f"{_W}/queue/status", + "observer_id", + False, + reason=_OBSERVER_OK, + build=_b_queue_status_observer, + allow_status=(200,), + ), + Case( + "GET", + f"{_W}/queue/status", + "sender_id", + False, + reason=( + "Filter only. `sender_id` reaches CRUD as `observed`, but it selects " + "existing queue rows rather than creating knowledge about a peer." + ), + build=_b_queue_status_sender, + allow_status=(200,), + ), + # ---- peer identity / membership mutation: never a scope ---- + Case( + "POST", + f"{_W}/peers", + _KEY_POSITION, + True, + refuse_missing=True, + build=_b_create_peer, + schema_level=True, + skip_squatter=( + "Creating any name in the reserved namespace is refused whether flagged " + "or not — that is what reserving it means. Covered by " + "test_scopes.py::test_peer_create_rejects_reserved_prefix." + ), + ), + Case( + "PUT", + f"{_W}/peers/{{peer_id}}", + "peer_id", + True, + refuse_missing=True, + build=_b_update_peer, + ), + Case( + "POST", + f"{_W}/sessions", + "peer_names", + True, + refuse_missing=True, + build=_b_session_create, + ), + Case( + "POST", + f"{_W}/sessions/{{session_id}}/messages", + "peer_name", + True, + refuse_missing=True, + build=_b_message, + ), + Case( + "POST", + f"{_W}/sessions/{{session_id}}/messages/upload", + "peer_id", + True, + refuse_missing=True, + build=_b_upload, + ), + Case( + "POST", + f"{_W}/sessions/{{session_id}}/peers", + _KEY_POSITION, + True, + refuse_missing=True, + build=_b_add_peers, + ), + Case( + "PUT", + f"{_W}/sessions/{{session_id}}/peers", + _KEY_POSITION, + True, + refuse_missing=True, + build=_b_set_peers, + ), + Case( + "DELETE", + f"{_W}/sessions/{{session_id}}/peers", + _KEY_POSITION, + True, + refuse_missing=False, + missing_reason=( + "Removal creates nothing and a name that does not exist has no " + "membership row, so the request is a no-op. Refusing here would give a " + "reserved name a different removal result than any other absent peer." + ), + missing_status=(200,), + build=_b_remove_peers, + ), + Case( + "PUT", + f"{_W}/sessions/{{session_id}}/peers/{{peer_id}}/config", + "peer_id", + True, + refuse_missing=False, + missing_reason=( + "The peer is resolved before the scope check, so a name that does not " + "exist is a 404 and never reaches the guard. Nothing is created, so " + "there is no window for the name to be claimed here." + ), + missing_status=(404,), + build=_b_peer_config, + ), + Case( + "GET", + f"{_W}/sessions/{{session_id}}/peers/{{peer_id}}/config", + "peer_id", + True, + refuse_missing=False, + missing_reason=( + "Same resolution order as the write side of this route: an absent peer " + "is a 404 before the scope check, and a read creates nothing." + ), + missing_status=(404,), + build=_b_peer_config_get, + ), + # ---- path peer on the dialectic surface ---- + Case( + "POST", + f"{_W}/peers/{{peer_id}}/chat", + "peer_id", + True, + refuse_missing=True, + build=_b_chat_observer, + ), + Case( + "POST", + f"{_W}/peers/{{peer_id}}/representation", + "peer_id", + True, + refuse_missing=True, + build=_b_repr_observer, + ), + # ---- reads that neither create nor mutate knowledge about a scope ---- + Case( + "POST", f"{_W}/peers/{{peer_id}}/search", "peer_id", False, reason=_READ_ONLY_OK + ), + Case( + "POST", + f"{_W}/peers/{{peer_id}}/sessions", + "peer_id", + False, + reason=( + "Read-only. A scope legitimately has member sessions; this is the " + "observer-mechanics view of POST /scopes/{scope_id}/sessions/list." + ), + ), + Case( + "GET", + f"{_W}/peers/{{peer_id}}/context", + "peer_id", + True, + refuse_missing=True, + build=_b_peer_context_observer, + ), + Case( + "GET", + f"{_W}/peers/{{peer_id}}/context", + "target", + True, + refuse_missing=True, + build=_b_peer_context_target, + ), + Case( + "GET", + f"{_W}/peers/{{peer_id}}/card", + "target", + False, + reason=( + "Read-only. The write side (PUT with target) IS refused, so this can only " + "return pre-existing rows, never create them." + ), + ), + Case( + "POST", + "/v3/keys", + "peer_id", + False, + reason=( + "Mints a scoped JWT rather than touching a peer, so no peer row is read " + "or written. Keys cannot be bound to a scope yet." + ), + ), +) + +_BY_KEY = {case.key: case for case in POLICY} + +# Routes whose peer names arrive as dict keys or an aliased body field, invisible +# to parameter-name detection and therefore matched by path shape. +_KEY_POSITION_PATHS = { + ("POST", f"{_W}/peers"), + ("POST", f"{_W}/sessions/{{session_id}}/peers"), + ("PUT", f"{_W}/sessions/{{session_id}}/peers"), + ("DELETE", f"{_W}/sessions/{{session_id}}/peers"), +} + + +def _nested_models(annotation: object, seen: set[object]) -> Iterator[type[BaseModel]]: + """Yield `annotation` and every pydantic model nested inside it.""" + if ( + not isinstance(annotation, type) + or not issubclass(annotation, BaseModel) + or annotation in seen + ): + return + model: type[BaseModel] = annotation + seen.add(model) + yield model + fields: dict[str, FieldInfo] = model.model_fields + for f in fields.values(): + stack = [f.annotation] + while stack: + current = stack.pop() + yield from _nested_models(current, seen) + stack.extend(getattr(current, "__args__", ()) or ()) + + +def _peer_positions(route: APIRoute) -> set[str]: + """Peer-name-carrying parameter names anywhere in a route's dependant tree. + + Walks sub-dependencies so `Form(...)` params behind a parser dependency are + seen — this is how `messages/upload` takes its `peer_id` — and descends into + request-body models so `MessageCreate.peer_name` is seen too. + """ + found: set[str] = set() + seen: set[object] = set() + stack = [route.dependant] + while stack: + dependant = stack.pop() + params = ( + dependant.path_params + + dependant.query_params + + dependant.header_params + + dependant.body_params + ) + for param in params: + if param.name in _PEER_PARAM_NAMES: + found.add(param.name) + annotations = [param.field_info.annotation] + while annotations: + annotation = annotations.pop() + for model in _nested_models(annotation, seen): + found |= set(model.model_fields) & _PEER_PARAM_NAMES + annotations.extend(getattr(annotation, "__args__", ()) or ()) + stack.extend(dependant.dependencies) + return found + + +def _derived_positions() -> set[tuple[str, str, str]]: + """Every (method, path, position) through which a peer name can be supplied.""" + found: set[tuple[str, str, str]] = set() + for route in app.routes: + if not isinstance(route, APIRoute): + continue + path = route.path.rstrip("/") or route.path + if path.startswith(_SCOPES_PREFIX): + continue + positions = _peer_positions(route) + for method in route.methods or set(): + if method in ("HEAD", "OPTIONS"): + continue + if (method, path) in _KEY_POSITION_PATHS: + found.add((method, path, _KEY_POSITION)) + for position in positions: + found.add((method, path, position)) + return found + + +def test_every_peer_position_is_classified(): + """Each (route, peer position) pair has an explicit scope policy. + + A new one fails here until classified. Decide whether a scope in that + *position* is harmful — the rule is that a scope may be an observer but never + observed — then add a Case with `refuse=True` and a builder, or `refuse=False` + and a reason. + """ + derived = _derived_positions() + classified = set(_BY_KEY) + + # _peer_positions walks FastAPI/Pydantic internals (route.dependant, its + # *_params lists, field_info.annotation). An upgrade that reshapes any of them + # would make derivation silently return nothing, and every assertion below + # would then pass vacuously. Anchor on a position that must always be found. + assert ( + "POST", + f"{_W}/sessions/{{session_id}}/messages", + "peer_name", + ) in derived, ( + "derived no peer positions for a route that certainly has one — the " + "FastAPI internals _peer_positions() traverses have probably changed shape" + ) + + unclassified = derived - classified + assert not unclassified, ( + "peer positions with no scope policy: " + + f"{sorted(unclassified)} — classify each as refuse or allow" + ) + + stale = classified - derived + assert not stale, f"classified positions that no longer exist: {sorted(stale)}" + + +def test_policy_entries_are_well_formed(): + assert len(_BY_KEY) == len(POLICY), "duplicate (method, path, position) in POLICY" + for case in POLICY: + if case.refuse: + assert case.build is not None, f"{case.key} refuses but has no builder" + assert not case.reason, f"{case.key} refuses; reason is for allow cases" + # The missing-name axis is not derivable from `refuse`, so it must be + # stated rather than defaulted — that gap is what this field closes. + assert case.refuse_missing is not None, ( + f"{case.key} refuses a real scope but does not say whether a " + "reserved name that does not exist yet is also refused" + ) + if case.refuse_missing: + assert not case.missing_status, ( + f"{case.key} refuses a missing reserved name, so the expected " + "status is 422 — missing_status is for the permissive cases" + ) + else: + assert ( + len(case.missing_reason.strip()) > 30 + ), f"{case.key} tolerates a missing reserved name; say why" + assert case.missing_status, ( + f"{case.key} tolerates a missing reserved name; name the exact " + "status(es) it should get, so a 5xx cannot satisfy the case" + ) + else: + assert len(case.reason.strip()) > 30, f"{case.key} needs a real reason" + assert bool(case.build) == bool(case.allow_status), ( + f"{case.key}: an allow case needs a builder and an expected " + "allow_status together, or neither" + ) + assert ( + case.refuse_missing is None + ), f"{case.key}: refuse_missing applies to REFUSE cases only" + + +_REFUSING = tuple(case for case in POLICY if case.refuse) +# Allow cases that additionally prove, behaviorally, that a real scope works here. +_ALLOWING_EXERCISED = tuple( + case for case in POLICY if not case.refuse and case.build is not None +) + + +def _setup(client: TestClient, workspace: str) -> tuple[str, str]: + """Create the counterparty peer and a session, returning (session, scope name).""" + assert client.post( + f"/v3/workspaces/{workspace}/peers", json={"id": _OTHER} + ).status_code in (200, 201) + session_name = str(generate_nanoid()) + assert client.post( + f"/v3/workspaces/{workspace}/sessions", json={"id": session_name} + ).status_code in (200, 201) + return session_name, str(generate_nanoid()) + + +def _real_scope( + client: TestClient, workspace: str, session_name: str, scope_name: str +) -> str: + """Create a scope, attach the session to it, and return its backing peer name.""" + assert ( + client.post( + f"/v3/workspaces/{workspace}/scopes", json={"id": scope_name} + ).status_code + == 201 + ) + assert ( + client.post( + f"/v3/workspaces/{workspace}/scopes/{scope_name}/sessions", + json={"session_ids": [session_name]}, + ).status_code + == 204 + ) + return scope_peer_name(scope_name) + + +@pytest.mark.parametrize("case", _REFUSING, ids=lambda c: f"{c.method}:{c.position}") +def test_refusing_position_rejects_a_real_scope( + client: TestClient, + sample_data: tuple[Workspace, Peer], + case: Case, +): + """A real scope is refused in every position marked REFUSE.""" + test_workspace, _ = sample_data + session_name, scope_name = _setup(client, test_workspace.name) + backing = _real_scope(client, test_workspace.name, session_name, scope_name) + + assert case.build is not None + result = case.build(client, test_workspace.name, session_name, backing) + status = result.status_code + assert status == 422, ( + f"{case.method} {case.path} accepted a scope in position " + f"{case.position!r} (got {status})" + ) + + # A 422 alone proves nothing — a malformed body would also produce one. + detail = result.text + if case.schema_level: + assert ( + "pattern" in detail + ), f"{case.key} expected a schema-level refusal; detail: {detail[:200]}" + else: + assert "scope" in detail.lower() and backing in detail, ( + f"{case.key} returned 422 but not because of the scope; " + f"detail: {detail[:200]}" + ) + + +@pytest.mark.parametrize( + "case", _ALLOWING_EXERCISED, ids=lambda c: f"{c.method}:{c.position}" +) +def test_allowing_position_accepts_a_real_scope( + client: TestClient, + sample_data: tuple[Workspace, Peer], + case: Case, +): + """A real scope works in every position marked ALLOW. + + The other half of the contract. Refusal tests alone would be satisfied by a + guard that rejected scopes everywhere, which would break the feature: scoped + conclusions, scoped dreams and scoped peer cards all require a scope in the + observer position. + """ + test_workspace, _ = sample_data + session_name, scope_name = _setup(client, test_workspace.name) + backing = _real_scope(client, test_workspace.name, session_name, scope_name) + + assert case.build is not None + result = case.build(client, test_workspace.name, session_name, backing) + assert result.status_code in case.allow_status, ( + f"{case.method} {case.path} refused a scope in the legitimate position " + f"{case.position!r}: expected {case.allow_status}, got " + f"{result.status_code} — {result.text[:200]}" + ) + + +@pytest.mark.parametrize( + "case", + tuple(c for c in _REFUSING if not c.skip_squatter), + ids=lambda c: f"{c.method}:{c.position}", +) +async def test_refusing_position_allows_unflagged_squatter( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], + case: Case, +): + """A peer merely occupying the reserved namespace is not a scope. + + Peer names were length-validated only before migration d429de0e5338, so + `scope.production` is a possible real user name. Such a peer has only the name + half of the invariant and must keep working — a guard keying off the prefix + alone locks a tenant out of its own data. + """ + test_workspace, _ = sample_data + session_name, _ = _setup(client, test_workspace.name) + squatter = scope_peer_name(str(generate_nanoid())) + db_session.add(models.Peer(workspace_name=test_workspace.name, name=squatter)) + await db_session.commit() + + # Give it a membership so config and removal have a row to act on. + assert ( + client.post( + f"/v3/workspaces/{test_workspace.name}/sessions/{session_name}/peers", + json={squatter: {}}, + ).status_code + == 200 + ) + + assert case.build is not None + result = case.build(client, test_workspace.name, session_name, squatter) + # Deliberately not `!= 422`: that also passes on a 5xx, so a guard regressing + # into an unhandled error (the psycopg DataError path this feature defends + # against) would keep this green. + assert result.status_code < 400, ( + f"{case.method} {case.path} did not accept an unflagged squatter in " + f"position {case.position!r} (got {result.status_code}) — a 422 means the " + "guard is keying off the name prefix rather than the scope flag; anything " + f"else means the request blew up. Body: {result.text[:200]}" + ) + + +@pytest.mark.parametrize("case", _REFUSING, ids=lambda c: f"{c.method}:{c.position}") +async def test_refusing_position_and_a_missing_reserved_name( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], + case: Case, +): + """The third axis: a reserved name that does not exist yet. + + Neither of the other two tests reaches it — both resolve an existing subject. + A permissive guard here is sometimes correct (the create path refuses the name + itself, or it simply resolves to nothing), which is why the expected verdict is + declared per case rather than assumed. + + What is NOT negotiable in either direction is that the request must not MINT + the reserved name. Minting it would let any caller squat a scope name before + the workspace owner can create it, and would leave a peer that the facade can + never adopt. + """ + test_workspace, _ = sample_data + session_name, _ = _setup(client, test_workspace.name) + missing = scope_peer_name(str(generate_nanoid())) + + assert case.build is not None + result = case.build(client, test_workspace.name, session_name, missing) + + if case.refuse_missing: + assert result.status_code == 422, ( + f"{case.method} {case.path} accepted a not-yet-existing reserved name " + f"in position {case.position!r} (got {result.status_code}) — it could " + f"become a scope later. Body: {result.text[:200]}" + ) + else: + # Exact, not `!= 422`: a permissive position still has one correct answer, + # and a 5xx must not read as tolerance. + assert result.status_code in case.missing_status, ( + f"{case.method} {case.path} gave {result.status_code} for a missing " + f"reserved name in position {case.position!r}; the policy expects " + f"{case.missing_status} because {case.missing_reason!r} — update the " + f"policy or the guard. Body: {result.text[:200]}" + ) + + minted = await db_session.scalar( + select(models.Peer) + .where(models.Peer.workspace_name == test_workspace.name) + .where(models.Peer.name == missing) + ) + assert minted is None, ( + f"{case.method} {case.path} minted the reserved name {missing!r} from " + f"position {case.position!r} — the scope namespace is now squatted" + ) diff --git a/tests/routes/test_scopes.py b/tests/routes/test_scopes.py new file mode 100644 index 00000000..a68a3725 --- /dev/null +++ b/tests/routes/test_scopes.py @@ -0,0 +1,1201 @@ +"""Tests for the scopes facade: scope-kind peers, guardrails, and CRUD routes. + +A scope is a named grouping of sessions, implemented as a peer named +``scope.`` carrying ``{"kind": "scope"}`` in ``internal_metadata`` and +``{"observe_me": false}`` in ``configuration``, that observes its member sessions +and never speaks. See src/utils/scopes.py. +""" + +import re +from typing import Any + +import pytest +from fastapi.testclient import TestClient +from nanoid import generate as generate_nanoid +from sqlalchemy import select +from sqlalchemy.ext.asyncio import AsyncSession + +from src import crud, models, schemas +from src.config import settings +from src.deriver.enqueue import enqueue +from src.exceptions import ValidationException +from src.models import Peer, QueueItem, Workspace +from src.schemas.api import RESOURCE_NAME_PATTERN +from src.security import JWTParams, create_jwt +from src.utils.scopes import ( + SCOPE_PEER_PREFIX, + is_scope_peer, + is_scope_peer_name, + scope_name_from_peer, + scope_peer_name, +) + + +def _create_scope( + client: TestClient, + workspace_name: str, + scope_name: str, + metadata: dict[str, Any] | None = None, +): + body: dict[str, Any] = {"id": scope_name} + if metadata is not None: + body["metadata"] = metadata + return client.post(f"/v3/workspaces/{workspace_name}/scopes", json=body) + + +def _create_session( + client: TestClient, + workspace_name: str, + session_name: str | None = None, + **extra: Any, +): + session_name = session_name or str(generate_nanoid()) + response = client.post( + f"/v3/workspaces/{workspace_name}/sessions", + json={"id": session_name, **extra}, + ) + assert response.status_code in [200, 201] + return session_name + + +def _add_sessions( + client: TestClient, + workspace_name: str, + scope_name: str, + session_names: list[str], +) -> int: + """Add sessions to a scope via the facade; returns the status code (204 on success).""" + return client.post( + f"/v3/workspaces/{workspace_name}/scopes/{scope_name}/sessions", + json={"session_ids": session_names}, + ).status_code + + +def _scope_sessions( + client: TestClient, workspace_name: str, scope_name: str +) -> list[str]: + """Names of a scope's member sessions, oldest membership first.""" + response = client.post( + f"/v3/workspaces/{workspace_name}/scopes/{scope_name}/sessions/list" + ) + assert response.status_code == 200, response.text + return [item["id"] for item in response.json()["items"]] + + +async def _get_session_peer( + db_session: AsyncSession, + workspace_name: str, + session_name: str, + peer_name: str, +) -> models.SessionPeer | None: + return await db_session.scalar( + select(models.SessionPeer) + .where(models.SessionPeer.workspace_name == workspace_name) + .where(models.SessionPeer.session_name == session_name) + .where(models.SessionPeer.peer_name == peer_name) + ) + + +def test_scope_namespace_helpers(): + assert scope_peer_name("therapy") == "scope.therapy" + assert is_scope_peer_name("scope.therapy") + assert not is_scope_peer_name("therapy") + assert scope_name_from_peer("scope.therapy") == "therapy" + # The prefix must stay outside the peer-name charset, or an existing peer + # could occupy the scope namespace. + assert not re.fullmatch(RESOURCE_NAME_PATTERN, SCOPE_PEER_PREFIX) + + +async def test_create_scope_creates_flagged_peer( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], +): + """Creating a scope creates a peer with the kind flag and observe_me=false.""" + test_workspace, _ = sample_data + scope_name = str(generate_nanoid()) + + response = _create_scope( + client, test_workspace.name, scope_name, metadata={"purpose": "testing"} + ) + assert response.status_code == 201 + data = response.json() + # The response id is the UNPREFIXED scope name + assert data["id"] == scope_name + assert data["metadata"] == {"purpose": "testing"} + assert "created_at" in data + + peer = await db_session.scalar( + select(models.Peer) + .where(models.Peer.workspace_name == test_workspace.name) + .where(models.Peer.name == scope_peer_name(scope_name)) + ) + assert peer is not None + # The kind flag lives in internal_metadata (not user-writable); only + # observe_me is in the user-visible configuration. + assert peer.internal_metadata == {"kind": "scope"} + assert peer.configuration == {"observe_me": False} + + +def test_create_scope_idempotent( + client: TestClient, sample_data: tuple[Workspace, Peer] +): + test_workspace, _ = sample_data + scope_name = str(generate_nanoid()) + + first = _create_scope(client, test_workspace.name, scope_name) + assert first.status_code == 201 + + second = _create_scope(client, test_workspace.name, scope_name) + assert second.status_code == 200 + assert second.json()["id"] == scope_name + + +def test_create_scope_rejects_invalid_names( + client: TestClient, sample_data: tuple[Workspace, Peer] +): + test_workspace, _ = sample_data + + # Names must match the resource name pattern + response = _create_scope(client, test_workspace.name, "bad name!") + assert response.status_code == 422 + + # Scope names are unprefixed: double-prefixing is rejected + response = _create_scope(client, test_workspace.name, "scope.therapy") + assert response.status_code == 422 + + +async def test_create_scope_rejects_legacy_collision( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], +): + """A pre-existing plain peer occupying the reserved name is never adopted.""" + test_workspace, _ = sample_data + scope_name = str(generate_nanoid()) + + legacy_peer = models.Peer( + workspace_name=test_workspace.name, + name=scope_peer_name(scope_name), + ) + db_session.add(legacy_peer) + await db_session.commit() + + response = _create_scope(client, test_workspace.name, scope_name) + assert response.status_code == 409 + + # And the collision peer is invisible to the scope read routes + response = client.get(f"/v3/workspaces/{test_workspace.name}/scopes/{scope_name}") + assert response.status_code == 404 + + +def test_get_scope(client: TestClient, sample_data: tuple[Workspace, Peer]): + test_workspace, _ = sample_data + scope_name = str(generate_nanoid()) + assert _create_scope(client, test_workspace.name, scope_name).status_code == 201 + + response = client.get(f"/v3/workspaces/{test_workspace.name}/scopes/{scope_name}") + assert response.status_code == 200 + assert response.json()["id"] == scope_name + + response = client.get( + f"/v3/workspaces/{test_workspace.name}/scopes/{generate_nanoid()}" + ) + assert response.status_code == 404 + + +def test_list_scopes(client: TestClient, sample_data: tuple[Workspace, Peer]): + """The scopes list contains only scopes, with unprefixed ids.""" + test_workspace, test_peer = sample_data + scope_names = {str(generate_nanoid()), str(generate_nanoid())} + for scope_name in scope_names: + assert _create_scope(client, test_workspace.name, scope_name).status_code == 201 + + response = client.post(f"/v3/workspaces/{test_workspace.name}/scopes/list") + assert response.status_code == 200 + items = response.json()["items"] + listed = {item["id"] for item in items} + assert listed == scope_names + assert test_peer.name not in listed + + +def test_scopes_routes_require_workspace_level_key( + client: TestClient, + sample_data: tuple[Workspace, Peer], + monkeypatch: pytest.MonkeyPatch, +): + """Scopes are an app-level admin surface: workspace keys work, peer- and + session-scoped keys are rejected.""" + test_workspace, test_peer = sample_data + monkeypatch.setattr(settings.AUTH, "USE_AUTH", True) + monkeypatch.setattr(settings.AUTH, "JWT_SECRET", "test-secret") + scope_name = str(generate_nanoid()) + scopes_url = f"/v3/workspaces/{test_workspace.name}/scopes" + + # Workspace-scoped key: allowed + client.headers["Authorization"] = ( + f"Bearer {create_jwt(JWTParams(w=test_workspace.name))}" + ) + assert client.post(scopes_url, json={"id": scope_name}).status_code == 201 + + # Peer-scoped key: rejected + client.headers["Authorization"] = ( + f"Bearer {create_jwt(JWTParams(w=test_workspace.name, p=test_peer.name))}" + ) + assert client.post(scopes_url, json={"id": scope_name}).status_code == 401 + assert client.post(f"{scopes_url}/list").status_code == 401 + assert client.get(f"{scopes_url}/{scope_name}").status_code == 401 + + # Session-scoped key: rejected + client.headers["Authorization"] = ( + f"Bearer {create_jwt(JWTParams(w=test_workspace.name, s='some-session'))}" + ) + assert client.post(f"{scopes_url}/{scope_name}/sessions/list").status_code == 401 + + +def test_session_create_scopes_requires_workspace_level_key( + client: TestClient, + sample_data: tuple[Workspace, Peer], + monkeypatch: pytest.MonkeyPatch, +): + """`scopes` on session create is the scopes routes by another door. + + It creates scope peers and attaches memberships, so it carries the same + workspace-level requirement. Session create is otherwise a self-authorizing + route that accepts peer- and session-scoped keys, which is exactly why this + needs its own check rather than the route's auth dependency. + """ + test_workspace, test_peer = sample_data + monkeypatch.setattr(settings.AUTH, "USE_AUTH", True) + monkeypatch.setattr(settings.AUTH, "JWT_SECRET", "test-secret") + sessions_url = f"/v3/workspaces/{test_workspace.name}/sessions" + scopes_url = f"/v3/workspaces/{test_workspace.name}/scopes" + scope_name = str(generate_nanoid()) + + def create_with_scope(session_name: str) -> int: + return client.post( + sessions_url, json={"id": session_name, "scopes": [scope_name]} + ).status_code + + def listed_scopes() -> set[str]: + response = client.post(f"{scopes_url}/list") + assert response.status_code == 200 + return {item["id"] for item in response.json()["items"]} + + # Peer-scoped key: rejected + client.headers["Authorization"] = ( + f"Bearer {create_jwt(JWTParams(w=test_workspace.name, p=test_peer.name))}" + ) + assert create_with_scope(str(generate_nanoid())) == 401 + + # Session-scoped key: rejected. The session name must match the token's `s` + # claim, or the handler's own session check would 401 first and this would + # pass without ever reaching the scopes gate. + own_session = str(generate_nanoid()) + client.headers["Authorization"] = ( + f"Bearer {create_jwt(JWTParams(w=test_workspace.name, s=own_session))}" + ) + assert create_with_scope(own_session) == 401 + + # Workspace-scoped key: allowed. The scope is absent until this call lands, + # which proves the rejected requests did not mint it on their way out. + client.headers["Authorization"] = ( + f"Bearer {create_jwt(JWTParams(w=test_workspace.name))}" + ) + assert scope_name not in listed_scopes() + assert create_with_scope(str(generate_nanoid())) == 201 + assert scope_name in listed_scopes() + + +def test_peer_create_rejects_reserved_prefix( + client: TestClient, sample_data: tuple[Workspace, Peer] +): + """User-created peers may not use the reserved scope prefix. + + The prefix sits outside RESOURCE_NAME_PATTERN, so PeerCreate's own charset + validation rejects it at the schema boundary — one layer earlier than the + route's validate_no_scope_peer_names guard. Either way the caller gets 422. + """ + test_workspace, _ = sample_data + + response = client.post( + f"/v3/workspaces/{test_workspace.name}/peers", + json={"name": f"{SCOPE_PEER_PREFIX}{generate_nanoid()}"}, + ) + assert response.status_code == 422 + assert RESOURCE_NAME_PATTERN in str(response.json()["detail"]) + + +def test_peers_list_kind_filtering( + client: TestClient, sample_data: tuple[Workspace, Peer] +): + """peers.list excludes scope peers by default; kind switches the view.""" + test_workspace, test_peer = sample_data + scope_name = str(generate_nanoid()) + assert _create_scope(client, test_workspace.name, scope_name).status_code == 201 + backing_peer_name = scope_peer_name(scope_name) + + # Default: scope peers excluded + response = client.post(f"/v3/workspaces/{test_workspace.name}/peers/list") + assert response.status_code == 200 + names = {item["id"] for item in response.json()["items"]} + assert test_peer.name in names + assert backing_peer_name not in names + + # kind=scope: only scope peers + response = client.post( + f"/v3/workspaces/{test_workspace.name}/peers/list", + json={"kind": "scope"}, + ) + assert response.status_code == 200 + names = {item["id"] for item in response.json()["items"]} + assert names == {backing_peer_name} + + # kind=all: everything + response = client.post( + f"/v3/workspaces/{test_workspace.name}/peers/list", + json={"kind": "all"}, + ) + assert response.status_code == 200 + names = {item["id"] for item in response.json()["items"]} + assert {test_peer.name, backing_peer_name} <= names + + +async def test_scope_sessions_add_list_remove( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], +): + test_workspace, _ = sample_data + workspace_name = test_workspace.name + scope_name = str(generate_nanoid()) + assert _create_scope(client, workspace_name, scope_name).status_code == 201 + session_1 = _create_session(client, workspace_name) + session_2 = _create_session(client, workspace_name) + + scope_base = f"/v3/workspaces/{workspace_name}/scopes/{scope_name}" + + # Add both sessions + assert ( + _add_sessions(client, workspace_name, scope_name, [session_1, session_2]) == 204 + ) + assert set(_scope_sessions(client, workspace_name, scope_name)) == { + session_1, + session_2, + } + + # Membership rows carry the observer shape: observe_others on, observe_me off + session_peer = await _get_session_peer( + db_session, workspace_name, session_1, scope_peer_name(scope_name) + ) + assert session_peer is not None + assert session_peer.left_at is None + assert session_peer.configuration["observe_others"] is True + assert session_peer.configuration["observe_me"] is False + + # Remove one membership (soft delete, like the generic remove-peer path) + response = client.delete(f"{scope_base}/sessions/{session_1}") + assert response.status_code == 204 + + assert _scope_sessions(client, workspace_name, scope_name) == [session_2] + + db_session.expire_all() + session_peer = await _get_session_peer( + db_session, workspace_name, session_1, scope_peer_name(scope_name) + ) + assert session_peer is not None + assert session_peer.left_at is not None + + +def test_scope_sessions_add_missing_session_404( + client: TestClient, sample_data: tuple[Workspace, Peer] +): + test_workspace, _ = sample_data + scope_name = str(generate_nanoid()) + assert _create_scope(client, test_workspace.name, scope_name).status_code == 201 + existing_session = _create_session(client, test_workspace.name) + + assert ( + _add_sessions( + client, + test_workspace.name, + scope_name, + [existing_session, str(generate_nanoid())], + ) + == 404 + ) + + +def test_scope_sessions_routes_404_on_unknown_scope( + client: TestClient, sample_data: tuple[Workspace, Peer] +): + test_workspace, _ = sample_data + session_name = _create_session(client, test_workspace.name) + unknown_scope = str(generate_nanoid()) + scope_base = f"/v3/workspaces/{test_workspace.name}/scopes/{unknown_scope}" + + assert ( + _add_sessions(client, test_workspace.name, unknown_scope, [session_name]) == 404 + ) + assert client.post(f"{scope_base}/sessions/list").status_code == 404 + assert client.delete(f"{scope_base}/sessions/{session_name}").status_code == 404 + + +async def test_session_create_with_scopes( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], +): + """`scopes` on session creation creates the scope peers and memberships.""" + test_workspace, _ = sample_data + scope_a = str(generate_nanoid()) + scope_b = str(generate_nanoid()) + session_name = str(generate_nanoid()) + + response = client.post( + f"/v3/workspaces/{test_workspace.name}/sessions", + json={"id": session_name, "scopes": [scope_a, scope_b]}, + ) + assert response.status_code == 201 + + for scope_name in (scope_a, scope_b): + peer = await db_session.scalar( + select(models.Peer) + .where(models.Peer.workspace_name == test_workspace.name) + .where(models.Peer.name == scope_peer_name(scope_name)) + ) + assert peer is not None + assert peer.internal_metadata == {"kind": "scope"} + assert peer.configuration == {"observe_me": False} + + session_peer = await _get_session_peer( + db_session, test_workspace.name, session_name, scope_peer_name(scope_name) + ) + assert session_peer is not None + assert session_peer.configuration["observe_others"] is True + assert session_peer.configuration["observe_me"] is False + + # And the memberships show up through the facade + assert _scope_sessions(client, test_workspace.name, scope_a) == [session_name] + + +def test_session_create_rejects_prefixed_scope_names( + client: TestClient, sample_data: tuple[Workspace, Peer] +): + test_workspace, _ = sample_data + + response = client.post( + f"/v3/workspaces/{test_workspace.name}/sessions", + json={"id": str(generate_nanoid()), "scopes": ["scope.x"]}, + ) + assert response.status_code == 422 + + response = client.post( + f"/v3/workspaces/{test_workspace.name}/sessions", + json={"id": str(generate_nanoid()), "scopes": ["bad name!"]}, + ) + assert response.status_code == 422 + + +async def test_scope_membership_equals_hand_built_observer( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], +): + """Facade-less equivalence: a scope membership row is exactly what a + hand-built observer peer would have (name and kind flag aside).""" + test_workspace, _ = sample_data + + # Hand-built observer peer added through the generic session-peer route + observer_name = str(generate_nanoid()) + observer_session = _create_session(client, test_workspace.name) + response = client.post( + f"/v3/workspaces/{test_workspace.name}/sessions/{observer_session}/peers", + json={observer_name: {"observe_others": True, "observe_me": False}}, + ) + assert response.status_code == 200 + + # Scope membership added through the facade + scope_name = str(generate_nanoid()) + assert _create_scope(client, test_workspace.name, scope_name).status_code == 201 + scope_session = _create_session(client, test_workspace.name) + assert ( + _add_sessions(client, test_workspace.name, scope_name, [scope_session]) == 204 + ) + + hand_built = await _get_session_peer( + db_session, test_workspace.name, observer_session, observer_name + ) + via_facade = await _get_session_peer( + db_session, test_workspace.name, scope_session, scope_peer_name(scope_name) + ) + assert hand_built is not None and via_facade is not None + assert hand_built.configuration == via_facade.configuration + assert hand_built.left_at is None and via_facade.left_at is None + + +async def test_scope_peer_observes_ingested_messages( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], +): + """End-to-end litmus: after adding a session to a scope, a message from a + real peer fans out a representation task with the scope peer as observer.""" + test_workspace, test_peer = sample_data + scope_name = str(generate_nanoid()) + assert _create_scope(client, test_workspace.name, scope_name).status_code == 201 + session_name = _create_session(client, test_workspace.name) + + # Add the speaking peer and the scope membership + response = client.post( + f"/v3/workspaces/{test_workspace.name}/sessions/{session_name}/peers", + json={test_peer.name: {}}, + ) + assert response.status_code == 200 + assert _add_sessions(client, test_workspace.name, scope_name, [session_name]) == 204 + + # Ingest a message from the real peer and run the deriver enqueue fan-out + message = models.Message( + workspace_name=test_workspace.name, + session_name=session_name, + peer_name=test_peer.name, + content="I love hiking in the mountains", + public_id=generate_nanoid(), + seq_in_session=1, + token_count=10, + ) + db_session.add(message) + await db_session.commit() + + await enqueue( + [ + { + "workspace_name": test_workspace.name, + "session_name": session_name, + "message_id": message.id, + "content": message.content, + "peer_name": test_peer.name, + "created_at": message.created_at, + "message_public_id": message.public_id, + "seq_in_session": message.seq_in_session, + } + ] + ) + + result = await db_session.execute( + select(QueueItem) + .where(QueueItem.task_type == "representation") + .where(QueueItem.message_id == message.id) + ) + representation_items = list(result.scalars().all()) + assert len(representation_items) == 1 + payload = representation_items[0].payload + assert payload.get("observed") == test_peer.name + observers = payload.get("observers") + assert observers is not None + assert test_peer.name in observers # self-observation + assert scope_peer_name(scope_name) in observers # the scope observes + + +# --------------------------------------------------------------------------- +# Dotted / legacy peer names must not 500 (regression for the PeerCreate-as-DTO +# chokepoint). Names were length-only validated before migration +# d429de0e5338, so pre-existing peers can contain '.' — and re-validating them +# against PeerCreate's charset pattern raised a raw pydantic error, i.e. a 500. +# --------------------------------------------------------------------------- + + +async def test_legacy_dotted_peer_name_is_fully_usable( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], +): + """A pre-existing dotted peer name must work end to end, not 500.""" + test_workspace, _ = sample_data + legacy_name = f"alice.smith.{generate_nanoid()}" + + db_session.add(models.Peer(workspace_name=test_workspace.name, name=legacy_name)) + await db_session.commit() + + session_name = _create_session(client, test_workspace.name) + + # Can author messages + response = client.post( + f"/v3/workspaces/{test_workspace.name}/sessions/{session_name}/messages", + json={"messages": [{"peer_id": legacy_name, "content": "hello"}]}, + ) + assert response.status_code in [200, 201], response.text + + # Can be added to a session through the generic route + response = client.post( + f"/v3/workspaces/{test_workspace.name}/sessions/{session_name}/peers", + json={legacy_name: {}}, + ) + assert response.status_code == 200, response.text + + # Can be updated through the generic peer route + response = client.put( + f"/v3/workspaces/{test_workspace.name}/peers/{legacy_name}", + json={"metadata": {"k": "v"}}, + ) + assert response.status_code == 200, response.text + + +async def test_legacy_prefixed_peer_without_flag_is_not_a_scope( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], +): + """A peer merely occupying the reserved namespace keeps working. + + Only the name half of the invariant is present, so it is not a scope: it can + still author messages and shows up in the default peers list, while the + scopes facade refuses to treat it as a scope. + """ + test_workspace, _ = sample_data + scope_name = str(generate_nanoid()) + squatter = scope_peer_name(scope_name) + + db_session.add(models.Peer(workspace_name=test_workspace.name, name=squatter)) + await db_session.commit() + + session_name = _create_session(client, test_workspace.name) + + # Not a scope, so the message-author guard must not fire + response = client.post( + f"/v3/workspaces/{test_workspace.name}/sessions/{session_name}/messages", + json={"messages": [{"peer_id": squatter, "content": "hello"}]}, + ) + assert response.status_code in [200, 201], response.text + + # Visible in the default (non-scope) peers list + response = client.post(f"/v3/workspaces/{test_workspace.name}/peers/list") + assert squatter in [p["id"] for p in response.json()["items"]] + + # But the facade does not recognise it + response = client.get(f"/v3/workspaces/{test_workspace.name}/scopes/{scope_name}") + assert response.status_code == 404 + + response = client.post(f"/v3/workspaces/{test_workspace.name}/scopes/list") + assert scope_name not in [s["id"] for s in response.json()["items"]] + + +async def test_forged_configuration_kind_does_not_make_a_scope( + client: TestClient, + sample_data: tuple[Workspace, Peer], +): + """`configuration` is user-writable, so it must not be load-bearing. + + A peer that forges `{"kind": "scope"}` in configuration has neither the + reserved name nor the internal flag, so it stays an ordinary peer. + """ + test_workspace, _ = sample_data + peer_name = str(generate_nanoid()) + + response = client.put( + f"/v3/workspaces/{test_workspace.name}/peers/{peer_name}", + json={"configuration": {"kind": "scope"}}, + ) + assert response.status_code == 200, response.text + + # Still an ordinary peer: present in the default list, absent from scopes + response = client.post(f"/v3/workspaces/{test_workspace.name}/peers/list") + assert peer_name in [p["id"] for p in response.json()["items"]] + + response = client.post(f"/v3/workspaces/{test_workspace.name}/scopes/list") + assert peer_name not in [s["id"] for s in response.json()["items"]] + + # And can still author messages + session_name = _create_session(client, test_workspace.name) + response = client.post( + f"/v3/workspaces/{test_workspace.name}/sessions/{session_name}/messages", + json={"messages": [{"peer_id": peer_name, "content": "hello"}]}, + ) + assert response.status_code in [200, 201], response.text + + +def test_internal_metadata_never_in_peer_response( + client: TestClient, sample_data: tuple[Workspace, Peer] +): + """The scope flag must not leak into any peer response body.""" + test_workspace, _ = sample_data + scope_name = str(generate_nanoid()) + assert _create_scope(client, test_workspace.name, scope_name).status_code == 201 + + response = client.post( + f"/v3/workspaces/{test_workspace.name}/peers/list", json={"kind": "all"} + ) + assert response.status_code == 200 + for peer in response.json()["items"]: + assert "internal_metadata" not in peer + assert "kind" not in peer.get("configuration", {}) + + +async def test_crud_get_peer_resolves_scope_and_dotted_names( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], +): + """crud.get_peer must accept names outside RESOURCE_NAME_PATTERN. + + This is the Dreamer's preflight path: DreamScheduler passes + ``collection.observer`` straight through, and scope peers have + ``observe_others=true``, so ``(scope.x, peer)`` collections exist and get + dreamt. While get_peer took a PeerCreate, every such dream died at preflight + on a raw pydantic ValidationError. + """ + test_workspace, _ = sample_data + scope_name = str(generate_nanoid()) + assert _create_scope(client, test_workspace.name, scope_name).status_code == 201 + + resolved = await crud.get_peer( + db_session, test_workspace.name, scope_peer_name(scope_name) + ) + assert resolved.name == scope_peer_name(scope_name) + assert is_scope_peer(resolved.name, resolved.internal_metadata) + + dotted = f"legacy.dotted.{generate_nanoid()}" + db_session.add(models.Peer(workspace_name=test_workspace.name, name=dotted)) + await db_session.commit() + + resolved = await crud.get_peer(db_session, test_workspace.name, dotted) + assert resolved.name == dotted + # Has neither half of the invariant + assert not is_scope_peer(resolved.name, resolved.internal_metadata) + + +# --------------------------------------------------------------------------- +# Name validation on the create path. PeerSpec carries no charset pattern so +# existing names can be *looked up*, but anything crud is about to INSERT is a +# new peer and must obey the public contract — otherwise request-controlled +# names (message authors, session peer maps) mint squatters. +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize( + "bad_name", + ["not a valid name!@#", "has spaces", "emoji-\U0001f600"], +) +def test_message_author_cannot_create_invalid_peer_name( + client: TestClient, sample_data: tuple[Workspace, Peer], bad_name: str +): + """A message author must not be able to create a non-conforming peer.""" + test_workspace, _ = sample_data + session_name = _create_session(client, test_workspace.name) + + response = client.post( + f"/v3/workspaces/{test_workspace.name}/sessions/{session_name}/messages", + json={"messages": [{"peer_id": bad_name, "content": "hello"}]}, + ) + assert response.status_code == 422, response.text + + response = client.post( + f"/v3/workspaces/{test_workspace.name}/peers/list", json={"kind": "all"} + ) + assert bad_name not in [p["id"] for p in response.json()["items"]] + + +async def test_resolved_scope_peer_rejected_at_membership_upsert( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], +): + """The last-line guard runs on resolved rows, closing the check-then-upsert race. + + Calls crud directly, bypassing the route-level name check, so the only thing + standing between the caller and a scope membership is the guard on the + resolved peer row — the guard the racing caller would hit. The unflagged → + flagged transition itself is not simulated here. + """ + test_workspace, _ = sample_data + scope_name = str(generate_nanoid()) + assert _create_scope(client, test_workspace.name, scope_name).status_code == 201 + backing = scope_peer_name(scope_name) + session_name = _create_session(client, test_workspace.name) + + # crud-level: the resolved row is a scope, so membership must be refused even + # though the peer already exists (no create-path validation fires). + with pytest.raises(ValidationException): + await crud.get_or_create_session( + db_session, + session=schemas.SessionCreate( + name=session_name, peers={backing: schemas.SessionPeerConfig()} + ), + workspace_name=test_workspace.name, + ) + + +async def test_set_peer_config_cannot_disable_a_scope( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], +): + """A scope's membership config belongs to the facade, not the caller. + + `observe_others=false` would silently stop all fan-out into the scope, and + `observe_me=true` would make Honcho form a representation *of* a scope. + + Verified against the row rather than the config route, because that read is + refused for a scope too — see the policy table. + """ + test_workspace, _ = sample_data + scope_name = str(generate_nanoid()) + assert _create_scope(client, test_workspace.name, scope_name).status_code == 201 + backing = scope_peer_name(scope_name) + session_name = _create_session(client, test_workspace.name) + assert _add_sessions(client, test_workspace.name, scope_name, [session_name]) == 204 + + response = client.put( + f"/v3/workspaces/{test_workspace.name}/sessions/{session_name}/peers/{backing}/config", + json={"observe_others": False, "observe_me": True}, + ) + assert response.status_code == 422, response.text + + # Observer semantics intact + membership = await _get_session_peer( + db_session, test_workspace.name, session_name, backing + ) + assert membership is not None + assert membership.configuration == {"observe_me": False, "observe_others": True} + + +def test_session_peers_listing_excludes_scopes( + client: TestClient, sample_data: tuple[Workspace, Peer] +): + """The generic session-peer surface must not expose the facade's observer. + + Without the filter this listing returns a peer literally named + `scope.` carrying `observe_others=true` — the exact mechanic the + facade exists to hide. The membership-config read is refused for the same + reason; ordinary members are unaffected by both. + """ + test_workspace, test_peer = sample_data + base = f"/v3/workspaces/{test_workspace.name}" + scope_name = str(generate_nanoid()) + assert _create_scope(client, test_workspace.name, scope_name).status_code == 201 + backing = scope_peer_name(scope_name) + session_name = _create_session(client, test_workspace.name) + + assert ( + client.post( + f"{base}/sessions/{session_name}/peers", json={test_peer.name: {}} + ).status_code + == 200 + ) + assert _add_sessions(client, test_workspace.name, scope_name, [session_name]) == 204 + + listed = client.get(f"{base}/sessions/{session_name}/peers") + assert listed.status_code == 200 + names = {item["id"] for item in listed.json()["items"]} + assert test_peer.name in names + assert backing not in names + + assert ( + client.get(f"{base}/sessions/{session_name}/peers/{backing}/config").status_code + == 422 + ) + assert ( + client.get( + f"{base}/sessions/{session_name}/peers/{test_peer.name}/config" + ).status_code + == 200 + ) + + +@pytest.mark.parametrize("bad_name", ["", "a" * 513]) +def test_degenerate_peer_names_are_422_not_500( + client: TestClient, sample_data: tuple[Workspace, Peer], bad_name: str +): + """Empty and over-long author names must not reach PeerSpec and 500. + + Request-bound peer names carry no length bound of their own, so these used to + raise a raw pydantic ValidationError inside crud, which the catch-all handler + turned into a 500. + """ + test_workspace, _ = sample_data + session_name = _create_session(client, test_workspace.name) + + response = client.post( + f"/v3/workspaces/{test_workspace.name}/sessions/{session_name}/messages", + json={"messages": [{"peer_id": bad_name, "content": "hello"}]}, + ) + assert response.status_code == 422, response.text + + +# --------------------------------------------------------------------------- +# Detach-by-omission. The replacement routes never name the scope, so there is +# no peer position for the route-policy table to classify — the caller detaches +# by leaving it out. Preservation has to be flag-based, not request-based. +# --------------------------------------------------------------------------- + + +def test_generic_replacement_preserves_scope_membership( + client: TestClient, sample_data: tuple[Workspace, Peer] +): + """`PUT /sessions/{id}/peers` replaces ordinary peers, never scopes. + + The guard cannot key off the request body: a caller detaches a scope by simply + *omitting* it from an otherwise valid replacement map, never naming it. + """ + test_workspace, test_peer = sample_data + scope_name = str(generate_nanoid()) + assert _create_scope(client, test_workspace.name, scope_name).status_code == 201 + session_name = _create_session(client, test_workspace.name) + assert _add_sessions(client, test_workspace.name, scope_name, [session_name]) == 204 + + # Replacement naming only an ordinary peer must succeed... + response = client.put( + f"/v3/workspaces/{test_workspace.name}/sessions/{session_name}/peers", + json={test_peer.name: {}}, + ) + assert response.status_code == 200, response.text + + # ...while leaving the scope's membership intact. + assert _scope_sessions(client, test_workspace.name, scope_name) == [session_name] + + +async def test_empty_replacement_preserves_scope_membership( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], +): + """An empty replacement map clears ordinary peers but not scopes.""" + test_workspace, test_peer = sample_data + scope_name = str(generate_nanoid()) + assert _create_scope(client, test_workspace.name, scope_name).status_code == 201 + session_name = _create_session(client, test_workspace.name) + client.post( + f"/v3/workspaces/{test_workspace.name}/sessions/{session_name}/peers", + json={test_peer.name: {}}, + ) + _add_sessions(client, test_workspace.name, scope_name, [session_name]) + + assert ( + client.put( + f"/v3/workspaces/{test_workspace.name}/sessions/{session_name}/peers", + json={}, + ).status_code + == 200 + ) + + assert _scope_sessions(client, test_workspace.name, scope_name) == [session_name] + + # The other half of the docstring: without this the test passes even if the + # empty replacement became a no-op for ordinary peers too. + session_peer = await _get_session_peer( + db_session, test_workspace.name, session_name, test_peer.name + ) + assert session_peer is not None + assert session_peer.left_at is not None + + +async def test_replacement_still_removes_unflagged_squatter( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], +): + """The preservation is flag-based: a squatter keeps ordinary semantics.""" + test_workspace, test_peer = sample_data + squatter = scope_peer_name(str(generate_nanoid())) + db_session.add(models.Peer(workspace_name=test_workspace.name, name=squatter)) + await db_session.commit() + + session_name = _create_session(client, test_workspace.name) + assert ( + client.post( + f"/v3/workspaces/{test_workspace.name}/sessions/{session_name}/peers", + json={squatter: {}}, + ).status_code + == 200 + ) + + assert ( + client.put( + f"/v3/workspaces/{test_workspace.name}/sessions/{session_name}/peers", + json={test_peer.name: {}}, + ).status_code + == 200 + ) + session_peer = await _get_session_peer( + db_session, test_workspace.name, session_name, squatter + ) + assert session_peer is not None + assert session_peer.left_at is not None, "squatter should be replaced normally" + + +# --------------------------------------------------------------------------- +# Observed-position guards below the HTTP surface. A scope may be an observer +# but must never be observed — and "observed" includes a reserved name that does +# not yet exist, since nothing on these paths creates the peer and the state +# would retroactively describe the scope once created. The route-level half of +# this is enumerated in test_scope_route_policy.py; these cover the crud entry +# points that no route table reaches, plus the side effects a status code alone +# would not catch. +# --------------------------------------------------------------------------- + + +async def test_set_peer_card_guard_covers_internal_callers( + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], +): + """The guard is in crud, so Dreamer and agent-tool paths are covered too.""" + test_workspace, test_peer = sample_data + with pytest.raises(ValidationException): + await crud.set_peer_card( + db_session, + test_workspace.name, + peer_card=["x"], + observer=test_peer.name, + observed=scope_peer_name(str(generate_nanoid())), + ) + + +async def test_dream_cannot_be_queued_for_a_future_scope( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], +): + """A refused dream leaves no queue row behind. + + The 422 itself is enumerated in test_scope_route_policy.py; what that cannot + see is the side effect. The route's own precheck cannot catch this — the peer + is not flagged yet — so the check has to sit in the transaction that inserts + the queue item, and a check placed after the insert would still 422. + """ + test_workspace, test_peer = sample_data + future = scope_peer_name(str(generate_nanoid())) + + response = client.post( + f"/v3/workspaces/{test_workspace.name}/schedule_dream", + json={"observer": test_peer.name, "observed": future, "dream_type": "omni"}, + ) + assert response.status_code == 422, response.text + + # No queue row was inserted for it + items = ( + await db_session.execute( + select(QueueItem).where(QueueItem.work_unit_key.contains(future)) + ) + ).all() + assert not items + + +async def test_enqueue_dream_guard_covers_internal_callers( + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], +): + """Direct enqueue_dream calls enforce the same invariant as the route.""" + from src.deriver.enqueue import enqueue_dream + from src.schemas.configuration import DreamType + + test_workspace, test_peer = sample_data + scope_name = str(generate_nanoid()) + await db_session.commit() + + with pytest.raises(ValidationException): + await enqueue_dream( + test_workspace.name, + observer=test_peer.name, + observed=scope_peer_name(scope_name), + dream_type=DreamType.OMNI, + ) + + +def test_prefixed_nul_name_is_422_not_500( + client: TestClient, sample_data: tuple[Workspace, Peer] +): + """A reserved-prefix name containing NUL must not reach a text comparison. + + It passes the request schemas and PeerSpec, so without pre-SQL rejection it + reaches psycopg inside the scope lookup and raises DataError — a 500. + """ + test_workspace, _ = sample_data + session_name = _create_session(client, test_workspace.name) + + response = client.post( + f"/v3/workspaces/{test_workspace.name}/sessions/{session_name}/messages", + json={"messages": [{"peer_id": "scope.future\x00name", "content": "hi"}]}, + ) + assert response.status_code == 422, response.text + + +# --------------------------------------------------------------------------- +# Observer limit. Scope memberships carry observe_others=True but must not +# consume SESSION_OBSERVERS_LIMIT: that would cap scopes-per-session at the +# limit and report it as an observer-shaped 400 through a facade whose whole +# job is hiding observers. +# --------------------------------------------------------------------------- + + +def test_scopes_do_not_count_toward_observer_limit( + client: TestClient, sample_data: tuple[Workspace, Peer] +): + """A session can join more scopes than SESSION_OBSERVERS_LIMIT allows observers.""" + test_workspace, _ = sample_data + session_name = _create_session(client, test_workspace.name) + scope_names = [ + str(generate_nanoid()) for _ in range(settings.SESSION_OBSERVERS_LIMIT + 2) + ] + + for scope_name in scope_names: + assert _create_scope(client, test_workspace.name, scope_name).status_code == 201 + assert ( + _add_sessions(client, test_workspace.name, scope_name, [session_name]) + == 204 + ) + + # Every membership is live, and the session-create path agrees. + for scope_name in scope_names: + assert _scope_sessions(client, test_workspace.name, scope_name) == [ + session_name + ] + + response = client.post( + f"/v3/workspaces/{test_workspace.name}/sessions", + json={"id": str(generate_nanoid()), "scopes": scope_names}, + ) + assert response.status_code == 201, response.text + + +def test_observer_limit_still_applies_to_real_peers( + client: TestClient, sample_data: tuple[Workspace, Peer] +): + """The exclusion is scope-only: real observers are still capped. + + Without this the scope carve-out could quietly disable the limit entirely. + """ + test_workspace, _ = sample_data + session_name = _create_session(client, test_workspace.name) + scope_name = str(generate_nanoid()) + assert _create_scope(client, test_workspace.name, scope_name).status_code == 201 + assert _add_sessions(client, test_workspace.name, scope_name, [session_name]) == 204 + + observers = { + str(generate_nanoid()): {"observe_others": True} + for _ in range(settings.SESSION_OBSERVERS_LIMIT + 1) + } + response = client.post( + f"/v3/workspaces/{test_workspace.name}/sessions/{session_name}/peers", + json=observers, + ) + assert response.status_code == 400, response.text + + +def test_session_create_scopes_list_is_capped( + client: TestClient, sample_data: tuple[Workspace, Peer] +): + """`scopes` is bounded like `session_ids` on the add-sessions route. + + Nothing downstream bounds it now that scopes are outside the observer limit, + so an unbounded list would mint a scope peer per element in one request. + """ + test_workspace, _ = sample_data + + response = client.post( + f"/v3/workspaces/{test_workspace.name}/sessions", + json={ + "id": str(generate_nanoid()), + "scopes": [str(generate_nanoid()) for _ in range(101)], + }, + ) + assert response.status_code == 422, response.text diff --git a/tests/routes/test_sessions.py b/tests/routes/test_sessions.py index 181336e1..27a4151a 100644 --- a/tests/routes/test_sessions.py +++ b/tests/routes/test_sessions.py @@ -7,7 +7,9 @@ from nanoid import generate as generate_nanoid from sqlalchemy.ext.asyncio import AsyncSession from src import models +from src.config import settings from src.models import Peer, Workspace +from src.security import JWTParams, create_jwt def test_get_or_create_session(client: TestClient, sample_data: tuple[Workspace, Peer]): @@ -1284,6 +1286,59 @@ def test_get_session_context_with_peer_perspective( assert "peer_card" in data +@pytest.mark.asyncio +async def test_get_session_context_peer_key_denied_for_co_member_perspective( + client: TestClient, + db_session: AsyncSession, + sample_data: tuple[Workspace, Peer], + monkeypatch: pytest.MonkeyPatch, +): + """`allow_member_read` gets a peer-scoped key onto this route, but it may only + read from its OWN perspective. A co-member's representation and peer card are + not session data, so membership must not hand them over.""" + test_workspace, alice = sample_data + bob = str(generate_nanoid()) + client.post( + f"/v3/workspaces/{test_workspace.name}/peers", + json={"name": bob, "metadata": {}}, + ) + session_id = str(generate_nanoid()) + client.post( + f"/v3/workspaces/{test_workspace.name}/sessions", + json={"id": session_id, "peer_names": {alice.name: {}, bob: {}}}, + ) + # Membership is read on a separate committed-only connection by the auth + # dependency, so it must be committed before a member-scoped read. + await db_session.commit() + + monkeypatch.setattr(settings.AUTH, "USE_AUTH", True) + monkeypatch.setattr(settings.AUTH, "JWT_SECRET", "test-secret") + client.headers["Authorization"] = ( + f"Bearer {create_jwt(JWTParams(w=test_workspace.name, p=alice.name))}" + ) + url = f"/v3/workspaces/{test_workspace.name}/sessions/{session_id}/context" + + # Bob's view of alice — alice is not the observer. + assert ( + client.get( + url, params={"peer_target": alice.name, "peer_perspective": bob} + ).status_code + == 401 + ) + # The omniscient view of bob — nobody's own perspective. + assert client.get(url, params={"peer_target": bob}).status_code == 401 + # Alice's own perspective on bob is hers to read, as is her own global view. + assert ( + client.get( + url, params={"peer_target": bob, "peer_perspective": alice.name} + ).status_code + == 200 + ) + assert client.get(url, params={"peer_target": alice.name}).status_code == 200 + # Session data itself is still readable by any member. + assert client.get(url).status_code == 200 + + def test_get_session_context_peer_perspective_without_target_fails( client: TestClient, sample_data: tuple[Workspace, Peer] ): diff --git a/tests/test_cache_redaction.py b/tests/test_cache_redaction.py index 3a8e6891..805b31c7 100644 --- a/tests/test_cache_redaction.py +++ b/tests/test_cache_redaction.py @@ -2,7 +2,9 @@ import pytest -from src.cache.client import _redact_cache_url # pyright: ignore[reportPrivateUsage] +from src.cache.client import ( + _redact_cache_url, # pyright: ignore[reportPrivateUsage] +) class TestRedactCacheUrl: diff --git a/tests/test_config.py b/tests/test_config.py index 7730c751..1a714e55 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -81,3 +81,50 @@ def test_representation_batch_target_input_cannot_exceed_max_input_tokens() -> N MAX_INPUT_TOKENS=1000, REPRESENTATION_BATCH_TARGET_INPUT_TOKENS=2048, ) + + +def _configured_with_timeout(timeout: object) -> ConfiguredModelSettings: + return ConfiguredModelSettings.model_validate( + { + "model": "gpt-5.4-mini", + "transport": "openai", + "overrides": {"provider_params": {"timeout": timeout}}, + } + ) + + +@pytest.mark.parametrize("timeout", [30, 42.5, "42.5", " 60 "]) +def test_provider_timeout_is_normalized_at_config_load(timeout: object) -> None: + settings = _configured_with_timeout(timeout) + + normalized = settings.overrides.provider_params["timeout"] + assert isinstance(normalized, float) + assert normalized == float(str(timeout).strip()) + + +@pytest.mark.parametrize( + "timeout", + ["slow", "", 0, -1, True, float("nan"), float("inf"), "nan", "inf", None, [30]], +) +def test_provider_timeout_is_rejected_at_config_load(timeout: object) -> None: + with pytest.raises( + ValueError, match=r"provider_params\.timeout must be a positive number" + ): + _configured_with_timeout(timeout) + + +def test_provider_timeout_on_fallback_overrides_is_validated_at_config_load() -> None: + with pytest.raises( + ValueError, match=r"provider_params\.timeout must be a positive number" + ): + ConfiguredModelSettings.model_validate( + { + "model": "gpt-5.4-mini", + "transport": "openai", + "fallback": { + "model": "gpt-4.1", + "transport": "openai", + "overrides": {"provider_params": {"timeout": "slow"}}, + }, + } + ) diff --git a/tests/unified/README.md b/tests/unified/README.md index dd40e038..0ef1d987 100644 --- a/tests/unified/README.md +++ b/tests/unified/README.md @@ -39,6 +39,9 @@ Tests are defined in JSON files. A test definition consists of a name, optional * `create_session`: Create a new session, optionally with peers and config. * `add_message`: Add a single message. * `add_messages`: Add multiple messages. + * `create_scope`: Create a scope and optionally add member sessions. Add the + sessions *before* the messages you want in scope — membership only affects + messages ingested after a session joins. 3. **Waiting**: * `wait`: Wait for duration or "queue_empty". @@ -46,6 +49,18 @@ Tests are defined in JSON files. A test definition consists of a name, optional 4. **Querying & Assertions**: * `query`: Perform an action and assert on the result. * `target`: "chat", "get_context", "get_peer_card", "get_representation" + * `scope`: confine the read to a scope (or, for chat/representation, to + the union of several). Valid for "chat", "get_representation" and + "get_context"; the latter takes a single scope and requires + `observed_peer_id`. + +### Raw HTTP vs the SDK + +Most steps drive the Honcho Python SDK. `create_scope` and any query carrying +`scope` go over raw HTTP instead, because the published SDK trails the API and +exposes neither. Calling the API directly also tests the contract the SDK is +generated from, so a wrong status code or response shape surfaces here rather +than being masked by client-side validation. ### Assertions diff --git a/tests/unified/runner.py b/tests/unified/runner.py index b99cd37a..6d78b06b 100644 --- a/tests/unified/runner.py +++ b/tests/unified/runner.py @@ -37,6 +37,7 @@ from tests.unified.schema import ( AddMessageAction, AddMessagesAction, ContainsAssertion, + CreateScopeAction, CreateSessionAction, ExactMatchAssertion, JsonMatchAssertion, @@ -215,6 +216,38 @@ class UnifiedTestExecutor: self.client: Honcho = honcho_client self.anthropic: AsyncAnthropic | None = anthropic_client + # --- raw HTTP ----------------------------------------------------------- + # Some surfaces (scopes, the `scope` read option) exist in the API before the + # published SDK exposes them. Calling them directly also tests the contract + # the SDK is generated from, so a wrong status or shape surfaces here instead + # of being masked by client-side validation. + + @property + def workspace_id(self) -> str: + workspace_id = getattr(self.client, "workspace_id", None) + if not workspace_id: + raise ValueError("Honcho client has no workspace_id") + return str(workspace_id) + + async def _request(self, method: str, path: str, **kwargs: Any) -> httpx.Response: + """Call a /v3 workspace-scoped path directly, raising on error status.""" + url = f"{str(self.client.base_url).rstrip('/')}/v3/workspaces/{self.workspace_id}{path}" + # Carry the same credential the SDK resolved (from `HONCHO_API_KEY`, unless + # passed explicitly). The harness sets no AUTH vars of its own, so auth is + # off by default — but it inherits `AUTH_USE_AUTH` from the environment, + # and these raw calls are the only ones here that would not be authorized. + headers: dict[str, str] = dict(kwargs.pop("headers", None) or {}) + api_key = getattr(getattr(self.client, "_http", None), "api_key", None) + if api_key: + headers.setdefault("Authorization", f"Bearer {api_key}") + async with httpx.AsyncClient(timeout=120.0) as raw: + response = await raw.request(method, url, headers=headers, **kwargs) + if response.is_error: + raise AssertionError( + f"{method} {path} failed: {response.status_code} {response.text[:400]}" + ) + return response + async def execute(self, test_def: TestDefinition, test_name: str) -> bool: logger.info(f"Starting test: {test_name}") @@ -311,6 +344,15 @@ class UnifiedTestExecutor: ) await session.aio.add_messages(msgs) + elif isinstance(step, CreateScopeAction): + await self._request("POST", "/scopes", json={"id": step.scope_id}) + if step.session_ids: + await self._request( + "POST", + f"/scopes/{step.scope_id}/sessions", + json={"session_ids": step.session_ids}, + ) + elif isinstance(step, WaitAction): if step.duration: await asyncio.sleep(step.duration) @@ -344,6 +386,9 @@ class UnifiedTestExecutor: raise TimeoutError("Deriver queue did not empty within timeout") async def perform_query(self, step: QueryAction) -> Any: + if step.scope is not None: + return await self._perform_scoped_query(step) + if step.target == "chat": if not step.observer_peer_id: raise ValueError("observer_peer_id required for chat") @@ -395,6 +440,60 @@ class UnifiedTestExecutor: return None + async def _perform_scoped_query(self, step: QueryAction) -> Any: + """Run a `scope`-confined read over raw HTTP (no SDK parameter for it).""" + if step.target == "chat": + if not step.observer_peer_id: + raise ValueError("observer_peer_id required for chat") + if step.input is None: + raise ValueError("input required for chat") + body: dict[str, Any] = {"query": step.input, "scope": step.scope} + if step.session_id: + body["session_id"] = step.session_id + if step.observed_peer_id: + body["target"] = step.observed_peer_id + if step.reasoning_level: + body["reasoning_level"] = step.reasoning_level + response = await self._request( + "POST", f"/peers/{step.observer_peer_id}/chat", json=body + ) + return response.json()["content"] + + if step.target == "get_representation": + if not step.observer_peer_id: + raise ValueError("observer_peer_id required for get_representation") + body = {"scope": step.scope} + if step.observed_peer_id: + body["target"] = step.observed_peer_id + if step.input: + body["search_query"] = step.input + response = await self._request( + "POST", f"/peers/{step.observer_peer_id}/representation", json=body + ) + return response.json()["representation"] + + if step.target == "get_context": + if not step.session_id: + raise ValueError("session_id required for get_context") + if not step.observed_peer_id: + raise ValueError("observed_peer_id required for a scoped get_context") + # `scope` on session context takes a single scope name. + if isinstance(step.scope, list): + raise ValueError("get_context accepts a single scope, not a list") + params: dict[str, Any] = { + "scope": step.scope, + "peer_target": step.observed_peer_id, + "summary": str(step.summary).lower(), + } + if step.max_tokens is not None: + params["tokens"] = step.max_tokens + response = await self._request( + "GET", f"/sessions/{step.session_id}/context", params=params + ) + return response.json() + + raise ValueError(f"`scope` is not supported for target {step.target!r}") + async def check_assertion(self, result: Any, assertion: Any): result_str = str(result) diff --git a/tests/unified/schema.py b/tests/unified/schema.py index 161d4f08..b0fa84b4 100644 --- a/tests/unified/schema.py +++ b/tests/unified/schema.py @@ -63,6 +63,22 @@ class AddMessagesAction(TestStep): messages: list[MessageItem] +class CreateScopeAction(TestStep): + """Create a scope and optionally add member sessions. + + Driven over raw HTTP rather than the SDK: scopes are a new API surface the + published SDK does not expose yet, and gating coverage on an SDK release + would leave the feature untested at exactly the point it needs testing. + """ + + step_type: Literal["create_scope"] = "create_scope" + scope_id: str = Field(..., description="Unprefixed scope name") + session_ids: list[str] = Field( + default_factory=list, + description="Existing sessions to add as members of the scope", + ) + + # --- Wait Actions --- @@ -152,6 +168,11 @@ class QueryAction(TestStep): # for chat - optional JSON Schema the response must conform to response_format: dict[str, Any] | None = None + # Confine the read to one scope (observer swap) or to the union of several + # scopes' member sessions. Forces the raw-HTTP path, since the SDK has no + # `scope` parameter. Valid for chat, get_representation and get_context. + scope: str | list[str] | None = None + assertions: list[ LLMJudgeAssertion | ContainsAssertion @@ -174,6 +195,7 @@ class TestDefinition(BaseModel): | CreateSessionAction | AddMessageAction | AddMessagesAction + | CreateScopeAction | WaitAction | ScheduleDreamAction | QueryAction, diff --git a/tests/unified/test_cases/scope_confines_recall.json b/tests/unified/test_cases/scope_confines_recall.json new file mode 100644 index 00000000..77589ca8 --- /dev/null +++ b/tests/unified/test_cases/scope_confines_recall.json @@ -0,0 +1,120 @@ +{ + "description": "A scope confines recall to its member sessions. Alice states one fact in a session that belongs to the 'work' scope and a different, contradictory-sounding fact in a session outside it. A scoped read must surface only the in-scope fact; the unscoped read sees both. This is the observer swap: the scope peer is the observer, so conclusion recall comes from the (scope, alice) collection and message recall from the scope's membership.", + "steps": [ + { + "step_type": "create_session", + "session_id": "work_session", + "description": "In-scope session", + "peer_configs": { + "alice": { "observe_me": true }, + "assistant": { "observe_others": true } + } + }, + { + "step_type": "create_session", + "session_id": "personal_session", + "description": "Out-of-scope session — must never leak into a scoped read", + "peer_configs": { + "alice": { "observe_me": true }, + "assistant": { "observe_others": true } + } + }, + { + "step_type": "create_scope", + "scope_id": "work", + "session_ids": ["work_session"], + "description": "Scope covers only work_session. Membership must precede the messages: it only affects messages ingested after the session joins." + }, + { + "step_type": "add_messages", + "session_id": "work_session", + "messages": [ + { + "peer_id": "alice", + "content": "I'm a staff platform engineer and I work primarily in Rust." + }, + { + "peer_id": "alice", + "content": "My current project is migrating our billing service off Postgres triggers." + } + ] + }, + { + "step_type": "add_messages", + "session_id": "personal_session", + "messages": [ + { + "peer_id": "alice", + "content": "Outside work I'm training for a marathon in Chicago this October." + }, + { + "peer_id": "alice", + "content": "I've been learning to play the upright bass on weekends." + } + ] + }, + { + "step_type": "wait", + "target": "queue_empty", + "flush": true + }, + { + "step_type": "query", + "target": "get_representation", + "observer_peer_id": "assistant", + "observed_peer_id": "alice", + "scope": "work", + "description": "Scoped representation: only what the scope observed.", + "assertions": [ + { + "assertion_type": "llm_judge", + "prompt": "Does this text describe Alice's professional life (engineering, Rust, or the billing/Postgres project) WITHOUT mentioning marathon running, Chicago, or the upright bass? Answer true only if the professional material is present and the personal material is entirely absent.", + "pass_if": true + }, + { + "assertion_type": "not_contains", + "text": "marathon" + }, + { + "assertion_type": "not_contains", + "text": "bass" + } + ] + }, + { + "step_type": "query", + "target": "chat", + "observer_peer_id": "assistant", + "observed_peer_id": "alice", + "scope": "work", + "input": "What do you know about Alice's hobbies outside of work?", + "description": "A scoped chat cannot answer from out-of-scope sessions, so it should report not knowing rather than surfacing the marathon or the bass.", + "assertions": [ + { + "assertion_type": "llm_judge", + "prompt": "Does this response indicate that it does not know about Alice's hobbies outside work, or only discuss her professional life? Answer false if it mentions marathon running, Chicago, or playing the bass.", + "pass_if": true + }, + { + "assertion_type": "not_contains", + "text": "marathon" + } + ] + }, + { + "step_type": "query", + "target": "chat", + "observer_peer_id": "assistant", + "observed_peer_id": "alice", + "input": "What do you know about Alice's hobbies outside of work?", + "description": "Control: the same question unscoped. Proves the scoped result above is the scope working, not the deriver simply having failed to record the personal session.", + "assertions": [ + { + "assertion_type": "llm_judge", + "prompt": "Does this response mention marathon running, Chicago, or playing the upright bass? Answer true if at least one of Alice's out-of-work hobbies is described.", + "pass_if": true + } + ] + } + ] +} diff --git a/uv.lock b/uv.lock index 0c2a9889..37e3985b 100644 --- a/uv.lock +++ b/uv.lock @@ -9,7 +9,7 @@ resolution-markers = [ ] [options] -exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values. +exclude-newer = "2026-08-07T23:49:08.393963Z" exclude-newer-span = "P5D" [manifest] @@ -731,19 +731,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/ab/84/02fc1827e8cdded4aa65baef11296a9bbe595c474f0d6d758af082d849fd/execnet-2.1.2-py3-none-any.whl", hash = "sha256:67fba928dd5a544b783f6056f449e5e3931a5c378b128bc18501f7ea79e296ec", size = 40708, upload-time = "2025-11-12T09:56:36.333Z" }, ] -[[package]] -name = "fakeredis" -version = "2.35.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "redis" }, - { name = "sortedcontainers" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/43/50/b748233c02fa77e5105238190cc9bb58b852eb1c8b1d0763230d3a5b745a/fakeredis-2.35.1.tar.gz", hash = "sha256:5bae5eba7b9d93cb968944ac40936373cf2397ff71667d4b595df65c3d2e413f", size = 189118, upload-time = "2026-04-12T17:05:58.539Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/6f/27/b8b057a23f7777177e92d3a602fd866751b6b45014964548997e92e048fd/fakeredis-2.35.1-py3-none-any.whl", hash = "sha256:67d97e11f562b7870e11e5c30cf182270bfb2dd37f6707dba47cc6d91628d1b9", size = 129678, upload-time = "2026-04-12T17:05:56.86Z" }, -] - [[package]] name = "fastapi" version = "0.136.1" @@ -761,10 +748,9 @@ wheels = [ ] [package.optional-dependencies] -standard = [ +standard-no-fastapi-cloud-cli = [ { name = "email-validator" }, - { name = "fastapi-cli", extra = ["standard"] }, - { name = "fastar" }, + { name = "fastapi-cli", extra = ["standard-no-fastapi-cloud-cli"] }, { name = "httpx" }, { name = "jinja2" }, { name = "pydantic-extra-types" }, @@ -788,30 +774,10 @@ wheels = [ ] [package.optional-dependencies] -standard = [ - { name = "fastapi-cloud-cli" }, +standard-no-fastapi-cloud-cli = [ { name = "uvicorn", extra = ["standard"] }, ] -[[package]] -name = "fastapi-cloud-cli" -version = "0.17.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "fastar" }, - { name = "httpx" }, - { name = "pydantic", extra = ["email"] }, - { name = "rich-toolkit" }, - { name = "rignore" }, - { name = "sentry-sdk" }, - { name = "typer" }, - { name = "uvicorn", extra = ["standard"] }, -] -sdist = { url = "https://files.pythonhosted.org/packages/96/57/cee8e91b83f39e75ae5562a2237261442a8179dcb3b631c7398113157398/fastapi_cloud_cli-0.17.1.tar.gz", hash = "sha256:0baece208fa88063bec46dccb5fb512f3199162092165e57654b44e64adbc44d", size = 47409, upload-time = "2026-04-27T13:38:07.094Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/e7/a0/e252b68cf155409afabea037ab2971f41509481838847f6503fe890884ea/fastapi_cloud_cli-0.17.1-py3-none-any.whl", hash = "sha256:325e0199bdac7cb86f5df4f4a1d2070054095588088ef7b923a60cec458dcd63", size = 34046, upload-time = "2026-04-27T13:38:08.319Z" }, -] - [[package]] name = "fastapi-pagination" version = "0.15.12" @@ -826,107 +792,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/d2/2f/644fd77ecac100da965221751ae4f7604e149c58c46c1d96c37e828bb5f7/fastapi_pagination-0.15.12-py3-none-any.whl", hash = "sha256:758e21157b2844feecb2409072f1433e24f2dc9526ae7906aa1a1b28622a970a", size = 60921, upload-time = "2026-03-28T12:51:04.288Z" }, ] -[[package]] -name = "fastar" -version = "0.11.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/03/0f/0aeb3fc50046617702acc0078b277b58367fd62eb727b9ec733ae0e8bbcc/fastar-0.11.0.tar.gz", hash = "sha256:aa7f100f7313c03fdb20f1385927ba95671071ba308ad0c1763fef295e1895ce", size = 70238, upload-time = "2026-04-13T17:11:17.143Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/11/7a/fb367bdaf4efa2c7952a45aeab2e87a564293ecffe150af673ec8edfda46/fastar-0.11.0-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:b82fd6f996e65a86f67a6bd64dd22ef3e8ae2dcaed0ae3b550e71f7e1bbb1df5", size = 709869, upload-time = "2026-04-13T17:09:55.62Z" }, - { url = "https://files.pythonhosted.org/packages/80/ff/b87efb0dcfd081c62c7c7601d7681dabe63103cd51fc16f8d57a1ab45961/fastar-0.11.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:27eed386fd0558e6daa29211111bbd7b740f7c7e881197f8a00ac7c0f3cdb1d7", size = 631668, upload-time = "2026-04-13T17:09:40.537Z" }, - { url = "https://files.pythonhosted.org/packages/24/7c/0ed6dd38b9adc04b3a8ec3b7045908e7c2170ba0ff6e6d2c51bc9fc770f3/fastar-0.11.0-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:a6931bebc1d8e95ddeef55732c195449e6b44ef33aa31b325505097ed3b4d6aa", size = 869663, upload-time = "2026-04-13T17:09:09.78Z" }, - { url = "https://files.pythonhosted.org/packages/58/ce/8b7fb3f23855accebaaf2d2637eac7f261a7a5d936f861a172079f1ef511/fastar-0.11.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:891f72ce42a5e28a74fbd4d5fbf1a3ac1a1163d13cbc200cbd005fb0fabc54bd", size = 762938, upload-time = "2026-04-13T17:07:54.51Z" }, - { url = "https://files.pythonhosted.org/packages/07/cc/5491e2b677bb841f768e3aba052d0344338a5c78aa5d4c18b443831a8e8d/fastar-0.11.0-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:5b83c1f61f7017d6e1498568038f8745440cfc16ca2f697ec81bac83050108f6", size = 759232, upload-time = "2026-04-13T17:08:08.864Z" }, - { url = "https://files.pythonhosted.org/packages/4e/b7/643630bdbd179e41e9fae31c03b4cf6061dbf4d6fbbae8425d16eb12545d/fastar-0.11.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:db73a9b765a516e73983b25341e7b5e0189733878279e278b2295131b0e3a21e", size = 926271, upload-time = "2026-04-13T17:08:23.68Z" }, - { url = "https://files.pythonhosted.org/packages/09/5d/37ade50003b4540e0a53ef100f6692d7ab2ac1122d5acf39920cc09a3e8b/fastar-0.11.0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:625827d52eb4e8fec942e0233f125ff8010fcf6a67c0a974a8e5f4666b771e3c", size = 818634, upload-time = "2026-04-13T17:08:54.268Z" }, - { url = "https://files.pythonhosted.org/packages/c3/ff/135d177de32cc1e837c99019e4643e6e79352bde49544d4ece5b5eebf56b/fastar-0.11.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d7f5fd8fa21ec0a88296a38dc5d7fc35efd3b26d46a17b8b7c73c5563925ca15", size = 822755, upload-time = "2026-04-13T17:09:25.01Z" }, - { url = "https://files.pythonhosted.org/packages/27/cb/b835dbe76ceac7fa6105851468c259ffd06830eb9c029402e499d0ec153b/fastar-0.11.0-cp311-cp311-manylinux_2_31_riscv64.whl", hash = "sha256:8c15af91b8cd87ddf23ea55355ae513c1de3ab67178f26dad017c9e9c0af6096", size = 887101, upload-time = "2026-04-13T17:08:39.248Z" }, - { url = "https://files.pythonhosted.org/packages/9e/54/aa8289eb57fc550535470397cb051f5a58a7c89ca4de31d5502b916dd894/fastar-0.11.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:03a112395a8b0bff251423bd1564c012f0cc058ad8b6bd8fba96f3d7fc117e44", size = 973606, upload-time = "2026-04-13T17:10:10.98Z" }, - { url = "https://files.pythonhosted.org/packages/1f/fd/776d50a0897c01dc6bfd0926772ee913436fdae91b9affaf0a0cbd09f0a1/fastar-0.11.0-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:f2994bb8f5f8c11eb12beae1e6e77a907173c9819236b8a4c8f0573652ceccce", size = 1036696, upload-time = "2026-04-13T17:10:28.502Z" }, - { url = "https://files.pythonhosted.org/packages/c8/f1/cf0f9b499fb37ac065c8a01ec642f96a3c5eb849c38ae983b59f3b3245e0/fastar-0.11.0-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:dcf99e4b5973d842c7f19c776c3a83cdc0977d505edce6206438505c0456b517", size = 1078182, upload-time = "2026-04-13T17:10:45.318Z" }, - { url = "https://files.pythonhosted.org/packages/f8/9e/21e4701aec4a1123d4dc4d31578dc18875582b5710e4725f7ceb752a248b/fastar-0.11.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:29c9c386dc0d5dda78845a8e6b1480d26ab861c1e0b68f42ae5735cb70ca07f1", size = 1032336, upload-time = "2026-04-13T17:11:02.364Z" }, - { url = "https://files.pythonhosted.org/packages/ce/e2/5872b28c72c27ec1a00760eace6ff35f714f41ebbd5208cf016b12e29250/fastar-0.11.0-cp311-cp311-win32.whl", hash = "sha256:030b2580fc394f2c9b7890b6735810404e9b9ed5e0344db150b945965b5482b7", size = 457368, upload-time = "2026-04-13T17:11:43.528Z" }, - { url = "https://files.pythonhosted.org/packages/fd/6e/ce6832a16193eb4466f4108be8809c249b51cb1f89dd7894545700d079d5/fastar-0.11.0-cp311-cp311-win_amd64.whl", hash = "sha256:83ab57ae067969cd0b483ac3b6dccc4b595fc77f5c820760998648d4c42822b5", size = 488605, upload-time = "2026-04-13T17:11:29.161Z" }, - { url = "https://files.pythonhosted.org/packages/15/5a/9cfb80661cf38fd7b0889224beb7d2746784d4ade2a931ed9775a18d8602/fastar-0.11.0-cp311-cp311-win_arm64.whl", hash = "sha256:27b1a4cee2298b704de8151d310462ee7335ed036011ca9aa6e784b30b6c73a9", size = 464580, upload-time = "2026-04-13T17:11:18.583Z" }, - { url = "https://files.pythonhosted.org/packages/0f/06/a5773706afc8bd496769786590bbc56d2d0ee419a299cc12ea3f5717fcf3/fastar-0.11.0-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:3c51f1c2cdddbd1420d2897ace7738e36c65e17f6ae84e0bfe763f8d1068bb97", size = 708394, upload-time = "2026-04-13T17:09:57.269Z" }, - { url = "https://files.pythonhosted.org/packages/cc/a6/d5e2a4e48495616440a21eed07558219ca90243ad00b0502586f95bd4833/fastar-0.11.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:0d9d6b052baf5380baea866675dab6ccd04ec2460d12b1c46f10ce3f4ee6a820", size = 628417, upload-time = "2026-04-13T17:09:42.145Z" }, - { url = "https://files.pythonhosted.org/packages/ab/69/9816d69ac8265c9e50456637a487ccfb7a9c566efd9dbcd673df9c2558c2/fastar-0.11.0-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:bd2f05666d4df7e14885b5c38fefd92a785917387513d33d837ff42ec143a22f", size = 863950, upload-time = "2026-04-13T17:09:11.506Z" }, - { url = "https://files.pythonhosted.org/packages/5b/0d/f88daad53aff2e754b6b5ff2a7113f72447a34f6ef17cc23ca99988117b7/fastar-0.11.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c1e6e74aba1ae77ca4aedcaf1697cd413319f4c88a5ccbe5b42c709517c5097e", size = 760737, upload-time = "2026-04-13T17:07:55.958Z" }, - { url = "https://files.pythonhosted.org/packages/2f/a6/82ef4ecd969d50d92ed3ed9dbd8fe77faa24be5e5736f716edc9f4ce8d62/fastar-0.11.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:38ef77fe940bbc9b37a98bd838727f844b11731cd39358a2640ff864fb385086", size = 757603, upload-time = "2026-04-13T17:08:10.623Z" }, - { url = "https://files.pythonhosted.org/packages/03/35/50249f0d827251f8ac511495e2eacccebda80a00a0ad73e9615b8113b84f/fastar-0.11.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:8955e61b32d6aff82c983217abf80933fd823b0e727586fc72f08043d996fd59", size = 923952, upload-time = "2026-04-13T17:08:25.526Z" }, - { url = "https://files.pythonhosted.org/packages/7b/d8/faee41659e9c379d906d24eaee6d6833ac8cfef0a5df480e5c2a8d3efb33/fastar-0.11.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:483532442cdb08fbff0169510224eae0836f2f672cea6aacb52847d90fefdc46", size = 816574, upload-time = "2026-04-13T17:08:56.076Z" }, - { url = "https://files.pythonhosted.org/packages/22/47/0448ea7992b997dad2bf004bfd98eca74b5858630eae080b50c7b17d9ddc/fastar-0.11.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ef5a6071121e05d8287fc75bccb054bcbac8bb0501200a0c0a8feeace5303ea4", size = 819382, upload-time = "2026-04-13T17:09:26.66Z" }, - { url = "https://files.pythonhosted.org/packages/33/ef/0d63eb43586831b7a6f8b22c4d77125a7c594423af1f4f090fa9541b9b40/fastar-0.11.0-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:e45e598af5afe8412197d4786efd6cf29be02e7d3d4f6a3461149eae5d7e94f1", size = 885254, upload-time = "2026-04-13T17:08:40.9Z" }, - { url = "https://files.pythonhosted.org/packages/01/25/edd584675d69e49a165052c3ee886df1c5d574f3e7d813c990306387c623/fastar-0.11.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:2e160919b1c47ddb8538e7e8eb4cd527281b40f0bf75110a75993838ef61f286", size = 971239, upload-time = "2026-04-13T17:10:12.997Z" }, - { url = "https://files.pythonhosted.org/packages/a5/37/e8bb24f506ba2b08fbaf36c5800e843bd4d542954e9331f00418e2d23349/fastar-0.11.0-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:4bb4dc0fc8f7a6807febcebce8a2f3626ba4955a9263d81ecc630aad83be84c0", size = 1035185, upload-time = "2026-04-13T17:10:30.207Z" }, - { url = "https://files.pythonhosted.org/packages/9a/bf/be753736296338149ee4cb3e92e2b5423d6ba17c7b951d15218fd7e99bbf/fastar-0.11.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:4ec95af56aa173f6e320e1183001bf108ba59beaf13edd1fc8200648db203588", size = 1072191, upload-time = "2026-04-13T17:10:47.072Z" }, - { url = "https://files.pythonhosted.org/packages/d2/cd/a81c1aaafb5a22ce57c98ae22f39c89413ed53e4ee6e1b1444b0bd666a6c/fastar-0.11.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:136cf342735464091c39dc3708168f9fdeb9ebea40b1ead937c61afaf46143d9", size = 1028054, upload-time = "2026-04-13T17:11:04.293Z" }, - { url = "https://files.pythonhosted.org/packages/ec/88/1ce4eed3d70627c95f49ca017f6bbbf2ddcc4b0c601d293259de7689bc20/fastar-0.11.0-cp312-cp312-win32.whl", hash = "sha256:35f23c11b556cc4d3704587faacbc0037f7bdf6c4525cd1d09c70bda4b1c6809", size = 454198, upload-time = "2026-04-13T17:11:45.168Z" }, - { url = "https://files.pythonhosted.org/packages/8f/1d/26ce92f4331cd61a69840db9ca6115829805eec24f285481a854f578e917/fastar-0.11.0-cp312-cp312-win_amd64.whl", hash = "sha256:920bc56c3c0b8a8ca492904941d1883c1c947c858cd93343356c29122a38f44c", size = 486697, upload-time = "2026-04-13T17:11:31.084Z" }, - { url = "https://files.pythonhosted.org/packages/ed/96/e6eda4480559c69b05d466e7b5ea9170e81fef3795a73e059959a3258319/fastar-0.11.0-cp312-cp312-win_arm64.whl", hash = "sha256:395248faf89e8a6bd5dc1fd544c8465113b627cb6d7c8b296796b60ebea33593", size = 462591, upload-time = "2026-04-13T17:11:20.577Z" }, - { url = "https://files.pythonhosted.org/packages/c9/d6/3be260037e86fb694e88d47f583bac3a0188c99cee1a6b257ac26cb6b53c/fastar-0.11.0-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:33f544b08b4541b678e53749b4552a44720d96761fb79c172b005b1089c443ed", size = 707975, upload-time = "2026-04-13T17:09:58.866Z" }, - { url = "https://files.pythonhosted.org/packages/e1/cd/7867aefb1784662554a335f2952c75a50f0c70585ed0d2210d6cc15e5627/fastar-0.11.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:91c1c792447e4a642745f347ff9847c52af39633071c57ee67ed53c157fc3506", size = 628460, upload-time = "2026-04-13T17:09:43.776Z" }, - { url = "https://files.pythonhosted.org/packages/e5/2b/d11d84bdd5e0e377771b955755771e3460b290da5809cb78c1b735ee2228/fastar-0.11.0-cp313-cp313-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:881247e6b6eaea59fc6569f9b61447aa6b9fc2ee864e048b4643d69c52745805", size = 863054, upload-time = "2026-04-13T17:09:13.048Z" }, - { url = "https://files.pythonhosted.org/packages/25/39/d3f428b318fa940b1b6e785b8d54fc895dfb5d5b945ef8d5442ffa904fb2/fastar-0.11.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:863b7929845c9fec92ef6c8d59579cf46af5136655e5342f8df5cebe46cab06c", size = 760247, upload-time = "2026-04-13T17:07:57.396Z" }, - { url = "https://files.pythonhosted.org/packages/9e/04/03949aee82aabb8ede06ac5a4a5579ffaf98a8fe59ce958494508ff15513/fastar-0.11.0-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:96b4a57df12bf3211662627a3ea29d62ecb314a2434a0d0843f9fc23e47536e5", size = 756512, upload-time = "2026-04-13T17:08:12.415Z" }, - { url = "https://files.pythonhosted.org/packages/3f/0c/2ca1ae0a3828ca51047962d932b80daca2522db73e8cb9d040cb6ebe28d5/fastar-0.11.0-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:ceef1c2c4df7b7b8ebd3f5d718bbf457b9bbdf25ce0bd07870211ec4fbd9aff4", size = 922183, upload-time = "2026-04-13T17:08:27.187Z" }, - { url = "https://files.pythonhosted.org/packages/65/68/7fe808b1f73a68e686f25434f538c6dc10ef4dfb3db0ace22cd861744bf8/fastar-0.11.0-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:b8e545918441910a779659d4759ad0eef349e935fbdb4668a666d3681567eb05", size = 816394, upload-time = "2026-04-13T17:08:57.657Z" }, - { url = "https://files.pythonhosted.org/packages/1f/17/07d086080f8a83b8d7966955e29bcdbd6a060f5bd949dc9d5abd3658cead/fastar-0.11.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:28095bb8f821e85fc2764e1a55f03e5e2876dee2abe7cd0ee9420d929905d643", size = 818983, upload-time = "2026-04-13T17:09:28.46Z" }, - { url = "https://files.pythonhosted.org/packages/fb/e2/2c4edf0910af2e814ff6d65b77a91196d472ca8a9fb2033bd983f6856caa/fastar-0.11.0-cp313-cp313-manylinux_2_31_riscv64.whl", hash = "sha256:0fafb95ecbe70f666a5e9b35dd63974ccdc9bb3d99ccdbd4014a823ec3e659b5", size = 884689, upload-time = "2026-04-13T17:08:42.763Z" }, - { url = "https://files.pythonhosted.org/packages/fa/ba/04fdcbd6558e60de4ced3b55230fac47675d181252582b2fcec3c74608e5/fastar-0.11.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:af48fed039b94016629dcdad1c95c90c486326dd068de2b0a4df419ee09b6821", size = 970677, upload-time = "2026-04-13T17:10:15.124Z" }, - { url = "https://files.pythonhosted.org/packages/df/b3/2b860a9658550167dbd5824c85e88d0b4b912bf493e42a6322544d6e483d/fastar-0.11.0-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:74cd96163f39b8638ab4e8d49708ca887959672a22871d8170d01f067319533b", size = 1034026, upload-time = "2026-04-13T17:10:32.318Z" }, - { url = "https://files.pythonhosted.org/packages/b7/9b/fa42ea1188b144bac4b1b60753dfd449974a4d5eda132029ee7711569f94/fastar-0.11.0-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:4e8b993cb5613bab495ed482810bedc0986633fcb9a3b55c37ec88e0d6714f6a", size = 1071147, upload-time = "2026-04-13T17:10:48.833Z" }, - { url = "https://files.pythonhosted.org/packages/95/c8/d2e501556dca9f1fbc9246111a31792fb49ad908fa4927f34938a97a3604/fastar-0.11.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:dfe39d91fc28e37e06162d94afe01050220edb7df554acb5b702b5503e564816", size = 1028377, upload-time = "2026-04-13T17:11:06.374Z" }, - { url = "https://files.pythonhosted.org/packages/db/33/5f11f23eca0a569cd052507bc45dda2e5468697f8665728d25be44120f7d/fastar-0.11.0-cp313-cp313-win32.whl", hash = "sha256:c5f63d4d99ff4bfb37c659982ec413358bdee747005348756cc50a04d412d989", size = 454089, upload-time = "2026-04-13T17:11:46.821Z" }, - { url = "https://files.pythonhosted.org/packages/da/2f/35ff03c939cba7a255a9132367873fec6c355fd06a7f84fedcbaf4c8129f/fastar-0.11.0-cp313-cp313-win_amd64.whl", hash = "sha256:8690ed1928d31ded3ada308e1086525fb3871f5fa81e1b69601a3f7774004583", size = 486312, upload-time = "2026-04-13T17:11:32.86Z" }, - { url = "https://files.pythonhosted.org/packages/ef/71/ee9246cbfcbfd4144558f35e7e9a306ffe0a7564730a5188c45f21d2dab8/fastar-0.11.0-cp313-cp313-win_arm64.whl", hash = "sha256:d977ded9d98a0719a305e0a4d5ee811f1d3e856d853a50acb8ae833c3cd6d5d2", size = 461975, upload-time = "2026-04-13T17:11:22.589Z" }, - { url = "https://files.pythonhosted.org/packages/7a/cd/3644c48ecac456f928c12d47ec3bed36c36555b17c3859856f1ff860265d/fastar-0.11.0-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:71375bd6f03c2a43eb47bd949ea38ff45434917f9cdac79675c5b9f60de4fa73", size = 707860, upload-time = "2026-04-13T17:10:00.371Z" }, - { url = "https://files.pythonhosted.org/packages/69/ca/dee04476ae3626b2b040a60ad84628f77e1ffd8444232f2426b0ca1e0d7e/fastar-0.11.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:eddfd9cab16e19ae247fe44bf992cb403ccfe27d3931d6de29a4695d95ad386c", size = 628216, upload-time = "2026-04-13T17:09:45.355Z" }, - { url = "https://files.pythonhosted.org/packages/dc/5e/9395c7353d079cb4f5be0f7982ce0dc9f2e7dec5fd175eef466729d6023a/fastar-0.11.0-cp314-cp314-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:7c371f1d4386c699018bb64eb2fa785feacf32785559049d2bb72fe4af023f53", size = 864378, upload-time = "2026-04-13T17:09:14.611Z" }, - { url = "https://files.pythonhosted.org/packages/fa/ba/1e4f67148223ff219612b6281a6000357abbcc2417964fa5c83f11d68fce/fastar-0.11.0-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cad7fa41e3e66554387481c1a09365e4638becd322904932674159d5f4046728", size = 760921, upload-time = "2026-04-13T17:07:59.138Z" }, - { url = "https://files.pythonhosted.org/packages/0f/82/09d11fb6d12f17993ffaf32ffd30c3c121a11e2966e84f19fb6f66430118/fastar-0.11.0-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:cf36652fa71b83761717c9899b98732498f8a2cb6327ff16bbf07f6be85c3437", size = 757012, upload-time = "2026-04-13T17:08:14.186Z" }, - { url = "https://files.pythonhosted.org/packages/52/1f/5aeeacc4cb65615e2c9292cd9c5b0cd6fb6d2e6ee472ca6adc6c1b1b22ef/fastar-0.11.0-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:f68ff8c17833053da4841720e95edde80ce45bb994b6b7d51418dddaac70ee47", size = 924510, upload-time = "2026-04-13T17:08:28.741Z" }, - { url = "https://files.pythonhosted.org/packages/bb/1a/1e5bdabbeaf2e856928956292609f2ff6a650f94480fb8afaca30229e483/fastar-0.11.0-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:4563ed37a12ea1cdc398af8571258d24b988bf342b7b3bf5451bd5891243280c", size = 816602, upload-time = "2026-04-13T17:08:59.461Z" }, - { url = "https://files.pythonhosted.org/packages/87/24/f960147910da3bed41a3adfcb026e17d5f50f4cf467a3324237a7088f61a/fastar-0.11.0-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:cee63c9875cba3b70dc44338c560facc5d6e763047dcc4a30501f9a68cf5f890", size = 819452, upload-time = "2026-04-13T17:09:29.926Z" }, - { url = "https://files.pythonhosted.org/packages/cc/f4/3e77d7901d5707fd7f8a352e153c8ae09ea974e6fabad0b7c4eb9944b8d4/fastar-0.11.0-cp314-cp314-manylinux_2_31_riscv64.whl", hash = "sha256:bd76bfffae6d0a91f4ac4a612f721e7aec108db97dccdd120ae063cd66959f27", size = 885254, upload-time = "2026-04-13T17:08:44.285Z" }, - { url = "https://files.pythonhosted.org/packages/47/01/1585edd5ec47782ae93cd94edf05828e0ab02ef00aec00aea4194a600464/fastar-0.11.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:8f5b707501ec01c1bc0518f741f01d322e50c9adc19a451aa24f67a2316e9397", size = 971496, upload-time = "2026-04-13T17:10:17.024Z" }, - { url = "https://files.pythonhosted.org/packages/f1/e9/6874c9d1236ded565a0bed54b320ac9f165f287b1d89490fb70f9f323c81/fastar-0.11.0-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:37c0b5a88a657839aad98b0a6c9e4ac4c2c15d6b49c44ee3935c6b08e9d3e479", size = 1034685, upload-time = "2026-04-13T17:10:34.063Z" }, - { url = "https://files.pythonhosted.org/packages/14/d8/4ab20613ce2983427aee958e39be878dba874aa227c530a845e32429c4f6/fastar-0.11.0-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:6c55f536c62a6efb180c1af0d5182948bff576bbfe6276e8e1359c9c7d2215d8", size = 1072675, upload-time = "2026-04-13T17:10:50.53Z" }, - { url = "https://files.pythonhosted.org/packages/1f/ae/5ac3b7c20ce4b08f011dd2b979f96caabe64f9b10b157f211ea91bdfadca/fastar-0.11.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:3082eeca59e189b9039335862f4c2780c0c8871d656bfdf559db4414a105b251", size = 1029330, upload-time = "2026-04-13T17:11:08.138Z" }, - { url = "https://files.pythonhosted.org/packages/8a/e7/37cd6a1d4e288292170b64e19d79ecce2a7de8bb76790323399a2abc4619/fastar-0.11.0-cp314-cp314-win32.whl", hash = "sha256:b201a0a4e29f9fec2a177e13154b8725ec65ab9f83bd6415483efaa2aa18344b", size = 453940, upload-time = "2026-04-13T17:11:48.713Z" }, - { url = "https://files.pythonhosted.org/packages/ff/1c/795c878b1ee29d79021cf8ed81f18f2b25ccde58453b0d34b9bdc7e025ea/fastar-0.11.0-cp314-cp314-win_amd64.whl", hash = "sha256:868fddb26072a43e870a8819134b9f80ee602931be5a76e6fb873e04da343637", size = 486334, upload-time = "2026-04-13T17:11:34.882Z" }, - { url = "https://files.pythonhosted.org/packages/ff/a4/113f104301df8bddcc0b3775b611a30cb7610baa3add933c7ccac9386467/fastar-0.11.0-cp314-cp314-win_arm64.whl", hash = "sha256:3db39c9cc42abb0c780a26b299f24dfbc8be455985e969e15336d70d7b2f833b", size = 461534, upload-time = "2026-04-13T17:11:24.329Z" }, - { url = "https://files.pythonhosted.org/packages/5a/a6/5c5f2c2c8e0c63e56a5636ebc7721589c889e94c0092cec7eb28ae7207e6/fastar-0.11.0-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:49c3299dec5e125e7ebaa27545714da9c7391777366015427e0ae62d548b442b", size = 707156, upload-time = "2026-04-13T17:10:02.176Z" }, - { url = "https://files.pythonhosted.org/packages/df/f7/982c01b61f0fc135ad2b16d01e6d0ee53cf8791e68827f5f7c5a65b2e5b1/fastar-0.11.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:3328ed1ed56d31f5198350b17dd60449b8d6b9d47abb4688bab6aef4450a165b", size = 627032, upload-time = "2026-04-13T17:09:46.978Z" }, - { url = "https://files.pythonhosted.org/packages/2b/c3/38f1dac77ae0c71c37b176277c96d830796b8ce2fe69705f917829b53829/fastar-0.11.0-cp314-cp314t-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:bd3eca3bbfec84a614bcb4143b4ad4f784d0895babc26cfc88436af88ca23c7a", size = 864403, upload-time = "2026-04-13T17:09:16.58Z" }, - { url = "https://files.pythonhosted.org/packages/6e/f0/e69c363bdb3e5a5848e937b662b5469581ee6682c51bc1c0556494773929/fastar-0.11.0-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ff86a967acb0d621dd24063dda090daa67bf4993b9570e97fe156de88a9006ca", size = 759480, upload-time = "2026-04-13T17:08:00.599Z" }, - { url = "https://files.pythonhosted.org/packages/3b/29/4d8737590c2a6357d614d7cc7288e8f68e7e449680b8922997cc4349e65e/fastar-0.11.0-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:86eaf7c0e985d93a7734168be2fb232b2a8cca53e41431c2782d7c12b12c03b1", size = 756219, upload-time = "2026-04-13T17:08:15.699Z" }, - { url = "https://files.pythonhosted.org/packages/bb/ec/400de7b3b7d48801908f19cf5462177104395799472671b3e8152b2b04ca/fastar-0.11.0-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:91f07b0b8eb67e2f177733a1f884edad7dfb9f8977ffef15927b20cb9604027d", size = 923669, upload-time = "2026-04-13T17:08:30.574Z" }, - { url = "https://files.pythonhosted.org/packages/5d/01/8926c53da923fed7ab4b96e7fbf7f73b663beb4f02095b654d6fab46f9ad/fastar-0.11.0-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:f85c896885eb4abf1a635d54dea22cac6ae48d04fc2ea26ae652fcf1febe1220", size = 815729, upload-time = "2026-04-13T17:09:01.204Z" }, - { url = "https://files.pythonhosted.org/packages/89/f0/5fef4c7946e352651b504b1a4235dac3505e7cfd24020788ab50552e84bf/fastar-0.11.0-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:075c07095c8de4b774ba8f28b9c0a02b1a2cd254da50cbe464dd3bb2432e9158", size = 819812, upload-time = "2026-04-13T17:09:31.907Z" }, - { url = "https://files.pythonhosted.org/packages/b3/c8/0ebc3298b4a45e7bddc50b169ae6a6f5b80c939394d4befe6e60de535ee7/fastar-0.11.0-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:07f028933820c65750baf3383b807ecce1cd9385cf00ce192b79d263ad6b856c", size = 884074, upload-time = "2026-04-13T17:08:45.802Z" }, - { url = "https://files.pythonhosted.org/packages/ae/9f/7baa4cdff8d6fbca41fa5c764b48a941fed8a9ec6c4cc92de65895a28299/fastar-0.11.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:039f875efa0f01fa43c20bf4e2fc7305489c61d0ac76eda991acfba7820a0e63", size = 969450, upload-time = "2026-04-13T17:10:18.667Z" }, - { url = "https://files.pythonhosted.org/packages/d4/dc/1ebbfb58a47056ba866494f19efbcdd2ba2897096b94f36e796594b4d05b/fastar-0.11.0-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:fff12452a9a5c6814a012445f26365541cc3d99dcca61f09762e6a389f7a32ea", size = 1033775, upload-time = "2026-04-13T17:10:36.165Z" }, - { url = "https://files.pythonhosted.org/packages/c2/5f/ce4e3914066f08c99eb8c32952cc07c1a013e81b1db1b0f598130bf6b974/fastar-0.11.0-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:2bf733e09f942b6fa876efe30a90508d1f4caef5630c00fb2a84fba355873712", size = 1072158, upload-time = "2026-04-13T17:10:52.497Z" }, - { url = "https://files.pythonhosted.org/packages/03/2a/6bca72992c84151c387cc6558f3867f5ebe5fb3684ee6fa9b76280ba4b8e/fastar-0.11.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:d1531fa848fdd3677d2dce0a4b436ea64d9ae38fb8babe2ddbc180dd153cb7a3", size = 1028577, upload-time = "2026-04-13T17:11:09.934Z" }, - { url = "https://files.pythonhosted.org/packages/83/18/7a7c15657a3da5569b26fc51cde6a80f8d84cb54b3b1aea6d74a103db4ad/fastar-0.11.0-cp314-cp314t-win32.whl", hash = "sha256:5744551bc67c6fc6581cbd0e34a0fd6e2cd0bd30b43e94b1c3119cf35064b162", size = 453601, upload-time = "2026-04-13T17:11:53.726Z" }, - { url = "https://files.pythonhosted.org/packages/6d/d8/331b59a6de279f3ad75c10c02c40a12f21d64a437d9c3d6f1af2dcbd7a76/fastar-0.11.0-cp314-cp314t-win_amd64.whl", hash = "sha256:f4ce44e3b56c47cf38244b98d29f269b259740a580c47a2552efa5b96a5458fb", size = 486436, upload-time = "2026-04-13T17:11:40.089Z" }, - { url = "https://files.pythonhosted.org/packages/6b/fd/5390ec4f49100f3ecb9968a392f9e6d039f1e3fe0ecd28443716ff01e589/fastar-0.11.0-cp314-cp314t-win_arm64.whl", hash = "sha256:76c1359314355eafbc6989f20fb1ad565a3d10200117923b9da765a17e2f6f11", size = 461049, upload-time = "2026-04-13T17:11:25.918Z" }, - { url = "https://files.pythonhosted.org/packages/cc/5c/9bbeffbf1905391446dd98aa520422ce7affde5c9a7c22d757cc5d7c1397/fastar-0.11.0-pp311-pypy311_pp73-macosx_10_12_x86_64.whl", hash = "sha256:1266d6a004f427b0d61bd6c7b544d84cc964691b2232c2f4d635a1b75f2f6d5e", size = 711644, upload-time = "2026-04-13T17:10:07.663Z" }, - { url = "https://files.pythonhosted.org/packages/7e/af/ae5cf39d4fb82d0c592705f5ec6db1b065be5265c151b108f86126ee8773/fastar-0.11.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:298a827ec04ade43733f6ca960d0faec38706aa1494175869ea7ea17f5bad5d3", size = 634371, upload-time = "2026-04-13T17:09:52.083Z" }, - { url = "https://files.pythonhosted.org/packages/7e/36/8d4569e26473c72ccb02d1c5df3ed710073f1c06eca09c26d52ea79fd815/fastar-0.11.0-pp311-pypy311_pp73-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:8800e2387e463a0e5799416a1cbe72dd0fde7270a20e4bde684145e7878f6516", size = 870850, upload-time = "2026-04-13T17:09:21.439Z" }, - { url = "https://files.pythonhosted.org/packages/bf/46/724dc796e1756d3977970f820d30d59bb8cab8e3671b285f1d82ab513aec/fastar-0.11.0-pp311-pypy311_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7496def0a2befd82d429cb004ef7ca831585cc887947bd6b9abb68a5ef852b0b", size = 764469, upload-time = "2026-04-13T17:08:05.638Z" }, - { url = "https://files.pythonhosted.org/packages/99/e3/74d6859e632e8fb9339a14f652fb9f800c2bd6aa53071e311c0be3fbab8b/fastar-0.11.0-pp311-pypy311_pp73-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:878eaf15463eb572e3538af7ca3a8534e5e279cf8196db902d24e5725c4af86e", size = 761375, upload-time = "2026-04-13T17:08:20.669Z" }, - { url = "https://files.pythonhosted.org/packages/a3/e7/cc70e2be5ef8731a7525552b1c35c1448cf9eae6a62cb3a56f12c1bf27ea/fastar-0.11.0-pp311-pypy311_pp73-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:0324ed1d1ef0186e1bbd843b17807d6d837d0906899d4c99378b02c5d86bdd9c", size = 928189, upload-time = "2026-04-13T17:08:35.663Z" }, - { url = "https://files.pythonhosted.org/packages/3c/33/c9a969e78dca323547276a6fee5f4f9588f7cd5ab45acec3778c67399589/fastar-0.11.0-pp311-pypy311_pp73-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:bdf9bd863205590beaf8ef6e66f315310196632180dceaf674985d01a876cac3", size = 820864, upload-time = "2026-04-13T17:09:06.366Z" }, - { url = "https://files.pythonhosted.org/packages/84/bd/6b9434b541fe55c125b5f2e017a565596a2d215aa09207e4555e4585064f/fastar-0.11.0-pp311-pypy311_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:59af8dbb683b24b90fb5b506de080faeab0a17a908e6c2a5d93a97260ed75d7b", size = 824060, upload-time = "2026-04-13T17:09:37.377Z" }, - { url = "https://files.pythonhosted.org/packages/24/8d/871d5f8cf4c6f13987119fb0a9ae8be131e34f2756c2524e9974adf33824/fastar-0.11.0-pp311-pypy311_pp73-manylinux_2_31_riscv64.whl", hash = "sha256:9f3df73a3c4292cfe15696cdf59cdb6c309ab59d30b34c733be13c6e32d9a264", size = 889217, upload-time = "2026-04-13T17:08:50.884Z" }, - { url = "https://files.pythonhosted.org/packages/d0/26/cca0fd2704f3ed20165e5613ed911549aef3aaf3b0b5b02fee0e8e23e6cc/fastar-0.11.0-pp311-pypy311_pp73-musllinux_1_2_aarch64.whl", hash = "sha256:aa3762cbb16e41a76b61f4a6914937a71aab3a7b6c2d82ca233bc686ebaf756b", size = 975418, upload-time = "2026-04-13T17:10:24.307Z" }, - { url = "https://files.pythonhosted.org/packages/99/94/8bbb0b13f5b6cbe2492f0b7cbba5103e6163976a3331466d010e781fa189/fastar-0.11.0-pp311-pypy311_pp73-musllinux_1_2_armv7l.whl", hash = "sha256:a8c7bc8ac74cb359bb546b199288c83236372d094b402e557c197e85527495cd", size = 1038492, upload-time = "2026-04-13T17:10:41.939Z" }, - { url = "https://files.pythonhosted.org/packages/ed/d3/5b7df222a30eac2822ffd00f82fd4c2ce84fba4b369d1e1a03732fd177fc/fastar-0.11.0-pp311-pypy311_pp73-musllinux_1_2_i686.whl", hash = "sha256:587cbd060a2699c5f66281081395bb4657b2b1e0eef5c206b1aabf740019d670", size = 1080210, upload-time = "2026-04-13T17:10:58.462Z" }, - { url = "https://files.pythonhosted.org/packages/ec/6d/56ef943ea524784598c035ccbd42e564e937da0438ae3f55f0e76cb95571/fastar-0.11.0-pp311-pypy311_pp73-musllinux_1_2_x86_64.whl", hash = "sha256:6a1c56957ac82408be37a3f63594bc83e0919e8760492a4475e542f9f1828778", size = 1034886, upload-time = "2026-04-13T17:11:15.617Z" }, -] - [[package]] name = "filelock" version = "3.29.0" @@ -1224,19 +1089,18 @@ wheels = [ [[package]] name = "honcho" -version = "3.0.11" +version = "3.0.12" source = { virtual = "." } dependencies = [ { name = "alembic" }, { name = "cashews", extra = ["redis"] }, { name = "cloudevents" }, - { name = "fastapi", extra = ["standard"] }, + { name = "fastapi", extra = ["standard-no-fastapi-cloud-cli"] }, { name = "fastapi-pagination" }, { name = "google-genai" }, { name = "greenlet" }, { name = "httpx" }, { name = "json-repair" }, - { name = "lancedb", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, { name = "langfuse" }, { name = "nanoid" }, { name = "openai" }, @@ -1244,7 +1108,6 @@ dependencies = [ { name = "pgvector" }, { name = "prometheus-client" }, { name = "psycopg", extra = ["binary"] }, - { name = "pyarrow" }, { name = "pydantic" }, { name = "pydantic-settings" }, { name = "pyjwt" }, @@ -1261,12 +1124,17 @@ dependencies = [ { name = "typing-extensions" }, ] +[package.optional-dependencies] +lancedb = [ + { name = "lancedb", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "pyarrow" }, +] + [package.dev-dependencies] dev = [ { name = "basedpyright" }, { name = "boto3" }, { name = "coverage" }, - { name = "fakeredis" }, { name = "honcho-ai" }, { name = "interrogate" }, { name = "pre-commit" }, @@ -1285,13 +1153,13 @@ requires-dist = [ { name = "alembic", specifier = ">=1.14.0" }, { name = "cashews", extras = ["redis"], specifier = "==7.5.0" }, { name = "cloudevents", specifier = ">=1.12.0,<2.0" }, - { name = "fastapi", extras = ["standard"], specifier = ">=0.131.0" }, + { name = "fastapi", extras = ["standard-no-fastapi-cloud-cli"], specifier = ">=0.131.0" }, { name = "fastapi-pagination", specifier = ">=0.14.2" }, { name = "google-genai", specifier = ">=1.32.0" }, { name = "greenlet", specifier = ">=3.0.3" }, { name = "httpx", specifier = ">=0.27.0" }, { name = "json-repair", specifier = ">=0.49.0" }, - { name = "lancedb", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'", specifier = ">=0.25.3" }, + { name = "lancedb", marker = "(platform_machine != 'x86_64' and extra == 'lancedb') or (sys_platform != 'darwin' and extra == 'lancedb')", specifier = ">=0.25.3" }, { name = "langfuse", specifier = ">=3.3.2" }, { name = "nanoid", specifier = ">=2.0.0" }, { name = "openai", specifier = ">=1.99.7" }, @@ -1299,7 +1167,7 @@ requires-dist = [ { name = "pgvector", specifier = ">=0.2.5" }, { name = "prometheus-client", specifier = ">=0.21.0" }, { name = "psycopg", extras = ["binary"], specifier = ">=3.1.19" }, - { name = "pyarrow", specifier = ">=19.0.0" }, + { name = "pyarrow", marker = "extra == 'lancedb'", specifier = ">=19.0.0" }, { name = "pydantic", specifier = ">=2.11.7" }, { name = "pydantic-settings", specifier = ">=2.10.1" }, { name = "pyjwt", specifier = ">=2.10.0" }, @@ -1315,13 +1183,13 @@ requires-dist = [ { name = "turbopuffer", specifier = ">=1.8.1" }, { name = "typing-extensions", specifier = ">=4.11.0" }, ] +provides-extras = ["lancedb"] [package.metadata.requires-dev] dev = [ { name = "basedpyright", specifier = ">=1.29.4" }, { name = "boto3", specifier = ">=1.42.5" }, { name = "coverage", specifier = ">=7.6.0" }, - { name = "fakeredis", specifier = ">=2.32.0" }, { name = "honcho-ai", editable = "sdks/python" }, { name = "interrogate", specifier = ">=1.7.0" }, { name = "pre-commit", specifier = ">=4.2.0" }, @@ -1337,7 +1205,7 @@ dev = [ [[package]] name = "honcho-ai" -version = "2.2.0" +version = "2.3.0" source = { editable = "sdks/python" } dependencies = [ { name = "httpx" }, @@ -2904,11 +2772,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/fd/7b/122376b1fd3c62c1ed9dc80c931ace4844b3c55407b6fb2d199377c9736f/pydantic-2.13.4-py3-none-any.whl", hash = "sha256:45a282cde31d808236fd7ea9d919b128653c8b38b393d1c4ab335c62924d9aba", size = 472262, upload-time = "2026-05-06T13:43:02.641Z" }, ] -[package.optional-dependencies] -email = [ - { name = "email-validator" }, -] - [[package]] name = "pydantic-core" version = "2.46.4" @@ -3446,101 +3309,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/fb/3c/c923619f6d2f5fafcc96fec0aaf9550a46cd5b6481f06e0c6b66a2a4fed0/rich_toolkit-0.19.7-py3-none-any.whl", hash = "sha256:0288e9203728c47c5a4eb60fd2f0692d9df7455a65901ab6f898437a2ba5989d", size = 32963, upload-time = "2026-02-24T16:06:22.066Z" }, ] -[[package]] -name = "rignore" -version = "0.7.6" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/e5/f5/8bed2310abe4ae04b67a38374a4d311dd85220f5d8da56f47ae9361be0b0/rignore-0.7.6.tar.gz", hash = "sha256:00d3546cd793c30cb17921ce674d2c8f3a4b00501cb0e3dd0e82217dbeba2671", size = 57140, upload-time = "2025-11-05T21:41:21.968Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/25/41/b6e2be3069ef3b7f24e35d2911bd6deb83d20ed5642ad81d5a6d1c015473/rignore-0.7.6-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:40be8226e12d6653abbebaffaea2885f80374c1c8f76fe5ca9e0cadd120a272c", size = 885285, upload-time = "2025-11-05T20:42:39.763Z" }, - { url = "https://files.pythonhosted.org/packages/52/66/ba7f561b6062402022887706a7f2b2c2e2e2a28f1e3839202b0a2f77e36d/rignore-0.7.6-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:182f4e5e4064d947c756819446a7d4cdede8e756b8c81cf9e509683fe38778d7", size = 823882, upload-time = "2025-11-05T20:42:23.488Z" }, - { url = "https://files.pythonhosted.org/packages/f5/81/4087453df35a90b07370647b19017029324950c1b9137d54bf1f33843f17/rignore-0.7.6-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:16b63047648a916a87be1e51bb5c009063f1b8b6f5afe4f04f875525507e63dc", size = 899362, upload-time = "2025-11-05T20:40:51.111Z" }, - { url = "https://files.pythonhosted.org/packages/fb/c9/390a8fdfabb76d71416be773bd9f162977bd483084f68daf19da1dec88a6/rignore-0.7.6-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:ba5524f5178deca4d7695e936604ebc742acb8958f9395776e1fcb8133f8257a", size = 873633, upload-time = "2025-11-05T20:41:06.193Z" }, - { url = "https://files.pythonhosted.org/packages/df/c9/79404fcb0faa76edfbc9df0901f8ef18568d1104919ebbbad6d608c888d1/rignore-0.7.6-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:62020dbb89a1dd4b84ab3d60547b3b2eb2723641d5fb198463643f71eaaed57d", size = 1167633, upload-time = "2025-11-05T20:41:22.491Z" }, - { url = "https://files.pythonhosted.org/packages/6e/8d/b3466d32d445d158a0aceb80919085baaae495b1f540fb942f91d93b5e5b/rignore-0.7.6-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:b34acd532769d5a6f153a52a98dcb81615c949ab11697ce26b2eb776af2e174d", size = 941434, upload-time = "2025-11-05T20:41:38.151Z" }, - { url = "https://files.pythonhosted.org/packages/e8/40/9cd949761a7af5bc27022a939c91ff622d29c7a0b66d0c13a863097dde2d/rignore-0.7.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1c5e53b752f9de44dff7b3be3c98455ce3bf88e69d6dc0cf4f213346c5e3416c", size = 959461, upload-time = "2025-11-05T20:42:08.476Z" }, - { url = "https://files.pythonhosted.org/packages/b5/87/1e1a145731f73bdb7835e11f80da06f79a00d68b370d9a847de979575e6d/rignore-0.7.6-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:25b3536d13a5d6409ce85f23936f044576eeebf7b6db1d078051b288410fc049", size = 985323, upload-time = "2025-11-05T20:41:52.735Z" }, - { url = "https://files.pythonhosted.org/packages/6c/31/1ecff992fc3f59c4fcdcb6c07d5f6c1e6dfb55ccda19c083aca9d86fa1c6/rignore-0.7.6-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:6e01cad2b0b92f6b1993f29fc01f23f2d78caf4bf93b11096d28e9d578eb08ce", size = 1079173, upload-time = "2025-11-05T21:40:12.007Z" }, - { url = "https://files.pythonhosted.org/packages/17/18/162eedadb4c2282fa4c521700dbf93c9b14b8842e8354f7d72b445b8d593/rignore-0.7.6-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:5991e46ab9b4868334c9e372ab0892b0150f3f586ff2b1e314272caeb38aaedb", size = 1139012, upload-time = "2025-11-05T21:40:29.399Z" }, - { url = "https://files.pythonhosted.org/packages/78/96/a9ca398a8af74bb143ad66c2a31303c894111977e28b0d0eab03867f1b43/rignore-0.7.6-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:6c8ae562e5d1246cba5eaeb92a47b2a279e7637102828dde41dcbe291f529a3e", size = 1118827, upload-time = "2025-11-05T21:40:46.6Z" }, - { url = "https://files.pythonhosted.org/packages/9f/22/1c1a65047df864def9a047dbb40bc0b580b8289a4280e62779cd61ae21f2/rignore-0.7.6-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:aaf938530dcc0b47c4cfa52807aa2e5bfd5ca6d57a621125fe293098692f6345", size = 1128182, upload-time = "2025-11-05T21:41:04.239Z" }, - { url = "https://files.pythonhosted.org/packages/bd/f4/1526eb01fdc2235aca1fd9d0189bee4021d009a8dcb0161540238c24166e/rignore-0.7.6-cp311-cp311-win32.whl", hash = "sha256:166ebce373105dd485ec213a6a2695986346e60c94ff3d84eb532a237b24a4d5", size = 646547, upload-time = "2025-11-05T21:41:49.439Z" }, - { url = "https://files.pythonhosted.org/packages/7c/c8/dda0983e1845706beb5826459781549a840fe5a7eb934abc523e8cd17814/rignore-0.7.6-cp311-cp311-win_amd64.whl", hash = "sha256:44f35ee844b1a8cea50d056e6a595190ce9d42d3cccf9f19d280ae5f3058973a", size = 727139, upload-time = "2025-11-05T21:41:34.367Z" }, - { url = "https://files.pythonhosted.org/packages/e3/47/eb1206b7bf65970d41190b879e1723fc6bbdb2d45e53565f28991a8d9d96/rignore-0.7.6-cp311-cp311-win_arm64.whl", hash = "sha256:14b58f3da4fa3d5c3fa865cab49821675371f5e979281c683e131ae29159a581", size = 657598, upload-time = "2025-11-05T21:41:23.758Z" }, - { url = "https://files.pythonhosted.org/packages/0b/0e/012556ef3047a2628842b44e753bb15f4dc46806780ff090f1e8fe4bf1eb/rignore-0.7.6-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:03e82348cb7234f8d9b2834f854400ddbbd04c0f8f35495119e66adbd37827a8", size = 883488, upload-time = "2025-11-05T20:42:41.359Z" }, - { url = "https://files.pythonhosted.org/packages/93/b0/d4f1f3fe9eb3f8e382d45ce5b0547ea01c4b7e0b4b4eb87bcd66a1d2b888/rignore-0.7.6-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:b9e624f6be6116ea682e76c5feb71ea91255c67c86cb75befe774365b2931961", size = 820411, upload-time = "2025-11-05T20:42:24.782Z" }, - { url = "https://files.pythonhosted.org/packages/4a/c8/dea564b36dedac8de21c18e1851789545bc52a0c22ece9843444d5608a6a/rignore-0.7.6-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:bda49950d405aa8d0ebe26af807c4e662dd281d926530f03f29690a2e07d649a", size = 897821, upload-time = "2025-11-05T20:40:52.613Z" }, - { url = "https://files.pythonhosted.org/packages/b3/2b/ee96db17ac1835e024c5d0742eefb7e46de60020385ac883dd3d1cde2c1f/rignore-0.7.6-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:b5fd5ab3840b8c16851d327ed06e9b8be6459702a53e5ab1fc4073b684b3789e", size = 873963, upload-time = "2025-11-05T20:41:07.49Z" }, - { url = "https://files.pythonhosted.org/packages/a5/8c/ad5a57bbb9d14d5c7e5960f712a8a0b902472ea3f4a2138cbf70d1777b75/rignore-0.7.6-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:ced2a248352636a5c77504cb755dc02c2eef9a820a44d3f33061ce1bb8a7f2d2", size = 1169216, upload-time = "2025-11-05T20:41:23.73Z" }, - { url = "https://files.pythonhosted.org/packages/80/e6/5b00bc2a6bc1701e6878fca798cf5d9125eb3113193e33078b6fc0d99123/rignore-0.7.6-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:a04a3b73b75ddc12c9c9b21efcdaab33ca3832941d6f1d67bffd860941cd448a", size = 942942, upload-time = "2025-11-05T20:41:39.393Z" }, - { url = "https://files.pythonhosted.org/packages/85/e5/7f99bd0cc9818a91d0e8b9acc65b792e35750e3bdccd15a7ee75e64efca4/rignore-0.7.6-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d24321efac92140b7ec910ac7c53ab0f0c86a41133d2bb4b0e6a7c94967f44dd", size = 959787, upload-time = "2025-11-05T20:42:09.765Z" }, - { url = "https://files.pythonhosted.org/packages/55/54/2ffea79a7c1eabcede1926347ebc2a81bc6b81f447d05b52af9af14948b9/rignore-0.7.6-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:73c7aa109d41e593785c55fdaa89ad80b10330affa9f9d3e3a51fa695f739b20", size = 984245, upload-time = "2025-11-05T20:41:54.062Z" }, - { url = "https://files.pythonhosted.org/packages/41/f7/e80f55dfe0f35787fa482aa18689b9c8251e045076c35477deb0007b3277/rignore-0.7.6-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:1734dc49d1e9501b07852ef44421f84d9f378da9fbeda729e77db71f49cac28b", size = 1078647, upload-time = "2025-11-05T21:40:13.463Z" }, - { url = "https://files.pythonhosted.org/packages/d4/cf/2c64f0b6725149f7c6e7e5a909d14354889b4beaadddaa5fff023ec71084/rignore-0.7.6-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:5719ea14ea2b652c0c0894be5dfde954e1853a80dea27dd2fbaa749618d837f5", size = 1139186, upload-time = "2025-11-05T21:40:31.27Z" }, - { url = "https://files.pythonhosted.org/packages/75/95/a86c84909ccc24af0d094b50d54697951e576c252a4d9f21b47b52af9598/rignore-0.7.6-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:8e23424fc7ce35726854f639cb7968151a792c0c3d9d082f7f67e0c362cfecca", size = 1117604, upload-time = "2025-11-05T21:40:48.07Z" }, - { url = "https://files.pythonhosted.org/packages/7f/5e/13b249613fd5d18d58662490ab910a9f0be758981d1797789913adb4e918/rignore-0.7.6-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:3efdcf1dd84d45f3e2bd2f93303d9be103888f56dfa7c3349b5bf4f0657ec696", size = 1127725, upload-time = "2025-11-05T21:41:05.804Z" }, - { url = "https://files.pythonhosted.org/packages/c7/28/fa5dcd1e2e16982c359128664e3785f202d3eca9b22dd0b2f91c4b3d242f/rignore-0.7.6-cp312-cp312-win32.whl", hash = "sha256:ccca9d1a8b5234c76b71546fc3c134533b013f40495f394a65614a81f7387046", size = 646145, upload-time = "2025-11-05T21:41:51.096Z" }, - { url = "https://files.pythonhosted.org/packages/26/87/69387fb5dd81a0f771936381431780b8cf66fcd2cfe9495e1aaf41548931/rignore-0.7.6-cp312-cp312-win_amd64.whl", hash = "sha256:c96a285e4a8bfec0652e0bfcf42b1aabcdda1e7625f5006d188e3b1c87fdb543", size = 726090, upload-time = "2025-11-05T21:41:36.485Z" }, - { url = "https://files.pythonhosted.org/packages/24/5f/e8418108dcda8087fb198a6f81caadbcda9fd115d61154bf0df4d6d3619b/rignore-0.7.6-cp312-cp312-win_arm64.whl", hash = "sha256:a64a750e7a8277a323f01ca50b7784a764845f6cce2fe38831cb93f0508d0051", size = 656317, upload-time = "2025-11-05T21:41:25.305Z" }, - { url = "https://files.pythonhosted.org/packages/b7/8a/a4078f6e14932ac7edb171149c481de29969d96ddee3ece5dc4c26f9e0c3/rignore-0.7.6-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:2bdab1d31ec9b4fb1331980ee49ea051c0d7f7bb6baa28b3125ef03cdc48fdaf", size = 883057, upload-time = "2025-11-05T20:42:42.741Z" }, - { url = "https://files.pythonhosted.org/packages/f9/8f/f8daacd177db4bf7c2223bab41e630c52711f8af9ed279be2058d2fe4982/rignore-0.7.6-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:90f0a00ce0c866c275bf888271f1dc0d2140f29b82fcf33cdbda1e1a6af01010", size = 820150, upload-time = "2025-11-05T20:42:26.545Z" }, - { url = "https://files.pythonhosted.org/packages/36/31/b65b837e39c3f7064c426754714ac633b66b8c2290978af9d7f513e14aa9/rignore-0.7.6-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c1ad295537041dc2ed4b540fb1a3906bd9ede6ccdad3fe79770cd89e04e3c73c", size = 897406, upload-time = "2025-11-05T20:40:53.854Z" }, - { url = "https://files.pythonhosted.org/packages/ca/58/1970ce006c427e202ac7c081435719a076c478f07b3a23f469227788dc23/rignore-0.7.6-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:f782dbd3a65a5ac85adfff69e5c6b101285ef3f845c3a3cae56a54bebf9fe116", size = 874050, upload-time = "2025-11-05T20:41:08.922Z" }, - { url = "https://files.pythonhosted.org/packages/d4/00/eb45db9f90137329072a732273be0d383cb7d7f50ddc8e0bceea34c1dfdf/rignore-0.7.6-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:65cece3b36e5b0826d946494734c0e6aaf5a0337e18ff55b071438efe13d559e", size = 1167835, upload-time = "2025-11-05T20:41:24.997Z" }, - { url = "https://files.pythonhosted.org/packages/f3/f1/6f1d72ddca41a64eed569680587a1236633587cc9f78136477ae69e2c88a/rignore-0.7.6-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d7e4bb66c13cd7602dc8931822c02dfbbd5252015c750ac5d6152b186f0a8be0", size = 941945, upload-time = "2025-11-05T20:41:40.628Z" }, - { url = "https://files.pythonhosted.org/packages/48/6f/2f178af1c1a276a065f563ec1e11e7a9e23d4996fd0465516afce4b5c636/rignore-0.7.6-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:297e500c15766e196f68aaaa70e8b6db85fa23fdc075b880d8231fdfba738cd7", size = 959067, upload-time = "2025-11-05T20:42:11.09Z" }, - { url = "https://files.pythonhosted.org/packages/5b/db/423a81c4c1e173877c7f9b5767dcaf1ab50484a94f60a0b2ed78be3fa765/rignore-0.7.6-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:a07084211a8d35e1a5b1d32b9661a5ed20669970b369df0cf77da3adea3405de", size = 984438, upload-time = "2025-11-05T20:41:55.443Z" }, - { url = "https://files.pythonhosted.org/packages/31/eb/c4f92cc3f2825d501d3c46a244a671eb737fc1bcf7b05a3ecd34abb3e0d7/rignore-0.7.6-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:181eb2a975a22256a1441a9d2f15eb1292839ea3f05606620bd9e1938302cf79", size = 1078365, upload-time = "2025-11-05T21:40:15.148Z" }, - { url = "https://files.pythonhosted.org/packages/26/09/99442f02794bd7441bfc8ed1c7319e890449b816a7493b2db0e30af39095/rignore-0.7.6-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:7bbcdc52b5bf9f054b34ce4af5269df5d863d9c2456243338bc193c28022bd7b", size = 1139066, upload-time = "2025-11-05T21:40:32.771Z" }, - { url = "https://files.pythonhosted.org/packages/2c/88/bcfc21e520bba975410e9419450f4b90a2ac8236b9a80fd8130e87d098af/rignore-0.7.6-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:f2e027a6da21a7c8c0d87553c24ca5cc4364def18d146057862c23a96546238e", size = 1118036, upload-time = "2025-11-05T21:40:49.646Z" }, - { url = "https://files.pythonhosted.org/packages/e2/25/d37215e4562cda5c13312636393aea0bafe38d54d4e0517520a4cc0753ec/rignore-0.7.6-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:ee4a18b82cbbc648e4aac1510066682fe62beb5dc88e2c67c53a83954e541360", size = 1127550, upload-time = "2025-11-05T21:41:07.648Z" }, - { url = "https://files.pythonhosted.org/packages/dc/76/a264ab38bfa1620ec12a8ff1c07778da89e16d8c0f3450b0333020d3d6dc/rignore-0.7.6-cp313-cp313-win32.whl", hash = "sha256:a7d7148b6e5e95035d4390396895adc384d37ff4e06781a36fe573bba7c283e5", size = 646097, upload-time = "2025-11-05T21:41:53.201Z" }, - { url = "https://files.pythonhosted.org/packages/62/44/3c31b8983c29ea8832b6082ddb1d07b90379c2d993bd20fce4487b71b4f4/rignore-0.7.6-cp313-cp313-win_amd64.whl", hash = "sha256:b037c4b15a64dced08fc12310ee844ec2284c4c5c1ca77bc37d0a04f7bff386e", size = 726170, upload-time = "2025-11-05T21:41:38.131Z" }, - { url = "https://files.pythonhosted.org/packages/aa/41/e26a075cab83debe41a42661262f606166157df84e0e02e2d904d134c0d8/rignore-0.7.6-cp313-cp313-win_arm64.whl", hash = "sha256:e47443de9b12fe569889bdbe020abe0e0b667516ee2ab435443f6d0869bd2804", size = 656184, upload-time = "2025-11-05T21:41:27.396Z" }, - { url = "https://files.pythonhosted.org/packages/9a/b9/1f5bd82b87e5550cd843ceb3768b4a8ef274eb63f29333cf2f29644b3d75/rignore-0.7.6-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:8e41be9fa8f2f47239ded8920cc283699a052ac4c371f77f5ac017ebeed75732", size = 882632, upload-time = "2025-11-05T20:42:44.063Z" }, - { url = "https://files.pythonhosted.org/packages/e9/6b/07714a3efe4a8048864e8a5b7db311ba51b921e15268b17defaebf56d3db/rignore-0.7.6-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:6dc1e171e52cefa6c20e60c05394a71165663b48bca6c7666dee4f778f2a7d90", size = 820760, upload-time = "2025-11-05T20:42:27.885Z" }, - { url = "https://files.pythonhosted.org/packages/ac/0f/348c829ea2d8d596e856371b14b9092f8a5dfbb62674ec9b3f67e4939a9d/rignore-0.7.6-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0ce2268837c3600f82ab8db58f5834009dc638ee17103582960da668963bebc5", size = 899044, upload-time = "2025-11-05T20:40:55.336Z" }, - { url = "https://files.pythonhosted.org/packages/f0/30/2e1841a19b4dd23878d73edd5d82e998a83d5ed9570a89675f140ca8b2ad/rignore-0.7.6-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:690a3e1b54bfe77e89c4bacb13f046e642f8baadafc61d68f5a726f324a76ab6", size = 874144, upload-time = "2025-11-05T20:41:10.195Z" }, - { url = "https://files.pythonhosted.org/packages/c2/bf/0ce9beb2e5f64c30e3580bef09f5829236889f01511a125f98b83169b993/rignore-0.7.6-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:09d12ac7a0b6210c07bcd145007117ebd8abe99c8eeb383e9e4673910c2754b2", size = 1168062, upload-time = "2025-11-05T20:41:26.511Z" }, - { url = "https://files.pythonhosted.org/packages/b9/8b/571c178414eb4014969865317da8a02ce4cf5241a41676ef91a59aab24de/rignore-0.7.6-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:2a2b2b74a8c60203b08452479b90e5ce3dbe96a916214bc9eb2e5af0b6a9beb0", size = 942542, upload-time = "2025-11-05T20:41:41.838Z" }, - { url = "https://files.pythonhosted.org/packages/19/62/7a3cf601d5a45137a7e2b89d10c05b5b86499190c4b7ca5c3c47d79ee519/rignore-0.7.6-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8fc5a531ef02131e44359419a366bfac57f773ea58f5278c2cdd915f7d10ea94", size = 958739, upload-time = "2025-11-05T20:42:12.463Z" }, - { url = "https://files.pythonhosted.org/packages/5f/1f/4261f6a0d7caf2058a5cde2f5045f565ab91aa7badc972b57d19ce58b14e/rignore-0.7.6-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:b7a1f77d9c4cd7e76229e252614d963442686bfe12c787a49f4fe481df49e7a9", size = 984138, upload-time = "2025-11-05T20:41:56.775Z" }, - { url = "https://files.pythonhosted.org/packages/2b/bf/628dfe19c75e8ce1f45f7c248f5148b17dfa89a817f8e3552ab74c3ae812/rignore-0.7.6-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:ead81f728682ba72b5b1c3d5846b011d3e0174da978de87c61645f2ed36659a7", size = 1079299, upload-time = "2025-11-05T21:40:16.639Z" }, - { url = "https://files.pythonhosted.org/packages/af/a5/be29c50f5c0c25c637ed32db8758fdf5b901a99e08b608971cda8afb293b/rignore-0.7.6-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:12ffd50f520c22ffdabed8cd8bfb567d9ac165b2b854d3e679f4bcaef11a9441", size = 1139618, upload-time = "2025-11-05T21:40:34.507Z" }, - { url = "https://files.pythonhosted.org/packages/2a/40/3c46cd7ce4fa05c20b525fd60f599165e820af66e66f2c371cd50644558f/rignore-0.7.6-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:e5a16890fbe3c894f8ca34b0fcacc2c200398d4d46ae654e03bc9b3dbf2a0a72", size = 1117626, upload-time = "2025-11-05T21:40:51.494Z" }, - { url = "https://files.pythonhosted.org/packages/8c/b9/aea926f263b8a29a23c75c2e0d8447965eb1879d3feb53cfcf84db67ed58/rignore-0.7.6-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:3abab3bf99e8a77488ef6c7c9a799fac22224c28fe9f25cc21aa7cc2b72bfc0b", size = 1128144, upload-time = "2025-11-05T21:41:09.169Z" }, - { url = "https://files.pythonhosted.org/packages/a4/f6/0d6242f8d0df7f2ecbe91679fefc1f75e7cd2072cb4f497abaab3f0f8523/rignore-0.7.6-cp314-cp314-win32.whl", hash = "sha256:eeef421c1782953c4375aa32f06ecae470c1285c6381eee2a30d2e02a5633001", size = 646385, upload-time = "2025-11-05T21:41:55.105Z" }, - { url = "https://files.pythonhosted.org/packages/d5/38/c0dcd7b10064f084343d6af26fe9414e46e9619c5f3224b5272e8e5d9956/rignore-0.7.6-cp314-cp314-win_amd64.whl", hash = "sha256:6aeed503b3b3d5af939b21d72a82521701a4bd3b89cd761da1e7dc78621af304", size = 725738, upload-time = "2025-11-05T21:41:39.736Z" }, - { url = "https://files.pythonhosted.org/packages/d9/7a/290f868296c1ece914d565757ab363b04730a728b544beb567ceb3b2d96f/rignore-0.7.6-cp314-cp314-win_arm64.whl", hash = "sha256:104f215b60b3c984c386c3e747d6ab4376d5656478694e22c7bd2f788ddd8304", size = 656008, upload-time = "2025-11-05T21:41:29.028Z" }, - { url = "https://files.pythonhosted.org/packages/ca/d2/3c74e3cd81fe8ea08a8dcd2d755c09ac2e8ad8fe409508904557b58383d3/rignore-0.7.6-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:bb24a5b947656dd94cb9e41c4bc8b23cec0c435b58be0d74a874f63c259549e8", size = 882835, upload-time = "2025-11-05T20:42:45.443Z" }, - { url = "https://files.pythonhosted.org/packages/77/61/a772a34b6b63154877433ac2d048364815b24c2dd308f76b212c408101a2/rignore-0.7.6-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:5b1e33c9501cefe24b70a1eafd9821acfd0ebf0b35c3a379430a14df089993e3", size = 820301, upload-time = "2025-11-05T20:42:29.226Z" }, - { url = "https://files.pythonhosted.org/packages/71/30/054880b09c0b1b61d17eeb15279d8bf729c0ba52b36c3ada52fb827cbb3c/rignore-0.7.6-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:bec3994665a44454df86deb762061e05cd4b61e3772f5b07d1882a8a0d2748d5", size = 897611, upload-time = "2025-11-05T20:40:56.475Z" }, - { url = "https://files.pythonhosted.org/packages/1e/40/b2d1c169f833d69931bf232600eaa3c7998ba4f9a402e43a822dad2ea9f2/rignore-0.7.6-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:26cba2edfe3cff1dfa72bddf65d316ddebf182f011f2f61538705d6dbaf54986", size = 873875, upload-time = "2025-11-05T20:41:11.561Z" }, - { url = "https://files.pythonhosted.org/packages/55/59/ca5ae93d83a1a60e44b21d87deb48b177a8db1b85e82fc8a9abb24a8986d/rignore-0.7.6-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:ffa86694fec604c613696cb91e43892aa22e1fec5f9870e48f111c603e5ec4e9", size = 1167245, upload-time = "2025-11-05T20:41:28.29Z" }, - { url = "https://files.pythonhosted.org/packages/a5/52/cf3dce392ba2af806cba265aad6bcd9c48bb2a6cb5eee448d3319f6e505b/rignore-0.7.6-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:48efe2ed95aa8104145004afb15cdfa02bea5cdde8b0344afeb0434f0d989aa2", size = 941750, upload-time = "2025-11-05T20:41:43.111Z" }, - { url = "https://files.pythonhosted.org/packages/ec/be/3f344c6218d779395e785091d05396dfd8b625f6aafbe502746fcd880af2/rignore-0.7.6-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8dcae43eb44b7f2457fef7cc87f103f9a0013017a6f4e62182c565e924948f21", size = 958896, upload-time = "2025-11-05T20:42:13.784Z" }, - { url = "https://files.pythonhosted.org/packages/c9/34/d3fa71938aed7d00dcad87f0f9bcb02ad66c85d6ffc83ba31078ce53646a/rignore-0.7.6-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:2cd649a7091c0dad2f11ef65630d30c698d505cbe8660dd395268e7c099cc99f", size = 983992, upload-time = "2025-11-05T20:41:58.022Z" }, - { url = "https://files.pythonhosted.org/packages/24/a4/52a697158e9920705bdbd0748d59fa63e0f3233fb92e9df9a71afbead6ca/rignore-0.7.6-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:42de84b0289d478d30ceb7ae59023f7b0527786a9a5b490830e080f0e4ea5aeb", size = 1078181, upload-time = "2025-11-05T21:40:18.151Z" }, - { url = "https://files.pythonhosted.org/packages/ac/65/aa76dbcdabf3787a6f0fd61b5cc8ed1e88580590556d6c0207960d2384bb/rignore-0.7.6-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:875a617e57b53b4acbc5a91de418233849711c02e29cc1f4f9febb2f928af013", size = 1139232, upload-time = "2025-11-05T21:40:35.966Z" }, - { url = "https://files.pythonhosted.org/packages/08/44/31b31a49b3233c6842acc1c0731aa1e7fb322a7170612acf30327f700b44/rignore-0.7.6-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:8703998902771e96e49968105207719f22926e4431b108450f3f430b4e268b7c", size = 1117349, upload-time = "2025-11-05T21:40:53.013Z" }, - { url = "https://files.pythonhosted.org/packages/e9/ae/1b199a2302c19c658cf74e5ee1427605234e8c91787cfba0015f2ace145b/rignore-0.7.6-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:602ef33f3e1b04c1e9a10a3c03f8bc3cef2d2383dcc250d309be42b49923cabc", size = 1127702, upload-time = "2025-11-05T21:41:10.881Z" }, - { url = "https://files.pythonhosted.org/packages/fc/d3/18210222b37e87e36357f7b300b7d98c6dd62b133771e71ae27acba83a4f/rignore-0.7.6-cp314-cp314t-win32.whl", hash = "sha256:c1d8f117f7da0a4a96a8daef3da75bc090e3792d30b8b12cfadc240c631353f9", size = 647033, upload-time = "2025-11-05T21:42:00.095Z" }, - { url = "https://files.pythonhosted.org/packages/3e/87/033eebfbee3ec7d92b3bb1717d8f68c88e6fc7de54537040f3b3a405726f/rignore-0.7.6-cp314-cp314t-win_amd64.whl", hash = "sha256:ca36e59408bec81de75d307c568c2d0d410fb880b1769be43611472c61e85c96", size = 725647, upload-time = "2025-11-05T21:41:44.449Z" }, - { url = "https://files.pythonhosted.org/packages/79/62/b88e5879512c55b8ee979c666ee6902adc4ed05007226de266410ae27965/rignore-0.7.6-cp314-cp314t-win_arm64.whl", hash = "sha256:b83adabeb3e8cf662cabe1931b83e165b88c526fa6af6b3aa90429686e474896", size = 656035, upload-time = "2025-11-05T21:41:31.13Z" }, - { url = "https://files.pythonhosted.org/packages/82/78/a6250ff0c49a3cdb943910ada4116e708118e9b901c878cfae616c80a904/rignore-0.7.6-pp311-pypy311_pp73-macosx_10_12_x86_64.whl", hash = "sha256:a20b6fb61bcced9a83dfcca6599ad45182b06ba720cff7c8d891e5b78db5b65f", size = 886470, upload-time = "2025-11-05T20:42:52.314Z" }, - { url = "https://files.pythonhosted.org/packages/35/af/c69c0c51b8f9f7914d95c4ea91c29a2ac067572048cae95dd6d2efdbe05d/rignore-0.7.6-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:392dcabfecbe176c9ebbcb40d85a5e86a5989559c4f988c2741da7daf1b5be25", size = 825976, upload-time = "2025-11-05T20:42:35.118Z" }, - { url = "https://files.pythonhosted.org/packages/f1/d2/1b264f56132264ea609d3213ab603d6a27016b19559a1a1ede1a66a03dcd/rignore-0.7.6-pp311-pypy311_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:22baa462abdc36fdd5a5e2dae423107723351b85ff093762f9261148b9d0a04a", size = 899739, upload-time = "2025-11-05T20:41:01.518Z" }, - { url = "https://files.pythonhosted.org/packages/55/e4/b3c5dfdd8d8a10741dfe7199ef45d19a0e42d0c13aa377c83bd6caf65d90/rignore-0.7.6-pp311-pypy311_pp73-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:53fb28882d2538cb2d231972146c4927a9d9455e62b209f85d634408c4103538", size = 874843, upload-time = "2025-11-05T20:41:17.687Z" }, - { url = "https://files.pythonhosted.org/packages/cc/10/d6f3750233881a2a154cefc9a6a0a9b19da526b19f7f08221b552c6f827d/rignore-0.7.6-pp311-pypy311_pp73-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:87409f7eeb1103d6b77f3472a3a0d9a5953e3ae804a55080bdcb0120ee43995b", size = 1170348, upload-time = "2025-11-05T20:41:34.21Z" }, - { url = "https://files.pythonhosted.org/packages/6e/10/ad98ca05c9771c15af734cee18114a3c280914b6e34fde9ffea2e61e88aa/rignore-0.7.6-pp311-pypy311_pp73-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:684014e42e4341ab3ea23a203551857fcc03a7f8ae96ca3aefb824663f55db32", size = 942315, upload-time = "2025-11-05T20:41:48.508Z" }, - { url = "https://files.pythonhosted.org/packages/de/00/ab5c0f872acb60d534e687e629c17e0896c62da9b389c66d3aa16b817aa8/rignore-0.7.6-pp311-pypy311_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:77356ebb01ba13f8a425c3d30fcad40e57719c0e37670d022d560884a30e4767", size = 961047, upload-time = "2025-11-05T20:42:19.403Z" }, - { url = "https://files.pythonhosted.org/packages/b8/86/3030fdc363a8f0d1cd155b4c453d6db9bab47a24fcc64d03f61d9d78fe6a/rignore-0.7.6-pp311-pypy311_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:6cbd8a48abbd3747a6c830393cd578782fab5d43f4deea48c5f5e344b8fed2b0", size = 986090, upload-time = "2025-11-05T20:42:03.581Z" }, - { url = "https://files.pythonhosted.org/packages/33/b8/133aa4002cee0ebbb39362f94e4898eec7fbd09cec9fcbce1cd65b355b7f/rignore-0.7.6-pp311-pypy311_pp73-musllinux_1_2_aarch64.whl", hash = "sha256:2673225dcec7f90497e79438c35e34638d0d0391ccea3cbb79bfb9adc0dc5bd7", size = 1079656, upload-time = "2025-11-05T21:40:24.89Z" }, - { url = "https://files.pythonhosted.org/packages/67/56/36d5d34210e5e7dfcd134eed8335b19e80ae940ee758f493e4f2b344dd70/rignore-0.7.6-pp311-pypy311_pp73-musllinux_1_2_armv7l.whl", hash = "sha256:c081f17290d8a2b96052b79207622aa635686ea39d502b976836384ede3d303c", size = 1139789, upload-time = "2025-11-05T21:40:42.119Z" }, - { url = "https://files.pythonhosted.org/packages/6b/5b/bb4f9420802bf73678033a4a55ab1bede36ce2e9b41fec5f966d83d932b3/rignore-0.7.6-pp311-pypy311_pp73-musllinux_1_2_i686.whl", hash = "sha256:57e8327aacc27f921968cb2a174f9e47b084ce9a7dd0122c8132d22358f6bd79", size = 1120308, upload-time = "2025-11-05T21:40:59.402Z" }, - { url = "https://files.pythonhosted.org/packages/ce/8b/a1299085b28a2f6135e30370b126e3c5055b61908622f2488ade67641479/rignore-0.7.6-pp311-pypy311_pp73-musllinux_1_2_x86_64.whl", hash = "sha256:d8955b57e42f2a5434670d5aa7b75eaf6e74602ccd8955dddf7045379cd762fb", size = 1129444, upload-time = "2025-11-05T21:41:17.906Z" }, -] - [[package]] name = "ruff" version = "0.15.12" @@ -3750,15 +3518,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/e9/44/75a9c9421471a6c4805dbf2356f7c181a29c1879239abab1ea2cc8f38b40/sniffio-1.3.1-py3-none-any.whl", hash = "sha256:2f6da418d1f1e0fddd844478f41680e794e6051915791a034ff65e5f100525a2", size = 10235, upload-time = "2024-02-25T23:20:01.196Z" }, ] -[[package]] -name = "sortedcontainers" -version = "2.4.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/e8/c4/ba2f8066cceb6f23394729afe52f3bf7adec04bf9ed2c820b39e19299111/sortedcontainers-2.4.0.tar.gz", hash = "sha256:25caa5a06cc30b6b83d11423433f65d1f9d76c4c6a0c90e3379eaa43b9bfdb88", size = 30594, upload-time = "2021-05-16T22:03:42.897Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/32/46/9cb0e58b2deb7f82b84065f37f3bffeb12413f947f9388e4cac22c4621ce/sortedcontainers-2.4.0-py2.py3-none-any.whl", hash = "sha256:a163dcaede0f1c021485e957a39245190e74249897e2ae4b2aa38595db237ee0", size = 29575, upload-time = "2021-05-16T22:03:41.177Z" }, -] - [[package]] name = "sqlalchemy" version = "2.0.49"