docs(integrations): rewrite Vercel AI SDK guide as cookbook style (DEV-1485) (#635)

* docs(integrations): add @honcho-ai/vercel-ai-sdk guide

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* docs(integrations): rewrite Vercel AI SDK guide as cookbook style (DEV-1485)

Reshapes the guide to cookbook formula, adds Full Script section, fixes
maxSteps → stopWhen for ai-sdk v5, renames package, and prunes stale notes.
See PR for full decision log.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* docs(integrations): lead Vercel AI SDK verification with direct-inspection check

- Restructure Verifying section: direct inspection (token delta + dashboard) is now step 1 so readers isolate Honcho's contribution before grading model behavior
- Behavioral tests (first turn, multi-turn, cross-session, tool calling) follow as steps 2-5
- Note `result.toolCalls` as the way to confirm which Honcho tool fired (tool names don't appear in `result.text`)
- Signpost the Full Script from Complete Example so the two snippets read as a staircase, not a duplicate

Addresses review comments on PR #635.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(tests): satisfy basedpyright in test_representation_manager

The save-representation tests added in #615 were structurally correct but
failed strict typing in two places. Static Analysis has been red on main
since the merge.

- `mock_save.await_args` is `_Call | None`; assert it's not None before
  reading `.kwargs` / `.args` so basedpyright can narrow the type
- `SimpleNamespace(...)` passed as `message_level_configuration` is an
  intentional duck-typed mock (only `.dream.enabled` is read by
  `save_representation`), so opt out at the call site with
  `# pyright: ignore[reportArgumentType]` rather than constructing a
  full `ResolvedConfiguration` (matches the existing `reportPrivateUsage`
  ignore pattern in this file)

No runtime behavior changes; `uv run basedpyright` is now clean
project-wide.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(tests): pad timestamp windows in test_messages for clock skew

Three timestamp tests captured `before_request` / `after_request` with
`datetime.now(UTC)` on the host and asserted the server's `created_at`
fell within. Under Docker, the Postgres container's clock can skew tens
of ms from the macOS host, flipping the assertion intermittently under
parallel pytest load.

Pad each window by 1 second on both sides — wide enough to absorb
realistic skew, narrow enough that the test still proves the timestamp
is server-current.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(integrations): tighten Verifying section after end-to-end smoke

Smoke-tested all five verification steps against a fresh Sonnet 4.6 + Honcho integration. Three findings, all reflected here:

- Cross-session recall (#4): added Note about DERIVER_REPRESENTATION_BATCH_MAX_TOKENS=1024 — short warmups don't accumulate enough content to flush observations, so cross-session recall returns empty even on a working integration.
- Tool calling prompt (#5): replaced the honcho_chat patterns prompt with a verbatim-retrieval honcho_search prompt. Sonnet skips honcho_chat when middleware-injected context already answers; verbatim retrieval forces a fire.
- Tool inspection (#5): replaced result.toolCalls reference with result.steps[i].toolCalls + flatMap snippet. Top-level toolCalls is empty in multi-step calls (stopWhen: stepCountIs(N)) — the fires are nested inside steps.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(integrations): make Step 4 cross-session test durable via honcho_search

Replace the prose-recall test ("Based on what we've talked about, what do you know about me?") with a forced honcho_search call. Prose recall depended on the model getting deriver-built representation/peer-card in its system prompt, which is gated behind DERIVER_REPRESENTATION_BATCH_MAX_TOKENS=1024 — short tutorial-length conversations don't trigger it, producing false negatives on a working integration.

honcho_search hits message embeddings, which are computed synchronously at message persist time (src/crud/message.py:262-276), so peer-scoped retrieval works regardless of how short the prior session was. Also folds the result.steps[i].toolCalls inspection snippet from the old Step 5 into Step 4 — same prompt, no need for two sections.

Drops Step 5 entirely.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Lily 2026-05-05 10:41:18 -04:00 committed by GitHub
parent f37338b855
commit e38085177c
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 404 additions and 14 deletions

View File

@ -104,6 +104,7 @@
"pages": [
"v3/guides/integrations/claude-code",
"v3/guides/integrations/opencode",
"v3/guides/integrations/vercel-ai-sdk",
"v3/guides/integrations/crewai",
"v3/guides/integrations/langgraph",
"v3/guides/integrations/mcp",

View File

@ -0,0 +1,377 @@
---
title: "Vercel AI SDK"
icon: "triangle"
iconType: "solid"
description: "Add persistent user memory and reasoning to any Vercel AI SDK app with Honcho"
sidebarTitle: "Vercel AI SDK"
---
Integrate Honcho with the Vercel AI SDK to build AI apps that remember users across sessions. The [Vercel AI SDK](https://sdk.vercel.ai) is an open-source TypeScript toolkit for building AI-powered apps with a unified API across providers. This guide shows you how to wrap any `generateText` or `streamText` call with Honcho's memory middleware and reasoning tools.
<Note>
The full package source and examples are available on [GitHub](https://github.com/plastic-labs/vercel-ai-sdk-package).
</Note>
## What We're Building
We'll wire Honcho into a Vercel AI SDK app so the model receives context from past conversations and can query what it knows about the user mid-generation. Here's how the pieces fit together:
- **Vercel AI SDK** handles model calls and streaming
- **Honcho** stores messages and retrieves user context before each generation
- **Your model provider** can be Anthropic, OpenAI, Google, etc.
The key benefit: you don't manually manage conversation history across sessions. Honcho handles persistence and context injection — the model always has a rich picture of who it's talking to. (New to Honcho's primitives? See [peers and sessions](/v3/documentation/core-concepts/architecture).)
## Setup
Install the package:
<CodeGroup>
```bash npm
npm install @honcho-ai/vercel-ai-sdk
```
```bash pnpm
pnpm add @honcho-ai/vercel-ai-sdk
```
```bash yarn
yarn add @honcho-ai/vercel-ai-sdk
```
```bash bun
bun add @honcho-ai/vercel-ai-sdk
```
</CodeGroup>
Get your API key at [app.honcho.dev](https://app.honcho.dev).
```bash
HONCHO_API_KEY=your-api-key
HONCHO_WORKSPACE_ID=your-workspace-id
```
## Create a Provider Instance
`createHoncho()` is the entry point. It reads your API key and workspace from environment variables and returns a provider object with `middleware()`, `tools()`, and `send()`.
```typescript
import { createHoncho } from '@honcho-ai/vercel-ai-sdk';
const honcho = createHoncho();
```
You can set a stable `defaultAssistantId` on the provider to identify the AI peer across all calls:
```typescript
const honcho = createHoncho({
defaultAssistantId: 'my-assistant',
});
```
## Add Middleware
`honcho.middleware()` is compatible with `wrapLanguageModel`. Two things happen on each call:
1. **Before generation** — Honcho fetches the user's representation, peer card, session summary, and recent messages and injects them into the system prompt
2. **After generation** — the user message and assistant response are stored back in Honcho with correct peer attribution
```typescript
import { createHoncho } from '@honcho-ai/vercel-ai-sdk';
import { wrapLanguageModel, generateText } from 'ai';
import { anthropic } from '@ai-sdk/anthropic';
const honcho = createHoncho();
const model = wrapLanguageModel({
model: anthropic('claude-sonnet-4-6'),
middleware: honcho.middleware({
userId: 'user-abc',
sessionId: 'session-123',
}),
});
const { text } = await generateText({
model,
prompt: 'What should I focus on today?',
});
```
Pass `userId` and `sessionId` per request — no session handles to construct. Both default to lazily generated IDs if omitted, which is fine for local scripts but not for multi-user server traffic.
## Add Tools
`honcho.tools()` gives the model six tools it can call mid-generation to query or update what it knows about the user:
| Tool | What it does |
| --- | --- |
| `honcho_chat` | Dialectic reasoning — ask natural-language questions about the user; answers synthesized from full interaction history |
| `honcho_context` | Short summary of recent context within the session |
| `honcho_search` | Semantic search over stored conversation messages |
| `honcho_search_conclusions` | Query derived conclusions: personality traits, preferences, behavioral patterns |
| `honcho_get_representation` | Full synthesized profile of the user |
| `honcho_save_conclusion` | Persist an observation about the user for future sessions |
Pass the same `userId` and `sessionId` to `honcho.tools()` so tool calls bind to the same peers as the middleware:
```typescript
import { generateText, stepCountIs } from 'ai';
const { text } = await generateText({
model,
tools: honcho.tools({
userId: 'user-abc',
sessionId: 'session-123',
}),
stopWhen: stepCountIs(3),
prompt: 'Based on our conversations, what do I care about most?',
});
```
## Complete Example
Here's a full working example combining middleware and tools.
Want a runnable end-to-end version? See the [Full Script](#full-script).
```typescript
import { createHoncho } from '@honcho-ai/vercel-ai-sdk';
import { wrapLanguageModel, generateText, stepCountIs } from 'ai';
import { anthropic } from '@ai-sdk/anthropic';
const honcho = createHoncho({
defaultAssistantId: 'assistant',
});
const userId = 'user-abc';
const sessionId = 'session-123';
const model = wrapLanguageModel({
model: anthropic('claude-sonnet-4-6'),
middleware: honcho.middleware({ userId, sessionId }),
});
const { text } = await generateText({
model,
tools: honcho.tools({ userId, sessionId }),
stopWhen: stepCountIs(3),
prompt: 'What should we work on today?',
});
console.log(text);
```
## Streaming
`streamText` works the same way — middleware handles persistence after the stream completes:
```typescript
import { createHoncho } from '@honcho-ai/vercel-ai-sdk';
import { wrapLanguageModel, streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
const honcho = createHoncho();
const userId = 'user-abc';
const sessionId = 'session-456';
const model = wrapLanguageModel({
model: openai('gpt-4o'),
middleware: honcho.middleware({ userId, sessionId }),
});
const result = streamText({
model,
tools: honcho.tools({ userId, sessionId }),
prompt: 'What should we work on today?',
});
for await (const chunk of result.textStream) {
process.stdout.write(chunk);
}
```
## Using with `messages`
If your app already manages conversation history and passes a `messages` array directly, set `injectHistory: false` to prevent Honcho from prepending duplicate history:
```typescript
honcho.middleware({
userId,
sessionId,
injectHistory: false, // don't prepend history — we're passing messages directly
})
```
Honcho still injects the user's representation and peer card into the system prompt, and still persists messages after generation. With `injectHistory: false` you must pass a `messages` array — without either `messages` or `prompt`, the Vercel AI SDK throws `Invalid prompt: prompt or messages must be defined`.
## Verifying the Integration
### 1. Isolate Honcho's Contribution
Let's confirm the memory is actually coming from Honcho and not your app's existing conversation history.
Two ways to check: 1) through a developer method 2) through the UI.
**Token delta (developer check).** On a session with a few prior turns, run the same prompt twice — once with `injectHistory: false` and once without.
Compare `result.usage.inputTokens`:
```typescript
const baseline = await generateText({
model: wrapLanguageModel({
model: anthropic('claude-sonnet-4-6'),
middleware: honcho.middleware({ userId, sessionId, injectHistory: false }),
}),
prompt: 'What do you know about my preferences?',
});
const injected = await generateText({
model: wrapLanguageModel({
model: anthropic('claude-sonnet-4-6'),
middleware: honcho.middleware({ userId, sessionId }),
}),
prompt: 'What do you know about my preferences?',
});
console.log(injected.usage.inputTokens - baseline.usage.inputTokens);
```
A positive delta is Honcho's representation, peer card, and session summary being injected into the system prompt. Expect ~0 on a fresh peer — the deriver runs asynchronously after messages persist, so injected context only populates after a few prior turns.
**Dashboard (UI check).** Open [app.honcho.dev/explore](https://app.honcho.dev/explore), select your workspace, and confirm your peer and session appear under the Peers and Sessions tables.
With Honcho's contribution isolated, the rest of this section shows what the integration feels like in practice.
### 2. First turn
Send any message. The model responds normally — nothing is stored yet. Context injection returns empty on the first turn.
### 3. Build memory across turns
Have a multi-turn conversation and share something about yourself:
```text
I prefer concise answers and I mostly work in TypeScript.
```
After a few turns, ask:
```text
What do you know about my preferences?
```
If the model references TypeScript and concise answers without being told again in this session, memory is working.
### 4. Cross-session recall
Start a new session (new `sessionId`) with the same `userId`. Ask:
```text
Call your honcho_search tool with the query 'TypeScript' and quote the exact verbatim message that contained TypeScript. Do not paraphrase.
```
If the search returns a message from the prior session word-for-word, peer-scoped retrieval is crossing session boundaries. `honcho_search` queries the user's messages across all their sessions and doesn't depend on the deriver, so it works regardless of how short the prior session was.
To confirm the tool actually fired, inspect `result.steps[i].toolCalls`:
```typescript
const toolFires = result.steps?.flatMap((step, i) =>
(step.toolCalls ?? []).map((tc) => ({ step: i, tool: tc.toolName, input: tc.input }))
) ?? [];
console.log(toolFires);
// [{ step: 0, tool: "honcho_search", input: { query: "TypeScript", limit: 10 } }]
```
When the model takes more than one turn (call a tool, see the result, then answer), the top-level `result.toolCalls` is empty — check inside each `step`.
## Full Script
<Accordion title="honcho_vercel_chat.ts">
```typescript
/**
* Multi-turn chat with Honcho memory + Vercel AI SDK.
*
* Prerequisites:
* 1. Install dependencies:
* npm install @honcho-ai/vercel-ai-sdk ai @ai-sdk/anthropic dotenv
* 2. Set environment variables in `.env`:
* HONCHO_API_KEY=your-honcho-api-key
* HONCHO_WORKSPACE_ID=your-workspace-id
* ANTHROPIC_API_KEY=your-anthropic-api-key
* 3. Run with: npx tsx honcho_vercel_chat.ts
*
* Pass a stable userId from your auth system and a sessionId for the conversation
* thread; Honcho handles persistence and context injection on every turn.
*/
import 'dotenv/config';
import { createHoncho } from '@honcho-ai/vercel-ai-sdk';
import { wrapLanguageModel, generateText, stepCountIs } from 'ai';
import { anthropic } from '@ai-sdk/anthropic';
import * as readline from 'node:readline/promises';
import { stdin as input, stdout as output } from 'node:process';
const honcho = createHoncho({
defaultAssistantId: 'assistant',
});
const userId = process.env.USER_ID ?? 'demo-user';
const sessionId = process.env.SESSION_ID ?? `session-${Date.now()}`;
const model = wrapLanguageModel({
model: anthropic('claude-sonnet-4-6'),
middleware: honcho.middleware({ userId, sessionId }),
});
async function chat(prompt: string): Promise<string> {
const { text } = await generateText({
model,
tools: honcho.tools({ userId, sessionId }),
stopWhen: stepCountIs(3),
prompt,
});
return text;
}
async function main() {
const rl = readline.createInterface({ input, output });
console.log(`Honcho session: ${sessionId} (user: ${userId})`);
console.log('Type a message, or "exit" to quit.\n');
while (true) {
const userMessage = (await rl.question('you > ')).trim();
if (!userMessage || userMessage === 'exit') break;
const reply = await chat(userMessage);
console.log(`bot > ${reply}\n`);
}
rl.close();
}
main().catch((err) => {
console.error(err);
process.exit(1);
});
```
</Accordion>
## Next Steps
<CardGroup cols={2}>
<Card title="Github Repository" icon="github" href="https://github.com/plastic-labs/vercel-ai-sdk-package">
Source, tests, and full API reference for @honcho-ai/vercel-ai-sdk.
</Card>
<Card title="Honcho Architecture" icon="sitemap" href="/v3/documentation/core-concepts/architecture">
Learn about peers, sessions, and dialectic reasoning.
</Card>
<Card title="Self-Hosting Guide" icon="server" href="/v3/contributing/self-hosting">
Run Honcho locally with your Vercel AI SDK app.
</Card>
<Card title="Vercel AI SDK Docs" icon="book" href="https://sdk.vercel.ai">
wrapLanguageModel, middleware, and tool use reference.
</Card>
</CardGroup>

View File

@ -24,6 +24,7 @@ async def _fake_tracked_db(_name: str):
def _saved_observations(mock_save: AsyncMock):
call = mock_save.await_args
assert call is not None, "mock was not awaited"
if "all_observations" in call.kwargs:
return call.kwargs["all_observations"]
if len(call.args) > 1:
@ -162,7 +163,9 @@ class TestRepresentationManagerSoftDelete:
class TestRepresentationManagerSave:
@pytest.mark.asyncio
async def test_save_representation_filters_blank_observations_before_embedding(self):
async def test_save_representation_filters_blank_observations_before_embedding(
self,
):
manager = RepresentationManager(
"workspace",
observer="observer",
@ -202,7 +205,7 @@ class TestRepresentationManagerSave:
message_ids=[1],
session_name="session",
message_created_at=datetime.now(timezone.utc),
message_level_configuration=SimpleNamespace(
message_level_configuration=SimpleNamespace( # pyright: ignore[reportArgumentType]
dream=SimpleNamespace(enabled=False)
),
)
@ -258,7 +261,7 @@ class TestRepresentationManagerSave:
message_ids=[1],
session_name="session",
message_created_at=datetime.now(timezone.utc),
message_level_configuration=SimpleNamespace(
message_level_configuration=SimpleNamespace( # pyright: ignore[reportArgumentType]
dream=SimpleNamespace(enabled=False)
),
)
@ -311,7 +314,7 @@ class TestRepresentationManagerSave:
message_ids=[1],
session_name="session",
message_created_at=datetime.now(timezone.utc),
message_level_configuration=SimpleNamespace(
message_level_configuration=SimpleNamespace( # pyright: ignore[reportArgumentType]
dream=SimpleNamespace(enabled=False)
),
)

View File

@ -1001,8 +1001,10 @@ async def test_create_message_without_timestamp_uses_default(
db_session.add(test_session)
await db_session.commit()
# Record time before request
before_request = datetime.datetime.now(datetime.timezone.utc)
# Pad the window to absorb client/Postgres clock skew under Docker.
before_request = datetime.datetime.now(datetime.timezone.utc) - datetime.timedelta(
seconds=1
)
response = client.post(
f"/v3/workspaces/{test_workspace.name}/sessions/{test_session.name}/messages",
@ -1017,8 +1019,9 @@ async def test_create_message_without_timestamp_uses_default(
},
)
# Record time after request
after_request = datetime.datetime.now(datetime.timezone.utc)
after_request = datetime.datetime.now(datetime.timezone.utc) + datetime.timedelta(
seconds=1
)
assert response.status_code == 201
data = response.json()
@ -1053,8 +1056,9 @@ async def test_create_batch_messages_with_mixed_timestamps(
timestamp1 = datetime.datetime(2023, 1, 1, 12, 0, 0, tzinfo=datetime.timezone.utc)
timestamp2 = datetime.datetime(2023, 1, 2, 12, 0, 0, tzinfo=datetime.timezone.utc)
# Record time before request for default timestamp
before_request = datetime.datetime.now(datetime.timezone.utc)
before_request = datetime.datetime.now(datetime.timezone.utc) - datetime.timedelta(
seconds=1
)
response = client.post(
f"/v3/workspaces/{test_workspace.name}/sessions/{test_session.name}/messages",
@ -1081,7 +1085,9 @@ async def test_create_batch_messages_with_mixed_timestamps(
},
)
after_request = datetime.datetime.now(datetime.timezone.utc)
after_request = datetime.datetime.now(datetime.timezone.utc) + datetime.timedelta(
seconds=1
)
assert response.status_code == 201
data = response.json()
@ -1124,8 +1130,9 @@ async def test_create_message_with_null_timestamp(
db_session.add(test_session)
await db_session.commit()
# Record time before request
before_request = datetime.datetime.now(datetime.timezone.utc)
before_request = datetime.datetime.now(datetime.timezone.utc) - datetime.timedelta(
seconds=1
)
response = client.post(
f"/v3/workspaces/{test_workspace.name}/sessions/{test_session.name}/messages",
@ -1141,7 +1148,9 @@ async def test_create_message_with_null_timestamp(
},
)
after_request = datetime.datetime.now(datetime.timezone.utc)
after_request = datetime.datetime.now(datetime.timezone.utc) + datetime.timedelta(
seconds=1
)
assert response.status_code == 201
data = response.json()