549 lines
16 KiB
Plaintext
549 lines
16 KiB
Plaintext
---
|
|
title: 'Get Context'
|
|
description: 'Learn how to use get_context() to retrieve and format conversation context for LLM integration'
|
|
icon: 'list-timeline'
|
|
---
|
|
|
|
The `get_context()` method is your one-stop-shop for solving memory in LLM applications. It curates the LLM's context window with everything needed for contextually-aware conversations: recent messages, relevant historical context, and conversation summaries. When you add a `peer_target`, it also includes peer cards and Honcho's reasoning about participants.
|
|
|
|
## The Simple Default
|
|
|
|
The simplest implementation is just calling `get_context()` with a `peer_target` - this gives you an optimized blend of everything Honcho knows about your conversation:
|
|
|
|
<CodeGroup>
|
|
```python Python
|
|
from honcho import Honcho
|
|
|
|
honcho = Honcho()
|
|
session = honcho.session("conversation-1")
|
|
user = honcho.peer("user-123")
|
|
assistant = honcho.peer("assistant")
|
|
|
|
# Add some conversation
|
|
session.add_messages([
|
|
user.message("I prefer concise responses"),
|
|
assistant.message("Understood! I'll keep it brief.")
|
|
])
|
|
|
|
# Get context with personalization - the recommended default
|
|
context = session.get_context(peer_target=user)
|
|
messages = context.to_openai(assistant=assistant)
|
|
|
|
# Ready to send to your LLM
|
|
```
|
|
|
|
```typescript TypeScript
|
|
import { Honcho } from "@honcho-ai/sdk";
|
|
|
|
(async () => {
|
|
const honcho = new Honcho({});
|
|
const session = await honcho.session("conversation-1");
|
|
const user = await honcho.peer("user-123");
|
|
const assistant = await honcho.peer("assistant");
|
|
|
|
// Add some conversation
|
|
await session.addMessages([
|
|
user.message("I prefer concise responses"),
|
|
assistant.message("Understood! I'll keep it brief.")
|
|
]);
|
|
|
|
// Get context with personalization - the recommended default
|
|
const context = await session.getContext({ peerTarget: user });
|
|
const messages = context.toOpenAI(assistant);
|
|
|
|
// Ready to send to your LLM
|
|
})();
|
|
```
|
|
</CodeGroup>
|
|
|
|
**What's included:**
|
|
- **Recent messages** from the conversation (token-limited)
|
|
- **Conversation summaries** for older history (automatically generated)
|
|
- **Working representation** of the user - Honcho's conclusions and insights
|
|
- **Peer card** - Structured metadata about the user
|
|
- Formatted for your target LLM (OpenAI, Anthropic, etc.)
|
|
|
|
This is the recommended default for most applications - it gives your LLM everything it needs to provide personalized, context-aware responses.
|
|
|
|
## Advanced Context Control
|
|
|
|
Build on the default by adding more sophisticated features to control what context your LLM receives.
|
|
|
|
### Semantic Retrieval
|
|
|
|
Use `last_user_message` to pull in relevant observations from past conversations:
|
|
|
|
<CodeGroup>
|
|
```python Python
|
|
# User asks about something from weeks ago
|
|
user_question = "What was that Italian restaurant I mentioned?"
|
|
|
|
context = session.get_context(
|
|
peer_target=user,
|
|
last_user_message=user_question,
|
|
tokens=2000
|
|
)
|
|
|
|
# Context includes observations semantically relevant to restaurants
|
|
# Even if the conversation was weeks ago
|
|
```
|
|
|
|
```typescript TypeScript
|
|
(async () => {
|
|
// User asks about something from weeks ago
|
|
const userQuestion = "What was that Italian restaurant I mentioned?";
|
|
|
|
const context = await session.getContext({
|
|
peerTarget: user,
|
|
lastUserMessage: userQuestion,
|
|
tokens: 2000
|
|
});
|
|
|
|
// Context includes observations semantically relevant to restaurants
|
|
// Even if the conversation was weeks ago
|
|
})();
|
|
```
|
|
</CodeGroup>
|
|
|
|
You can pass either a string or a Message object. This is particularly useful for recall-style queries where users reference past conversations.
|
|
|
|
### Perspective-Based Views
|
|
|
|
In multi-agent scenarios, get context from a specific agent's perspective:
|
|
|
|
<CodeGroup>
|
|
```python Python
|
|
# Different agents, different perspectives on the same user
|
|
sales_agent = honcho.peer("sales-bot")
|
|
support_agent = honcho.peer("support-bot")
|
|
|
|
# Sales agent's view - includes conclusions about purchase intent
|
|
sales_context = session.get_context(
|
|
peer_target=user,
|
|
peer_perspective=sales_agent
|
|
)
|
|
|
|
# Support agent's view - includes conclusions about technical needs
|
|
support_context = session.get_context(
|
|
peer_target=user,
|
|
peer_perspective=support_agent
|
|
)
|
|
|
|
# Each agent reasons independently about the user
|
|
```
|
|
|
|
```typescript TypeScript
|
|
(async () => {
|
|
// Different agents, different perspectives on the same user
|
|
const salesAgent = await honcho.peer("sales-bot");
|
|
const supportAgent = await honcho.peer("support-bot");
|
|
|
|
// Sales agent's view - includes conclusions about purchase intent
|
|
const salesContext = await session.getContext({
|
|
peerTarget: user,
|
|
peerPerspective: salesAgent
|
|
});
|
|
|
|
// Support agent's view - includes conclusions about technical needs
|
|
const supportContext = await session.getContext({
|
|
peerTarget: user,
|
|
peerPerspective: supportAgent
|
|
});
|
|
|
|
// Each agent reasons independently about the user
|
|
})();
|
|
```
|
|
</CodeGroup>
|
|
|
|
Use this pattern when you have multiple specialized agents that need different mental models of the same user.
|
|
|
|
### Combining Advanced Features
|
|
|
|
Stack multiple advanced parameters for maximum context awareness:
|
|
|
|
<CodeGroup>
|
|
```python Python
|
|
current_message = "Can you recommend a restaurant for tonight?"
|
|
|
|
context = session.get_context(
|
|
peer_target=user, # User's representation & card
|
|
last_user_message=current_message, # Relevant observations
|
|
peer_perspective=assistant, # Assistant's perspective
|
|
tokens=3000 # Generous limit
|
|
)
|
|
|
|
# Includes: user insights, relevant past observations,
|
|
# assistant's conclusions, recent messages, summaries
|
|
```
|
|
|
|
```typescript TypeScript
|
|
(async () => {
|
|
const currentMessage = "Can you recommend a restaurant for tonight?";
|
|
|
|
const context = await session.getContext({
|
|
peerTarget: user, // User's representation & card
|
|
lastUserMessage: currentMessage, // Relevant observations
|
|
peerPerspective: assistant, // Assistant's perspective
|
|
tokens: 3000 // Generous limit
|
|
});
|
|
|
|
// Includes: user insights, relevant past observations,
|
|
// assistant's conclusions, recent messages, summaries
|
|
})();
|
|
```
|
|
</CodeGroup>
|
|
|
|
## Tuning & Simplification
|
|
|
|
When you need to adjust the default behavior or reduce context complexity.
|
|
|
|
### Adjusting Token Limits
|
|
|
|
Control how much context to include by setting a token budget:
|
|
|
|
<CodeGroup>
|
|
```python Python
|
|
# Adjust context size for your model's limits
|
|
context = session.get_context(peer_target=user, tokens=1500) # Smaller models
|
|
context = session.get_context(peer_target=user, tokens=4000) # Larger models
|
|
```
|
|
|
|
```typescript TypeScript
|
|
(async () => {
|
|
// Adjust context size for your model's limits
|
|
const context = await session.getContext({ peerTarget: user, tokens: 1500 });
|
|
const context = await session.getContext({ peerTarget: user, tokens: 4000 });
|
|
})();
|
|
```
|
|
</CodeGroup>
|
|
|
|
**When to adjust:** When you're hitting model context limits or want more/less conversation history.
|
|
|
|
### Disabling Summaries
|
|
|
|
Turn off summaries to get only raw messages:
|
|
|
|
<CodeGroup>
|
|
```python Python
|
|
# Get more recent messages instead of summaries
|
|
context = session.get_context(peer_target=user, summary=False, tokens=2000)
|
|
```
|
|
|
|
```typescript TypeScript
|
|
(async () => {
|
|
// Get more recent messages instead of summaries
|
|
const context = await session.getContext({
|
|
peerTarget: user,
|
|
summary: false,
|
|
tokens: 2000
|
|
});
|
|
})();
|
|
```
|
|
</CodeGroup>
|
|
|
|
**When to use:** Short conversations where full message history fits in context, or when you prefer verbatim exchanges over summarized history.
|
|
|
|
### Removing Personalization
|
|
|
|
Omit `peer_target` for just messages and summaries without peer reasoning:
|
|
|
|
<CodeGroup>
|
|
```python Python
|
|
# Just messages and summaries, no peer-specific reasoning
|
|
context = session.get_context()
|
|
messages = context.to_openai(assistant=assistant)
|
|
```
|
|
|
|
```typescript TypeScript
|
|
(async () => {
|
|
// Just messages and summaries, no peer-specific reasoning
|
|
const context = await session.getContext();
|
|
const messages = context.toOpenAI(assistant);
|
|
})();
|
|
```
|
|
</CodeGroup>
|
|
|
|
**When to use:** When you explicitly don't want personalization or need to reduce context size. Most applications benefit from including `peer_target`.
|
|
|
|
## Complete Integration Examples
|
|
|
|
### OpenAI Integration
|
|
|
|
<CodeGroup>
|
|
```python Python
|
|
import openai
|
|
from honcho import Honcho
|
|
|
|
honcho = Honcho()
|
|
openai_client = openai.OpenAI()
|
|
|
|
session = honcho.session("chat")
|
|
user = honcho.peer("user-123")
|
|
assistant = honcho.peer("assistant")
|
|
|
|
# Get new user input
|
|
user_input = "Can you help me with Python?"
|
|
session.add_messages([user.message(user_input)])
|
|
|
|
# Get context with personalization
|
|
context = session.get_context(
|
|
peer_target=user,
|
|
last_user_message=user_input,
|
|
tokens=2000
|
|
)
|
|
|
|
# Convert to OpenAI format (specifies which peer is the assistant)
|
|
messages = context.to_openai(assistant=assistant)
|
|
|
|
# Get AI response
|
|
response = openai_client.chat.completions.create(
|
|
model="gpt-4",
|
|
messages=messages
|
|
)
|
|
|
|
# Save response back to Honcho
|
|
ai_response = response.choices[0].message.content
|
|
session.add_messages([assistant.message(ai_response)])
|
|
```
|
|
|
|
```typescript TypeScript
|
|
import OpenAI from 'openai';
|
|
import { Honcho } from "@honcho-ai/sdk";
|
|
|
|
(async () => {
|
|
const honcho = new Honcho({});
|
|
const openai = new OpenAI();
|
|
|
|
const session = await honcho.session("chat");
|
|
const user = await honcho.peer("user-123");
|
|
const assistant = await honcho.peer("assistant");
|
|
|
|
// Get new user input
|
|
const userInput = "Can you help me with Python?";
|
|
await session.addMessages([user.message(userInput)]);
|
|
|
|
// Get context with personalization
|
|
const context = await session.getContext({
|
|
peerTarget: user,
|
|
lastUserMessage: userInput,
|
|
tokens: 2000
|
|
});
|
|
|
|
// Convert to OpenAI format (specifies which peer is the assistant)
|
|
const messages = context.toOpenAI(assistant);
|
|
|
|
// Get AI response
|
|
const response = await openai.chat.completions.create({
|
|
model: "gpt-4",
|
|
messages: messages
|
|
});
|
|
|
|
// Save response back to Honcho
|
|
const aiResponse = response.choices[0].message.content;
|
|
await session.addMessages([assistant.message(aiResponse)]);
|
|
})();
|
|
```
|
|
</CodeGroup>
|
|
|
|
### Anthropic Integration
|
|
|
|
<CodeGroup>
|
|
```python Python
|
|
import anthropic
|
|
from honcho import Honcho
|
|
|
|
honcho = Honcho()
|
|
anthropic_client = anthropic.Anthropic()
|
|
|
|
session = honcho.session("chat")
|
|
user = honcho.peer("user-123")
|
|
assistant = honcho.peer("assistant")
|
|
|
|
user_input = "Tell me about quantum computing"
|
|
session.add_messages([user.message(user_input)])
|
|
|
|
context = session.get_context(peer_target=user)
|
|
|
|
# Convert to Anthropic format
|
|
messages = context.to_anthropic(assistant=assistant)
|
|
|
|
response = anthropic_client.messages.create(
|
|
model="claude-3-5-sonnet-20241022",
|
|
max_tokens=1024,
|
|
messages=messages
|
|
)
|
|
|
|
ai_response = response.content[0].text
|
|
session.add_messages([assistant.message(ai_response)])
|
|
```
|
|
|
|
```typescript TypeScript
|
|
import Anthropic from '@anthropic-ai/sdk';
|
|
import { Honcho } from "@honcho-ai/sdk";
|
|
|
|
(async () => {
|
|
const honcho = new Honcho({});
|
|
const anthropic = new Anthropic();
|
|
|
|
const session = await honcho.session("chat");
|
|
const user = await honcho.peer("user-123");
|
|
const assistant = await honcho.peer("assistant");
|
|
|
|
const userInput = "Tell me about quantum computing";
|
|
await session.addMessages([user.message(userInput)]);
|
|
|
|
const context = await session.getContext({ peerTarget: user });
|
|
|
|
// Convert to Anthropic format
|
|
const messages = context.toAnthropic(assistant);
|
|
|
|
const response = await anthropic.messages.create({
|
|
model: "claude-3-5-sonnet-20241022",
|
|
max_tokens: 1024,
|
|
messages: messages
|
|
});
|
|
|
|
const aiResponse = response.content[0].text;
|
|
await session.addMessages([assistant.message(aiResponse)]);
|
|
})();
|
|
```
|
|
</CodeGroup>
|
|
|
|
### Chat Loop Example
|
|
|
|
<CodeGroup>
|
|
```python Python
|
|
def chat_loop():
|
|
session = honcho.session("chat")
|
|
user = honcho.peer("user")
|
|
assistant = honcho.peer("assistant")
|
|
|
|
while True:
|
|
user_input = input("You: ")
|
|
if user_input.lower() in ['quit', 'exit']:
|
|
break
|
|
|
|
session.add_messages([user.message(user_input)])
|
|
|
|
context = session.get_context(
|
|
peer_target=user,
|
|
last_user_message=user_input,
|
|
tokens=2000
|
|
)
|
|
|
|
response = openai_client.chat.completions.create(
|
|
model="gpt-4",
|
|
messages=context.to_openai(assistant=assistant)
|
|
)
|
|
|
|
ai_response = response.choices[0].message.content
|
|
print(f"Assistant: {ai_response}")
|
|
session.add_messages([assistant.message(ai_response)])
|
|
|
|
chat_loop()
|
|
```
|
|
|
|
```typescript TypeScript
|
|
(async () => {
|
|
async function chatLoop() {
|
|
const session = await honcho.session("chat");
|
|
const user = await honcho.peer("user");
|
|
const assistant = await honcho.peer("assistant");
|
|
|
|
// In a real app, use actual input handling
|
|
const userInputs = ["Hello!", "What's the weather?", "Tell me a joke"];
|
|
|
|
for (const userInput of userInputs) {
|
|
console.log(`You: ${userInput}`);
|
|
await session.addMessages([user.message(userInput)]);
|
|
|
|
const context = await session.getContext({
|
|
peerTarget: user,
|
|
lastUserMessage: userInput,
|
|
tokens: 2000
|
|
});
|
|
|
|
const response = await openai.chat.completions.create({
|
|
model: "gpt-4",
|
|
messages: context.toOpenAI(assistant)
|
|
});
|
|
|
|
const aiResponse = response.choices[0].message.content;
|
|
console.log(`Assistant: ${aiResponse}`);
|
|
await session.addMessages([assistant.message(aiResponse)]);
|
|
}
|
|
}
|
|
|
|
await chatLoop();
|
|
})();
|
|
```
|
|
</CodeGroup>
|
|
|
|
## Best Practices
|
|
|
|
### Start with peer_target
|
|
Use `get_context(peer_target=user)` as your default - it gives your LLM personalized context with minimal code.
|
|
|
|
### Include peer_target for almost all use cases
|
|
Most applications benefit from including `peer_target=user` to get Honcho's reasoning about the user. Only omit it if you explicitly don't want personalization.
|
|
|
|
### Set token limits based on your model
|
|
Match your token limit to your LLM's context window:
|
|
- Small models: `tokens=1500`
|
|
- GPT-4 / Claude: `tokens=3000-4000`
|
|
- Remember: peer cards and representations use some of these tokens
|
|
|
|
### Use last_user_message for recall queries
|
|
When users ask about past conversations ("What did I say about...?"), add `last_user_message` for semantic retrieval.
|
|
|
|
### Perspective requires a target
|
|
`peer_perspective` only works when combined with `peer_target` - you need both to specify whose view of whom.
|
|
|
|
### Cache context objects
|
|
If you need multiple formats (OpenAI and Anthropic), get context once and convert twice:
|
|
```python
|
|
context = session.get_context()
|
|
openai_msgs = context.to_openai(assistant)
|
|
anthropic_msgs = context.to_anthropic(assistant)
|
|
```
|
|
|
|
## Reference
|
|
|
|
### What's Actually Included in Context
|
|
|
|
When you call `get_context()` with default settings:
|
|
|
|
1. **Recent messages** - Token-limited conversation history
|
|
2. **Summaries** (if `summary=True`) - Auto-generated at intervals (every ~20 messages)
|
|
3. **Working representation** (if `peer_target` set) - Honcho's conclusions about the target peer:
|
|
- Observations from interactions
|
|
- Inferred insights and preferences
|
|
- Things explicitly stated with certainty
|
|
4. **Peer card** (if `peer_target` set) - Structured metadata:
|
|
- User preferences and settings
|
|
- Demographics
|
|
- Custom fields
|
|
|
|
### Parameter Reference
|
|
|
|
| Parameter | Type | Default | Description |
|
|
|-----------|------|---------|-------------|
|
|
| `tokens` | int | 1500 | Maximum tokens for the context |
|
|
| `summary` | bool | True | Include auto-generated summaries |
|
|
| `peer_target` | Peer | None | Include this peer's representation & card |
|
|
| `peer_perspective` | Peer | None | Get target peer from this peer's POV (requires `peer_target`) |
|
|
| `last_user_message` | str \| Message | None | Retrieve observations relevant to this message (works best with `peer_target`) |
|
|
|
|
### Format Conversion Methods
|
|
|
|
**`context.to_openai(assistant)`** - Converts to OpenAI format
|
|
- Requires: `assistant` peer to determine role mapping
|
|
- Returns: List of dicts with `{"role": "...", "content": "..."}`
|
|
- Messages from `assistant` → `role: "assistant"`
|
|
- All other peers → `role: "user"`
|
|
|
|
**`context.to_anthropic(assistant)`** - Converts to Anthropic format
|
|
- Requires: `assistant` peer to determine role mapping
|
|
- Returns: List of dicts in Claude's message format
|
|
- Same role mapping as OpenAI
|