RomanNum3ral
/
honcho
mirror of https://github.com/plastic-labs/honcho.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
honcho/docs/v3/contributing/configuration.mdx

789 lines
29 KiB
Plaintext
Raw Blame History

---
title: "Configuration Guide"
description: "Complete reference for configuring Honcho providers, features, and infrastructure"
icon: "gear"
---
<Info>
Most users only need the setup from the [Self-Hosting Guide](./self-hosting#llm-setup). This page is the full reference for customizing providers, tuning features, and hardening your deployment.
</Info>
Honcho loads configuration in this priority order (highest wins):
1. **Environment variables** (always take precedence)
2. **`.env` file**
3. **`config.toml` file**
4. **Built-in defaults**
Use `.env` for secrets and overrides, `config.toml` for base settings. Or use environment variables exclusively — whatever fits your deployment. Copy the examples to get started:
```bash
cp .env.template .env
cp config.toml.example config.toml
```
### Environment Variable Naming
All config values map to environment variables:
- `{SECTION}_{KEY}` for top-level section settings (e.g., `DB_CONNECTION_URI` → `[db].CONNECTION_URI`)
- `{KEY}` for app-level settings (e.g., `LOG_LEVEL` → `[app].LOG_LEVEL`)
- Use `__` inside `{KEY}` for nested settings (e.g., `DIALECTIC_LEVELS__minimal__MODEL_CONFIG__TRANSPORT`, `DERIVER_MODEL_CONFIG__OVERRIDES__BASE_URL`)
## LLM Configuration
The [Self-Hosting Guide](./self-hosting#llm-setup) covers the basic setup: either the built-in OpenAI defaults or one OpenAI-compatible endpoint/model for all features. This section covers recommended model tiers, using multiple providers, and per-feature tuning.
<Note>
All Honcho agents (deriver, dialectic, dream) require tool calling. Your models must support the OpenAI tool calling format.
</Note>
### Choosing Models
Model choice matters more for tool-use reliability than raw intelligence:
| Tier | Example models | Use case | Notes |
|---|---|---|---|
| **Light** | Gemini 2.5 Flash, GLM-4.7-Flash | Deriver, summary, dialectic minimal/low | High throughput, cheap, reliable tool use |
| **Medium** | Claude Haiku 4.5, Grok 4.1 Fast | Dialectic medium/high | Good reasoning + tool use balance |
| **Heavy** | Claude Sonnet 4, GLM-5 | Dream, dialectic max | Best quality for rare/complex tasks |
You can mix providers freely — for example, use Gemini for the deriver and Claude for dreaming.
### Provider Types
| Transport value | What it connects to | API key env var |
|---|---|---|
| `openai` | OpenAI or any OpenAI-compatible endpoint (OpenRouter, Together, Fireworks, LiteLLM, vLLM, Ollama) | `LLM_OPENAI_API_KEY` |
| `anthropic` | Anthropic Claude (direct) | `LLM_ANTHROPIC_API_KEY` |
| `gemini` | Google Gemini (direct) | `LLM_GEMINI_API_KEY` |
For OpenAI-compatible proxies (OpenRouter, vLLM, Ollama, etc.), use `transport = "openai"` and set `MODEL_CONFIG__OVERRIDES__BASE_URL` on each feature to point at your endpoint.
<Note>
Some OpenAI-compatible providers don't support OpenAI Structured Outputs (`json_schema`). Set `DERIVER_MODEL_CONFIG__STRUCTURED_OUTPUT_MODE=json_object` to request loose JSON mode and inject the schema into the prompt instead.
This setting only applies to the **deriver** on the **`openai`** transport — it is the only feature that uses structured output. The dialectic, summarizer, and dreamer don't request structured output, so the setting has no effect there, and the anthropic/gemini transports reject it.
</Note>
### Tiered Model Setup
Once you're past initial setup, you can assign different models per feature for better cost/quality tradeoffs. This example uses OpenRouter with light/medium/heavy tiers:
```bash
LLM_OPENAI_API_KEY=sk-or-v1-...
# All features route through OpenRouter via overrides.base_url
# (You can set this on each feature's MODEL_CONFIG)
# Light tier — high throughput, cheap
DERIVER_MODEL_CONFIG__TRANSPORT=openai
DERIVER_MODEL_CONFIG__MODEL=google/gemini-2.5-flash-lite
DERIVER_MODEL_CONFIG__OVERRIDES__BASE_URL=https://openrouter.ai/api/v1
SUMMARY_MODEL_CONFIG__TRANSPORT=openai
SUMMARY_MODEL_CONFIG__MODEL=google/gemini-2.5-flash
DIALECTIC_LEVELS__minimal__MODEL_CONFIG__TRANSPORT=openai
DIALECTIC_LEVELS__minimal__MODEL_CONFIG__MODEL=google/gemini-2.5-flash-lite
DIALECTIC_LEVELS__low__MODEL_CONFIG__TRANSPORT=openai
DIALECTIC_LEVELS__low__MODEL_CONFIG__MODEL=google/gemini-2.5-flash-lite
# Medium tier — better reasoning
DIALECTIC_LEVELS__medium__MODEL_CONFIG__TRANSPORT=openai
DIALECTIC_LEVELS__medium__MODEL_CONFIG__MODEL=anthropic/claude-haiku-4-5
DIALECTIC_LEVELS__high__MODEL_CONFIG__TRANSPORT=openai
DIALECTIC_LEVELS__high__MODEL_CONFIG__MODEL=anthropic/claude-haiku-4-5
DIALECTIC_LEVELS__max__MODEL_CONFIG__TRANSPORT=openai
DIALECTIC_LEVELS__max__MODEL_CONFIG__MODEL=anthropic/claude-haiku-4-5
# Heavy tier — best quality for complex tasks
DREAM_DEDUCTION_MODEL_CONFIG__TRANSPORT=openai
DREAM_DEDUCTION_MODEL_CONFIG__MODEL=anthropic/claude-haiku-4-5
DREAM_INDUCTION_MODEL_CONFIG__TRANSPORT=openai
DREAM_INDUCTION_MODEL_CONFIG__MODEL=anthropic/claude-haiku-4-5
```
### Direct Vendor Keys
Instead of an OpenAI-compatible proxy, you can use vendor APIs directly. Each transport picks up its own `LLM_{TRANSPORT}_API_KEY`.
If you keep the built-in defaults, only `LLM_OPENAI_API_KEY` is required:
```bash
LLM_OPENAI_API_KEY=...
# Built-in model defaults
# - deriver: openai / gpt-5.4-mini
# - dialectic (all levels): openai / gpt-5.4-mini
# - summary: openai / gpt-5.4-mini
# - dream specialists: openai / gpt-5.4-mini
# - embeddings: openai / text-embedding-3-small
```
To use Gemini or Anthropic directly, override the features you want to move:
```bash
LLM_GEMINI_API_KEY=...
DERIVER_MODEL_CONFIG__TRANSPORT=gemini
DERIVER_MODEL_CONFIG__MODEL=gemini-2.5-flash
LLM_ANTHROPIC_API_KEY=...
DREAM_DEDUCTION_MODEL_CONFIG__TRANSPORT=anthropic
DREAM_DEDUCTION_MODEL_CONFIG__MODEL=claude-haiku-4-5
```
### Self-Hosted (vLLM / Ollama)
Use `transport = "openai"` and set `MODEL_CONFIG__OVERRIDES__BASE_URL` on each feature:
```bash
# vLLM
LLM_OPENAI_API_KEY=not-needed
DERIVER_MODEL_CONFIG__TRANSPORT=openai
DERIVER_MODEL_CONFIG__MODEL=your-model-name
DERIVER_MODEL_CONFIG__OVERRIDES__BASE_URL=http://localhost:8000/v1
# Ollama
LLM_OPENAI_API_KEY=ollama
DERIVER_MODEL_CONFIG__TRANSPORT=openai
DERIVER_MODEL_CONFIG__MODEL=llama3.3:70b
DERIVER_MODEL_CONFIG__OVERRIDES__BASE_URL=http://localhost:11434/v1
```
Set `MODEL_CONFIG__TRANSPORT`, `MODEL_CONFIG__MODEL`, and `MODEL_CONFIG__OVERRIDES__BASE_URL` for each feature the same way.
The same overrides are available in `config.toml`:
```toml
[deriver.model_config]
transport = "openai"
model = "my-local-model"
[deriver.model_config.overrides]
base_url = "http://localhost:8000/v1"
api_key_env = "DERIVER_LOCAL_API_KEY"
```
### Thinking Budget
Built-in defaults do not set `MODEL_CONFIG__THINKING_BUDGET_TOKENS` or `MODEL_CONFIG__THINKING_EFFORT`. Add one only when your chosen model supports it.
Use `MODEL_CONFIG__THINKING_EFFORT` for OpenAI reasoning models:
```bash
DERIVER_MODEL_CONFIG__THINKING_EFFORT=minimal
DIALECTIC_LEVELS__max__MODEL_CONFIG__THINKING_EFFORT=medium
```
Use `MODEL_CONFIG__THINKING_BUDGET_TOKENS` for Anthropic and Gemini models. Set it to `0` or omit it for providers that don't support extended thinking:
```bash
SUMMARY_MODEL_CONFIG__THINKING_BUDGET_TOKENS=1024
DREAM_DEDUCTION_MODEL_CONFIG__THINKING_BUDGET_TOKENS=1024
```
### Provider-Specific Parameters
Each model config supports an `overrides.provider_params` dict for passing arbitrary parameters to the underlying provider SDK. Use this for vendor-specific features that aren't part of the standard config:
```toml
[deriver.model_config.overrides.provider_params]
# These are passed directly to the provider SDK
verbosity = "low"
# Per-request timeout in seconds; useful for queued workers that can wait longer
timeout = 3600.0
```
Because provider params live on each model config, background workers such as
the Deriver and Dreamer can use longer request timeouts while synchronous
chat paths keep tighter defaults.
`timeout` gotchas:
- The value is validated **at config load**: it must coerce to a positive,
finite number of seconds (numbers or numeric strings like `"3600"`), or the
process refuses to start with an error naming the offending config path.
This applies to both the primary model config and its `fallback.overrides`.
- The unit is always **seconds**, regardless of transport. OpenAI and
Anthropic receive it as the SDK's `timeout` kwarg; Gemini has no such
kwarg, so Honcho converts it to milliseconds on `http_options.timeout`.
- When unset, nothing is forwarded and each SDK's default applies — adding
this key is opt-in and changes no existing behavior.
- A too-tight timeout doesn't fail once: the aborted request goes through the
normal retry/fallback chain before the caller sees an error, so the
observed latency is several multiples of the timeout.
#### Transport passthrough keys
Three keys inside `provider_params` are recognized as request-level escape hatches and forwarded to the underlying transport. Where a transport actually validates and merges one of these keys, its value must be a mapping — a non-mapping value raises a configuration error (see the per-transport behavior below; a key a transport ignores is not validated):
- **`extra_body`** — merged into the request body
- **`extra_headers`** — extra HTTP headers
- **`extra_query`** — extra URL query parameters
How each transport forwards them differs:
- **OpenAI and Anthropic** forward all three as identically-named SDK kwargs (`extra_body`, `extra_headers`, `extra_query`).
- **Gemini** has no SDK kwargs for these. It merges `extra_body` into the `GenerateContentConfig` dict and folds `extra_headers` into `http_options.headers`; `extra_query` is **unsupported and silently ignored**.
The merge is shallow and **operator-wins**: if Honcho and your config both set the same top-level key inside `extra_body`, your value replaces Honcho's. You are responsible for choosing a coherent combination — e.g. unset `thinking_budget_tokens` when supplying an `extra_body.thinking` for Anthropic-via-proxy, since Honcho will not translate between the two shapes.
Because Gemini merges `extra_body` directly into `GenerateContentConfig` (rather than a nested request body), an `extra_body` written for OpenAI/Anthropic generally will not transfer to Gemini unchanged — and a key collision there can overwrite a field Honcho manages (`thinking_config`, `response_schema`, `tools`, …).
```toml
# Example: route an OpenAI-compatible proxy and tag requests for tracing
[deriver.model_config.overrides.provider_params.extra_headers]
X-Proxy-Route = "vertex"
[deriver.model_config.overrides.provider_params.extra_body]
# Provider-native body fields the standard config doesn't expose
anthropic_beta = ["context-1m-2025-01-15"]
```
### Changing Transport
When changing a feature's `transport`, always specify `model` explicitly. Partial overrides that change transport without model will keep the previous model name, which may not be valid for the new provider.
### General LLM Settings
```bash
LLM_DEFAULT_MAX_TOKENS=2500
# Tool output limits (to prevent token explosion)
LLM_MAX_TOOL_OUTPUT_CHARS=10000 # ~2500 tokens at 4 chars/token
LLM_MAX_MESSAGE_CONTENT_CHARS=2000 # Max chars per message in tool results
```
### Embedding Configuration
Embeddings use their own nested model config, separate from the main text-generation LLM settings.
```bash
# Embedding vector settings
EMBEDDING_VECTOR_DIMENSIONS=1536
EMBEDDING_MAX_INPUT_TOKENS=8192
EMBEDDING_MAX_TOKENS_PER_REQUEST=300000
# Embedding transport/model selection
EMBEDDING_MODEL_CONFIG__TRANSPORT=openai # openai, gemini
EMBEDDING_MODEL_CONFIG__MODEL=text-embedding-3-small
EMBEDDING_MODEL_CONFIG__MAX_BATCH_SIZE=10 # optional per-request input cap
EMBEDDING_MODEL_CONFIG__TIMEOUT=90.0 # optional client HTTP timeout (seconds)
# Optional endpoint overrides
EMBEDDING_MODEL_CONFIG__OVERRIDES__BASE_URL=http://localhost:8000/v1
EMBEDDING_MODEL_CONFIG__OVERRIDES__API_KEY_ENV=EMBEDDING_CUSTOM_API_KEY
```
`EMBEDDING_MODEL_CONFIG__MAX_BATCH_SIZE` defaults to 2048 for OpenAI. For
Gemini the client applies a conservative default of 100 — Gemini does not
document a per-request limit. Set it when an OpenAI-compatible embedding
provider accepts fewer inputs per request, such as DashScope
`text-embedding-v4` with a limit of 10.
`EMBEDDING_MODEL_CONFIG__TIMEOUT` is an optional client HTTP timeout in
seconds. OpenAI-compatible transports receive it as the SDK `timeout` kwarg
(omitted when unset, so the SDK default applies). Gemini converts it to
milliseconds on `http_options.timeout`, and keeps its existing 10-minute
default when unset. The value is validated at config load the same way as
LLM `provider_params.timeout` (positive, finite number of seconds).
Forwarding `dimensions=` to OpenAI-compatible providers is controlled by `EMBEDDING_MODEL_CONFIG__DIMENSIONS_MODE`:
- `auto` (default): forwards `dimensions=` when **the operator has explicitly set `EMBEDDING_VECTOR_DIMENSIONS`** — provenance, not value — and the configured model is not on the known-rejecting list (currently `text-embedding-ada-002`). Explicit `EMBEDDING_VECTOR_DIMENSIONS=1536` *does* trigger the forward; this is how `text-embedding-3-large` truncation to 1536 is expressed. Deployments that leave the setting unset get their existing behavior (`dimensions=` is not forwarded).
- `always`: always forward, regardless of whether `EMBEDDING_VECTOR_DIMENSIONS` was set. Use for OpenAI-compatible self-hosted providers that require it. Do not pick `always` *just* for same-as-default truncation — `auto` handles that case correctly as long as you set `EMBEDDING_VECTOR_DIMENSIONS=1536` explicitly in your environment. `always` is the right answer when your config layer might strip explicit "default-valued" envs, or when you want defense-in-depth.
- `never`: never forward. Explicit opt-out for providers that reject the parameter (e.g. `text-embedding-ada-002` if it slips past the known-rejecting allowlist).
The embedding wire format is controlled by `EMBEDDING_MODEL_CONFIG__ENCODING_FORMAT_MODE`. The `openai` SDK sends `encoding_format=base64` when the caller passes nothing, and some OpenAI-compatible providers answer that with an error or with empty data, so Honcho always sends the format explicitly:
- `auto` (default): `base64` when no `EMBEDDING_MODEL_CONFIG__OVERRIDES__BASE_URL` is set or it points at `api.openai.com`, `float` otherwise. base64 is roughly 3.6x smaller on the wire than JSON floats, so this keeps the compact format for real OpenAI and only pays the larger payload where compatibility requires it.
- `float`: always request floats. Use for a provider that rejects base64 but sits behind a host `auto` reads as OpenAI-compatible-but-capable.
- `base64`: always request base64. Use for a proxy that fronts real OpenAI (Azure OpenAI, LiteLLM) where `auto` cannot tell from the host that base64 is safe, and you want the smaller payload.
Both formats decode to identical vectors, so switching modes does not require re-embedding.
#### Bootstrapping non-default dimensions
`EMBEDDING_VECTOR_DIMENSIONS` is treated as immutable for the life of a deployment. The pgvector schema is dim-pinned by Alembic at `1536` by default; if you want a different dim, you must ALTER the empty columns once at bootstrap time.
Install order for a non-default dim:
```bash
# 1. Apply migrations (creates default vector(1536) schema)
alembic upgrade head
# 2. Set the dim you want
export EMBEDDING_VECTOR_DIMENSIONS=768
# 3. ALTER the empty columns to the target dim
uv run python scripts/configure_embeddings.py --dry-run # preview
uv run python scripts/configure_embeddings.py --yes # apply
# 4. Start API and deriver — both run the startup validator and refuse
# to serve traffic if the schema and EMBEDDING_VECTOR_DIMENSIONS disagree.
```
Existing deployments at 1536 with `text-embedding-3-small` need no action — step 3 detects matching dims and skips.
The script refuses to ALTER tables that already contain non-null embeddings. To switch dim or model on a populated deployment, stand up a new deployment at the new configuration and migrate data out of band; there is no in-place re-embedding affordance. See [Changing Embeddings](./changing-embeddings) for the destroy + rebuild recipe and the same-dim model-swap caveat.
External vector stores (Turbopuffer, LanceDB) do not need bootstrap setup. Namespaces are per-workspace and lazy-created on first write at whatever dim the embedding client returns. Use `--report` to inventory the existing namespaces against the configured dim:
```bash
uv run python scripts/configure_embeddings.py --report
```
The startup validator at `src/startup/embedding_validator.py` enforces the dim invariant at boot for both the API (`src/main.py` lifespan) and the deriver (`src/deriver/__main__.py`). A mismatch crashes the process with an actionable error before any HTTP route is served or any queue task is processed.
`VECTOR_STORE_DIMENSIONS` is **deprecated**. `EMBEDDING_VECTOR_DIMENSIONS` is the single source of truth; setting `VECTOR_STORE_DIMENSIONS` explicitly emits a startup warning and is otherwise ignored. The field will be removed in a future release; drop it from your `.env` to silence the warning.
The `VECTOR_STORE_MIGRATED` flag still exists and still controls dual-write / cutover semantics for legacy tenants moving between storage backends (pgvector ↔ turbopuffer ↔ lancedb). It is unrelated to dimension configuration after this release.
### Feature-Specific Model Configuration
Each feature can use a different provider and model. Below are all the tuning knobs.
**Dialectic API:**
The Dialectic API provides theory-of-mind informed responses. It uses a tiered reasoning system with five levels:
```bash
# Global dialectic settings
DIALECTIC_MAX_OUTPUT_TOKENS=8192
DIALECTIC_MAX_INPUT_TOKENS=100000
DIALECTIC_HISTORY_TOKEN_LIMIT=8192
DIALECTIC_SESSION_HISTORY_MAX_TOKENS=4096
```
**Per-Level Configuration:**
Each reasoning level has its own provider, model, and settings:
```toml
# config.toml example
[dialectic.levels.minimal]
MAX_TOOL_ITERATIONS = 1
MAX_OUTPUT_TOKENS = 250
TOOL_CHOICE = "any"
[dialectic.levels.minimal.model_config]
transport = "openai"
model = "gpt-5.4-mini"
[dialectic.levels.low]
MAX_TOOL_ITERATIONS = 5
TOOL_CHOICE = "any"
[dialectic.levels.low.model_config]
transport = "openai"
model = "gpt-5.4-mini"
[dialectic.levels.medium]
MAX_TOOL_ITERATIONS = 2
[dialectic.levels.medium.model_config]
transport = "openai"
model = "gpt-5.4-mini"
[dialectic.levels.high]
MAX_TOOL_ITERATIONS = 4
[dialectic.levels.high.model_config]
transport = "openai"
model = "gpt-5.4-mini"
[dialectic.levels.max]
MAX_TOOL_ITERATIONS = 10
[dialectic.levels.max.model_config]
transport = "openai"
model = "gpt-5.4-mini"
```
Environment variables for nested levels use double underscores:
```bash
DIALECTIC_LEVELS__minimal__MODEL_CONFIG__TRANSPORT=openai
DIALECTIC_LEVELS__minimal__MODEL_CONFIG__MODEL=gpt-5.4-mini
DIALECTIC_LEVELS__minimal__MAX_TOOL_ITERATIONS=1
DIALECTIC_LEVELS__minimal__MAX_OUTPUT_TOKENS=250
DIALECTIC_LEVELS__minimal__TOOL_CHOICE=any
```
**Deriver (Theory of Mind):**
The Deriver extracts facts from messages and builds theory-of-mind representations of peers.
```bash
DERIVER_ENABLED=true
# LLM settings
DERIVER_MODEL_CONFIG__TRANSPORT=openai
DERIVER_MODEL_CONFIG__MODEL=gpt-5.4-mini
DERIVER_MAX_INPUT_TOKENS=25000
DERIVER_MAX_CUSTOM_INSTRUCTIONS_TOKENS=2000
# DERIVER_MODEL_CONFIG__THINKING_EFFORT=minimal
# DERIVER_MODEL_CONFIG__THINKING_BUDGET_TOKENS=1024
# DERIVER_MODEL_CONFIG__TEMPERATURE=0.7 # Optional temperature override
# DERIVER_MODEL_CONFIG__STRUCTURED_OUTPUT_MODE=json_object # for providers without json_schema support
# Backup model (optional)
# DERIVER_MODEL_CONFIG__FALLBACK__MODEL=claude-haiku-4-5
# DERIVER_MODEL_CONFIG__FALLBACK__TRANSPORT=anthropic
# Worker settings
DERIVER_WORKERS=1 # Increase for higher throughput
DERIVER_POLLING_SLEEP_INTERVAL_SECONDS=1.0
# Adaptive polling: when idle/erroring, the sleep interval grows from the base
# toward DERIVER_POLLING_SLEEP_MAX_INTERVAL_SECONDS by the multiplier each cycle,
# then snaps back to base when work is found. Cuts steady-state query load.
DERIVER_POLLING_BACKOFF_ENABLED=true
DERIVER_POLLING_SLEEP_MAX_INTERVAL_SECONDS=30.0
DERIVER_POLLING_BACKOFF_MULTIPLIER=2.0
# Jitter so instances that start together don't poll in lockstep. Startup: sleep
# a random delay in [0, value] before the first poll (0.0 disables). Per-cycle:
# multiply every poll sleep by a random factor in [1-ratio, 1+ratio] (0.0 disables).
DERIVER_POLLING_STARTUP_JITTER_SECONDS=30.0
DERIVER_POLLING_JITTER_RATIO=0.5
DERIVER_STALE_SESSION_TIMEOUT_MINUTES=5
# Queue management
DERIVER_QUEUE_ERROR_RETENTION_SECONDS=2592000 # 30 days
# Observation settings
DERIVER_DEDUPLICATE=true
DERIVER_LOG_OBSERVATIONS=false
DERIVER_WORKING_REPRESENTATION_MAX_OBSERVATIONS=100
DERIVER_REPRESENTATION_BATCH_WORK_UNIT_TARGET_TOKENS=512
DERIVER_REPRESENTATION_BATCH_TARGET_INPUT_TOKENS=1024
DERIVER_REPRESENTATION_BATCH_MAX_AGE_SECONDS=1800
```
**Peer Card:**
```bash
PEER_CARD_ENABLED=true
```
**Summary Generation:**
Session summaries provide compressed context for long conversations — short summaries (frequent) and long summaries (comprehensive).
```bash
SUMMARY_ENABLED=true
SUMMARY_MODEL_CONFIG__TRANSPORT=openai
SUMMARY_MODEL_CONFIG__MODEL=gpt-5.4-mini
SUMMARY_MAX_TOKENS_SHORT=1000
SUMMARY_MAX_TOKENS_LONG=4000
# SUMMARY_MODEL_CONFIG__THINKING_EFFORT=minimal
# SUMMARY_MODEL_CONFIG__THINKING_BUDGET_TOKENS=1024
SUMMARY_MESSAGES_PER_SHORT_SUMMARY=20
SUMMARY_MESSAGES_PER_LONG_SUMMARY=60
```
**Dream Processing:**
Dream processing consolidates and refines peer representations during idle periods.
```bash
DREAM_ENABLED=true
DREAM_DOCUMENT_THRESHOLD=50
DREAM_IDLE_TIMEOUT_MINUTES=60
DREAM_MIN_HOURS_BETWEEN_DREAMS=8
DREAM_ENABLED_TYPES=["omni"]
DREAM_MAX_TOOL_ITERATIONS=20
DREAM_HISTORY_TOKEN_LIMIT=16384
# Specialist model configs (each is independent)
DREAM_DEDUCTION_MODEL_CONFIG__TRANSPORT=openai
DREAM_DEDUCTION_MODEL_CONFIG__MODEL=gpt-5.4-mini
DREAM_INDUCTION_MODEL_CONFIG__TRANSPORT=openai
DREAM_INDUCTION_MODEL_CONFIG__MODEL=gpt-5.4-mini
```
**Surprisal-Based Sampling (Advanced):**
Optional subsystem for identifying unusual observations during dreaming:
```bash
DREAM_SURPRISAL__ENABLED=false
DREAM_SURPRISAL__TREE_TYPE=kdtree
DREAM_SURPRISAL__TREE_K=5
DREAM_SURPRISAL__SAMPLING_STRATEGY=recent
DREAM_SURPRISAL__SAMPLE_SIZE=200
DREAM_SURPRISAL__TOP_PERCENT_SURPRISAL=0.10
DREAM_SURPRISAL__MIN_HIGH_SURPRISAL_FOR_REPLACE=10
DREAM_SURPRISAL__INCLUDE_LEVELS=["explicit", "deductive"]
```
## Core Configuration
### Application Settings
```bash
LOG_LEVEL=INFO # DEBUG, INFO, WARNING, ERROR, CRITICAL
SESSION_OBSERVERS_LIMIT=10
GET_CONTEXT_MAX_TOKENS=100000
REPRESENTATION_INJECTION_ORDER=explicit,deductive,inductive,contradiction
MAX_MESSAGE_SIZE=25000
MAX_FILE_SIZE=5242880 # 5MB
EMBED_MESSAGES=true
EMBEDDING_MAX_INPUT_TOKENS=8192
EMBEDDING_MAX_TOKENS_PER_REQUEST=300000
NAMESPACE=honcho
```
`REPRESENTATION_INJECTION_ORDER` controls the order of the explicit, deductive,
inductive, and contradiction blocks injected into prompts and returned in rendered
representations. It must contain each section exactly once. The default preserves
the existing order; to put higher-level patterns first, use:
```bash
REPRESENTATION_INJECTION_ORDER=inductive,explicit,deductive,contradiction
```
**Optional Integrations:**
```bash
LANGFUSE_HOST=https://cloud.langfuse.com
LANGFUSE_PUBLIC_KEY=your-langfuse-public-key
COLLECT_METRICS_LOCAL=false
LOCAL_METRICS_FILE=metrics.jsonl
REASONING_TRACES_FILE=traces.jsonl
```
### Database
```bash
# Connection (required)
DB_CONNECTION_URI=postgresql+psycopg://postgres:postgres@localhost:5432/postgres
# Pool settings
DB_SCHEMA=public
DB_POOL_PRE_PING=true
DB_POOL_SIZE=10
DB_MAX_OVERFLOW=20
DB_POOL_TIMEOUT=5
DB_POOL_RECYCLE=300
DB_POOL_USE_LIFO=true
DB_SQL_DEBUG=false
# Per-connection establish timeout (seconds) so a single connection attempt
# fails fast instead of hanging when the server/pooler is unreachable.
DB_CONNECT_TIMEOUT_SECONDS=2
```
### Authentication
```bash
AUTH_USE_AUTH=false # Set to true to require JWT tokens
AUTH_JWT_SECRET=your-super-secret-jwt-key # Required when auth is enabled
```
Generate a secret: `python scripts/generate_jwt_secret.py`
### Cache (Redis)
Redis caching is optional. Honcho works without it but benefits from caching in high-traffic scenarios.
```bash
CACHE_ENABLED=false
CACHE_URL=redis://localhost:6379/0?suppress=true
CACHE_NAMESPACE=honcho
CACHE_DEFAULT_TTL_SECONDS=300
CACHE_DEFAULT_LOCK_TTL_SECONDS=5 # Cache stampede prevention
```
### Webhooks
```bash
WEBHOOK_SECRET=your-webhook-signing-secret
WEBHOOK_MAX_WORKSPACE_LIMIT=10
```
### Vector Store
```bash
VECTOR_STORE_TYPE=pgvector # Options: pgvector, turbopuffer, lancedb
VECTOR_STORE_MIGRATED=false
VECTOR_STORE_NAMESPACE=honcho
# Embedding dim is configured via EMBEDDING_VECTOR_DIMENSIONS — see the
# Embedding Configuration section. VECTOR_STORE_DIMENSIONS is deprecated.
# Turbopuffer-specific
VECTOR_STORE_TURBOPUFFER_API_KEY=your-turbopuffer-api-key
VECTOR_STORE_TURBOPUFFER_REGION=us-east-1
# LanceDB-specific
VECTOR_STORE_LANCEDB_PATH=./lancedb_data
```
LanceDB is an optional extra and is not included in the default Docker image. Build with `docker build --build-arg INSTALL_LANCEDB=true .` (or `INSTALL_LANCEDB=true docker compose up -d --build`), or run `uv sync --extra lancedb` for manual setups. Note the extra is unavailable on Intel macOS.
## Monitoring
### Prometheus Metrics
Honcho exposes `/metrics` endpoints for scraping:
- **API process**: Port 8000
- **Deriver process**: Port 9090
```bash
METRICS_ENABLED=false
METRICS_NAMESPACE=honcho
```
### CloudEvents Telemetry
```bash
TELEMETRY_ENABLED=false
TELEMETRY_ENDPOINT=https://telemetry.honcho.dev/v1/events
TELEMETRY_HEADERS='{"Authorization": "Bearer your-token"}'
TELEMETRY_BATCH_SIZE=100
TELEMETRY_FLUSH_INTERVAL_SECONDS=1.0
TELEMETRY_MAX_RETRIES=3
TELEMETRY_MAX_BUFFER_SIZE=10000
```
### Sentry
```bash
SENTRY_ENABLED=false
SENTRY_DSN=https://your-sentry-dsn@sentry.io/project-id
SENTRY_ENVIRONMENT=production
SENTRY_TRACES_SAMPLE_RATE=0.1
SENTRY_PROFILES_SAMPLE_RATE=0.1
```
## Reference config.toml
A complete config.toml with all defaults. Copy and modify what you need:
```toml
[app]
LOG_LEVEL = "INFO"
SESSION_OBSERVERS_LIMIT = 10
REPRESENTATION_INJECTION_ORDER = ["explicit", "deductive", "inductive", "contradiction"]
EMBED_MESSAGES = true
NAMESPACE = "honcho"
[db]
CONNECTION_URI = "postgresql+psycopg://postgres:postgres@localhost:5432/postgres"
POOL_SIZE = 10
MAX_OVERFLOW = 20
[auth]
USE_AUTH = false
[cache]
ENABLED = false
URL = "redis://localhost:6379/0?suppress=true"
DEFAULT_TTL_SECONDS = 300
[deriver]
ENABLED = true
WORKERS = 1
[deriver.model_config]
transport = "openai"
model = "gpt-5.4-mini"
[peer_card]
ENABLED = true
[dialectic]
MAX_OUTPUT_TOKENS = 8192
[dialectic.levels.minimal]
MAX_TOOL_ITERATIONS = 1
MAX_OUTPUT_TOKENS = 250
TOOL_CHOICE = "any"
[dialectic.levels.minimal.model_config]
transport = "openai"
model = "gpt-5.4-mini"
[dialectic.levels.low]
MAX_TOOL_ITERATIONS = 5
TOOL_CHOICE = "any"
[dialectic.levels.low.model_config]
transport = "openai"
model = "gpt-5.4-mini"
[dialectic.levels.medium]
MAX_TOOL_ITERATIONS = 2
[dialectic.levels.medium.model_config]
transport = "openai"
model = "gpt-5.4-mini"
[dialectic.levels.high]
MAX_TOOL_ITERATIONS = 4
[dialectic.levels.high.model_config]
transport = "openai"
model = "gpt-5.4-mini"
[dialectic.levels.max]
MAX_TOOL_ITERATIONS = 10
[dialectic.levels.max.model_config]
transport = "openai"
model = "gpt-5.4-mini"
[summary]
ENABLED = true
MAX_TOKENS_SHORT = 1000
MAX_TOKENS_LONG = 4000
[summary.model_config]
transport = "openai"
model = "gpt-5.4-mini"
[dream]
ENABLED = true
[dream.deduction_model_config]
transport = "openai"
model = "gpt-5.4-mini"
[dream.induction_model_config]
transport = "openai"
model = "gpt-5.4-mini"
[webhook]
MAX_WORKSPACE_LIMIT = 10
[metrics]
ENABLED = false
[telemetry]
ENABLED = false
[vector_store]
TYPE = "pgvector"
[sentry]
ENABLED = false
```
## Database Migrations
```bash
uv run alembic current # Check status
uv run alembic upgrade head # Upgrade to latest
uv run alembic downgrade <rev> # Downgrade to specific revision
uv run alembic revision --autogenerate -m "Description" # Create new migration
```
## Troubleshooting
1. **Database connection errors** — Ensure `DB_CONNECTION_URI` uses `postgresql+psycopg://` prefix. Verify database is running and pgvector extension is installed.
2. **Authentication issues** — Generate and set `AUTH_JWT_SECRET` when `AUTH_USE_AUTH=true`. Use `python scripts/generate_jwt_secret.py`.
3. **LLM provider errors** — Verify API keys are set. Check model names match your provider's format. Ensure models support tool calling.
4. **Deriver not processing** — Check logs. Increase `DERIVER_WORKERS` for throughput. Verify database and LLM connectivity.
5. **Dialectic level issues** — Unset level fields inherit from the built-in defaults. For Anthropic, `THINKING_BUDGET_TOKENS` must be >= 1024 when enabled. For providers without budgeted thinking, omit it or set it to `0`. `MAX_OUTPUT_TOKENS` must exceed `THINKING_BUDGET_TOKENS`.
6. **Vector store issues** — For Turbopuffer, set the API key. Check that `EMBEDDING_VECTOR_DIMENSIONS` matches your embedding model — the startup validator will refuse to boot on a mismatch.
Reference in New Issue View Git Blame Copy Permalink