fix: Inconsistencies in Docs, health endpoint, troubleshooting guide

This commit is contained in:
Vineeth Voruganti 2026-03-28 18:49:48 -04:00
parent cc3483bfaa
commit 3b0a71b4bd
10 changed files with 327 additions and 66 deletions

View File

@ -32,7 +32,7 @@ LOG_LEVEL=INFO
# =============================================================================
# Connection URI for PostgreSQL database with pgvector support
# Must use postgresql+psycopg prefix for SQLAlchemy compatibility
DB_CONNECTION_URI=postgresql+psycopg://postgres:postgres@localhost:5432/postgres
DB_CONNECTION_URI=postgresql+psycopg://postgres:postgres@localhost:5432/honcho
# Optional database settings
# DB_SCHEMA=public

View File

@ -106,7 +106,7 @@ git commit -m "docs(readme): update installation instructions"
### Python Code Style
- Follow [PEP 8](https://www.python.org/dev/peps/pep-0008/) style guidelines
- Use [Black](https://black.readthedocs.io/) for code formatting (we may add this to CI in the future)
- Use [ruff](https://docs.astral.sh/ruff/) for linting and code formatting
- Use type hints where possible
- Write docstrings for functions and classes using Google style docstrings

View File

@ -51,6 +51,6 @@ USER app
EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/openapi.json')" || exit 1
CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')" || exit 1
CMD ["fastapi", "run", "--host", "0.0.0.0", "src/main.py"]

View File

@ -162,8 +162,8 @@ Server.
Honcho is developed using [python](https://www.python.org/) and [uv](https://docs.astral.sh/uv/).
The minimum python version is `3.9`
The minimum uv version is `0.4.9`
The minimum python version is `3.10`
The minimum uv version is `0.5.0`
### Setup
@ -221,11 +221,11 @@ Below are the required configurations:
```env
DB_CONNECTION_URI= # Connection uri for a postgres database (with postgresql+psycopg prefix)
# LLM Provider API Keys (at least one required depending on your configuration)
LLM_ANTHROPIC_API_KEY= # API Key for Anthropic (used for dialectic by default)
LLM_OPENAI_API_KEY= # API Key for OpenAI (optional, for embeddings if EMBED_MESSAGES=true)
LLM_GEMINI_API_KEY= # API Key for Google Gemini (used for summary/deriver by default)
LLM_GROQ_API_KEY= # API Key for Groq (used for query generation by default)
# LLM Provider API Keys
LLM_GEMINI_API_KEY= # API Key for Google Gemini (used for deriver, summary, and dialectic minimal/low by default)
LLM_ANTHROPIC_API_KEY= # API Key for Anthropic (used for dialectic medium/high/max and dream by default)
LLM_OPENAI_API_KEY= # API Key for OpenAI (used for embeddings when EMBED_MESSAGES=true)
LLM_GROQ_API_KEY= # API Key for Groq (optional)
```
> Note that the `DB_CONNECTION_URI` must have the prefix `postgresql+psycopg` to

View File

@ -48,7 +48,7 @@ services:
- 5432:5432
command: ["postgres", "-c", "max_connections=800"]
environment:
- POSTGRES_DB=postgres
- POSTGRES_DB=honcho
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_HOST_AUTH_METHOD=trust
@ -57,7 +57,7 @@ services:
- ./database/init.sql:/docker-entrypoint-initdb.d/init.sql
- pgdata:/var/lib/postgresql/data/
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres -d postgres"]
test: ["CMD-SHELL", "pg_isready -U postgres -d honcho"]
interval: 5s
timeout: 5s
retries: 5

View File

@ -131,7 +131,8 @@
"group": "Self-Hosting",
"pages": [
"v3/contributing/self-hosting",
"v3/contributing/configuration"
"v3/contributing/configuration",
"v3/contributing/troubleshooting"
]
},
{

View File

@ -184,8 +184,7 @@ DB_TRACING=false # Enable query tracing
**Docker Compose for PostgreSQL:**
```yaml
# docker-compose.yml
version: '3.8'
# From docker-compose.yml.example
services:
database:
image: pgvector/pgvector:pg15
@ -193,14 +192,20 @@ services:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: honcho
POSTGRES_HOST_AUTH_METHOD: trust
ports:
- "5432:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
- ./init.sql:/docker-entrypoint-initdb.d/init.sql
- pgdata:/var/lib/postgresql/data
- ./database/init.sql:/docker-entrypoint-initdb.d/init.sql
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres -d honcho"]
interval: 5s
timeout: 5s
retries: 5
volumes:
postgres_data:
pgdata:
```
### Authentication Configuration

View File

@ -32,6 +32,23 @@ You'll need a PostgreSQL database with the pgvector extension. Choose one:
- **Railway** - Simple cloud PostgreSQL hosting
- **Your own PostgreSQL server**
## Which API Keys Do I Need?
Honcho uses different LLM providers for different features. The table below shows which keys are needed for the **default** configuration. All providers are configurable per feature (see the [Configuration Guide](./configuration)).
| Feature | Default Provider | Environment Variable | Required? |
|---------|-----------------|---------------------|-----------|
| Deriver (memory extraction) | Google Gemini | `LLM_GEMINI_API_KEY` | Yes (core feature) |
| Summary generation | Google Gemini | `LLM_GEMINI_API_KEY` | Yes (core feature) |
| Dialectic minimal/low | Google Gemini | `LLM_GEMINI_API_KEY` | If using Chat API |
| Dialectic medium/high/max | Anthropic Claude | `LLM_ANTHROPIC_API_KEY` | If using Chat API |
| Dream (memory consolidation) | Anthropic Claude | `LLM_ANTHROPIC_API_KEY` | Optional |
| Message embeddings | OpenAI | `LLM_OPENAI_API_KEY` | If `EMBED_MESSAGES=true` (default) |
**Minimum for basic usage**: `LLM_GEMINI_API_KEY` + `LLM_OPENAI_API_KEY`
**Full functionality**: All three keys (Gemini, Anthropic, OpenAI)
## Docker Setup (Recommended)
The easiest way to get started is using Docker Compose, which handles both the database and Honcho server.
@ -51,15 +68,16 @@ Copy the example environment file and configure it:
cp .env.template .env
```
Edit `.env` and set your API keys (if using LLM features):
Edit `.env` and set your API keys (see [Which API Keys Do I Need?](#which-api-keys-do-i-need) above):
```bash
# Optional API keys (required for LLM features)
OPENAI_API_KEY=your-openai-api-key
ANTHROPIC_API_KEY=your-anthropic-api-key
# Database (matches docker-compose.yml.example credentials)
DB_CONNECTION_URI=postgresql+psycopg://postgres:postgres@database:5432/honcho
# Database will be created automatically by Docker
DB_CONNECTION_URI=postgresql+psycopg://postgres:postgres@database:5432/postgres
# LLM API keys (see table above for which features use which provider)
LLM_GEMINI_API_KEY=your-gemini-api-key
LLM_ANTHROPIC_API_KEY=your-anthropic-api-key
LLM_OPENAI_API_KEY=your-openai-api-key
# Disable auth for local development
AUTH_USE_AUTH=false
@ -71,11 +89,21 @@ AUTH_USE_AUTH=false
# Copy the example docker-compose file
cp docker-compose.yml.example docker-compose.yml
# Start PostgreSQL and Honcho
# Start PostgreSQL, Redis, Honcho API, and the deriver background worker
docker compose up -d
```
### 4. Verify It's Working
> **Note:** The docker-compose file starts both the API server and the deriver (background worker) automatically. It also includes Redis (for caching, disabled by default) and Grafana (for monitoring on port 3000 when `METRICS_ENABLED=true`).
### 4. Run Database Migrations
After the containers are running, create the database tables:
```bash
docker compose exec api uv run alembic upgrade head
```
### 5. Verify It's Working
Check that both services are running:
@ -153,7 +181,6 @@ psql -U postgres
CREATE DATABASE honcho;
\c honcho
CREATE EXTENSION IF NOT EXISTS vector;
CREATE EXTENSION IF NOT EXISTS pg_trgm;
\q
```
@ -165,15 +192,16 @@ Create a `.env` file with your settings:
cp .env.template .env
```
Edit `.env` with your configuration:
Edit `.env` with your configuration (see [Which API Keys Do I Need?](#which-api-keys-do-i-need) above):
```bash
# Database connection
DB_CONNECTION_URI=postgresql+psycopg://postgres:postgres@localhost:5432/honcho
# Optional API keys (required for LLM features)
OPENAI_API_KEY=your-openai-api-key
ANTHROPIC_API_KEY=your-anthropic-api-key
# LLM API keys (see table above for which features use which provider)
LLM_GEMINI_API_KEY=your-gemini-api-key
LLM_ANTHROPIC_API_KEY=your-anthropic-api-key
LLM_OPENAI_API_KEY=your-openai-api-key
# Development settings
AUTH_USE_AUTH=false
@ -191,11 +219,21 @@ uv run alembic upgrade head
```bash
# Start the development server
fastapi dev src/main.py
uv run fastapi dev src/main.py
```
The server will be available at `http://localhost:8000`.
### 7. Start the Background Worker (Deriver)
In a **separate terminal**, start the deriver background worker:
```bash
uv run python -m src.deriver
```
The deriver is essential for Honcho's core functionality. It processes incoming messages to extract observations, build peer representations, generate session summaries, and run dream consolidation. Without it, messages will be stored but no memory or reasoning will occur.
## Cloud Database Setup
If you prefer to use a managed PostgreSQL service:
@ -206,7 +244,6 @@ If you prefer to use a managed PostgreSQL service:
2. **Enable pgvector extension** in the SQL editor:
```sql
CREATE EXTENSION IF NOT EXISTS vector;
CREATE EXTENSION IF NOT EXISTS pg_trgm;
```
3. **Get your connection string** from Settings > Database
4. **Update your `.env` file** with the connection string
@ -241,7 +278,10 @@ Create a simple test script:
from honcho import Honcho
# Connect to your local instance
client = Honcho(base_url="http://localhost:8000")
client = Honcho(
base_url="http://localhost:8000",
workspace_id="my-app-testing"
)
# Create a test peer
peer = client.peer("test-user")
@ -283,42 +323,51 @@ const client = new Honcho({
## Troubleshooting
### Common Issues
Running into issues? See the [Troubleshooting Guide](./troubleshooting) for detailed solutions to common problems including:
**Database Connection Errors**
- Ensure PostgreSQL is running
- Verify the connection string format: `postgresql+psycopg://...`
- Check that pgvector extension is installed
- Startup failures (missing API keys, database issues)
- Runtime errors ("An unexpected error occurred" on every request)
- Deriver not processing messages
- Database connection and migration issues
- Docker and Redis problems
**API Key Issues**
- Verify your OpenAI and Anthropic API keys are valid
- Check that the keys have sufficient credits/quota
**Port Already in Use**
- Pass a different port to FastAPI or stop other services using port 8000
**Docker Issues**
- Ensure Docker is running
- Check container logs: `docker compose logs`
- Restart containers: `docker compose down && docker compose up -d`
**Migration Errors**
- Ensure the database exists and pgvector is enabled
- Check database permissions
- Run migrations manually: `uv run alembic upgrade head`
### Getting Help
- **GitHub Issues**: [Report bugs](https://github.com/plastic-labs/honcho/issues)
- **Discord**: [Join our community](https://discord.gg/plasticlabs)
- **Documentation**: Check the [Configuration Guide](./configuration) for detailed settings
**Quick checks:**
- Verify the server is running: `curl http://localhost:8000/health`
- Check logs: `docker compose logs api` (Docker) or check terminal output (manual setup)
- Ensure migrations ran: `uv run alembic upgrade head`
## Production Considerations
When self-hosting for production, consider:
- **Security**: Enable authentication, use HTTPS, secure your database
- **Scaling**: Use connection pooling, consider load balancing
- **Monitoring**: Set up logging, error tracking, health checks
- **Backups**: Regular database backups, disaster recovery plan
- **Updates**: Keep Honcho and dependencies updated
### Security
- Set `AUTH_USE_AUTH=true` and generate a JWT secret with `python scripts/generate_jwt_secret.py`
- Use HTTPS via a reverse proxy (e.g., nginx, Caddy) in front of Honcho
- Secure your database with strong credentials and restrict network access
### Scaling the Deriver
- Increase `DERIVER_WORKERS` (default: 1) for higher message throughput
- You can also run multiple deriver processes across machines — they coordinate via the database queue
- Monitor deriver logs for processing backlog
### Caching
- Enable Redis caching for high-traffic environments: `CACHE_ENABLED=true`
- Requires a Redis instance (included in docker-compose, or use a managed Redis service)
- Configure `CACHE_URL` to point to your Redis instance
### Database Migrations
- Always run `uv run alembic upgrade head` after updating Honcho before starting the server
- Check current migration status with `uv run alembic current`
### API Keys
- Ensure you have the correct API keys for your configured providers (see [Which API Keys Do I Need?](#which-api-keys-do-i-need))
- All providers are configurable — see the [Configuration Guide](./configuration) to use different models or providers
### Monitoring
- Enable Prometheus metrics with `METRICS_ENABLED=true` (scraped at `/metrics` on port 8000 for API, port 9090 for deriver)
- Enable Sentry error tracking with `SENTRY_ENABLED=true`
- The docker-compose includes Grafana on port 3000 for dashboards
### Backups
- Set up regular PostgreSQL backups (pg_dump or continuous archiving)
- Back up your `.env` or `config.toml` configuration files

View File

@ -0,0 +1,200 @@
---
title: 'Troubleshooting'
sidebarTitle: 'Troubleshooting'
description: 'Common issues and solutions when self-hosting Honcho'
icon: 'wrench'
---
This page covers common issues you may encounter when self-hosting Honcho, what causes them, and how to fix them.
## Startup Failures
### Server won't start: "Missing client for ..."
```
ValueError: Missing client for Deriver: google
```
**Cause:** The server validates at startup that all configured LLM providers have API keys. If a provider is referenced in your configuration but the corresponding API key isn't set, the server refuses to start.
**Fix:** Set the API keys for your configured providers. With default configuration, you need:
```bash
LLM_GEMINI_API_KEY=... # Used by deriver, summary, dialectic minimal/low
LLM_ANTHROPIC_API_KEY=... # Used by dialectic medium/high/max, dream
LLM_OPENAI_API_KEY=... # Used by embeddings (when EMBED_MESSAGES=true)
```
See the [API Keys table](/v3/contributing/self-hosting#which-api-keys-do-i-need) for a full breakdown. Alternatively, you can change which providers are used in your `config.toml` or environment variables (see [Configuration Guide](./configuration)).
### Server won't start: "JWT_SECRET must be set"
```
ValueError: JWT_SECRET must be set if USE_AUTH is true
```
**Cause:** You enabled authentication (`AUTH_USE_AUTH=true`) but didn't provide a JWT secret.
**Fix:** Generate a secret and set it:
```bash
python scripts/generate_jwt_secret.py
# Then set the output as:
AUTH_JWT_SECRET=<generated_secret>
```
Or disable authentication for local development: `AUTH_USE_AUTH=false`
## Runtime Errors
### API returns "An unexpected error occurred" on every request
**Cause:** This is almost always a database issue. The health endpoint (`/health`) will return `{"status": "ok"}` even when the database is unreachable because it doesn't check the database connection. The actual error appears in the server logs.
**Common causes and fixes:**
1. **Database is unreachable** — Check that PostgreSQL is running and the `DB_CONNECTION_URI` is correct
2. **Migrations haven't been run** — The server starts successfully without tables, but every API call will fail. Run:
```bash
uv run alembic upgrade head
```
In Docker:
```bash
docker compose exec api uv run alembic upgrade head
```
3. **pgvector extension not installed** — The `vector` extension must be enabled in your database:
```sql
CREATE EXTENSION IF NOT EXISTS vector;
```
**How to diagnose:** Check the server logs for the actual error. Look for:
- `sqlalchemy.exc.OperationalError` — database connection issue
- `sqlalchemy.exc.ProgrammingError` with "relation does not exist" — migrations not run
- `psycopg.OperationalError` — connection refused or authentication failed
### Health check passes but API calls fail
The `/health` endpoint is a lightweight check that confirms the server process is running. It does **not** verify:
- Database connectivity
- That migrations have been run
- That LLM providers are reachable
To verify full functionality, try creating a workspace:
```bash
curl -X POST http://localhost:8000/v3/workspaces \
-H "Content-Type: application/json" \
-d '{"name": "test"}'
```
If this succeeds, your database connection and migrations are working.
### Deriver not processing messages
Messages are stored but no observations, summaries, or representations are being generated.
**Common causes:**
1. **Deriver isn't running** — In manual setup, the deriver is a separate process:
```bash
uv run python -m src.deriver
```
In Docker, it starts automatically via `docker compose up`.
2. **Deriver can't reach the database** — Check deriver logs for connection errors. The deriver uses the same `DB_CONNECTION_URI` as the API server.
3. **Missing LLM API key for deriver provider** — By default the deriver uses Google Gemini (`LLM_GEMINI_API_KEY`). Check deriver logs for API errors.
4. **Processing backlog** — With `DERIVER_WORKERS=1` (default), high message volume can cause a backlog. Increase workers:
```bash
DERIVER_WORKERS=4
```
## Database Issues
### Connection string format
The connection URI **must** use the `postgresql+psycopg` prefix:
```bash
# Correct
DB_CONNECTION_URI=postgresql+psycopg://postgres:postgres@localhost:5432/honcho
# Wrong - will fail
DB_CONNECTION_URI=postgresql://postgres:postgres@localhost:5432/honcho
DB_CONNECTION_URI=postgres://postgres:postgres@localhost:5432/honcho
```
### Checking migration status
```bash
# See current migration version
uv run alembic current
# See migration history
uv run alembic history
# Upgrade to latest
uv run alembic upgrade head
```
## Cache & Redis
### Redis is optional
Redis is used for caching when `CACHE_ENABLED=true` (default: `false`). If Redis is unreachable, Honcho **gracefully falls back to in-memory caching** and logs a warning. This means:
- The server and deriver will still start and function normally
- Performance may be reduced under high load without Redis
- You do not need Redis for local development or testing
### Redis connection issues
If you see Redis connection warnings in logs but `CACHE_ENABLED=false`, they can be safely ignored. If you want caching:
```bash
# Start Redis via Docker
docker run -d -p 6379:6379 redis:latest
# Configure Honcho
CACHE_ENABLED=true
CACHE_URL=redis://localhost:6379/0
```
## Docker Issues
### Containers start but API fails
1. Check container status: `docker compose ps`
2. Check API logs: `docker compose logs api`
3. Check database logs: `docker compose logs database`
4. Ensure migrations ran: `docker compose exec api uv run alembic upgrade head`
### Port conflicts
If port 8000 is already in use:
```bash
# Check what's using the port
lsof -i :8000
# Or change the port mapping in docker-compose.yml
ports:
- "8001:8000" # Map to a different host port
```
### Rebuilding after code changes
```bash
docker compose build --no-cache
docker compose up -d
```
## Getting Help
If your issue isn't covered here:
- **Check the logs** — most issues are diagnosed from server or deriver logs
- **GitHub Issues** — [Report bugs](https://github.com/plastic-labs/honcho/issues)
- **Discord** — [Join our community](https://discord.gg/plasticlabs)
- **Configuration** — See the [Configuration Guide](./configuration) for all available settings

View File

@ -196,6 +196,12 @@ app.include_router(webhooks.router, prefix="/v3")
app.add_route("/metrics", metrics_endpoint, methods=["GET"])
@app.get("/health")
async def health_check():
"""Health check endpoint for monitoring and container orchestration."""
return {"status": "ok"}
# Global exception handlers
@app.exception_handler(HonchoException)
async def honcho_exception_handler(_request: Request, exc: HonchoException):