* feat(harness-plugin-core): simplify telemetry headers and HOME-first config path
- Identity is three headers: X-Honcho-Host `name/version (platform)`,
X-Honcho-Plugin `name/version`, X-Honcho-Agent-Model. X-Honcho-Runtime is
dropped. TelemetryIdentity gains `plugin` and `platform`.
- configPath() resolves env.HOME (then USERPROFILE) before os.homedir(), since
Bun's homedir() ignores in-process HOME changes and plugin tests were hitting
the real ~/.honcho/config.json.
- Extensionless internal imports so consumers no longer need
allowImportingTsExtensions to type-check against the source exports.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* chore: minor nits
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* fix(tests): repair unsatisfiable unified-test assertions and surface traces
Five of the eight persistent `unified-tests` failures assert things the code
cannot produce. None are regressions.
Raise the queue-drain timeout to 600s on the three large longmem fixtures.
They ingest 484-550 messages across ~50 sessions, then wait on the 60s
`WaitAction` default; the deriver is still working normally when the timer
fires. Matches the sibling 550-message case that already passes.
Raise `max_tokens` to 2500 in the two config-summary fixtures. Context
allocates 40% of the limit to the summary, so the previous 400 gave a
160-token budget while `SUMMARY.MAX_TOKENS_SHORT` is 1000 — no conforming
summary could ever fit, and the query returned `summary=None` even though the
summary was created.
Drop `session_id` from the dream test's `get_representation` step. A bare
session id becomes a one-element allowlist, and an allowlist narrows levels to
`ALLOWLIST_SAFE_LEVELS` (`explicit`), so the deductive and inductive
observations the step asserts on are excluded by design. The unscoped
representation is where the dreamer's conclusions are actually served.
Delete `WaitAction.flush`. Flush is process-wide — the harness starts the
deriver with `DERIVER_FLUSH_ENABLED=true` — and there is no per-request flush,
so the field never had an effect despite being set in 47 places. `TestStep`
now forbids extra fields so a dead knob cannot silently accumulate again.
Presign the reasoning traces alongside `results.json` and report both to the
Discord webhook and a GitHub job summary. The traces hold the full prompts and
model outputs and were already uploaded, but only `results.json` was surfaced.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(tests): record why a unified test failed, not just that it did
`results.json` carried only name, status and duration, so a red run said
which test failed and nothing about why. The reason existed solely in the job
log, where the secrets action's masking can render it unreadable — diagnosing
a failure meant re-reading GHA logs that had digits redacted out of them.
`execute` now returns the `StepFailure` that stopped the test (step index,
step type, and the exception message) instead of a bare bool. Assertion
failures already raised useful text, including the LLM judge's own reasoning;
that text now reaches `results.json`, the console output, the job summary and
the Discord message rather than being discarded at the call site.
`results` moves from a `(status, duration)` tuple to a `TestOutcome` with
named fields so the failure can ride along.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(tests): keep the Discord report inside the webhook size limit
The failure reasons added to the Discord message pushed it past Discord's
2000-character content limit, and the webhook answered 400 — run
33779689337 sent no notification at all. Six LLM-judge verdicts run to
~2760 characters; capping the count at ten did nothing because the length
was never the count.
Reasons are now clipped per line for Discord only; the job summary, the
console and results.json keep them whole. `send_discord_message` also clamps
the assembled content, so an over-long report loses its tail rather than the
entire notification.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(tests): keep the Discord report short and link to the Actions run
The Discord message restated every failure, which pushed it past Discord's
2000-character limit and returned a 400 — run 33779689337 sent no
notification at all.
The report is now the headline, the results link, an Actions run link, and
the traces S3 key. Per-test failure reasons stay in the job summary that the
Actions link points at, along with both presigned URLs, so nothing is lost by
not repeating them in chat.
Restating failures was not the only size risk. A presigned URL carries an
OIDC session token and can run past a thousand characters by itself, so two
of them exceeded the limit unaided — which is why the traces go in as their
S3 key, the `aws s3 cp` path, at ~90 characters instead of ~1500.
`clamp_lines` drops whole lines rather than characters, since half a
presigned URL is useless and renders as broken markdown, and drops the
longest line first so an overlong URL cannot evict the short Actions link
that leads to everything else.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(api): say when a session summary is dropped for budget
`get_context` allocates 40% of the token limit to the summary, but that limit
is what remains *after* the peer representation and peer card are subtracted,
not the `tokens` the caller asked for. When nothing fits, the caller receives
`summary: null` — indistinguishable from a session that has no summary — and
the only trace was a debug line in a different module.
`_select_summary_for_context` now logs at info when summaries exist and none
was chosen, with the budget and the sizes that missed it.
The two `config_summary_control` fixtures go to 4000. Measured against CI run
33779689337, their 12 messages produce 12 explicit observations costing ~1176
tokens, so the original `max_tokens: 400` left a budget of -776: no summary of
any size could have been served, and the earlier reading of this failure — a
160-token budget against a 388-token summary — had the mechanism wrong. 2500
was also short, leaving 529 against a `SUMMARY.MAX_TOKENS_SHORT` of 1000; 3676
is the minimum that guarantees a conforming summary fits.
Tests cover the budget arithmetic at each of those limits, the new log line,
and that a stored summary is served through the route with and without an
observer — the retrieval path itself was never at fault and had no coverage.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(api): report a dropped summary on the path get_context actually takes
The previous commit added this log to `_select_summary_for_context`, which
only runs when `get_context` is given a `peer_target`. The unified
`config_summary` fixtures set `observer_peer_id`, but the runner does not
forward it, so those requests take `summarizer.get_session_context` instead —
where the same outcome was reported at debug and stayed invisible.
That also retracts the representation-budget explanation for those fixtures.
Nothing is subtracted from the limit on this path: the summary gets 40% of the
requested tokens outright, so at `max_tokens: 4000` a 99-token summary has a
1600-token budget and fits comfortably. The reason it is still absent is not
the budget, and the log now says so on the right path.
Tests cover both paths, and record that the fixtures exercise the one without
a representation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(tests): drop the ignored observer from the config_summary fixtures
`observer_peer_id` has no effect on a `get_context` step — the runner does not
forward it — so it read as scoping a request that was never scoped. The step
description now records that these are unscoped reads and what naming an
observer would change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(mock-provider): deterministic OpenAI-compatible endpoint for local and CI use
Adds src/mock_provider/, a standalone ASGI app that lets Honcho run with no
model provider, no API key, and no spend. It answers /v1/chat/completions and
/v1/embeddings with obviously-synthetic content derived from the request, so
the same request always produces the same response.
It runs as its own service from the standard Honcho image with a different
entrypoint, the way api and deriver already differ, so there is no second image
to build or keep in digest-sync. The app imports nothing from src.config or
src.db, so it boots even when the rest of the stack is misconfigured.
The chat endpoint generates from the JSON Schema it is sent rather than
answering with prose. That matters because a prose answer does not fail loudly:
repair_response_model_json swallows the parse error and returns an empty
PromptRepresentation, which reads as "the deriver found nothing" rather than
"the mock is wrong". Generation resolves $ref/$defs indirection, caps recursion
for reasoning-tree schemas, and covers json_object mode by recovering the
schema Honcho injects into the prompt. Embeddings are hash-derived, so
identical input yields an identical vector.
Tests drive the production OpenAIBackend and _EmbeddingClient against the app
over ASGI, including the strict json_schema transform that
chat.completions.parse() applies. Verified end to end against a real stack:
messages in, conclusions and 1536-dim embeddings written to pgvector, with no
calls to any real provider.
Mock embeddings carry no semantic similarity, so recall against this provider
must use lexical search. CONTRIBUTING notes that, and the load_dotenv(override=
True) behaviour that lets a stale repo .env win over exported environment
variables.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(mock-provider): validate requests with Pydantic models
Review feedback: hand-coercing the request bodies was defended on the grounds
that FastAPI answers a malformed body with a 422, and a 422 mid-deriver-run
reads as a Honcho bug. That argues against the default handler, not against the
models. Registering an exception handler fixes it — and the resulting behaviour
is more faithful, not less, because the real API answers a bad request with a
400 and an `error` envelope, which is now exactly what the mock returns.
Adds src/mock_provider/schemas.py with ChatCompletionRequest and
EmbeddingsRequest. Every model allows extra fields and every field is optional,
so validation fires on a wrong type rather than on a parameter the mock has not
heard of — a new upstream parameter must not turn a working setup into a hard
failure. dimensions is a StrictInt because bool is an int subclass and a JSON
`true` would otherwise mean a one-dimensional vector.
coerce.py stays, narrowed to serving schema_gen, which walks arbitrary
caller-supplied JSON Schema and is untyped by nature. response_format likewise
stays dict[str, Any]: only its envelope is worth typing.
Also records why schema_gen does not reuse src/utils/schema_conversion.py
despite the overlapping $ref/$defs handling — it builds a model class rather
than an instance, raises by contract where a mock must degrade, and rejects
both allOf and the recursive $ref that reasoning-tree schemas rely on.
Documents that LLM_OPENAI_API_KEY is only tested for truthiness; the previous
wording read as though the value had to be the literal string "sandbox".
Re-verified end to end after the refactor: 6 messages in, 4 conclusions and 6
1536-dim embeddings out, every real request answered 200, no calls to any real
provider.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(mock-provider): honour include_usage, generate prefixItems tuples
Three fidelity gaps where the mock answered a request differently from the
API it stands in for:
- The usage chunk was emitted on every stream. The real API sends it only
when stream_options.include_usage is set, so a caller that did not opt in
had to skip a trailing chunk with an empty choices array. stream_options
is now a typed model, which also rejects a non-boolean include_usage
instead of reading it as truthy.
- A fixed-length tuple is prefixItems with no items, which is what Pydantic
emits for tuple[str, int]. Reading only items returned [], failing the
minItems the same schema carries — the silent-empty failure schema_gen
exists to avoid.
- A zero or negative dimensions was silently replaced with 1536, answering
a bad request with a plausible-looking vector rather than a 400.
Three further deviations from JSON Schema are left in place and documented
where they occur: allOf merges properties first-wins, oneOf is treated as
anyOf, and string pattern is ignored. None is reachable from a Honcho
response model — no model emits prefixItems or oneOf, and the only pattern
constraints are on API request models — and each fix costs more than the
unreachable path is worth.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(mock-provider): strict request booleans, bounded recursion, multipleOf
Second CodeRabbit pass. All four findings reproduced first; none is reachable
from a Honcho response model, but two trace back to the previous commit.
- `include_usage` and `stream` were plain `bool`, which Pydantic coerces from
"yes"/"on"/"true"/"1". The comment added last commit claimed a string had to
fail here, and it did not — the test only passed because "definitely" is not
a recognised bool literal. Both are StrictBool now, matching why `dimensions`
is StrictInt, and the tests cover the truthy strings that actually coerced.
- `_generate_array` returned the prefix alone when `items` was absent, so
prefixItems plus a larger minItems undershot its own schema. Absent `items`
leaves those positions unconstrained rather than disallowed, so the shortfall
is filled to minItems — a bare `{"type": "array"}` still generates nothing.
- A required, non-nullable recursive $ref hit RecursionError: MAX_DEPTH only
terminates a cycle that offers a `default` or a nullable branch, and
`_generate_object` keeps descending into required properties. HARD_MAX_DEPTH
degrades to an empty container instead, since a mock must not turn its own
defect into a 500. Bounded, not plumbed into an error response — the
unreachable path does not justify touching the request path.
- `_bounded_int` ignored `multipleOf` while honouring minimum, maximum and both
exclusive bounds; 9 of 12 sampled paths produced a non-multiple. Values now
snap onto a multiple inside the bounds, and an unsatisfiable window keeps the
bounds. A fractional `multipleOf` is still ignored, as documented.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(mock-provider): correct the reason fractional multipleOf is dropped
The docstring claimed honouring it would mean returning a non-integer from an
integer schema. That is wrong: 3 is an integer and a multiple of 1.5. The real
reason is that it needs exact-decimal arithmetic to keep float drift from
deciding validity, and no Honcho response model emits multipleOf at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(mcp): add stdio host for local clients
* feat(mcp): add Streamable HTTP host and image
Long-lived HTTP entry for Docker and other process hosts, reusing
createServer(). Dedicated mcp/Dockerfile; compose service beside api.
* fix(mcp): stdio launcher cwd/silent and HTTP session bounds
Pin bun --cwd so bunfig loads. Silence bun run. Require Bearer on HTTP.
Idle-expire and cap in-memory MCP sessions.
* fix(mcp): re-check bearer on established HTTP sessions
Session lookup returned early without Authorization, so a missing or
wrong token still 200'd after initialize. Bind each session to the
init key and 401 on mismatch.
* fix: nit cleaning claude command
---------
Co-authored-by: ajspig <dragon@monstercode.com>
* chore: scaffold @honcho-ai/harness-core
* feat(harness-core): resolve shared root config
* feat(harness-core): send client identity headers on SDK requests
* feat(harness-core): drop cloud vs custom api header
* feat(harness-core): migrating v0 config to schema v1 on read
* chore(harness-core): clean up
* feat(config): describe oauth and host overrides in the v1 schema
* chore: rename to harness-plugin-core
* feat(harness-plugin-core): update telemetry headers on a live client.
The workspace agent's prefetch is an orientation overview — scale, active
peers, their cards — not the corpus. `low` is the only reasoning level that
explicitly sets TOOL_CHOICE="auto", so the model was free to skip tools
entirely, and it did: every workspace_chat call in CI run 33662772219 made
zero tool calls. It answered when the overview happened to carry the fact and
otherwise wrote out the search it should have run, then asked the caller which
option to take — at an endpoint with no caller to answer.
Add a `_tool_choice` seam alongside `_select_tools` and override it on
WorkspaceDialecticAgent to require a tool call. `execute_tool_loop` already
relaxes "required"/"any" to "auto" after the first iteration, so this costs one
search round rather than pinning the loop, and the model can still stop and
synthesize. Any value a level configures other than None/"auto" passes through.
The pair agent is unaffected: it prefetches the observations for its query and
can legitimately answer from context alone.
Also tell the workspace prompt it is non-interactive. It had "Do not narrate
tool use" but never said the caller cannot reply, and three of the five traced
responses ended in a menu of lookups.
Unified subset goes 1/5 -> 5/5, and search_memory — the recall path that never
once ran — now fires on 6 of 7 workspace queries. workspace_chat_scope is the
notable one: its two not_contains assertions were passing vacuously because
nothing was ever retrieved, and it now recalls the in-scope fact while still
excluding the out-of-scope vault code.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(docker): make API worker count configurable
Add API_WORKERS with a single-worker default and document database pool sizing.
Refs #1063
* fix(docker): address API worker review feedback
* chore(docs): Add section about harness integrations and deepseek harness to docs
* chore: Add section about harness integrations and deepseek harness to docs
* fix(deriver): eliminate create_documents deadlock and stop silently burning batches on transient errors
Two concurrent work units writing the same (workspace, observer, observed)
collection deadlocked on times_derived reinforcement UPDATEs issued in
batch order (DEV-1975, 682 events in 90 days). The deadlock was swallowed
per-document, the loop cascaded PendingRollbackErrors against the dead
session, the whole batch was lost, and the queue item was marked processed.
- serialize writers per collection with a transaction-scoped advisory lock
(pg_advisory_xact_lock + SET LOCAL lock_timeout), skipped for insert-only
batches; covers all three row-lock sites in one move
- hoist external-vector-store dup-candidate resolution ahead of the first
DB statement so the lock's critical section contains no network calls
- abort the batch on SQLAlchemyError instead of continuing through an
aborted transaction; per-document skip semantics kept for non-DB errors
- classify transient errors (new src/utils/retryable_errors.py) and retry
them via a bounded in-process counter instead of marking items errored
* fix(deriver): replace create_documents advisory lock with id-ordered row locks
Advisory locks are database-scoped and would serialize every writer to a
collection, including across Groudon tenants that share names. Collect
reinforcement and replace ops during the loop, lock target rows with
SELECT ... ORDER BY id FOR UPDATE, then apply. populate_existing reloads
times_derived so a prefetched identity-map row cannot lose a concurrent
increment.
* fix(deriver): harden create_documents candidate hoist and test isolation
Skip empty embeddings on the external-store path, isolate per-document
resolve failures, and keep replacement times_derived in the in-batch
ledger. Patch get_external_vector_store in the hoist test and cover
in-loop SQLAlchemyError abort.
* fix(deriver): address CodeRabbit findings on create_documents deadlock fix
- Distinguish external resolve failure ([] skip) from pgvector fallback (None)
so _semantic_dup_decision never re-enters external I/O under an open session
- Bound external candidate hoist concurrency with a semaphore
- Map in-loop IntegrityError to ValidationException for a uniform contract
- Persist transient retry attempts on the oldest unprocessed queue item so
every deriver instance shares one MAX_RETRYABLE_ATTEMPTS budget
- Cover resolve-failure skip and multi-manager reclaim of the retry budget
* fix(deriver): harden retry metadata cleanup and stale reinforce fallback
- Strip _retry_attempts from payloads in the same transaction as
mark_queue_items_as_processed / mark_queue_item_as_errored
- Clear shared retry metadata only after a successful terminal mark
- On reinforce, if the locked target is gone or soft-deleted, insert the
incoming document instead of dropping it
- Skip pgvector semantic lookup when embedding is empty so query_documents
cannot embed under an open session
* fix(deriver): address review on deadlock retry and row-lock apply
Strip _retry_attempts before payload validation so non-representation
tasks are not burned as extra_forbidden. Re-raise retryable observer
save errors after telemetry so the queue actually retries. Skip
same-batch reinforce fallbacks after a replace. Revert unordered
FOR UPDATE on mark processed/errored and drop post-commit retry
cleanup from the success path.
* fix: add test and simplify queue query
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* fix(deriver): chunk scope backfill so large sessions don't OOM the worker
_run_backfill embedded, wrote, and synced every planned copy at once, holding
one Python float list per document. A 14k-document session is ~580MB of
vectors alone, and several backfills run concurrently, which OOM-killed the
deriver at its 1000Mi limit and crash-looped it since the work units never
completed. Phases 2-4 now run per chunk of 500 specs and drop each chunk's
embeddings once synced.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(deriver): hydrate backfill embeddings per chunk
Phase 1 no longer materializes every source embedding into plans.
load_only skips the vector column on the plan queries, and each chunk
reloads only its source embeddings before embed/write/sync.
* fix(deriver): lock scope membership across backfill chunk writes
SELECT ... FOR UPDATE on the active SessionPeer row so a concurrent
leave cannot commit between the membership check and the copy inserts.
Adds a concurrency test that asserts the leave blocks until commit.
* fix: add test for memory bound
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* fix(deriver): strip NUL bytes from model-generated observations
Postgres rejects NUL (0x00) in text columns and in jsonb strings. API
ingress has always stripped it from user-supplied content, but the
deriver's own output did not go through any equivalent: a model can emit
a \u0000 escape in its tool-call arguments, which the JSON parser decodes
into a real NUL byte. Seen in production when models transcribe shell
output (`tr '\x00' '\n'`) or Windows paths (`c:\<NUL>users\amal`).
The NUL reached the exact-content dedup pre-fetch in create_documents as
a bind parameter, so the query raised DataError before any row was
written and the whole batch for that observer was dropped.
Strip in _normalized_observation and _normalized_observation_input --
the points that already normalize text for persistence and embedding --
so the embedded text matches the stored text. premises and sources are
covered too, since they ride along in internal_metadata. The emptiness
check now runs after normalization, because str.strip() does not remove
NUL and all-NUL content would otherwise be stored as an empty string.
DocumentCreate.content gets a mode="before" validator as a backstop for
callers that bypass those paths; running before the length constraint
makes all-NUL content fail min_length rather than silently empty out.
The NUL helpers move out of schemas/api.py into utils/sanitization.py as
a single recursive strip_nul, so ingress and internal paths share one
implementation. It is overloaded to keep str -> str for the callers that
chain .strip(), and passes None through so optional fields need no guard.
Fixes HONCHO-4XZ
* fix: broaden nul strip check
* chore: code simplification
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* chore(docs): Add Documentation for Scopes
* chore(docs): add scopes to API reference, architecture, and design patterns
- Add the seven /scopes routes and their schemas to openapi.json, plus the
scope/kind fields on chat, representation, session-create, and peer-list
schemas; generate the endpoint pages and register a scopes nav group
- Add a Scopes subsection and diagram node to the architecture data model
- Add scope guidance to design patterns: quick-reference rows, an isolation
boundary comparison (workspace / scope / session allowlist), and common
mistakes (scope-per-reader, scopes-as-access-control)
- Replace the "Underneath the Facade" section in scopes.mdx with behavioral
guardrails and pointers to the implementation source
* chore(docs): tighten scopes doc to decision-level detail
- Drop the recall-resolution diagram (restated the Two Arms table)
- Replace the enumerated Rules table with prose; caps and error shapes
now live in the API reference schema descriptions
- Trim backfill/removal internals to observable behavior and note that a
backfilled scope deepens through subsequent dreams
* chore(docs): reserve "scope" for the scopes feature
Using it as a verb for session design, recall filters, and CLI targeting
collides with the named-session-set feature.
* chore(docs): clarify the scopes page and document create/status responses
The page now leads with projection rather than partition and points at the
scopes API; OpenAPI declares the 201/409/404 those routes actually return.
* chore(docs): fix broken anchor and core-concepts link
The rebase reintroduced a link to a renamed anchor in scopes.mdx, and
unified-memory-setup pointed at /core-concepts/, which has no index page.
* chore(docs): correct scope arms, listing, and read-surface pointers
The Accepts row mixed named-scope with the allowlist arm, kind=scope on
the peers list does not return facade ids, and chat/context/search never
mentioned scope=.
* chore(docs): drop the 1k-token session batching narrative
Reasoning no longer waits on a per-session token threshold, so product
docs should not tell people to size sessions around that gate.
* chore: minor fix
If consent is withdrawn while array.js downloads, sync() runs before window.posthog exists and the opt-out is skipped. onload now re-checks granted() and resets loaded so a later re-grant retries.
Mintlify swaps pages without a reload, so capture_pageview: 'history_change' records navigation past the landing page.
* fix: stop top_k=0 from reaching Turbopuffer on message search
HONCHO-19Q: dreamer search_messages passed LLM limit=0 through to
Turbopuffer (top_k must be 1..10000). #970 guarded documents; this
closes the message path and floors tool limits at 1.
* fix: preserve pgvector None sentinel on zero top_k
query_external_vector_document_ids must return None when on the
pgvector path before applying the top_k<=0 empty-list guard.
The gate job only runs on `pull_request: labeled`, so it is skipped on
push. A skipped ancestor propagates down the needs chain unless a job
opts out, which `unified-tests` never did — so the suite has been
skipped on every merge to main while still burning a Fly machine.
* feat(cli): fix API key typing
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore: bump honcho-cli to 0.1.4
Ship the masked --setup API key prompt plus the openai-compatible embedding base URL already on main.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(cli): check for newer version
* docs: nit
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(llm): preserve null content on OpenAI tool-call turns
OpenAI-compatible providers can return assistant tool-call messages with
content=null. Coercing that to "" before history replay breaks providers
that bind opaque reasoning state to the exact assistant message shape.
Keep null only when the normalized response has tool calls; tool-less
null still becomes "", and content_override stays authoritative.
Fixes#1061
* test(live_llm): cover OpenAI null content tool-call replay
Add a live multi-turn tool replay that asserts provider content=null stays
null through normalize + OpenAIHistoryAdapter and that the continuation
still answers. Mark gpt_4/gpt_5 families as supports_tool_replay.
* docs(llm): note content_override None sentinel semantics
None means no override, not force-null content. Addresses review on #1064.
author_association on the webhook is CONTRIBUTOR when org membership is
private, so maintainers with write (e.g. ajspig) were labelled
needs-approved-issue. Skip on admin/maintain/write from
getCollaboratorPermissionLevel instead; 404 stays gated.
* docs: make honcho start the documented local path
honcho-cli 0.1.3 can run a personal stack without cloning the repo; point the README, self-hosting, and CLI reference at that, and drop the community installer callouts.
Co-authored-by: Cursor <cursoragent@cursor.com>
* docs: updating with new CLI language
* docs: drop compatibility-guide changes from this PR
Leave that file on main; CLI version cards are updated at release time.
Co-authored-by: Cursor <cursoragent@cursor.com>
* docs: small language changes
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Review findings on #1071. The cookie match now requires the exact
CookieConsent name boundary. A failed array.js request resets the
loaded flag so later consent events retry. And consent events now run
a full sync: withdrawal opts an already running instance out, and a
re-grant opts it back in — same behavior as the landing site's gate.
The runtime stage copies application code but not pyproject.toml, so
src/_version.py cannot find the file it reads the version from. The
image also installs dependencies with --no-install-project, so there is
no honcho distribution for the importlib.metadata fallback to find.
Both lookups fail, so the service falls back to reporting its version as
"unknown" in the OpenAPI schema and in telemetry events.
Copying the file into the runtime stage restores an accurate version.
The file is under 4 KB, so the image size is unchanged.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
DEV-2465 open question 5, option d. Mintlify's built-in integration
loaded PostHog unconditionally on all 249 docs pages — a visitor who
declined on the homepage was tracked one click later in the docs. The
integration key comes out of docs.json; docs/posthog-consent.js loads
PostHog directly instead, only when the CookieConsent cookie grants
Statistics (or holds Cookiebot's -1 marker), and listens for the
consent events so a grant on the docs banner itself loads it too.
Trade recorded on the ticket: this bypasses the ph.mintlify.com proxy,
so ad blockers reduce docs PostHog volume. Verify after deploy that
Mintlify's page CSP allows us-assets.i.posthog.com; if it blocks,
fall back to option c.
DEV-2465 step 1. Mintlify injects gtm.js on all docs pages; the
container is audited to be inert on /docs before this merges, so the
snippet loads and nothing fires. Cookiebot and GA4 arrive later as
container publishes, consent first.
Merging this publishes the docs within minutes, so it stays unmerged
until Marc confirms the container audit.
`get_observation_context` resolved scope by fetching every session name the
observer has a membership record in, then expanding that list into
`session_name IN (...)` twice in one statement — once in the CTE and once in
the outer select. That puts psycopg's 65535-bind-parameter ceiling at roughly
32,765 sessions, and the count only ever grows: the loose membership
definition (`active_only=False`) counts sessions the peer has since left, so
leaving a session does not shrink the scope. A workspace with tens of
thousands of sessions for one peer produced a statement the driver could not
serialize at all.
Two new helpers in `crud.message` express the observer half as a correlated
EXISTS over `session_peers`. Scope now costs two bind parameters regardless of
membership size, and the membership query disappears (two round trips become
one). The `session_peers` primary key is `(workspace_name, session_name,
peer_name)`, so the correlated probe is an exact-match index hit.
The caller-supplied allowlist stays an IN clause — it is route-capped at 1000
entries and carries none of the unbounded-growth risk. `resolve_session_scope`
is left in place: three other callers still need the materialized list,
including `_search_messages_external`, which sends session names to the vector
store as a filter payload and cannot take SQL.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
On Redis Cluster a key's slot comes from the substring inside the first
{...}, when one is present. Untagged, one deployment's keys spread over
every slot, so its client opens and holds a connection to every node in
the cluster. Wrapping the namespace in braces puts them all on one slot,
and therefore one node, cutting each deployment's connection count to
the cluster by a factor of the shard count. Namespaces still hash
independently of each other, so keys stay spread across the cluster and
no shard becomes a hotspot.
The tag needs two spellings, because the two ways a key gets built treat
the string differently. cashews runs `prefix=` through format
substitution, so braces have to be doubled there to survive as literals;
keys built by concatenation need them single. A single brace passed to
cashews is read as an empty substitution field and the namespace is
dropped entirely, which would let two deployments collide on one key --
hence two clearly named helpers rather than one string, and a test that
the two paths produce identical bytes.
No key format change for a non-cluster backend, and no migration: the
old keys simply age out by TTL.
* feat(cli): add honcho start/stop/status for a local Docker stack
* feat(cli): fix status command
* feat(cli): improving how we pull docker images and writing a config,toml
* feat(cli): add honcho start --setup wizard for local stack config
* feat(cli): cleaning up unnecessary func, and error throwing
* feat(cli): minor clean up in stack.py
* feat(cli): read setup wizard defaults from the image config.toml
* feat(cli): cleaning up unused commands
* feat(cli): adding ignored docker-compose.yml
* feat(cli): forward host LLM env into honcho start
* feat(cli): share start/stop progress helpers via output.py and cleaning up language
Teach both agents what Honcho, peers, and the harness are instead of comparing them to each other. Render only the tools the request actually offers, and drop the pair prompt's call to a write tool that is not in the loadout.
* fix(crud): preserve joined_at for active session peers
Re-adding an already-active peer no longer advances the membership
window, so peer_perspective search keeps messages from the original
join. Genuine rejoins still start a new window.
* docs: document set_peers membership window and wrap test docstrings
* fix: preserve session observer limit
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
Callers wrapped every ValueError from the embedding client in a
"exceeds maximum token limit" message, so provider and configuration
failures (dimension mismatch, empty response, upstream error) surfaced
to users as though their input were too long.
Add EmbeddingTokenLimitError, raised only by the pre-flight token checks
in embed() and simple_batch_embed(), and narrow the remaps in search.py,
agent_tools.py, document.py and representation.py to catch it. It
subclasses ValueError so existing broad handlers keep working.
Both simple_batch_embed() remap sites pass on_oversize="truncate" and so
could never raise a token-limit error at all; their handlers only ever
mislabelled provider failures.
Fixes#568
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* chore: update issue templates and docs
* chore: slim bug/quality forms and add integration template
Drop high-friction required fields from bug and quality issue forms.
Add an integration-request form (app stores, plugins, frameworks) labeled
integration, and point contributing docs at it.
* chore: route security mail to support@ and polish docs intake
Use support@honcho.dev for private vulnerability email. List the
documentation template in contributing guides, rename Media prove,
and add public-issue redaction/security redirects on the docs form.
* fix: address render issue in templates and add version field
* feat(docs): Initial draft of new contributing policies
* feat(ci): defer issue-gate closes to a scheduled sweeper
Addresses review feedback on #1041.
The gate now reads GitHub's resolved closing references
(closingIssuesReferences) instead of regex-parsing the pull request body,
so an issue linked through the sidebar Development panel counts, and a
bare `#123` mention no longer does.
It also no longer closes on the pull request event. It labels and
explains; pr-sweeper.yml re-checks every six hours and closes only what is
still failing 72 hours after the notice. That re-check is load-bearing:
linking an issue via the sidebar fires no webhook, so an event-only gate
could never observe a contributor complying that way. The sweeper also
closes drafts from outside the org after 30 days.
The shared check lives in .github/scripts/issue-gate.js so both workflows
run identical logic, with a dependency-free self-check wired into static
analysis. Its one regression guard: author_association CONTRIBUTOR stays
gated, since GitHub assigns it to anyone who has previously committed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore(codeowners): drop the third reviewer from most areas
Discussed with @akattelu. Also reassigns SECURITY.md to @Rajat-Ahuja1997
and strips trailing whitespace from the deployment block.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(v2): port the issue gate policy into the v2 contributing guide
The v2 guide is still published (v2.5.1 in docs.json) but carried no
mention of the issue gate, so a contributor reading it would not learn
that a pull request needs an approved issue until the bot labelled theirs.
Ports the policy, both linking routes, and the gate's place among the
automated checks, keeping the v2 guide's own structure and unwrapped
prose rather than importing the v3 rewrite wholesale.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(ci): count only bot-authored gate notices, share the exemption list
Two review findings on the issue gate, with a common root cause.
MARKER is an invisible HTML comment, so anyone who can comment on a
public repository can paste it. findNotices accepted any comment
containing it, so a third party could post one on someone else's pull
request: runGate posts a notice only when none exists, so the author
would never be told, and runSweep would then measure the 72-hour grace
window from the stranger's timestamp and close them unwarned. Notices now
require bot authorship.
The stale-draft sweep re-listed the gate's exemptions and had lost the
bot case, so a bot's long-lived draft was closable despite checkGate
exempting bots. Both callers now share one exemptReason(pr) rather than
keeping parallel lists that drift.
Not changed: closingIssuesReferences(first: 20) truncation. It needs a
pull request with 21+ closing references where only a later one carries
the label, and the outcome would be a label plus the grace window, not a
close.
Coverage goes 11 -> 20 cases, including the stale-draft close path, which
had none. Both fixes were confirmed to fail their tests when reverted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Aakash Kattelu <aakash@plasticlabs.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: update issue templates and docs
* chore: slim bug/quality forms and add integration template
Drop high-friction required fields from bug and quality issue forms.
Add an integration-request form (app stores, plugins, frameworks) labeled
integration, and point contributing docs at it.
* chore: route security mail to support@ and polish docs intake
Use support@honcho.dev for private vulnerability email. List the
documentation template in contributing guides, rename Media prove,
and add public-issue redaction/security redirects on the docs form.
* fix: address render issue in templates and add version field
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* Add workspace-level chat (DEV-1326)
POST /v3/workspaces/{workspace_id}/chat: agentic dialectic over the whole
workspace instead of a single (observer, observed) pair. Salvaged from
plastic-labs/honcho#373 and re-grown on today's DialecticAgent:
- WorkspaceDialecticAgent subclasses DialecticAgent via four new seams
(_get_tools, _create_tool_executor, _prefetch_intro, _trace_name) instead
of a base-class extraction; observer/observed use empty-string sentinels.
- Routing-accelerated prefetch: workspace stats + top-5 active peers with
their self peer-cards (pure DB, ~7ms measured) so routing-obvious queries
resolve without a discovery tool round.
- Observation search stays pair-scoped (matches per-pair vector namespaces;
avoids workspace-flat top-k dilution): search_memory/get_peer_card take
observer/observed as tool arguments, with pair attribution in results.
- workspace_chat / workspace_chat_stream orchestrators, WorkspaceChatOptions
schema (scope param seam left for the #897 scopes facade), SSE streaming,
structured output via response_format.
- crud: get_workspace_stats, get_active_peers; format_documents_with_attribution.
- SDKs: Python Honcho.chat/chat_stream + HonchoAio mirrors; TypeScript
honcho.chat/chatStream.
- 46 tests (route, orchestrator preflight, tool handlers, executor routing,
attribution formatting) + unified test cases + docs.
Co-Authored-By: doria <93405247+dr-frmr@users.noreply.github.com>
Co-Authored-By: Benjamin McCormick <docterformer@protonmail.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: type SSE stream wrapper as AsyncIterator (basedpyright)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: silence unused db_session fixture warnings (basedpyright failOnWarnings)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: drop docs changes from this PR (defer to follow-up)
Restores docs/v3/documentation/features/chat.mdx to main's version. This
also puts back the peer-chat Structured Outputs section (#896) that the
workspace-chat commit removed as a rebase artifact.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: workspace message tools deny-all under rebased session scoping
The #882 rebase changed the unscoped-observer contract from falsy to
'observer is None': resolve_session_scope looked up the workspace
executor's observer='' sentinel as a real peer with no session
memberships and denied every workspace-flat message read (search, grep,
date-range, temporal, observation context) whenever no session was
pinned — the primary workspace-chat shape. Normalize the sentinel to
None at the five read-handler crud boundaries and add regression tests
that run the tools unpinned (verified to fail without the fix).
Also from review:
- wrap the workspace prefetch in the same degrade-to-None protection
the base agent has (an overview query error no longer 500s the
request or kills the SSE stream after headers)
- thread session_allowlist through create_workspace_tool_executor so
the agent-level allowlist seam is honored end to end when scopes
(#897) wire it up; allowlisted grep is covered by a test
- deterministic name tie-break in get_active_peers ordering
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* review: SDK response_format parity, shared query sanitizer, annotations
- TS SDK: WorkspaceChatParams gains response_format; _workspaceChat/
_workspaceChatStream consume the shared interface instead of inline
duplicates; chat/chatStream expose responseFormat.
- Consolidate the three identical sanitize_query validators into one
NulStripped annotation.
- workspace_chat_stream: return annotation + full docstring.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* review: fold active peers into workspace stats; trace + query bounds
- Merge get_active_peers into get_workspace_stats (one discovery round
instead of two); minimal loadout keeps a discovery tool via the merged
stats tool. Fixed top-10 by recent activity; deeper discovery routes
through search_messages.
- get_active_peers CRUD now aggregates over a trailing 90-day window so
the chat-path prefetch never scans a workspace's full message history.
- Workspace agent inherits the "dialectic_chat" trace name; scope stays
distinguished by agent_type/track_name (workspace name was already in
telemetry context).
- Prefetch failure logs carry workspace + traceback; prompt no longer
contrasts against a peer-level agent the model has no concept of;
drop ticket identifiers from comments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: add `scope` to workspace chat and exclude scope peers from stats
Workspace chat is peer-unanchored, so `scope` is always a session-union
allowlist (single name or list), fail-closed when empty. Stats and
active-peer prefetch drop scope-kind peers and honor the same allowlist.
* test: teach the unified runner `workspace_chat` and parse every case
QueryAction now accepts target=workspace_chat (SDK path, including
scope). A pytest over tests/unified/test_cases/*.json keeps the four
existing workspace-chat cases — and a new scoped one — from rotting
against the schema again.
* docs: tighten workspace-chat scope docs and judge prompt
Scoped workspace_chat uses the SDK, not raw HTTP. The scope fixture's
judge now requires the in-scope tea fact, not merely the absence of the
leak. format_sse_stream matches the peer-chat one-liner.
* fix(dialectic): restore the empty-memory fallback for workspace chat
`search_memory` auto-searches messages when a pair has no observations,
but the gate only admitted `agent_type == "dialectic"`. The workspace
executor passes `workspace_dialectic`, so workspace chat got a bare
"No observations found" and answered that it knew nothing rather than
falling through to message search.
Also fixes the two unified cases that never ran: `deriver` is not a
field on `WorkspaceConfiguration`, so both aborted at load with
`extra_forbidden`. `workspace_chat_scope` additionally enables reasoning,
since it asserts scope isolation and has no reason to depend on the
fallback path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(tests/unified): fail CI when unified tests fail
`runner.run()` tallied failures into `failed_count` and printed them, but
returned nothing, and both entrypoints ignored the result. The workflow
invokes `python -m tests.unified.run` bare, so the job has gone green on
failing and unrunnable cases since it was wired up in #291.
Return the count and exit non-zero on it. `INVALID SCHEMA` already counts
toward the tally, so a malformed case now fails the job instead of being
skipped silently.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(unified): assert scope peers stay out of workspace chat answers
Scope peers are real peer rows, so a regression in the `scope_peer_clause`
exclusion would surface `scope.therapy` through workspace stats or the
routing prefetch. Nothing asserted against that.
Adds the check to the existing scoped query and a new unscoped one, since
the two exercise different `get_active_peers` branches. Verified by
removing the exclusion, which fails the unscoped query.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix: Remove dead code references
---------
Co-authored-by: doria <93405247+dr-frmr@users.noreply.github.com>
Co-authored-by: Benjamin McCormick <docterformer@protonmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Aakash Kattelu <aakash@plasticlabs.ai>
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
Bug (verified vs SQLAlchemy 2.0.49): on GC-cleanup of an abandoned async connection, _finalize_fairy routes through fairy.detach(), which nulls the record's dbapi_connection so NullPool's close is a no-op (the `close` event never fires) then emits `detach` with the record. Listening only to close/invalidate left the marker unpopped, so db_connections_open leaked upward and never reset until restart. Listen to `detach` too — it carries the ConnectionRecord and the marker dedupes, so exactly one decrement occurs.
Also: strip a bare PR-number provenance tag from the test docstring (plastic-labs comment-reconciliation rule); disambiguate db_connections_open from the existing db_pool_connections; note in initialize_bounded_metrics that DB-instrumentation metrics zero-init in their registrar.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add db_connections_open (gauge) and db_connections_established (counter), driven by SQLAlchemy connection-lifecycle events so they report real connections under every pool class — including NullPool, where the pool-object collector (db_pool_connections) reads zero. Under NullPool the establishment rate approximates request rate.
DBConnectionTracker mirrors DBQueryInflightTracker: a ConnectionRecord.info marker makes each physical connection increment once and decrement at most once (no leak, no negative). Registered per-process in the API and deriver; zero-init via the pre-resolved labeled children, matching db_queries_in_flight_gauge.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(filter): make ne on jsonb metadata keys null-safe
* test(filter): cover null-safe ne on nested metadata keys
* test(filter): count actual rows for nested-metadata ne null-safety
Compile-only checks lock the operator map entry but don't catch wrong
row sets under three-valued logic. Adds a live messages/list case with
a message missing the key and one with empty metadata, following the
scalar-column pattern in test_negation_includes_conclusions_with_no_session.
* test(filter): type message_configs with a TypedDict
basedpyright couldn't narrow the heterogeneous metadata dict literals,
so indexing message_configs["content"] came back partially unknown and
broke the sorted() calls under type checking.
* fix(deriver): truncate oversize observations so one cannot drop the batch
simple_batch_embed raised ValueError when any input exceeded the per-input
token cap, which failed the entire deriver save when a single observation
was over-length. Add on_oversize="truncate": oversize inputs are embedded
from a token-capped prefix (re-encoded until it fits, with a warning),
preserving one vector per input. Default stays "raise" so existing callers
are unchanged. RepresentationManager opts into truncate.
Also add a live embedding test that fails on main (raise / missing kwarg)
and passes once a mixed short+oversize batch survives.
Refs #569
* fix(deriver): surface failure when all observer saves fail
When every observer's save_representation failed (e.g. embedding retries
exhausted under a sustained 429), the deriver logged the error and returned
normally, so the queue marked the work unit processed with zero documents
saved. Collect per-observer errors and, after telemetry is emitted, raise
RepresentationSaveError when no observer succeeded. Partial failures stay
processed (saved observers must not be discarded) and are recorded via an
additive failed_observer_count on RepresentationCompletedEvent.
Refs #728
* fix(embedding): guarantee truncation progress and truncate on re-embed
The retry slice in _truncate_to_token_limit always recomputed the same
keep count, so a slice whose re-encode grew past the cap could oscillate.
Decrement keep after each unsuccessful retry.
Document re-embed in the reconciler used the default on_oversize="raise",
so one oversize document failed every other document in the batch.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* chore: drop ticket ids and shrink comments to one sentence
Comments and docstrings describe current behavior, not the PR that
introduced them. Ticket numbers stay in the commit/PR.
* chore: annotate RepresentationSaveError and assert truncate on re-embed
* fix(embedding): truncate on conclusion create paths and document BPE loop
Storage callers in create_observations (API + agent tools) now pass
on_oversize="truncate" so a single oversize item cannot drop the batch.
Docstring on _truncate_to_token_limit notes why decode/re-encode is load-bearing.
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* telemetry: materialize dropped-event counter children at 0
A labeled Prometheus counter exports no series until its first labels()
call, so telemetry_events_dropped stayed invisible until an event was
actually dropped — impossible to alert on or graph, and "no drops" was
indistinguishable from "metric missing / scrape broken".
Pre-create the (namespace, reason) children at 0 on emitter start, for
each reason the emitter can emit, so the metric is always present.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* telemetry: generalize counter zero-init to all bounded-label counters
Extends #927 (which zero-inited telemetry_events_dropped) to every counter
whose label domain is bounded and known at startup, so metrics are present in
Prometheus before their first event — a missing series then signals a broken
scrape rather than "nothing happened yet".
- add initialize_bounded_metrics(instance_type) on PrometheusMetrics; call it
per-process from main.py (api) and deriver/__main__.py (deriver).
- extract a shared _touch() helper; refactor initialize_telemetry_dropped_metrics
onto it (that one stays per-emitter in start() — it's prefix-dependent).
- explicit ALL_EVENT_TYPES / HIGH_VOLUME_EVENT_TYPES registry in telemetry.events,
drift-guarded by tests that walk BaseEvent subclasses.
- only VALID (task_type, token_type, component) tuples for deriver_tokens (the
cartesian product would fabricate impossible always-0 series); only high-volume
event types for sampled_out; high-cardinality labels (endpoint, workspace_name)
left open.
- gauges: zero-init embed_now_tasks_in_flight + telemetry_buffer_size; add a new
message_embeddings_pending backlog gauge, set each reconciliation cycle and
zero-inited at deriver startup (Rajat's pending/in-flight ask).
- backfills the tests #927 shipped without.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* review: task-aware deriver combos + fail-soft gauge zero-init
I1: _DERIVER_TOKEN_COMBOS was factored task-independently, materializing the
impossible (ingestion, input, previous_summary) series — previous_summary is
summary-only. Make combos task-aware (_DERIVER_TOKEN_COMBOS_BY_TASK) so no
always-0 impossible series is fabricated, matching the PR's own goal. Tests
tightened to assert the ingestion/previous_summary series is absent.
I2: the three gauge .set(0) zero-inits were bare while the counter inits go
through the fail-soft _touch. Add _set_gauge_zero() so a gauge init can't
propagate an exception into process startup either.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(telemetry): isolate zero-init namespaces, add deriver-to-api guard
Global-REGISTRY assertions used a fixed "test" namespace, which several
other suites also pin, so another test's materialized children could
satisfy a presence assertion or break an absence one. Each test now runs
under a unique namespace resolved from settings at read time.
Adds the inverse per-process isolation test: deriver-only init must not
materialize API-only series (dialectic tokens, embed_now).
Co-Authored-By: Claude <noreply@anthropic.com>
* review: per-replica backlog gauge, drop duplicated constants and .meta refs
Addresses Vineeth's review on #927.
Blocking:
- message_embeddings_pending is a DB-global count, so drive it from
ReconcilerScheduler._scheduler_loop (runs on every replica, every
interval) instead of run_vector_reconciliation_cycle (runs off the
queue behind work-unit dedup, so one replica per cycle). Combined with
the zero-init, the old placement made every replica that never won the
work unit export a confident permanent 0. Help string now names the
owner so dashboards don't reach for sum().
- guard initialize_telemetry_dropped_metrics on METRICS.ENABLED,
matching its sibling initializer.
- drop the duplicate REASONING_LEVELS; import the one in src/config.
Non-blocking:
- walk BaseSpecialist recursively via a shared utils.types.walk_subclasses
(replaces the direct-children-only __subclasses__() and the test's
private copy of the same helper).
- derive the specialist assertion from the subclasses instead of
hardcoding two names — the hardcoded pair kept passing after
CardRefreshSpecialist landed, leaving it uncovered.
- inline the zero-init rationale and the multi-instance bucket taxonomy;
removes both pointers to a .meta design doc that is not in the repo.
Tests: new tests/reconciler/test_pending_backlog_gauge.py pins both
halves of the relocation (verified it fails when reverted).
* review: fix inert test guard, stale comments, and the REASONING_LEVELS drift claim
Second review pass on the branch. Findings, most severe first:
- tests/reconciler/test_pending_backlog_gauge.py: the _try_enqueue_task stub
was patched onto the class but declared without `self`, so calling it
raised TypeError — which _scheduler_loop swallows. The guard was inert and
the test passed for the wrong reason. Fixed the arity.
- metrics.py still commented that the backlog gauge is "set live each
reconciliation cycle". That is the exact claim the previous commit
overturned; it now contradicted the help string, the bucket-3 docstring
and sync_vectors.py.
- metrics.py claimed REASONING_LEVELS is "derived from the config Literal so
it never drifts", but config.py hand-listed it, so the earlier dedup had
quietly traded away the guarantee the original get_args() call provided.
Made it true instead: config.REASONING_LEVELS = list(get_args(...)), which
keeps the dedup and restores the invariant.
- dropped _set_gauge_zero: all three gauges it zeroed already have identical
fail-soft setters, so it was a second way to do one thing. Using the
setters also makes _handle_metric_error name the actual gauge.
- record_pending_embeddings_backlog's docstring oversold the covering index
as making the COUNT "negligible". The index makes cost proportional to the
pending backlog, not to the table — which is worst precisely when the
backlog matters. Stated honestly.
- _scheduler_loop's docstring said it only enqueues; it also refreshes the
gauge, at a cadence set by the shortest task interval.
- comment reconciliation: stripped #927 / "the generalization" temporal
anchoring, a CardRefreshSpecialist change-narration clause, and
reviewer-directed phrasing from the test file; disambiguated the
src/utils/summarizer.py path.
- CLAUDE.md had no Prometheus section at all, so the new "add a BaseEvent
subclass -> update ALL_EVENT_TYPES" obligation and the never-sum() rule
for non-additive gauges were undiscoverable from the architecture doc.
Verified: ruff + basedpyright clean (0 errors), tests/telemetry + reconciler
+ dialectic + llm 497 passed, full suite 1768 passed with only the 4
pre-existing test_document failures (OpenAI key required, reproduced on
clean origin/main). Re-confirmed the relocation guard fails when reverted.
* fix: silence the two basedpyright warnings inherited from main
CI runs `uv run basedpyright` bare, and basedpyright exits non-zero on any
warning — so these two have been failing the staticanalysis job on every
branch cut from current main, not just this one:
- src/vector_store/__init__.py:209 implicit string concatenation (#496)
- tests/test_cache_redaction.py:5 private import (#869)
Both predate this branch and are unrelated to the telemetry work; fixed
here only because they block this PR from going green. Verified: clean
origin/main also reports "0 errors, 2 warnings" and exits 1.
basedpyright now 0 errors, 0 warnings, exit 0.
* docs(telemetry): make the bucket-3 aggregation rule precise
The multi-instance taxonomy said a service-scoped non-additive metric has
"no aggregation correct once they disagree", then immediately mandated that
every instance refresh on its own timer. Those undercut each other: staggered
timers ALWAYS disagree slightly, so as written the rule reads as "ensure they
don't", which is unachievable, and it leaves the reader unsure whether max()
and avg() survived the fix.
The actual rule is bounded disagreement plus a scale-preserving aggregator.
Instances are N witnesses to one fact, not N parts of one whole, so sum() can
never be correct (it scales with replica count) while max()/avg()/quantiles
are correct precisely because the per-instance timer bounds the spread.
Wording only; no behavior change. The gauge help string already said
"max() or avg(), never sum()" — this makes the normative docstring agree
with it. Surfaced walking Vineeth's comment 3668208059 for comprehension.
* refactor(bench): import REASONING_LEVELS from config instead of re-listing
Third copy of the constant, missed when ee781c0/694e07f deduped the other
two. This one re-declared the ReasoningLevel Literal as well as the list,
so the type alias could diverge from config's with nothing to catch it —
and the list was hand-written, the variant that typechecks clean while
missing a member.
No import barrier justified it: this module already imports from src, as do
seven of its siblings in tests/bench. Concrete effect of the drift was that
a newly added sixth reasoning level would be rejected by the bench CLI's
argparse choices=.
src.config.REASONING_LEVELS is now the single definition repo-wide.
* test(telemetry): pin the METRICS.ENABLED guard on the per-emitter initializer
initialize_telemetry_dropped_metrics gained a METRICS.ENABLED guard in
ee781c0, addressing Vineeth's asymmetry comment, but nothing asserted it —
it had only the enabled half of the pair its sibling has. Deleting the guard
left the suite green, so the fix closed the asymmetry in the guards and
reproduced it one level up in the tests.
Mirrors test_init_noop_when_metrics_disabled. Verified live rather than
assumed: deleting the two guard lines turns this test red.
Uses a unique namespace, without which the absence assertion would be
satisfied by the enabled test's children rather than by the guard.
* docs(telemetry): fold zero-init why-prose behind # region ai markers
Comment/docstring-only pass over the changed files, per the groudon
comment-marker standard: the terse human-facing "what" stays visible, and
load-bearing "why" (the zero-init / absent-series-means-broken-scrape
rationale, gotchas, receipts) folds into # region ai / # ai: blocks.
Behavior-preserving: AST-identical modulo docstrings/comments vs the
pre-pass merge; ruff, ruff format --check, and basedpyright all clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: honor DERIVER_DEDUPLICATE in create_observations
The agent-tool path hardcoded deduplicate=True, so DERIVER_DEDUPLICATE=false
could not disable dedup for observations created through this path. Pass
settings.DERIVER.DEDUPLICATE, matching crud/representation.py.
* test: cover deduplicate setting is forwarded in create_observations
docker/entrypoint.sh runs under dash inside the container image. A Windows checkout with core.autocrlf=true rewrites it to CRLF, and dash aborts at startup because the carriage return becomes part of the -e flag argument.
* feat: scopes SDK surface and session allowlist on session context
Exposes the Scopes v1 facade in both SDKs, which until now was reachable
only by hand-rolled HTTP, and closes the Phase 1 gap where the session
allowlist never landed on the context route.
SDKs (DEV-2001, folds in DEV-1996)
New Scope class in both SDKs — addSessions / removeSession / sessions /
status — plus honcho.scope() and honcho.scopes() entry points, a
`scopes` option on session creation, and `scope` + `sessions` read
options on chat, chatStream, representation, and session.context.
`scope` on workspace search. Python covers sync and .aio equally.
`sessions` is sugar, not a new wire field: on the recall endpoints it
goes out as the constrained `filters: {session_id: [...]}` body, never
as a key of its own. Kept separate from the `filters` parameter on the
list/search methods on purpose — that one is the full filter DSL,
whereas the recall endpoints accept a single key and 422 on anything
else, so one name for two grammars would be a trap.
Server (DEV-2357)
`GET /sessions/{id}/context` accepts a `sessions` allowlist confining
the target's representation. Two deliberate choices worth review:
- Sent as a repeated query parameter rather than the `filters` body the
issue specced. The route is a GET and `session_id` is the only
supported key, so a JSON blob in a query string buys nothing.
- The peer card is omitted under an allowlist. Cards key on
(workspace, observer, observed) with no session dimension, so they
cannot be narrowed; returning one would leak exactly what the
allowlist exists to exclude. Same reasoning as ALLOWLIST_SAFE_LEVELS.
`scope` needs no carve-out — it swaps the observer to the scope peer,
so the card read is the scope's own.
`extract_session_allowlist` now delegates to a shared
`normalize_session_allowlist`, so the cap, id charset, and must_include
rule have one implementation across both entry points. Existing error
messages are unchanged.
Also in here
- ConclusionScope renamed to ConclusionsView in both SDKs. "Scope" now
means a named set of sessions, which that class is not — it is a view
over one observer/observed pair. ConclusionScope kept as a deprecated
alias; the package-level import path only.
- The TS HTTP client comma-joined array query params, so any list-valued
parameter arrived as one malformed entry. Fixed at buildURL rather
than the call site.
Not in this PR: the "How Scopes Work" docs guide and the Groudon
dashboard tab (both DEV-2001), and CHANGELOG entries for Phases 2a-2c,
which are still merged-but-unrecorded.
Verified: ruff, basedpyright, tsc --noEmit, biome all clean. New unit
tests cover the SDK option translation and the Scope client, but the
context route's own behavior — the 422s, the 401 membership gate, the
dropped peer card — has no test yet; the analogous chat/representation
cases in tests/test_session_allowlist.py are the place for it.
Refs DEV-2001, DEV-1996, DEV-2357
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Exposes the Scopes v1 facade in both SDKs, which until now was reachable
only by hand-rolled HTTP, and closes the Phase 1 gap where the session
allowlist never landed on the context route.
SDKs (DEV-2001, folds in DEV-1996)
New Scope class in both SDKs — addSessions / removeSession / sessions /
status — plus honcho.scope() and honcho.scopes() entry points, a
`scopes` option on session creation, and `scope` + `sessions` read
options on chat, chatStream, representation, and session.context.
`scope` on workspace search. Python covers sync and .aio equally.
`sessions` is sugar, not a new wire field: on the recall endpoints it
goes out as the constrained `filters: {session_id: [...]}` body, never
as a key of its own. Kept separate from the `filters` parameter on the
list/search methods on purpose — that one is the full filter DSL,
whereas the recall endpoints accept a single key and 422 on anything
else, so one name for two grammars would be a trap.
Server (DEV-2357)
`GET /sessions/{id}/context` accepts a `sessions` allowlist confining
the target's representation. Two deliberate choices worth review:
- Sent as a repeated query parameter rather than the `filters` body the
issue specced. The route is a GET and `session_id` is the only
supported key, so a JSON blob in a query string buys nothing.
- The peer card is omitted under an allowlist. Cards key on
(workspace, observer, observed) with no session dimension, so they
cannot be narrowed; returning one would leak exactly what the
allowlist exists to exclude. Same reasoning as ALLOWLIST_SAFE_LEVELS.
`scope` needs no carve-out — it swaps the observer to the scope peer,
so the card read is the scope's own.
`extract_session_allowlist` now delegates to a shared
`normalize_session_allowlist`, so the cap, id charset, and must_include
rule have one implementation across both entry points. Existing error
messages are unchanged.
Also in here
- ConclusionScope renamed to ConclusionsView in both SDKs. "Scope" now
means a named set of sessions, which that class is not — it is a view
over one observer/observed pair. ConclusionScope kept as a deprecated
alias; the package-level import path only.
- The TS HTTP client comma-joined array query params, so any list-valued
parameter arrived as one malformed entry. Fixed at buildURL rather
than the call site
* fix(scopes): close peer-card leak under limit_to_session, harden SDK inputs
Addresses review findings on the scopes work. All four were verified by
reproducing them, not by reading.
Peer card no longer leaks under any allowlist
The card was dropped when `sessions` was set but returned when
`limit_to_session=true` produced the identical allowlist, so a control
meant to fail closed was defeated by swapping one query parameter. It is
now gated on the effective allowlist, computed once and shared by the
representation call and the card read — the duplicated inline
conditional is what let the two drift apart.
`POST /peers/{id}/chat` still injects an unscoped card under an
allowlist (src/dialectic/chat.py fetches it on peer_card.use alone, with
no reference to session_allowlist). Left alone deliberately: that is a
behavior change to the shipped dialectic and
Scope validation messages survive the option union
ScopeOptionSchema is a union, and Zod collapses a failing union into one
`invalid_union` / "Invalid input" issue, burying the branch errors. Every
invalid scope on chat/representation reported "Invalid input" and told
the caller nothing — including the reserved-prefix case the check order
exists to surface. The rules are now a plain function applied after the
union resolves, so the specific message reaches the caller for bad
charset, reserved prefix, empty and over-cap lists alike.
Empty scope no longer fails open
`session.context({scope: ''})` and `honcho.search(q, {scope: ''})` used
truthiness checks, so an invalid scope was dropped and the call returned
*unscoped* results. Both now test against undefined so the value reaches
the schema.
Session IDs validated before reaching a URL path
`scope.removeSession('valid-session?typo')` addressed `valid-session`
with a stray query string: the wrong session removed, and reconciliation
run against it. Both SDKs now validate the charset first. Python's
`add_sessions` was unvalidated too — harmless in a JSON body, but
leaving one path checked and its sibling unchecked is how this recurs.
Also
- Corrected the `limit_to_session` description: it claimed "only used if
search_query is provided", but the allowlist reaches
_query_documents_recent unconditionally.
- Corrected the documented 1,000-session cap on `sessions`, which is
unreachable via repeated query params — the request line exceeds h11's
16 KB and nginx's 8 KB defaults at a few hundred entries, giving an
opaque 414/431 instead of a 422.
- Removed a dead route builder.
Tests: 7 new TypeScript cases and 2 new Python classes covering all four
findings. The TypeScript unit suite passes 146/146. The context route
itself is still unexercised — the card gate and the 401 membership check
remain verified by reading only.
Refs DEV-2001, DEV-1996, DEV-2357, DEV-2201
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(sdk): align the conclusions-view error message across both SDKs
The ConclusionScope -> ConclusionsView rename updated the identifier but
not the prose inside the thrown message, which the rename pattern
(\bConclusionScope\b) does not match. TypeScript ended up throwing
"managed by this conclusions view" while Python still threw "managed by
this conclusion scope" — the same error, different text per SDK.
Three server-backed conclusions.test.ts cases assert that message by
regex and failed under `pytest -k typescript`. The four equivalent Python
assertions were passing, because they matched Python's unchanged string —
so fixing only the TypeScript tests would have made the suite green with
the divergence still in place.
Brings Python's message, comments and docstring in line with TypeScript,
and updates the assertions in both suites. `grep -ri 'conclusion scope'`
is now empty.
Verified: 64 passed across tests/sdk_typescript/, tests/sdk/test_conclusions.py
and tests/sdk/test_scope_options.py — the last of which had never been run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(llm): forward provider_params.timeout to the OpenAI-compatible embedding client
#832 and #903 added a configurable request timeout for the LLM registry
and the Gemini embedding client respectively, but the OpenAI-compatible
embedding client (src/embedding_client.py) was never wired up. It
constructed AsyncOpenAI with no timeout at all, so a stalled socket
against a slow or contended OpenAI-compatible backend (e.g. a
self-hosted embedding model under load) wedges the deriver worker's
event loop indefinitely — the exact failure #785/#903 describe, just
via a code path #903 didn't cover.
EmbeddingModelConfig now carries provider_params through from
resolve_embedding_model_config, mirroring how resolve_model_config
already does it for ModelConfig, and the OpenAI branch of
_EmbeddingClient.__init__ extracts `timeout` via the existing
request_timeout_from_extra_params helper. Unset stays unset — no
existing behavior changes.
Reproduced and verified against a real self-hosted deployment (local
Ollama backend under load): before this fix, a single stuck embedding
call blocked all deriver queue processing for 20+ minutes with no
error logged, twice in one session.
* fix(embedding): use first-class timeout on embedding model config
provider_params is the LLM per-request escape hatch; embedding timeouts are
client-construction knobs and belong next to max_batch_size. Wire the field
for OpenAI and Gemini, omit the OpenAI kwarg when unset so the SDK default
stays, and keep Gemini's 10-minute floor when unset.
* test(embedding): live coverage for first-class embedding timeout
Exercise EmbeddingModelConfig.timeout on one representative OpenAI and
Gemini model: configured timeout lands on the SDK client, and a near-zero
timeout aborts before the provider answers.
---------
Co-authored-by: Aakash Kattelu <aakash@plasticlabs.ai>
* fix(deriver): delimit examples and forbid example leakage
Wrap the minimal deriver EXAMPLES block in <examples> tags and add an explicit negative instruction: the examples are fabricated format illustrations only, and every conclusion must be grounded in the <messages> block. Soft mitigation for example-content leakage; real fixes (sources_indices + SFT / structural enforcement) are follow-ups.
* test(deriver): drop low-value examples-delimiter unit test
* fix: enforce explicit-document session purity in dedup/merge paths
Audit for DEV-2000 (Scopes RFC prerequisite): explicit-level documents must
stay session-pure so scope memory can be built by copying explicit documents
between collections. Two classes of violation were possible:
- Exact-content and semantic dedup in crud/document.py matched candidates
with no level or session scoping, so an explicit document could be
reinforced by — or soft-deleted in favor of — a same-content document from
a different session or a different level (silently merging cross-session
derivations into one row).
- The generic create_observations tool handler accepted level='explicit'
from agents with no message context (dreamer/dialectic), which would mint
session-less explicit documents.
Enforcement (refuse, never rewrite):
- create_documents refuses explicit documents with a null session_name
- exact dedup keys on (content, level, session-for-explicit); derived levels
keep cross-session consolidation
- is_rejected_duplicate scopes candidate search to the same level, and the
same session for explicit documents
- the create_observations tool rejects explicit-level input outside message
ingestion (deriver) context
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: add card_refresh dream type for event-driven peer-card updates
Adds a lightweight dream variant (DEV-2000, Scopes RFC prerequisite) that
runs ONLY the peer-card update — for event-driven refreshes such as scope
membership changes and cold starts:
- DreamType.CARD_REFRESH alongside OMNI; dispatched by process_dream to a
new run_card_refresh_dream orchestration
- CardRefreshSpecialist: restricted to get_recent_observations,
search_memory, and update_peer_card (no observation-mutating tools), with
a low tool-iteration cap of min(6, DREAM.MAX_TOOL_ITERATIONS)
- rebuild=True mode carried in the dream payload: the existing card is NOT
injected into the prompt and the specialist rebuilds it solely from
observations present in the collection (for use after removals)
- enqueue-able via the manual enqueue_dream path (bypasses volume gates);
the work-unit key already embeds the dream type so a card refresh never
collides with a pending omni dream. POST /v3/workspaces/{id}/schedule_dream
accepts dream_type=card_refresh plus the rebuild flag
- card refreshes never advance the omni dream guard pair
(last_dream_at / last_dream_document_count)
- shared PEER CARD prompt section extracted (verbatim) from
DeductionSpecialist for reuse; CallPurpose gains dream.card_refresh
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: reserve scope__ peer namespace with kind flag and guardrails
Introduce the scope peer namespace (scope__<name>) and the authoritative
{"kind": "scope"} configuration flag, plus the server-side guardrails:
- src/utils/scopes.py: single source of truth for the prefix, kind flag,
and name helpers (scope_peer_name / is_scope_peer_name /
scope_name_from_peer / validate_no_scope_peer_names)
- reject reserved-prefix names on peer get-or-create (422)
- reject scope peers as message authors in crud.create_messages (422)
- reject scope peers as chat/representation targets (422); a scope peer
as the path-level observer is deferred to Phase 2b
- reject scope peers on the generic session-peer add/set/remove routes
and the session-create peers mapping (422, directing to scopes routes)
- peers.list excludes scope peers by default; new PeerGet.kind option
("scope" | "all") switches the view via a configuration JSONB filter
- schemas: Scope / ScopeCreate / ScopeSessions(Add) and
SessionCreate.scopes (unprefixed scope names, validated)
Part of DEV-1997 (Scopes RFC DEV-1970).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: add scopes CRUD routes and session-create scopes wiring
New /v3/workspaces/{workspace_id}/scopes facade (workspace-level auth;
peer- and session-scoped keys are rejected):
POST "" create-or-get (201/200)
POST /list paginated scope list
GET /{scope_id} single scope
POST /{scope_id}/sessions add memberships
DELETE /{scope_id}/sessions/{session_id} remove membership
GET /{scope_id}/sessions list member session ids
- crud/scope.py: get_or_create_scopes stamps the backing peer with
{"kind": "scope", "observe_me": false} and refuses to adopt a
legacy peer occupying the reserved name without the flag (409)
- memberships are session_peers rows with observe_others=true /
observe_me=false — identical to a hand-built observer peer
- SessionCreate.scopes: create-or-get each scope peer and add the
membership at session creation (the no-backfill common path)
- crud/session.py: public upsert_session_peers wrapper so the facade
bypasses the route-level guardrails without reaching into privates
Backfill of pre-existing documents and reconciliation on removal land in
DEV-1999; membership only affects messages ingested after the change.
Part of DEV-1997 (Scopes RFC DEV-1970).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: cover scopes facade, guardrails, and observer semantics
- create-or-get idempotency, list/get, name validation, legacy-collision
rejection (409), auth scoping (workspace key ok, peer/session keys 401)
- reserved prefix rejected on peer create; peers.list kind filtering
- scope peers rejected as message authors, chat/representation targets,
and on the generic session-peer routes
- membership add/list/remove with observe_others=true / observe_me=false
row shape asserted via DB, and facade-less equivalence with a
hand-built observer peer
- end-to-end litmus: after adding a session to a scope, the deriver
enqueue fan-out includes the scope peer as an observer
- session creation with scopes: [a, b] creates both memberships
Part of DEV-1997 (Scopes RFC DEV-1970).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: scope backfill-by-copy and removal reconciliation jobs
Retroactive scope membership changes (DEV-1999):
- New queue task types scope_backfill / scope_removal with payloads,
work-unit keys ({task}:{workspace}:{scope_peer}:{session}), deduped
enqueue (mirrors enqueue_dream), and consumer dispatch.
- Backfill copies a session's explicit documents from each sender's
global (P, P) collection into the scope's (scope_peer, P) collection
with internal_metadata.copied_from as the idempotency marker;
soft-deleted copies from an earlier removal are restored, so
add -> remove -> re-add converges on exactly one live copy. Completion
enqueues one manual omni dream per touched collection.
- Removal soft-deletes the session's explicit documents in the scope's
collections and cascades (fail-closed, transitively) to derived
documents whose source_ids intersect anything removed, deletes the
vectors from the external store, then enqueues a card_refresh dream
with rebuild=True plus a manual omni dream per touched collection.
- Zero LLM re-derivation: explicit documents are session-pure (DEV-2000
invariant); the only external call is re-embedding rows whose
embedding column is NULL (external-store deployments).
- Per-session job status lives in the scope peer's internal_metadata
under backfill_status, written via single-statement JSONB merges
(concurrent-writer safe) and surfaced at
GET /v3/workspaces/{w}/scopes/{scope_id}/status.
- Enqueued from the scopes add-sessions route and SessionCreate.scopes
handling, only when the session already has messages.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: add scope backfill/removal test coverage, fix backfill status JSONB bug
Adds tests/deriver/test_scope_backfill.py covering the DEV-1999 scope
backfill-by-copy and removal reconciliation jobs implemented in a prior
commit: explicit-doc copying, copied_from idempotency (including
add->remove->re-add), multi-peer collection routing, removal cascade to
dependent derived docs, dream enqueues (manual omni on backfill;
card_refresh rebuild + omni on removal), the status route, and add-sessions
route wiring (backfill enqueued only when the session already has messages).
Fixes a real production bug surfaced by these tests: update_scope_backfill_status
passed json.dumps()'d strings through SQLAlchemy cast(..., JSONB), which
double-encodes (psycopg re-serializes the already-JSON string), producing a
JSONB string scalar instead of an object. Postgres's `||` between two
non-array jsonb scalars doesn't merge — it silently wraps both into a
2-element array, corrupting backfill_status into a list. This crashed
clear_scope_backfill_status's `#-` path delete (called on every removal)
with "path element is not an integer" once a session had ever completed a
backfill. Fixed by passing raw Python dicts to cast() instead, mirroring the
working pattern already used in update_collection_internal_metadata.
Also adds src.deriver.scope_backfill.tracked_db to conftest's tracked_db
patch list — the module was missing from that per-import-site allowlist, so
its DB work ran against the real configured database instead of the
isolated per-test database.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(crud): preserve cache invalidation across get_or_create retry
`get_or_create_peers` and `get_or_create_scopes` mutate existing rows, then
insert new ones inside `db.begin_nested()`. When a concurrent writer creates
one of those rows first, the insert raises IntegrityError and the function
retries.
`begin_nested()` autoflushes the pending UPDATEs *before* opening the
savepoint, so the rollback neither undoes them nor expires the now-clean ORM
state. The retry then compared already-updated values, found no change, and
dropped those peers from `changed_peers` — skipping the cache purge while the
row change committed anyway, leaving entries stale until the 300s TTL.
Carry the mutated names into the retry via `_pending_invalidation` so the
purge cannot be lost.
The scopes facade mirrors `get_or_create_peers`, so both copies carried this.
The peer path is pre-existing and runs on every message ingest.
Also add the missing /v3 prefix to `_SCOPES_ROUTE_GUIDANCE`, which pointed
callers at a 404.
Adds tests/crud/test_get_or_create_retry_invalidation.py, which drives a real
racing session and fails without this change.
* fix(scopes): make scope identity unforgeable, unblock non-pattern peer names
Three coupled changes to the scopes facade.
1. `PeerCreate` no longer gates internal lookups. It exists to validate a new,
user-supplied peer id at the API boundary, but crud used it as a DTO for
names that already exist, so any name outside RESOURCE_NAME_PATTERN raised a
raw pydantic ValidationError — which is not a HonchoException, so it fell
through to the catch-all handler as an HTTP 500. Adds `PeerSpec` (same
fields, no charset pattern) as `PeerCreate`'s base, widens
`get_or_create_peers` to accept it, and changes `get_peer` to take a plain
str. All 13 construction sites converted; the create route keeps full
validation.
This unbreaks the Dreamer: DreamScheduler passes `collection.observer`
straight into the specialist preflight, and scope peers have
`observe_others=true`, so every `(scope.x, peer)` dream died there — the
feature scopes exist to enable. It also fixes a pre-existing bug unrelated
to scopes: a peer named `alice.smith` (legal before d429de0e5338, which
validated names by length alone) 500s on message create, session peer add,
and peer update.
2. The `kind` flag moves from `configuration` to `internal_metadata`.
`configuration` is user-writable — `PeerCreate`/`PeerUpdate` accept a
free-form dict and `update_peer` replaces it wholesale — so a legitimate
`{"observe_me": true}` update silently dropped the flag, and a forged
`{"kind": "scope"}` injected an ordinary peer into `POST /scopes/list`.
`internal_metadata` appears in no API schema. `observe_me: false` stays in
`configuration`, where it belongs.
3. Scope identity requires prefix AND flag, via `is_scope_peer()` and
`scope_peer_clause()`. Neither half is forgeable: the prefix sits outside
RESOURCE_NAME_PATTERN, `internal_metadata` is unreachable. Usage-site guards
become flag-based so a legacy peer merely occupying the namespace keeps
working rather than 422-ing on its own traffic; peer create and update stay
name-based, since those must stop new names entering the namespace.
`update_peer` now returns 422 instead of 500.
Also swaps the reserved prefix from `scope__` to `scope.`: `_` is inside
RESOURCE_NAME_PATTERN, so any tenant could already own a `scope__x` peer.
No DB migration — `internal_metadata` already exists on `peers`, and no scope
peers exist in any deployment yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): validate peer names on create, close namespace squatting and upsert race
Addresses three review findings against 48047a6a.
1. `PeerSpec` let API callers create invalid and reserved-prefix peers.
Widening `get_or_create_peers` to accept a pattern-free schema fixed the
lookup 500s but also removed validation from the *insert* path, and
request-controlled names reach it via message authors, session peer maps, and
the chat observer path — none of which carry a charset pattern of their own.
Confirmed: `POST /sessions/{id}/messages` with `peer_id: "scope.x"` returned
201 and minted an unflagged squatter, after which `POST /scopes {id: x}` was
permanently 409-blocked — namespace denial of service by any caller able to
post a message. `peer_id: "not a valid name!@#"` was likewise created.
Fixed by validating only names about to be INSERTed
(`_validate_new_peer_names`), so already-existing names — legacy dotted
names, scope peers — still resolve without a spurious 422. That keeps the
Dreamer fix intact, since it reads through `get_peer`.
2. Existing reserved-prefix squatters could not be updated. The name-based guard
on `PUT /peers/{peer_id}` refused every `scope.` name, contradicting the
invariant that an unflagged squatter stays a normal peer. Now flag-based, so
behavior is three-way: a real scope is refused, an existing unflagged peer
updates, and a missing reserved-prefix name is refused by (1) rather than
minted.
3. Scope checks raced with get-or-create and the membership upsert. The
route-level guards run before peers are resolved, so a scope created
concurrently in that window would be attached by the generic path with a
default `SessionPeerConfig()`, clobbering its observer membership config.
Adds `_reject_resolved_scope_peers`, which runs on the resolved rows in the
same transaction as the upsert — no window, no extra query. The early checks
stay for better error messages.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): guard scope membership config, move checks to the mutation point
Addresses a second review pass against 10655792.
1. Scope membership configuration was directly user-mutable.
`PUT /sessions/{id}/peers/{peer_id}/config` had no scope guard at all, and
`crud.set_peer_config` resolved the peer only to discard the row. Confirmed:
posting `{"observe_others": false, "observe_me": true}` for a scope returned
204 and persisted, which silently stops all fan-out into the scope and makes
Honcho form a representation *of* a scope — neither of which is reachable
through the facade. Deterministic, no race required. Now checked on the row
`get_peer` already returns, so it costs nothing and cannot race.
2. Empty and over-long names were still 500s. Removing the charset pattern from
`PeerSpec` fixed one trap but left its length bounds, and request-bound peer
names carry no length limits of their own — so `peer_id: ""` or a 513-char
name reached `PeerSpec(...)` and raised a raw pydantic ValidationError that
the catch-all turned into a 500. `PeerSpec` now carries no constraints at all
(matching its documented purpose) and every rule for a new name lives in
`_validate_new_peer_names` on the insert path.
3. Resolved-row protection generalized. The previous pass applied it only to
membership upserts, leaving check-then-use windows elsewhere: peer update
could have a concurrently-created scope's configuration replaced wholesale
(create-path validation does not fire for a peer that now exists), the chat
observer get-or-create could resolve a fresh scope as its observer, and the
generic session-peer removal could silently detach a scope from its sessions.
Each now inspects the resolved peer immediately before acting; the redundant
name-level guard on the update route is dropped in favor of the race-free one.
`remove_peers_from_session` grows an internal `_allow_scope_peers` flag because
the scopes facade ends membership through that same path and must not be blocked
by its own guard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(scopes): enumerate every peer-touching route against a scope policy
Three review passes each found the same class of defect: a route nobody had
checked, rather than logic that was subtly wrong. One of them — `PUT
/sessions/{id}/peers/{id}/config`, which let any caller set a scope to
`observe_others=false` and silently stop all fan-out into it — predated this work
entirely, because the guardrail set was assembled guardrail-by-guardrail instead
of derived from the route list. Sampling review cannot close that kind of gap;
enumeration can.
Derives every route through which a peer name can reach the system and requires
each to be classified as GUARDED or EXEMPT-with-a-reason, so a newly added
peer-touching route fails the suite until someone classifies it. Detection is the
union of path shape and a walk of the dependant tree (including sub-dependency
`Form(...)` params and nested request-body models), because neither signal alone
suffices: parameter names miss `POST /sessions/{id}/peers`, whose peer names are
dict keys, and path shape misses `messages/upload`, whose `peer_id` arrives as a
form field behind a parser dependency.
Both invariants are then asserted behaviorally, by calling the routes rather than
inspecting annotations — the guards deliberately live in crud, which is what makes
`messages/upload` guarded for free via `crud.create_messages`:
- a real scope is refused on all 11 guarded routes, and the rejection must name
the scope, so an unrelated 422 (a malformed body) cannot pass the assertion;
- an *unflagged* peer merely occupying the reserved namespace is unaffected. That
half regressed once already when `update_peer` used a name-based check.
Mutation-tested all three failure modes: disabling the `set_peer_config` guard
fails the guarded test naming that route; regressing `update_peer` to name-based
fails the squatter test; adding an unclassified peer route fails the enumeration.
Covers the HTTP surface only. Peer names also reach the system through the
deriver, dreamer, and queue, which have no route table to enumerate — noted in
the module docstring rather than implied to be covered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): refuse a scope in every observed position; enumerate per position
Addresses a fourth review pass against 62681e4d. The headline finding is that the
previous commit's enumeration test had the wrong *model*, not a missing entry.
1. Manual conclusions could create knowledge about a scope. `POST /conclusions`
validated only that observer_id and observed_id exist, so a scope as
`observed_id` persisted a conclusion about a peer carrying observe_me=false and
created an (observer, scope) collection for it. Confirmed: 201, and it read
back. `POST /schedule_dream` had the same hole via `observed`.
The fix is positional, because the invariant is:
A scope may be an OBSERVER. A scope may never be OBSERVED.
A scope as `observer_id` is how scoped conclusions are stored and must keep
working (verified still 201); as `observed_id` it is now refused. Same split
applied to schedule_dream `observed`, the peer-card `target` (which also covers
a scope's self-card, since target-omitted collapses observed to peer_id), and
session-context `peer_target`.
2. Chat target and both representation roles kept check-to-use races. Only the
chat path-level observer was re-checked on its resolved row; the target was
checked by name and then resolved without inspecting scope identity. Both are
now checked at the dialectic preflight, where observer and observed are already
resolved — an absent name has already failed by then, and an existing squatter
cannot retroactively become a scope.
3. Generic membership removal was still racy. The adjacent SELECT narrowed the
window but could not close it under READ COMMITTED. The UPDATE now carries its
own correlated NOT EXISTS against scope_peer_clause(), so Postgres evaluates
the exclusion as part of the statement and a scope committed after the advisory
check still cannot be detached.
4. New-name validation ran after the name reached Postgres. A NUL byte passed the
request schemas and PeerSpec, then raised psycopg.DataError inside the lookup —
a 500. Values that cannot correspond to a stored row by construction (NUL
bytes, over-length names) are now refused before the query.
(Over-length names already returned 422; only the wasted query was real there.)
The enumeration test is rekeyed from (method, path) to (method, path, position).
A binary per-route verdict cannot express finding 1 at all: `POST /conclusions` is
one route with two positions and opposite verdicts. Detection widens to observer /
observed / target / peer_target / peer_perspective, which surfaced four routes the
previous version never saw — conclusions, schedule_dream, queue/status, and
session context.
Also registers `src.routers.workspaces.tracked_db` in the conftest patch list; the
new guard there would otherwise have run against the real configured database
instead of the per-test one.
Mutation-tested: disabling the conclusions observed-guard fails the positional
test naming that position; adding an unclassified `observed_id` param fails
enumeration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): refuse future scopes in observed positions, preserve scope membership
Addresses a fifth review pass against 14136e5b. All six findings reproduced
locally before fixing.
1. High — generic peer replacement removed scope memberships.
`set_peers_for_session` soft-deleted every active SessionPeer row, and the
request-level guard only inspected names *present* in the replacement map. A
caller detached a scope by simply omitting it, never naming it — so no
request-level guard could ever see it. Reproduced: scope sessions went
['<id>'] -> [] on a 200. The exclusion now lives in the UPDATE itself
(correlated NOT EXISTS against scope_peer_clause), so replacement means
"replace ordinary peers" regardless of request contents or concurrent creation.
2. High — peer cards could be pre-seeded for future scopes.
`set_peer_card` resolves only the observer and writes a JSONB key derived from
an unchecked observed name, and the route guard rejected only *existing*
flagged scopes. Reproduced: PUT card with target=scope.<missing> returned 200,
creating that scope then returned 201, and the card described the real scope.
3. High — dreams could be queued for future scopes.
The route checked `observed` in a read-only session that closed before
`enqueue_dream`, and a missing reserved name passes any is-it-a-scope check.
Reproduced: 204 with observed=scope.<missing>.
2 and 3 share a root cause, so they share a fix: a new `reject_scope_observed`
that is stricter than `reject_scope_peers` in exactly one case — a *missing*
reserved name is refused, because nothing on these paths creates the peer, so
nothing else would ever catch it. Existing unflagged squatters still pass.
Both guards moved to the mutation point: card validation into
`crud.set_peer_card` (same transaction as the JSONB write, so Dreamer and
agent-tool callers are covered), dream validation into `enqueue_dream` (same
transaction as the queue insert). The redundant route-level checks are dropped
rather than left as weaker duplicates.
4. Medium — prefixed NUL names still reached PostgreSQL.
`reject_scope_peers` filtered for the reserved prefix and sent matches to a
text comparison, so "scope.future\0name" raised psycopg.DataError — a 500.
Both guards now share `_reserved_name_candidates`, which materializes the input
once and rejects impossible values before any SQL. Materializing matters
independently: the message-author path passes a generator, and validation
iterates separately from the prefix filter, so a generator would be
half-consumed. `_reject_impossible_peer_names` now takes a Collection so the
type checker enforces that.
5. Medium — representation kept a check-to-use race.
The previous commit claimed both representation roles were rechecked after
resolution; that was wrong — only the dialectic preflight got that check, and
the representation route never goes through it. It now opens one short
read-only session *after* the embedding call, checks both positions, and passes
that same session to `get_working_representation`, so no connection is held
across external work and a scope committed later cannot have conclusions in the
collection being read.
6. Low — policy coverage was not exhaustive. `sender_id` reaches CRUD as
`observed` but was missing from the detected parameter set. ALLOW cases could
also not carry builders, so the suite never proved the other half of the
contract — that legitimate scope *observers* keep working, which a guard
rejecting scopes everywhere would satisfy. Both fixed; observer positions on
conclusions, dreams, cards, session context and queue status are now asserted
behaviorally.
Deliberately not implemented: the scope-creation backstop scanning for
pre-existing card keys and queue items naming a future backing peer. Reasoning is
recorded in `get_or_create_scopes` — no new such state can be created now, any
pre-existing row is coincidental since `scope.` was never a meaningful namespace,
the consequence is inert, and detecting card keys means a full table scan per
scope creation.
Mutation-tested each new guard: removing the replacement exclusion fails both
membership-preservation tests; weakening either observed guard to existing-only
fails the pre-seeding tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): exclude scope memberships from the session observer limit
Scope memberships carry observe_others=true, so every scope counted against
SESSION_OBSERVERS_LIMIT (default 10) — capping scopes-per-session at the limit
minus the session's real observers, and reporting the failure as
`400 Cannot create session <name> with 11 observers. ... Observers are peers
with 'observe_others' set to true.` on a membership call. Wrong on three counts:
the ceiling is undocumented and contradicts RFC §5.1 ("sessions belong to any
number of scopes"), the message describes session creation, and it leaks the
word "observer" through a facade whose entire job is hiding observers (RFC
goal 5). The limit exists to bound per-observer deriver fan-out for real peers;
a scope costs document rows, not LLM calls (RFC §5.2), so it does not belong in
that budget.
Excluded from both halves of the check in `_get_or_add_peers_to_session`: the
incoming names via a flag-based lookup, existing memberships via a correlated
NOT EXISTS on `scope_peer_clause()` — the same pattern the replacement and
removal paths already use, so the exclusion holds regardless of concurrent
scope creation. The early `count_observers_in_config(session.peer_names)` check
in `get_or_create_session` is left alone: `peer_names` cannot contain a scope,
and `scopes` is a separate field.
`reject_scope_peers` is split into a `scope_peer_names()` query helper plus a
two-line raiser so the observer count reuses the authoritative name-AND-flag
predicate instead of growing a third copy of it. Still costs nothing on the
common path — no reserved-prefix name in the input means no query at all.
Also caps `SessionCreate.scopes` at 100, matching `ScopeSessionsAdd.session_ids`.
This belongs in the same commit: the observer limit was the only thing bounding
that list, so removing it turns an unbounded `scopes` array into a peer row and
a membership row per element, committed — the single-request path to the
cardinality anti-pattern RFC §8 warns about. Partly answers OQ6: no per-session
cap, 100 per request.
Tests: a session joins SESSION_OBSERVERS_LIMIT + 2 scopes through both the
facade and session creation; real observers over the limit still 400, so the
carve-out cannot quietly disable the limit; 101 scopes is a 422.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): skip semantic retrieval when the embedding precompute failed
Addresses three open review comments.
1. Major — the representation read could embed inside its DB session. The
route's precompute is suppressed, and both
`RepresentationManager.get_working_representation` and
`crud.query_documents` fall back to embedding when a query arrives without
one, so a failed precompute meant an external call inside the read session
this branch opens for the scope re-check — the connection-holding rule the
route's own comment claimed to satisfy. The innermost fallback also only
catches ValueError, so a provider outage surfaced as a 500. The semantic
query is now passed only when an embedding exists, degrading to
derived+recent retrieval. (`crud.query_documents` embedding inside a caller's
session predates this branch and is left alone.)
2. Minor — `test_resolved_scope_peer_rejected_at_membership_upsert` described a
race it does not perform. It creates an already-flagged scope and calls crud
directly; the unflagged → flagged transition is not simulated. Docstring now
says what the test actually pins.
3. Minor — `test_empty_replacement_preserves_scope_membership` asserted only
half its docstring. It passed if the empty PUT left every ordinary
membership intact; now asserts the ordinary peer's left_at is set.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): close auth and observed-position gaps, paginate membership
Review response for #884.
Security:
- gate `SessionCreate.scopes` behind a workspace-level key; the session-create
route is self-authorizing, so a peer- or session-scoped token could mint scope
peers and join sessions to scopes it had no access to via `POST /scopes`
- refuse a reserved-but-nonexistent name in two observed positions that used the
permissive guard: chat `target` and session-context `peer_target`. Both let a
caller act on `scope.X` before it existed, then create the scope
Facade:
- exclude scope peers from `GET /sessions/{id}/peers` and refuse the membership
-config read for a real scope, matching its write side
- replace `GET /scopes/{id}/sessions` with `POST /scopes/{id}/sessions/list`
returning `Page[Session]`; the add route now returns 204. Membership was
unbounded on both, while every other list surface paginates
- rename `crud.get_scope` to `get_scope_or_raise`
Tests:
- add a missing-name axis to the route-policy table (`Case.refuse_missing`), which
is what surfaced the two guard gaps above
- delete 14 hand-written tests the table now enumerates; 52 -> 39 functions in
test_scopes.py with more cases covered
- tighten the squatter assertion from `!= 422` to `< 400`, which was passing on 5xx
- assert the FastAPI-internals traversal still derives positions, so a framework
upgrade can't silently empty the suite
Docs:
- drop internal ticket and RFC references from the published OpenAPI descriptions
and surrounding comments; state the behavior instead
- move implementation reasoning out of the `PUT /peers/{id}` docstring, which
FastAPI publishes, into a comment
* test(scopes): assert exact statuses for permissive missing-name cases
Follow-up review pass on #884.
- add `Case.missing_status` so a permissive missing-name position asserts the
status it should actually get (404, or 200 for the no-op removal) instead of
`!= 422`, which also passed on a 5xx — the same hole already closed in the
squatter assertion
- require it whenever `refuse_missing` is False, and require its absence when
True, so the policy table can't drift from the assertion
- repoint a stale allow-reason at POST /scopes/{scope_id}/sessions/list; the GET
it named was removed
- document the membership list's ordering under `reverse`
* chore: clean up stale docstring language
* fix: don't backfill a session that left the scope
scope_backfill and scope_removal carry different work-unit keys, so
nothing orders them: a removal enqueued right after the add — or one
that lands while the backfill is embedding — sweeps the scope before
the copies exist, leaving a departed session's documents live in the
scope forever.
_run_backfill now re-checks SessionPeer membership inside the write
transaction and returns None; process_scope_backfill then skips both
the dream enqueues and the status write, so a skipped backfill can't
resurrect the status entry removal just cleared.
Adds coverage for the skip, the NULL-embedding re-embed path, and the
failed-status write. Handler-driven tests now stand up the membership
row the guard requires.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(mcp): search conclusions via search tool
Port of plastic-labs/claude-honcho#113 to the standalone MCP worker.
The search tool only queried the message store, so saved conclusions
were unreachable through search (findable only by paging
list_conclusions or via query_conclusions with a known scope).
search now also queries conclusions in parallel with the message
search, returning {messages, conclusions}. Since the conclusions
query API requires an explicit (observer, observed) pair, the
conclusion leg runs only when peer_id is given (self-conclusions,
matching list_conclusions/query_conclusions defaults) and degrades
to [] on error so search never gets worse than before. Conclusion
results include IDs usable with delete_conclusion.
Also syncs mcp/bun.lock with package.json's @honcho-ai/sdk ^2.1.0
(the lock still recorded ^2.0.0) and points delete_conclusion's
description at query_conclusions/list_conclusions for ID discovery.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: add support for filtering to mcp
* chore: minor nits
* fix: adding better error handling
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(filter): reject unknown operator dicts on scalar columns
An unrecognized operator dict on a non-JSONB column (e.g.
{"session_id": {"operator": "null"}}) fell through to `column == value`,
binding a dict to a VARCHAR parameter. That compiles, then fails in the
driver at execute time with "cannot adapt type 'dict'" — an unhandled
500 for what is invalid input.
Raise FilterError (422) instead. The guard lives in the shared
_build_field_condition, so every route through apply_filter is covered.
It keys on the actual column type rather than the JSONB_COLUMNS name
list, so dict equality still works on JSONB columns reachable through
Document's raw-key fallback (e.g. source_ids), where the driver adapts
dicts fine.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore(filter): name psycopg explicitly in comment
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(filter): handle null operands and non-numeric columns in comparisons
Two defects in _build_comparison_conditions, both reachable from any
route that accepts filters:
1. A null operand hit float(None), raising TypeError where only
ValueError was caught — an unhandled 500. A null operand is a null
check, not a value comparison, so {"ne": null} now compiles to
IS NOT NULL and the other operators reject null with a 422. Equality
against null already produced IS NULL via _build_field_condition.
2. Numeric operators float()-cast on every column type, so a string
inequality on a text column ({"session_id": {"ne": "abc"}}) was
rejected as an invalid number. Coercion is now gated on the column
actually being numeric; text columns compare as text. Numeric columns
still validate, and TypeError is caught alongside ValueError.
Existing ne coverage only exercised the JSONB metadata path, which uses
_safe_numeric_cast and handles strings — the scalar column path was
untested. Adds cases for both.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(filter): fail closed on unrecognized filter shapes
The filter body is arbitrary client JSON with no schema, so validation
was emergent: any shape the DSL didn't recognize surfaced as an
unhandled 500 from somewhere in SQLAlchemy or psycopg. Fixing individual
shapes doesn't converge — a fuzz over the DSL found five more families
beyond the three already fixed here:
{"AND": [None]} TypeError, non-dict in a logical list
{"AND": [[]]} AttributeError on .items()
{"session_id": {"gte": true}} SQLAlchemy ArgumentError
{"embedding": []} NotImplementedError, no python_type
{"session_id": {"ne": {...}}} execute-time "cannot adapt type 'dict'"
Two generic guards instead:
1. Any operand bound to a non-JSONB column must be a scalar, checked
element-wise for `in`. A dict or list bound to a scalar column
compiles cleanly and only fails in the driver at execute time, so it
has to be rejected during construction. JSONB columns are exempt —
a dict there is a containment match.
2. apply_filter fails closed: FilterError propagates, anything else is
logged with logger.exception (filter shape included) and re-raised as
FilterError. Unknown filter failures become 422s while staying fully
visible as errors rather than being swallowed.
Adds two invariant tests over a generated matrix of filter shapes: every
shape either compiles or raises FilterError, and no non-scalar is ever
bound to a scalar column. Both fail without the guards above. They cover
shapes nobody enumerated, so the next unimagined body fails in CI rather
than in production.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: clear the two remaining basedpyright warnings
`uv run basedpyright tests/ src/` reported two warnings in files that
predate this change. The pre-commit hook is file-scoped, so neither was
visible unless the owning file was touched.
- src/vector_store/__init__.py: join the lancedb error message with
explicit `+` instead of adjacent literals (reportImplicitStringConcatenation).
- tests/test_cache_redaction.py: the test covers a private helper
deliberately, so annotate the import (reportPrivateUsage).
No behavior change; whole-tree check is now clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(filter): keep numeric operands exact instead of coercing to float
float() rounds any integer past 2**53 and flattens a Decimal, so
{"token_count": {"gt": 9007199254740993}} silently compared against
9007199254740992 — a different row set than the client asked for.
_coerce_numeric passes already-numeric operands through untouched and
only parses strings, trying int() before float() so "5" stays exact
while "5.5" still parses. bool narrows to int: it is an int subclass,
but binding it as a boolean against a numeric column produces SQL
Postgres has no operator for.
Not coerced to the column's own type: int(5.5) would turn
{"token_count": {"lt": 5.5}} into `lt 5`, changing which rows match.
Also fixes a vacuous assertion in test_dict_on_jsonb_column_still_works.
It checked for "internal_metadata" in the whole statement, but that name
is in the SELECT projection either way, so the test passed even when no
WHERE clause was applied. Now asserts on stmt.whereclause and that the
filter payload is actually bound.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(filter): bind boolean columns as booleans, numerics without a cast
Boolean columns were treated as numeric because bool subclasses int, so
`{"is_active": {"ne": true}}` coerced true to 1 and Postgres rejected
`boolean <> integer` at execute time. Confirmed against a live database:
every form except bare equality with a native boolean was a 500.
Boolean columns now get their own branch: native true/false bind as
booleans, and any other operand is a FilterError. SQLAlchemy types the
bind from the operand rather than the column, so "true" renders
`is_active = %(param)s::VARCHAR` and Postgres has no such operator — a
422 is the honest answer. String booleans have never worked, are absent
from the docs (every documented boolean is inside metadata, which is
JSONB containment and unaffected), and produced no Sentry events in 90
days, so nothing can depend on the current behavior.
Also corrects the previous commit. Coercing operands to exact ints made
SQLAlchemy render an ::INTEGER cast, so any value past int4 — not 2**53
— started failing with "integer out of range" where float() had silently
compared as a double. Decimal keeps the value exact and renders no cast,
matching what float() did. The `in` branch never went through coercion
at all, so {"token_count": {"in": [1, 2147483648]}} was a 500 before
this PR too; it now takes the same path.
Verified end to end against the live database, not just at compile time.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(filter): coerce every operand against its column's type in one place
The DSL had two operand paths with different rules. Comparison operators
parsed datetimes and coerced numbers; bare equality bound whatever it was
handed. SQLAlchemy types a bind from the operand rather than the column
and the psycopg dialect renders that type as an explicit cast, so a
mismatch compiled into valid-looking SQL and failed at execute time:
"operator does not exist: timestamp with time zone = character varying".
A matrix of column type x operand type x operator against a live
database found 462 combinations, of which 55 built cleanly and then
failed. The most plausible was a filter someone would write first try:
{"created_at": "2026-01-01"} 500
{"created_at": {"gte": "2026-01-01"}} worked
_coerce_operand now handles every operand, whatever the operator, keyed
on the column's real type: JSONB takes an object, boolean takes only
true/false, datetime parses strings, numeric goes through _coerce_numeric,
text requires a string, and a column with no python_type (pgvector) is
not filterable. eq/ne/gt/in cannot drift apart because they share the
one call; `in` coerces element-wise, since a single element's type
decides the cast rendered for that parameter. The matrix is now clean.
This is a net deletion: the separate datetime, numeric, in-datetime and
boolean branches, plus _require_bindable_operand, all collapse into it.
Two more execute-time failures fixed on the way. `contains` was keyed on
column_name == "h_metadata", so Document's equally-JSONB
internal_metadata fell through to ILIKE and produced `jsonb ~~* text`;
it now keys on the column type. And {"source_ids": "abc"} was
`jsonb = character varying`.
Closed-set columns are validated against the Literal that defines them,
so declaring a new level or sync state updates filter validation with no
change here. {"level": "banana"} was silently matching nothing.
Empty IN is now always applied rather than skipped. Unifying the branches
inherited a guard that had only ever wrapped the datetime path, which
dropped the condition entirely and widened the query to every row —
fail-open on an empty allowlist, which session scoping relies on to fail
closed (see extract_session_allowlist). Caught by an existing test that
asserts returned rows; the fuzz and the type matrix only check for
errors, so neither would have seen it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(filter): make NOT and ne include rows where the field is unset
NOT (col = v) and col <> v are NULL when col is NULL, so negation
dropped rows whose column is unset — a conclusion with no session is
not "some other session", but was excluded anyway. IS NOT TRUE and
IS DISTINCT FROM behave identically when no NULL is involved.
Adds a row-count test, since this class of bug builds and executes
cleanly, and documents {"ne": null} for excluding unset fields.
* fix(filter): reject session ids that can't name a real session
extract_session_allowlist accepted any non-empty string, so "*" reached
the three consumers of the allowlist — direct IN, the filter DSL, and a
Python membership test — which disagree about it. The DSL reads "*" as
"drop the condition" and matches every session; the others treat it as a
literal name and match none. One /chat request could have some recall
sources unscoped and others scoped to nothing.
Entries are now validated against RESOURCE_NAME_PATTERN, the same pattern
the API requires of session ids, so no session could ever be named "*"
anyway. Wildcards were never part of this endpoint's documented contract
(an id, a list of ids, or {"in": [...]}), and a wildcard alongside a
top-level session_id already 422'd via must_include.
* fix(filter): treat a bare null operand as a null check
Routing every operand through _coerce_operand made bare `None` a type
error rather than a null check, so {"session_id": null} raised FilterError
where it previously built IS NULL: _build_field_condition used to end in
`column == value`, which SQLAlchemy renders as IS NULL. Confirmed 422 on
all five column families (text, numeric, boolean, datetime, JSONB).
Nothing caught it. The docs added in this branch promise
`{"session_id": None}` matches unset rows, the comment in
_build_comparison_conditions claimed the equality path already covered it,
and the DSL-wide invariant test accepts "compiles OR raises FilterError",
so a 422 passed. _coerce_operand's docstring already stated the contract
its caller wasn't honoring — "Callers handle None (a null check) and `*`
(a wildcard) before calling" — so the guard restores that rather than
adding a new rule.
The three null forms now agree: {"col": null} is IS NULL, {"col": {"ne":
null}} is IS NOT NULL, NOT [{"col": null}] is (IS NULL) IS NOT true.
Also from review of #947:
- Log filter keys, not the body. That log line is new in this branch and
operands carry peer/session ids and free-text `contains` values; the
traceback plus the entry shape is what locates a builder bug.
- Assert whereclause in the _where test helper, so a dropped condition
fails instead of returning the whole statement to substring-match.
- Cover the raw-key JSONB path via source_ids, a JSONB column outside
JSONB_COLUMNS reachable through Document's raw-key fallback.
- Drop the orphaned comment left above ENUM_COLUMN_VALUES when
_coerce_operand replaced SCALAR_OPERAND_TYPES.
- Document that a JSONB column takes an object bare or under `contains`
and nothing else. Bare {"metadata": X} is containment, so the `ne` this
branch removed was never its inverse: a row with {"status":"done","x":1}
satisfied both it and {"metadata": {"status":"done"}}. Per-key operators
and NOT cover the real intents.
- Rewrite "Negation and Unset Fields" to lead with the operator rule and a
truth table, forward-linking to Filtering Conclusions instead of using
conclusions ~525 lines before they are introduced. A conclusion's
session_id is the only nullable documented filterable field, verified
across Message/Document/Session/Peer/Workspace.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Import anthropic/openai/google-genai only when a provider is first used
instead of at module import. CLIENTS is now populated lazily via
default_client(), which preserves the patch.dict test seam. The
embedding client defers its SDK imports the same way and dispatches on
transport instead of isinstance.
Cuts idle RSS by ~60MiB per process with all three providers configured
but unused at startup; a process that only ever calls one provider also
never pays for the other two.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix: apply session scoping to all working-representation query paths
session_name was only applied to the recent-documents query in
RepresentationManager; the semantic and most-derived paths ignored it,
so limit_to_session leaked cross-session conclusions into perspectives.
- Thread a session allowlist (session_names) uniformly through all
three query paths; pushed down to pgvector and external vector stores
- Accept a list so the upcoming session-allowlist API reuses this path
- Fail closed on an empty allowlist (downstream stores drop empty IN
clauses, which would silently widen scope)
Fixes DEV-1994
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: bare-list membership sugar in the filter DSL
{"session_id": ["s1", "s2"]} is now shorthand for
{"session_id": {"in": [...]}} on regular columns, generically
(peer_id, etc.). JSONB metadata columns are excluded — a bare list
there keeps JSONB containment semantics, unchanged.
Previously a bare list on a regular column compiled to a type-mismatched
equality that matched nothing, so this is strictly additive.
Also translates the same shape in the turbopuffer/lancedb filter
builders, and fixes lancedb dropping empty IN clauses (fail-open) —
an empty membership list now emits an always-false condition.
Groundwork for DEV-1995 (session allowlist via the existing filters
DSL, no new API params)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: session allowlist on dialectic and representation via filters
Adds a constrained 'filters' body to peer.chat and /representation —
the same DSL search and conclusions already accept, supporting only the
session_id key (a session id, a bare list, or {"in": [...]}).
Unsupported keys and shapes are rejected with 422, never silently
ignored. Composes with session_id (must be included in the allowlist
when both are given). Capped at 1,000 sessions per request.
Enforcement is uniform at every recall chokepoint, fail-closed:
- dialectic prefetch + search_memory: conclusion recall restricted to
the allowlist; dream docs (session_name IS NULL) excluded
- message tools (search/grep/date-range/temporal/context/history):
strict intersection of allowlist and observer session membership
- get_reasoning_chain: unavailable under an allowlist (chains traverse
provenance across sessions and cannot be scoped without leaking)
- empty allowlist short-circuits to empty results everywhere
Auth: workspace keys pass the allowlist as-given; peer-scoped JWTs must
be a member of every allowlisted session (403 otherwise), mirroring the
existing single-session check.
Fixes DEV-1995
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: reserve scope__ peer namespace with kind flag and guardrails
Introduce the scope peer namespace (scope__<name>) and the authoritative
{"kind": "scope"} configuration flag, plus the server-side guardrails:
- src/utils/scopes.py: single source of truth for the prefix, kind flag,
and name helpers (scope_peer_name / is_scope_peer_name /
scope_name_from_peer / validate_no_scope_peer_names)
- reject reserved-prefix names on peer get-or-create (422)
- reject scope peers as message authors in crud.create_messages (422)
- reject scope peers as chat/representation targets (422); a scope peer
as the path-level observer is deferred to Phase 2b
- reject scope peers on the generic session-peer add/set/remove routes
and the session-create peers mapping (422, directing to scopes routes)
- peers.list excludes scope peers by default; new PeerGet.kind option
("scope" | "all") switches the view via a configuration JSONB filter
- schemas: Scope / ScopeCreate / ScopeSessions(Add) and
SessionCreate.scopes (unprefixed scope names, validated)
Part of DEV-1997 (Scopes RFC DEV-1970).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: add scopes CRUD routes and session-create scopes wiring
New /v3/workspaces/{workspace_id}/scopes facade (workspace-level auth;
peer- and session-scoped keys are rejected):
POST "" create-or-get (201/200)
POST /list paginated scope list
GET /{scope_id} single scope
POST /{scope_id}/sessions add memberships
DELETE /{scope_id}/sessions/{session_id} remove membership
GET /{scope_id}/sessions list member session ids
- crud/scope.py: get_or_create_scopes stamps the backing peer with
{"kind": "scope", "observe_me": false} and refuses to adopt a
legacy peer occupying the reserved name without the flag (409)
- memberships are session_peers rows with observe_others=true /
observe_me=false — identical to a hand-built observer peer
- SessionCreate.scopes: create-or-get each scope peer and add the
membership at session creation (the no-backfill common path)
- crud/session.py: public upsert_session_peers wrapper so the facade
bypasses the route-level guardrails without reaching into privates
Backfill of pre-existing documents and reconciliation on removal land in
DEV-1999; membership only affects messages ingested after the change.
Part of DEV-1997 (Scopes RFC DEV-1970).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: cover scopes facade, guardrails, and observer semantics
- create-or-get idempotency, list/get, name validation, legacy-collision
rejection (409), auth scoping (workspace key ok, peer/session keys 401)
- reserved prefix rejected on peer create; peers.list kind filtering
- scope peers rejected as message authors, chat/representation targets,
and on the generic session-peer routes
- membership add/list/remove with observe_others=true / observe_me=false
row shape asserted via DB, and facade-less equivalence with a
hand-built observer peer
- end-to-end litmus: after adding a session to a scope, the deriver
enqueue fan-out includes the scope peer as an observer
- session creation with scopes: [a, b] creates both memberships
Part of DEV-1997 (Scopes RFC DEV-1970).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(scopes): scope resolution helper and read-route schemas
Add resolve_scope_peers (crud/scope.py) mapping unprefixed scope names to
their backing scope peers, 404 when missing and 422 when a non-scope peer
squats the reserved name. Add the `scope` option to DialecticOptions and
PeerRepresentationGet, and a WorkspaceMessageSearchOptions with `scope`.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(scopes): wire the `scope` read option into the routes
Chat and representation: a single scope swaps the observer to the scope
peer (recall confined to the scoped collection and the scope's sessions by
existing observer semantics); a scope list resolves to the union of member
sessions and rides the DEV-1995 dynamic allowlist arm with the path peer as
observer. `scope` is mutually exclusive with `filters`/`session_id` (422)
and requires a workspace/admin key (403 for peer-scoped JWTs).
Session context: `scope` swaps the perspective source for both the working
representation and the peer-card fetch. Workspace search: `scope` injects
the scope's session set into the message-search filter (empty scope -> no
results). Close the carry-over guardrail gap: scope peers are rejected as
peer_target/peer_perspective in session context and as the peer/target in
GET /peers/{id}/context.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(scopes): cover the `scope` read option end-to-end
Validation (404/422/403), single-scope observer swap vs. union allowlist,
scoped representation and session-context (scoped collection + scoped card),
workspace search restricted to a scope's sessions, and guardrail closure for
scope peers on the peer/session context surfaces. Patch the workspaces
tracked_db import site so scope resolution reads the per-test database.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(crud): preserve cache invalidation across get_or_create retry
`get_or_create_peers` and `get_or_create_scopes` mutate existing rows, then
insert new ones inside `db.begin_nested()`. When a concurrent writer creates
one of those rows first, the insert raises IntegrityError and the function
retries.
`begin_nested()` autoflushes the pending UPDATEs *before* opening the
savepoint, so the rollback neither undoes them nor expires the now-clean ORM
state. The retry then compared already-updated values, found no change, and
dropped those peers from `changed_peers` — skipping the cache purge while the
row change committed anyway, leaving entries stale until the 300s TTL.
Carry the mutated names into the retry via `_pending_invalidation` so the
purge cannot be lost.
The scopes facade mirrors `get_or_create_peers`, so both copies carried this.
The peer path is pre-existing and runs on every message ingest.
Also add the missing /v3 prefix to `_SCOPES_ROUTE_GUIDANCE`, which pointed
callers at a 404.
Adds tests/crud/test_get_or_create_retry_invalidation.py, which drives a real
racing session and fails without this change.
* fix(scopes): make scope identity unforgeable, unblock non-pattern peer names
Three coupled changes to the scopes facade.
1. `PeerCreate` no longer gates internal lookups. It exists to validate a new,
user-supplied peer id at the API boundary, but crud used it as a DTO for
names that already exist, so any name outside RESOURCE_NAME_PATTERN raised a
raw pydantic ValidationError — which is not a HonchoException, so it fell
through to the catch-all handler as an HTTP 500. Adds `PeerSpec` (same
fields, no charset pattern) as `PeerCreate`'s base, widens
`get_or_create_peers` to accept it, and changes `get_peer` to take a plain
str. All 13 construction sites converted; the create route keeps full
validation.
This unbreaks the Dreamer: DreamScheduler passes `collection.observer`
straight into the specialist preflight, and scope peers have
`observe_others=true`, so every `(scope.x, peer)` dream died there — the
feature scopes exist to enable. It also fixes a pre-existing bug unrelated
to scopes: a peer named `alice.smith` (legal before d429de0e5338, which
validated names by length alone) 500s on message create, session peer add,
and peer update.
2. The `kind` flag moves from `configuration` to `internal_metadata`.
`configuration` is user-writable — `PeerCreate`/`PeerUpdate` accept a
free-form dict and `update_peer` replaces it wholesale — so a legitimate
`{"observe_me": true}` update silently dropped the flag, and a forged
`{"kind": "scope"}` injected an ordinary peer into `POST /scopes/list`.
`internal_metadata` appears in no API schema. `observe_me: false` stays in
`configuration`, where it belongs.
3. Scope identity requires prefix AND flag, via `is_scope_peer()` and
`scope_peer_clause()`. Neither half is forgeable: the prefix sits outside
RESOURCE_NAME_PATTERN, `internal_metadata` is unreachable. Usage-site guards
become flag-based so a legacy peer merely occupying the namespace keeps
working rather than 422-ing on its own traffic; peer create and update stay
name-based, since those must stop new names entering the namespace.
`update_peer` now returns 422 instead of 500.
Also swaps the reserved prefix from `scope__` to `scope.`: `_` is inside
RESOURCE_NAME_PATTERN, so any tenant could already own a `scope__x` peer.
No DB migration — `internal_metadata` already exists on `peers`, and no scope
peers exist in any deployment yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): validate peer names on create, close namespace squatting and upsert race
Addresses three review findings against 48047a6a.
1. `PeerSpec` let API callers create invalid and reserved-prefix peers.
Widening `get_or_create_peers` to accept a pattern-free schema fixed the
lookup 500s but also removed validation from the *insert* path, and
request-controlled names reach it via message authors, session peer maps, and
the chat observer path — none of which carry a charset pattern of their own.
Confirmed: `POST /sessions/{id}/messages` with `peer_id: "scope.x"` returned
201 and minted an unflagged squatter, after which `POST /scopes {id: x}` was
permanently 409-blocked — namespace denial of service by any caller able to
post a message. `peer_id: "not a valid name!@#"` was likewise created.
Fixed by validating only names about to be INSERTed
(`_validate_new_peer_names`), so already-existing names — legacy dotted
names, scope peers — still resolve without a spurious 422. That keeps the
Dreamer fix intact, since it reads through `get_peer`.
2. Existing reserved-prefix squatters could not be updated. The name-based guard
on `PUT /peers/{peer_id}` refused every `scope.` name, contradicting the
invariant that an unflagged squatter stays a normal peer. Now flag-based, so
behavior is three-way: a real scope is refused, an existing unflagged peer
updates, and a missing reserved-prefix name is refused by (1) rather than
minted.
3. Scope checks raced with get-or-create and the membership upsert. The
route-level guards run before peers are resolved, so a scope created
concurrently in that window would be attached by the generic path with a
default `SessionPeerConfig()`, clobbering its observer membership config.
Adds `_reject_resolved_scope_peers`, which runs on the resolved rows in the
same transaction as the upsert — no window, no extra query. The early checks
stay for better error messages.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): guard scope membership config, move checks to the mutation point
Addresses a second review pass against 10655792.
1. Scope membership configuration was directly user-mutable.
`PUT /sessions/{id}/peers/{peer_id}/config` had no scope guard at all, and
`crud.set_peer_config` resolved the peer only to discard the row. Confirmed:
posting `{"observe_others": false, "observe_me": true}` for a scope returned
204 and persisted, which silently stops all fan-out into the scope and makes
Honcho form a representation *of* a scope — neither of which is reachable
through the facade. Deterministic, no race required. Now checked on the row
`get_peer` already returns, so it costs nothing and cannot race.
2. Empty and over-long names were still 500s. Removing the charset pattern from
`PeerSpec` fixed one trap but left its length bounds, and request-bound peer
names carry no length limits of their own — so `peer_id: ""` or a 513-char
name reached `PeerSpec(...)` and raised a raw pydantic ValidationError that
the catch-all turned into a 500. `PeerSpec` now carries no constraints at all
(matching its documented purpose) and every rule for a new name lives in
`_validate_new_peer_names` on the insert path.
3. Resolved-row protection generalized. The previous pass applied it only to
membership upserts, leaving check-then-use windows elsewhere: peer update
could have a concurrently-created scope's configuration replaced wholesale
(create-path validation does not fire for a peer that now exists), the chat
observer get-or-create could resolve a fresh scope as its observer, and the
generic session-peer removal could silently detach a scope from its sessions.
Each now inspects the resolved peer immediately before acting; the redundant
name-level guard on the update route is dropped in favor of the race-free one.
`remove_peers_from_session` grows an internal `_allow_scope_peers` flag because
the scopes facade ends membership through that same path and must not be blocked
by its own guard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(scopes): enumerate every peer-touching route against a scope policy
Three review passes each found the same class of defect: a route nobody had
checked, rather than logic that was subtly wrong. One of them — `PUT
/sessions/{id}/peers/{id}/config`, which let any caller set a scope to
`observe_others=false` and silently stop all fan-out into it — predated this work
entirely, because the guardrail set was assembled guardrail-by-guardrail instead
of derived from the route list. Sampling review cannot close that kind of gap;
enumeration can.
Derives every route through which a peer name can reach the system and requires
each to be classified as GUARDED or EXEMPT-with-a-reason, so a newly added
peer-touching route fails the suite until someone classifies it. Detection is the
union of path shape and a walk of the dependant tree (including sub-dependency
`Form(...)` params and nested request-body models), because neither signal alone
suffices: parameter names miss `POST /sessions/{id}/peers`, whose peer names are
dict keys, and path shape misses `messages/upload`, whose `peer_id` arrives as a
form field behind a parser dependency.
Both invariants are then asserted behaviorally, by calling the routes rather than
inspecting annotations — the guards deliberately live in crud, which is what makes
`messages/upload` guarded for free via `crud.create_messages`:
- a real scope is refused on all 11 guarded routes, and the rejection must name
the scope, so an unrelated 422 (a malformed body) cannot pass the assertion;
- an *unflagged* peer merely occupying the reserved namespace is unaffected. That
half regressed once already when `update_peer` used a name-based check.
Mutation-tested all three failure modes: disabling the `set_peer_config` guard
fails the guarded test naming that route; regressing `update_peer` to name-based
fails the squatter test; adding an unclassified peer route fails the enumeration.
Covers the HTTP surface only. Peer names also reach the system through the
deriver, dreamer, and queue, which have no route table to enumerate — noted in
the module docstring rather than implied to be covered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): refuse a scope in every observed position; enumerate per position
Addresses a fourth review pass against 62681e4d. The headline finding is that the
previous commit's enumeration test had the wrong *model*, not a missing entry.
1. Manual conclusions could create knowledge about a scope. `POST /conclusions`
validated only that observer_id and observed_id exist, so a scope as
`observed_id` persisted a conclusion about a peer carrying observe_me=false and
created an (observer, scope) collection for it. Confirmed: 201, and it read
back. `POST /schedule_dream` had the same hole via `observed`.
The fix is positional, because the invariant is:
A scope may be an OBSERVER. A scope may never be OBSERVED.
A scope as `observer_id` is how scoped conclusions are stored and must keep
working (verified still 201); as `observed_id` it is now refused. Same split
applied to schedule_dream `observed`, the peer-card `target` (which also covers
a scope's self-card, since target-omitted collapses observed to peer_id), and
session-context `peer_target`.
2. Chat target and both representation roles kept check-to-use races. Only the
chat path-level observer was re-checked on its resolved row; the target was
checked by name and then resolved without inspecting scope identity. Both are
now checked at the dialectic preflight, where observer and observed are already
resolved — an absent name has already failed by then, and an existing squatter
cannot retroactively become a scope.
3. Generic membership removal was still racy. The adjacent SELECT narrowed the
window but could not close it under READ COMMITTED. The UPDATE now carries its
own correlated NOT EXISTS against scope_peer_clause(), so Postgres evaluates
the exclusion as part of the statement and a scope committed after the advisory
check still cannot be detached.
4. New-name validation ran after the name reached Postgres. A NUL byte passed the
request schemas and PeerSpec, then raised psycopg.DataError inside the lookup —
a 500. Values that cannot correspond to a stored row by construction (NUL
bytes, over-length names) are now refused before the query.
(Over-length names already returned 422; only the wasted query was real there.)
The enumeration test is rekeyed from (method, path) to (method, path, position).
A binary per-route verdict cannot express finding 1 at all: `POST /conclusions` is
one route with two positions and opposite verdicts. Detection widens to observer /
observed / target / peer_target / peer_perspective, which surfaced four routes the
previous version never saw — conclusions, schedule_dream, queue/status, and
session context.
Also registers `src.routers.workspaces.tracked_db` in the conftest patch list; the
new guard there would otherwise have run against the real configured database
instead of the per-test one.
Mutation-tested: disabling the conclusions observed-guard fails the positional
test naming that position; adding an unclassified `observed_id` param fails
enumeration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): refuse future scopes in observed positions, preserve scope membership
Addresses a fifth review pass against 14136e5b. All six findings reproduced
locally before fixing.
1. High — generic peer replacement removed scope memberships.
`set_peers_for_session` soft-deleted every active SessionPeer row, and the
request-level guard only inspected names *present* in the replacement map. A
caller detached a scope by simply omitting it, never naming it — so no
request-level guard could ever see it. Reproduced: scope sessions went
['<id>'] -> [] on a 200. The exclusion now lives in the UPDATE itself
(correlated NOT EXISTS against scope_peer_clause), so replacement means
"replace ordinary peers" regardless of request contents or concurrent creation.
2. High — peer cards could be pre-seeded for future scopes.
`set_peer_card` resolves only the observer and writes a JSONB key derived from
an unchecked observed name, and the route guard rejected only *existing*
flagged scopes. Reproduced: PUT card with target=scope.<missing> returned 200,
creating that scope then returned 201, and the card described the real scope.
3. High — dreams could be queued for future scopes.
The route checked `observed` in a read-only session that closed before
`enqueue_dream`, and a missing reserved name passes any is-it-a-scope check.
Reproduced: 204 with observed=scope.<missing>.
2 and 3 share a root cause, so they share a fix: a new `reject_scope_observed`
that is stricter than `reject_scope_peers` in exactly one case — a *missing*
reserved name is refused, because nothing on these paths creates the peer, so
nothing else would ever catch it. Existing unflagged squatters still pass.
Both guards moved to the mutation point: card validation into
`crud.set_peer_card` (same transaction as the JSONB write, so Dreamer and
agent-tool callers are covered), dream validation into `enqueue_dream` (same
transaction as the queue insert). The redundant route-level checks are dropped
rather than left as weaker duplicates.
4. Medium — prefixed NUL names still reached PostgreSQL.
`reject_scope_peers` filtered for the reserved prefix and sent matches to a
text comparison, so "scope.future\0name" raised psycopg.DataError — a 500.
Both guards now share `_reserved_name_candidates`, which materializes the input
once and rejects impossible values before any SQL. Materializing matters
independently: the message-author path passes a generator, and validation
iterates separately from the prefix filter, so a generator would be
half-consumed. `_reject_impossible_peer_names` now takes a Collection so the
type checker enforces that.
5. Medium — representation kept a check-to-use race.
The previous commit claimed both representation roles were rechecked after
resolution; that was wrong — only the dialectic preflight got that check, and
the representation route never goes through it. It now opens one short
read-only session *after* the embedding call, checks both positions, and passes
that same session to `get_working_representation`, so no connection is held
across external work and a scope committed later cannot have conclusions in the
collection being read.
6. Low — policy coverage was not exhaustive. `sender_id` reaches CRUD as
`observed` but was missing from the detected parameter set. ALLOW cases could
also not carry builders, so the suite never proved the other half of the
contract — that legitimate scope *observers* keep working, which a guard
rejecting scopes everywhere would satisfy. Both fixed; observer positions on
conclusions, dreams, cards, session context and queue status are now asserted
behaviorally.
Deliberately not implemented: the scope-creation backstop scanning for
pre-existing card keys and queue items naming a future backing peer. Reasoning is
recorded in `get_or_create_scopes` — no new such state can be created now, any
pre-existing row is coincidental since `scope.` was never a meaningful namespace,
the consequence is inert, and detecting card keys means a full table scan per
scope creation.
Mutation-tested each new guard: removing the replacement exclusion fails both
membership-preservation tests; weakening either observed guard to existing-only
fails the pre-seeding tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): exclude scope memberships from the session observer limit
Scope memberships carry observe_others=true, so every scope counted against
SESSION_OBSERVERS_LIMIT (default 10) — capping scopes-per-session at the limit
minus the session's real observers, and reporting the failure as
`400 Cannot create session <name> with 11 observers. ... Observers are peers
with 'observe_others' set to true.` on a membership call. Wrong on three counts:
the ceiling is undocumented and contradicts RFC §5.1 ("sessions belong to any
number of scopes"), the message describes session creation, and it leaks the
word "observer" through a facade whose entire job is hiding observers (RFC
goal 5). The limit exists to bound per-observer deriver fan-out for real peers;
a scope costs document rows, not LLM calls (RFC §5.2), so it does not belong in
that budget.
Excluded from both halves of the check in `_get_or_add_peers_to_session`: the
incoming names via a flag-based lookup, existing memberships via a correlated
NOT EXISTS on `scope_peer_clause()` — the same pattern the replacement and
removal paths already use, so the exclusion holds regardless of concurrent
scope creation. The early `count_observers_in_config(session.peer_names)` check
in `get_or_create_session` is left alone: `peer_names` cannot contain a scope,
and `scopes` is a separate field.
`reject_scope_peers` is split into a `scope_peer_names()` query helper plus a
two-line raiser so the observer count reuses the authoritative name-AND-flag
predicate instead of growing a third copy of it. Still costs nothing on the
common path — no reserved-prefix name in the input means no query at all.
Also caps `SessionCreate.scopes` at 100, matching `ScopeSessionsAdd.session_ids`.
This belongs in the same commit: the observer limit was the only thing bounding
that list, so removing it turns an unbounded `scopes` array into a peer row and
a membership row per element, committed — the single-request path to the
cardinality anti-pattern RFC §8 warns about. Partly answers OQ6: no per-session
cap, 100 per request.
Tests: a session joins SESSION_OBSERVERS_LIMIT + 2 scopes through both the
facade and session creation; real observers over the limit still 400, so the
carve-out cannot quietly disable the limit; 101 scopes is a 422.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): skip semantic retrieval when the embedding precompute failed
Addresses three open review comments.
1. Major — the representation read could embed inside its DB session. The
route's precompute is suppressed, and both
`RepresentationManager.get_working_representation` and
`crud.query_documents` fall back to embedding when a query arrives without
one, so a failed precompute meant an external call inside the read session
this branch opens for the scope re-check — the connection-holding rule the
route's own comment claimed to satisfy. The innermost fallback also only
catches ValueError, so a provider outage surfaced as a 500. The semantic
query is now passed only when an embedding exists, degrading to
derived+recent retrieval. (`crud.query_documents` embedding inside a caller's
session predates this branch and is left alone.)
2. Minor — `test_resolved_scope_peer_rejected_at_membership_upsert` described a
race it does not perform. It creates an already-flagged scope and calls crud
directly; the unflagged → flagged transition is not simulated. Docstring now
says what the test actually pins.
3. Minor — `test_empty_replacement_preserves_scope_membership` asserted only
half its docstring. It passed if the empty PUT left every ordinary
membership intact; now asserts the ordinary peer's left_at is set.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): close auth and observed-position gaps, paginate membership
Review response for #884.
Security:
- gate `SessionCreate.scopes` behind a workspace-level key; the session-create
route is self-authorizing, so a peer- or session-scoped token could mint scope
peers and join sessions to scopes it had no access to via `POST /scopes`
- refuse a reserved-but-nonexistent name in two observed positions that used the
permissive guard: chat `target` and session-context `peer_target`. Both let a
caller act on `scope.X` before it existed, then create the scope
Facade:
- exclude scope peers from `GET /sessions/{id}/peers` and refuse the membership
-config read for a real scope, matching its write side
- replace `GET /scopes/{id}/sessions` with `POST /scopes/{id}/sessions/list`
returning `Page[Session]`; the add route now returns 204. Membership was
unbounded on both, while every other list surface paginates
- rename `crud.get_scope` to `get_scope_or_raise`
Tests:
- add a missing-name axis to the route-policy table (`Case.refuse_missing`), which
is what surfaced the two guard gaps above
- delete 14 hand-written tests the table now enumerates; 52 -> 39 functions in
test_scopes.py with more cases covered
- tighten the squatter assertion from `!= 422` to `< 400`, which was passing on 5xx
- assert the FastAPI-internals traversal still derives positions, so a framework
upgrade can't silently empty the suite
Docs:
- drop internal ticket and RFC references from the published OpenAPI descriptions
and surrounding comments; state the behavior instead
- move implementation reasoning out of the `PUT /peers/{id}` docstring, which
FastAPI publishes, into a comment
* test(scopes): assert exact statuses for permissive missing-name cases
Follow-up review pass on #884.
- add `Case.missing_status` so a permissive missing-name position asserts the
status it should actually get (404, or 200 for the no-op removal) instead of
`!= 422`, which also passed on a 5xx — the same hole already closed in the
squatter assertion
- require it whenever `refuse_missing` is False, and require its absence when
True, so the policy table can't drift from the assertion
- repoint a stale allow-reason at POST /scopes/{scope_id}/sessions/list; the GET
it named was removed
- document the membership list's ordering under `reverse`
* fix: Address Coderabbit Comments
* fix: Address more coderabbit comments
* chore: remove ai artifacting
* fix: remove unused fakeredis client in lieu of in-memory taskless cache for test suite
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat: reserve scope__ peer namespace with kind flag and guardrails
Introduce the scope peer namespace (scope__<name>) and the authoritative
{"kind": "scope"} configuration flag, plus the server-side guardrails:
- src/utils/scopes.py: single source of truth for the prefix, kind flag,
and name helpers (scope_peer_name / is_scope_peer_name /
scope_name_from_peer / validate_no_scope_peer_names)
- reject reserved-prefix names on peer get-or-create (422)
- reject scope peers as message authors in crud.create_messages (422)
- reject scope peers as chat/representation targets (422); a scope peer
as the path-level observer is deferred to Phase 2b
- reject scope peers on the generic session-peer add/set/remove routes
and the session-create peers mapping (422, directing to scopes routes)
- peers.list excludes scope peers by default; new PeerGet.kind option
("scope" | "all") switches the view via a configuration JSONB filter
- schemas: Scope / ScopeCreate / ScopeSessions(Add) and
SessionCreate.scopes (unprefixed scope names, validated)
Part of DEV-1997 (Scopes RFC DEV-1970).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: add scopes CRUD routes and session-create scopes wiring
New /v3/workspaces/{workspace_id}/scopes facade (workspace-level auth;
peer- and session-scoped keys are rejected):
POST "" create-or-get (201/200)
POST /list paginated scope list
GET /{scope_id} single scope
POST /{scope_id}/sessions add memberships
DELETE /{scope_id}/sessions/{session_id} remove membership
GET /{scope_id}/sessions list member session ids
- crud/scope.py: get_or_create_scopes stamps the backing peer with
{"kind": "scope", "observe_me": false} and refuses to adopt a
legacy peer occupying the reserved name without the flag (409)
- memberships are session_peers rows with observe_others=true /
observe_me=false — identical to a hand-built observer peer
- SessionCreate.scopes: create-or-get each scope peer and add the
membership at session creation (the no-backfill common path)
- crud/session.py: public upsert_session_peers wrapper so the facade
bypasses the route-level guardrails without reaching into privates
Backfill of pre-existing documents and reconciliation on removal land in
DEV-1999; membership only affects messages ingested after the change.
Part of DEV-1997 (Scopes RFC DEV-1970).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: cover scopes facade, guardrails, and observer semantics
- create-or-get idempotency, list/get, name validation, legacy-collision
rejection (409), auth scoping (workspace key ok, peer/session keys 401)
- reserved prefix rejected on peer create; peers.list kind filtering
- scope peers rejected as message authors, chat/representation targets,
and on the generic session-peer routes
- membership add/list/remove with observe_others=true / observe_me=false
row shape asserted via DB, and facade-less equivalence with a
hand-built observer peer
- end-to-end litmus: after adding a session to a scope, the deriver
enqueue fan-out includes the scope peer as an observer
- session creation with scopes: [a, b] creates both memberships
Part of DEV-1997 (Scopes RFC DEV-1970).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(crud): preserve cache invalidation across get_or_create retry
`get_or_create_peers` and `get_or_create_scopes` mutate existing rows, then
insert new ones inside `db.begin_nested()`. When a concurrent writer creates
one of those rows first, the insert raises IntegrityError and the function
retries.
`begin_nested()` autoflushes the pending UPDATEs *before* opening the
savepoint, so the rollback neither undoes them nor expires the now-clean ORM
state. The retry then compared already-updated values, found no change, and
dropped those peers from `changed_peers` — skipping the cache purge while the
row change committed anyway, leaving entries stale until the 300s TTL.
Carry the mutated names into the retry via `_pending_invalidation` so the
purge cannot be lost.
The scopes facade mirrors `get_or_create_peers`, so both copies carried this.
The peer path is pre-existing and runs on every message ingest.
Also add the missing /v3 prefix to `_SCOPES_ROUTE_GUIDANCE`, which pointed
callers at a 404.
Adds tests/crud/test_get_or_create_retry_invalidation.py, which drives a real
racing session and fails without this change.
* fix(scopes): make scope identity unforgeable, unblock non-pattern peer names
Three coupled changes to the scopes facade.
1. `PeerCreate` no longer gates internal lookups. It exists to validate a new,
user-supplied peer id at the API boundary, but crud used it as a DTO for
names that already exist, so any name outside RESOURCE_NAME_PATTERN raised a
raw pydantic ValidationError — which is not a HonchoException, so it fell
through to the catch-all handler as an HTTP 500. Adds `PeerSpec` (same
fields, no charset pattern) as `PeerCreate`'s base, widens
`get_or_create_peers` to accept it, and changes `get_peer` to take a plain
str. All 13 construction sites converted; the create route keeps full
validation.
This unbreaks the Dreamer: DreamScheduler passes `collection.observer`
straight into the specialist preflight, and scope peers have
`observe_others=true`, so every `(scope.x, peer)` dream died there — the
feature scopes exist to enable. It also fixes a pre-existing bug unrelated
to scopes: a peer named `alice.smith` (legal before d429de0e5338, which
validated names by length alone) 500s on message create, session peer add,
and peer update.
2. The `kind` flag moves from `configuration` to `internal_metadata`.
`configuration` is user-writable — `PeerCreate`/`PeerUpdate` accept a
free-form dict and `update_peer` replaces it wholesale — so a legitimate
`{"observe_me": true}` update silently dropped the flag, and a forged
`{"kind": "scope"}` injected an ordinary peer into `POST /scopes/list`.
`internal_metadata` appears in no API schema. `observe_me: false` stays in
`configuration`, where it belongs.
3. Scope identity requires prefix AND flag, via `is_scope_peer()` and
`scope_peer_clause()`. Neither half is forgeable: the prefix sits outside
RESOURCE_NAME_PATTERN, `internal_metadata` is unreachable. Usage-site guards
become flag-based so a legacy peer merely occupying the namespace keeps
working rather than 422-ing on its own traffic; peer create and update stay
name-based, since those must stop new names entering the namespace.
`update_peer` now returns 422 instead of 500.
Also swaps the reserved prefix from `scope__` to `scope.`: `_` is inside
RESOURCE_NAME_PATTERN, so any tenant could already own a `scope__x` peer.
No DB migration — `internal_metadata` already exists on `peers`, and no scope
peers exist in any deployment yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): validate peer names on create, close namespace squatting and upsert race
Addresses three review findings against 48047a6a.
1. `PeerSpec` let API callers create invalid and reserved-prefix peers.
Widening `get_or_create_peers` to accept a pattern-free schema fixed the
lookup 500s but also removed validation from the *insert* path, and
request-controlled names reach it via message authors, session peer maps, and
the chat observer path — none of which carry a charset pattern of their own.
Confirmed: `POST /sessions/{id}/messages` with `peer_id: "scope.x"` returned
201 and minted an unflagged squatter, after which `POST /scopes {id: x}` was
permanently 409-blocked — namespace denial of service by any caller able to
post a message. `peer_id: "not a valid name!@#"` was likewise created.
Fixed by validating only names about to be INSERTed
(`_validate_new_peer_names`), so already-existing names — legacy dotted
names, scope peers — still resolve without a spurious 422. That keeps the
Dreamer fix intact, since it reads through `get_peer`.
2. Existing reserved-prefix squatters could not be updated. The name-based guard
on `PUT /peers/{peer_id}` refused every `scope.` name, contradicting the
invariant that an unflagged squatter stays a normal peer. Now flag-based, so
behavior is three-way: a real scope is refused, an existing unflagged peer
updates, and a missing reserved-prefix name is refused by (1) rather than
minted.
3. Scope checks raced with get-or-create and the membership upsert. The
route-level guards run before peers are resolved, so a scope created
concurrently in that window would be attached by the generic path with a
default `SessionPeerConfig()`, clobbering its observer membership config.
Adds `_reject_resolved_scope_peers`, which runs on the resolved rows in the
same transaction as the upsert — no window, no extra query. The early checks
stay for better error messages.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): guard scope membership config, move checks to the mutation point
Addresses a second review pass against 10655792.
1. Scope membership configuration was directly user-mutable.
`PUT /sessions/{id}/peers/{peer_id}/config` had no scope guard at all, and
`crud.set_peer_config` resolved the peer only to discard the row. Confirmed:
posting `{"observe_others": false, "observe_me": true}` for a scope returned
204 and persisted, which silently stops all fan-out into the scope and makes
Honcho form a representation *of* a scope — neither of which is reachable
through the facade. Deterministic, no race required. Now checked on the row
`get_peer` already returns, so it costs nothing and cannot race.
2. Empty and over-long names were still 500s. Removing the charset pattern from
`PeerSpec` fixed one trap but left its length bounds, and request-bound peer
names carry no length limits of their own — so `peer_id: ""` or a 513-char
name reached `PeerSpec(...)` and raised a raw pydantic ValidationError that
the catch-all turned into a 500. `PeerSpec` now carries no constraints at all
(matching its documented purpose) and every rule for a new name lives in
`_validate_new_peer_names` on the insert path.
3. Resolved-row protection generalized. The previous pass applied it only to
membership upserts, leaving check-then-use windows elsewhere: peer update
could have a concurrently-created scope's configuration replaced wholesale
(create-path validation does not fire for a peer that now exists), the chat
observer get-or-create could resolve a fresh scope as its observer, and the
generic session-peer removal could silently detach a scope from its sessions.
Each now inspects the resolved peer immediately before acting; the redundant
name-level guard on the update route is dropped in favor of the race-free one.
`remove_peers_from_session` grows an internal `_allow_scope_peers` flag because
the scopes facade ends membership through that same path and must not be blocked
by its own guard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(scopes): enumerate every peer-touching route against a scope policy
Three review passes each found the same class of defect: a route nobody had
checked, rather than logic that was subtly wrong. One of them — `PUT
/sessions/{id}/peers/{id}/config`, which let any caller set a scope to
`observe_others=false` and silently stop all fan-out into it — predated this work
entirely, because the guardrail set was assembled guardrail-by-guardrail instead
of derived from the route list. Sampling review cannot close that kind of gap;
enumeration can.
Derives every route through which a peer name can reach the system and requires
each to be classified as GUARDED or EXEMPT-with-a-reason, so a newly added
peer-touching route fails the suite until someone classifies it. Detection is the
union of path shape and a walk of the dependant tree (including sub-dependency
`Form(...)` params and nested request-body models), because neither signal alone
suffices: parameter names miss `POST /sessions/{id}/peers`, whose peer names are
dict keys, and path shape misses `messages/upload`, whose `peer_id` arrives as a
form field behind a parser dependency.
Both invariants are then asserted behaviorally, by calling the routes rather than
inspecting annotations — the guards deliberately live in crud, which is what makes
`messages/upload` guarded for free via `crud.create_messages`:
- a real scope is refused on all 11 guarded routes, and the rejection must name
the scope, so an unrelated 422 (a malformed body) cannot pass the assertion;
- an *unflagged* peer merely occupying the reserved namespace is unaffected. That
half regressed once already when `update_peer` used a name-based check.
Mutation-tested all three failure modes: disabling the `set_peer_config` guard
fails the guarded test naming that route; regressing `update_peer` to name-based
fails the squatter test; adding an unclassified peer route fails the enumeration.
Covers the HTTP surface only. Peer names also reach the system through the
deriver, dreamer, and queue, which have no route table to enumerate — noted in
the module docstring rather than implied to be covered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): refuse a scope in every observed position; enumerate per position
Addresses a fourth review pass against 62681e4d. The headline finding is that the
previous commit's enumeration test had the wrong *model*, not a missing entry.
1. Manual conclusions could create knowledge about a scope. `POST /conclusions`
validated only that observer_id and observed_id exist, so a scope as
`observed_id` persisted a conclusion about a peer carrying observe_me=false and
created an (observer, scope) collection for it. Confirmed: 201, and it read
back. `POST /schedule_dream` had the same hole via `observed`.
The fix is positional, because the invariant is:
A scope may be an OBSERVER. A scope may never be OBSERVED.
A scope as `observer_id` is how scoped conclusions are stored and must keep
working (verified still 201); as `observed_id` it is now refused. Same split
applied to schedule_dream `observed`, the peer-card `target` (which also covers
a scope's self-card, since target-omitted collapses observed to peer_id), and
session-context `peer_target`.
2. Chat target and both representation roles kept check-to-use races. Only the
chat path-level observer was re-checked on its resolved row; the target was
checked by name and then resolved without inspecting scope identity. Both are
now checked at the dialectic preflight, where observer and observed are already
resolved — an absent name has already failed by then, and an existing squatter
cannot retroactively become a scope.
3. Generic membership removal was still racy. The adjacent SELECT narrowed the
window but could not close it under READ COMMITTED. The UPDATE now carries its
own correlated NOT EXISTS against scope_peer_clause(), so Postgres evaluates
the exclusion as part of the statement and a scope committed after the advisory
check still cannot be detached.
4. New-name validation ran after the name reached Postgres. A NUL byte passed the
request schemas and PeerSpec, then raised psycopg.DataError inside the lookup —
a 500. Values that cannot correspond to a stored row by construction (NUL
bytes, over-length names) are now refused before the query.
(Over-length names already returned 422; only the wasted query was real there.)
The enumeration test is rekeyed from (method, path) to (method, path, position).
A binary per-route verdict cannot express finding 1 at all: `POST /conclusions` is
one route with two positions and opposite verdicts. Detection widens to observer /
observed / target / peer_target / peer_perspective, which surfaced four routes the
previous version never saw — conclusions, schedule_dream, queue/status, and
session context.
Also registers `src.routers.workspaces.tracked_db` in the conftest patch list; the
new guard there would otherwise have run against the real configured database
instead of the per-test one.
Mutation-tested: disabling the conclusions observed-guard fails the positional
test naming that position; adding an unclassified `observed_id` param fails
enumeration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): refuse future scopes in observed positions, preserve scope membership
Addresses a fifth review pass against 14136e5b. All six findings reproduced
locally before fixing.
1. High — generic peer replacement removed scope memberships.
`set_peers_for_session` soft-deleted every active SessionPeer row, and the
request-level guard only inspected names *present* in the replacement map. A
caller detached a scope by simply omitting it, never naming it — so no
request-level guard could ever see it. Reproduced: scope sessions went
['<id>'] -> [] on a 200. The exclusion now lives in the UPDATE itself
(correlated NOT EXISTS against scope_peer_clause), so replacement means
"replace ordinary peers" regardless of request contents or concurrent creation.
2. High — peer cards could be pre-seeded for future scopes.
`set_peer_card` resolves only the observer and writes a JSONB key derived from
an unchecked observed name, and the route guard rejected only *existing*
flagged scopes. Reproduced: PUT card with target=scope.<missing> returned 200,
creating that scope then returned 201, and the card described the real scope.
3. High — dreams could be queued for future scopes.
The route checked `observed` in a read-only session that closed before
`enqueue_dream`, and a missing reserved name passes any is-it-a-scope check.
Reproduced: 204 with observed=scope.<missing>.
2 and 3 share a root cause, so they share a fix: a new `reject_scope_observed`
that is stricter than `reject_scope_peers` in exactly one case — a *missing*
reserved name is refused, because nothing on these paths creates the peer, so
nothing else would ever catch it. Existing unflagged squatters still pass.
Both guards moved to the mutation point: card validation into
`crud.set_peer_card` (same transaction as the JSONB write, so Dreamer and
agent-tool callers are covered), dream validation into `enqueue_dream` (same
transaction as the queue insert). The redundant route-level checks are dropped
rather than left as weaker duplicates.
4. Medium — prefixed NUL names still reached PostgreSQL.
`reject_scope_peers` filtered for the reserved prefix and sent matches to a
text comparison, so "scope.future\0name" raised psycopg.DataError — a 500.
Both guards now share `_reserved_name_candidates`, which materializes the input
once and rejects impossible values before any SQL. Materializing matters
independently: the message-author path passes a generator, and validation
iterates separately from the prefix filter, so a generator would be
half-consumed. `_reject_impossible_peer_names` now takes a Collection so the
type checker enforces that.
5. Medium — representation kept a check-to-use race.
The previous commit claimed both representation roles were rechecked after
resolution; that was wrong — only the dialectic preflight got that check, and
the representation route never goes through it. It now opens one short
read-only session *after* the embedding call, checks both positions, and passes
that same session to `get_working_representation`, so no connection is held
across external work and a scope committed later cannot have conclusions in the
collection being read.
6. Low — policy coverage was not exhaustive. `sender_id` reaches CRUD as
`observed` but was missing from the detected parameter set. ALLOW cases could
also not carry builders, so the suite never proved the other half of the
contract — that legitimate scope *observers* keep working, which a guard
rejecting scopes everywhere would satisfy. Both fixed; observer positions on
conclusions, dreams, cards, session context and queue status are now asserted
behaviorally.
Deliberately not implemented: the scope-creation backstop scanning for
pre-existing card keys and queue items naming a future backing peer. Reasoning is
recorded in `get_or_create_scopes` — no new such state can be created now, any
pre-existing row is coincidental since `scope.` was never a meaningful namespace,
the consequence is inert, and detecting card keys means a full table scan per
scope creation.
Mutation-tested each new guard: removing the replacement exclusion fails both
membership-preservation tests; weakening either observed guard to existing-only
fails the pre-seeding tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): exclude scope memberships from the session observer limit
Scope memberships carry observe_others=true, so every scope counted against
SESSION_OBSERVERS_LIMIT (default 10) — capping scopes-per-session at the limit
minus the session's real observers, and reporting the failure as
`400 Cannot create session <name> with 11 observers. ... Observers are peers
with 'observe_others' set to true.` on a membership call. Wrong on three counts:
the ceiling is undocumented and contradicts RFC §5.1 ("sessions belong to any
number of scopes"), the message describes session creation, and it leaks the
word "observer" through a facade whose entire job is hiding observers (RFC
goal 5). The limit exists to bound per-observer deriver fan-out for real peers;
a scope costs document rows, not LLM calls (RFC §5.2), so it does not belong in
that budget.
Excluded from both halves of the check in `_get_or_add_peers_to_session`: the
incoming names via a flag-based lookup, existing memberships via a correlated
NOT EXISTS on `scope_peer_clause()` — the same pattern the replacement and
removal paths already use, so the exclusion holds regardless of concurrent
scope creation. The early `count_observers_in_config(session.peer_names)` check
in `get_or_create_session` is left alone: `peer_names` cannot contain a scope,
and `scopes` is a separate field.
`reject_scope_peers` is split into a `scope_peer_names()` query helper plus a
two-line raiser so the observer count reuses the authoritative name-AND-flag
predicate instead of growing a third copy of it. Still costs nothing on the
common path — no reserved-prefix name in the input means no query at all.
Also caps `SessionCreate.scopes` at 100, matching `ScopeSessionsAdd.session_ids`.
This belongs in the same commit: the observer limit was the only thing bounding
that list, so removing it turns an unbounded `scopes` array into a peer row and
a membership row per element, committed — the single-request path to the
cardinality anti-pattern RFC §8 warns about. Partly answers OQ6: no per-session
cap, 100 per request.
Tests: a session joins SESSION_OBSERVERS_LIMIT + 2 scopes through both the
facade and session creation; real observers over the limit still 400, so the
carve-out cannot quietly disable the limit; 101 scopes is a 422.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): skip semantic retrieval when the embedding precompute failed
Addresses three open review comments.
1. Major — the representation read could embed inside its DB session. The
route's precompute is suppressed, and both
`RepresentationManager.get_working_representation` and
`crud.query_documents` fall back to embedding when a query arrives without
one, so a failed precompute meant an external call inside the read session
this branch opens for the scope re-check — the connection-holding rule the
route's own comment claimed to satisfy. The innermost fallback also only
catches ValueError, so a provider outage surfaced as a 500. The semantic
query is now passed only when an embedding exists, degrading to
derived+recent retrieval. (`crud.query_documents` embedding inside a caller's
session predates this branch and is left alone.)
2. Minor — `test_resolved_scope_peer_rejected_at_membership_upsert` described a
race it does not perform. It creates an already-flagged scope and calls crud
directly; the unflagged → flagged transition is not simulated. Docstring now
says what the test actually pins.
3. Minor — `test_empty_replacement_preserves_scope_membership` asserted only
half its docstring. It passed if the empty PUT left every ordinary
membership intact; now asserts the ordinary peer's left_at is set.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scopes): close auth and observed-position gaps, paginate membership
Review response for #884.
Security:
- gate `SessionCreate.scopes` behind a workspace-level key; the session-create
route is self-authorizing, so a peer- or session-scoped token could mint scope
peers and join sessions to scopes it had no access to via `POST /scopes`
- refuse a reserved-but-nonexistent name in two observed positions that used the
permissive guard: chat `target` and session-context `peer_target`. Both let a
caller act on `scope.X` before it existed, then create the scope
Facade:
- exclude scope peers from `GET /sessions/{id}/peers` and refuse the membership
-config read for a real scope, matching its write side
- replace `GET /scopes/{id}/sessions` with `POST /scopes/{id}/sessions/list`
returning `Page[Session]`; the add route now returns 204. Membership was
unbounded on both, while every other list surface paginates
- rename `crud.get_scope` to `get_scope_or_raise`
Tests:
- add a missing-name axis to the route-policy table (`Case.refuse_missing`), which
is what surfaced the two guard gaps above
- delete 14 hand-written tests the table now enumerates; 52 -> 39 functions in
test_scopes.py with more cases covered
- tighten the squatter assertion from `!= 422` to `< 400`, which was passing on 5xx
- assert the FastAPI-internals traversal still derives positions, so a framework
upgrade can't silently empty the suite
Docs:
- drop internal ticket and RFC references from the published OpenAPI descriptions
and surrounding comments; state the behavior instead
- move implementation reasoning out of the `PUT /peers/{id}` docstring, which
FastAPI publishes, into a comment
* test(scopes): assert exact statuses for permissive missing-name cases
Follow-up review pass on #884.
- add `Case.missing_status` so a permissive missing-name position asserts the
status it should actually get (404, or 200 for the no-op removal) instead of
`!= 422`, which also passed on a 5xx — the same hole already closed in the
squatter assertion
- require it whenever `refuse_missing` is False, and require its absence when
True, so the policy table can't drift from the assertion
- repoint a stale allow-reason at POST /scopes/{scope_id}/sessions/list; the GET
it named was removed
- document the membership list's ordering under `reverse`
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* test(embedding): add reproducer for missing encoding_format on openai paths
The openai SDK defaults encoding_format to base64 when it is not passed. OpenAI-compatible providers that don't support base64 embeddings (e.g. OpenRouter with nvidia/nemotron-3-embed-1b:free) return HTTP 200 with empty data, and every embedding call fails with 'No embedding data received'.
* fix(embedding): request float encoding_format on openai embedding calls
The openai SDK defaults encoding_format to base64 when the caller does not pass one. OpenAI-compatible providers that don't support base64 embeddings (e.g. OpenRouter hosting nvidia/nemotron-3-embed-1b:free) answer HTTP 200 with empty embedding data, and every embedding call fails with 'No embedding data received', breaking conclusions, semantic search, and the deriver. Pass encoding_format='float' explicitly on both the single-query and batch call paths.
* test(embedding): cover openai-compatible providers in the live embedding matrix
The existing openai family runs against real OpenAI, which serves base64
embeddings happily, so the matrix passes with or without the #932 fix. Adds an
`openai_compatible_embedding` family (openai transport, third-party base_url)
so the matrix can reach a provider that rejects base64. Empty default_models
keeps it skipped unless LIVE_EMBEDDING_OPENAI_COMPATIBLE_MODELS is set.
Also adds test_live_openai_float_encoding_matches_base64, which pins the other
direction: switching the wire format must not move vectors on real OpenAI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(embedding): keep an explicit embedding-count check on the openai paths
Passing `encoding_format` disables the openai SDK's own empty-data guard, so a
provider answering 200 with missing embeddings surfaced as `IndexError: list
index out of range` on the single path and `zip() argument 2 is shorter than
argument 1` on the batch path. The latter is also #745's signature, which would
have left it with two unrelated causes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(live-llm): correct the openai-compatible embedding matrix env docs
The documented default dimensions said 2048 after the family moved to 3072, and
LIVE_EMBEDDING_OPENAI_COMPATIBLE_SEND_DIMENSIONS was missing entirely. Also
points the example and the coverage note at a model that is actually reachable,
and records that OpenRouter load-balances, so the base64 failure is per-attempt.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(embedding): resolve openai encoding_format by mode instead of pinning float
Requesting float unconditionally costs ~3.6x the response bytes of base64 and up
to +83% latency on a 500-item batch, which the default deployment on real OpenAI
pays for nothing: only third-party OpenAI-compatible providers reject base64.
Adds EMBEDDING_MODEL_CONFIG__ENCODING_FORMAT_MODE, mirroring dimensions_mode.
`auto` keeps base64 when no base_url override is set or it points at
api.openai.com, and picks float elsewhere. The format is still always sent
explicitly, since the SDK otherwise injects base64 on its own.
Also corrects the _validate_embedding_count docstring, which said "fewer" where
the guard is an inequality.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(embedding): request base64 embeddings by omission, not by name
The openai SDK decodes a base64 response only when it injected the default
itself; naming any format makes it skip the decode and hand back the raw string,
which then fails the dimension check with "Expected 1536, got 8192". base64 mode
therefore has to omit the kwarg rather than pass it.
The unit fake returned float lists whatever was asked for, so it could not catch
this. It now mirrors the SDK and returns a base64 string for a named base64
request, which fails against the previous commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Aakash Kattelu <aakash@plasticlabs.ai>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* Fix Gemini batch embedding for gemini-embedding-2* models
The google-genai SDK treats embed_content(contents=[list_of_strings]) as a
single multi-part document for gemini-embedding-2* models, silently returning
exactly 1 embedding regardless of input count. This caused a zip(...,
strict=True) ValueError in _process_batch.
Wrap each text in genai_types.Content(parts=[genai_types.Part(text=...)])
so the SDK treats each string as a separate content item. This matches the
workaround used by pydantic-ai (#4873) and graphiti (#1474).
Upstream SDK issue: googleapis/python-genai#2523Fixesplastic-labs/honcho#744
* test(embedding): live embedding coverage for every Gemini and OpenAI model
Adds tests/live_llm/test_live_embeddings.py plus an env-driven embedding
matrix alongside the existing LLM one. Covers single embed, batched embed,
batch-vs-single alignment, chunk-to-id mapping, and the batch-split path.
Only a live call catches the gemini-embedding-2* collapse: the SDK folds a
list of bare strings into one document and returns a single embedding.
Reverting the Content wrapping fails all four batch tests for
gemini-embedding-2-preview and gemini-embedding-2 with the reported
`zip() argument 2 is shorter than argument 1`, while gemini-embedding-001
and text-embedding-3-small stay green.
Also makes the concatenation in the conclusions semantic-search validation
message explicit, so the repo-wide basedpyright pre-push hook passes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(embedding): drop the preview twin from the default embedding matrix
gemini-embedding-2 is the GA release of gemini-embedding-2-preview and
behaves identically, so running both by default doubles the Gemini cost for
no extra coverage. The preview stays reachable through
LIVE_EMBEDDING_GEMINI_MODELS.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Aakash Kattelu <aakash@plasticlabs.ai>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The error raised when observer/observed are missing from a semantic-search
query states the requirement but not where the values go, so callers can't
tell whether they belong at the top level or inside `filters`. Point at the
`filters` object explicitly, show a minimal well-formed payload, and note
that both the bare and `_id`-suffixed key spellings are accepted (the code
already reads either).
Co-authored-by: hermes <hermes@local>
* feat(cli): add `honcho session view` transcript command
Adds a read-only transcript view for a session, with three paging modes:
a tail window (`--last N`, the default), server pages (`--page N --size M`),
and the whole conversation (`--all`). `--reverse` selects newest-first in
every mode, `--ids` exposes message IDs, and `-p` scopes to one peer.
JSON mode emits the same shape as `message list`.
The renderer is deliberately literal: content and identifiers go through
`rich.text.Text` rather than Markdown or console markup, so newlines, tag
delimiters like `<thinking>`, and bracketed text survive intact — this is a
debugging surface, so it has to show what was actually stored. Timestamps
are converted to UTC (not just stripped of their offset) and keep
millisecond precision. Nothing is truncated with an ellipsis: a displayed
message ID is always usable with `honcho message get`.
Flags are validated before the client is built, and the session is
constructed directly instead of via the get-or-create `client.session()`,
so an invalid or mistyped invocation never reaches — or creates — anything
server-side. `--size` is bounded locally to the server's 100-item ceiling
rather than surfacing a raw 422, and the "more:" hint echoes back the size
and ordering actually in use so following it lands on the adjacent window.
Also fixes `honcho message list --last N`, which stopped at the first page
of 50: both commands now share the page-walking helper, so the same flag
returns the same window either way.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(cli): regenerate command reference for `session view`
Adds the generated `session view` accordion to the docs snippet and points
the session-debugging workflows at it. Trims the docstring to plain prose —
the RST double-backticks were rendering literally in `--help`, where every
other command uses unmarked flag names — and stops the generator emitting a
trailing blank line that tripped end-of-file-fixer on every regeneration.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(cli): carry invocation scope into the next-page hint
Addresses CodeRabbit review on #1006.
The "more:" hint echoed only `--page`, `--size`, and `--reverse`, so copying
it off a scoped invocation dropped `-w`, `-p`, and `--ids` — landing on a
different workspace or an unfiltered transcript. Hint construction moves into
`_next_page_command` in the command module, which knows the invocation; the
renderer now just prints the string it's handed and no longer needs to know
CLI flag syntax. Only flags passed explicitly are echoed, since anything from
the environment or config resolves the same way on the next run.
Also rejects non-positive `--last` on `honcho message list`, which slice
semantics turned into a silently empty result. `session view` already errored
on it; the two now agree.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(cli): read next-page hint scope from effective flag overrides
Addresses the second CodeRabbit pass on #1006.
`-w`/`-p` parse at group and top level as well as command level, all landing
in `_global_overrides`, so reading the command-level params dropped the scope
from `honcho session -w ws2 view ...`. The hint now reads the effective
overrides via a new `get_flag_overrides()`, which deliberately excludes
environment and config values since those resolve the same way on the next run.
Also shell-quotes the hint's identifiers with `shlex.join`. Note this is
hardening rather than a live injection fix: the API constrains IDs to
`^[a-zA-Z0-9_-]+$`, so an ID carrying a space or metacharacter fails the fetch
before any hint is printed. `validate_resource_id` is looser than the server
though, so quoting is the cheaper invariant to hold locally.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* docs: adding honcho-memory skill
* fix: skills to point at llm friendly content
* docs: split honcho-mcp skill out of honcho-memory; address PR review
Restructure honcho-memory into a concepts/strategy hub that routes to
per-connection path skills, and add a dedicated honcho-mcp skill holding
the MCP-tool mechanics that previously lived inline.
Addresses review feedback on #784:
- honcho-memory step 2 now leads with fast context reads, with chat as
the slower escalation
- honcho-mcp adds a "Speed: reads vs reasoning" section, describes what
each context call returns, and a reasoning-levels table
- get_representation framed as a contextualized snapshot insertable into
a system prompt
- drop schedule_dream from the tool table (manual escape hatch, not
routine guidance)
- prune queue-status references from honcho-cli; document honcho-mcp in
vibecoding skill registry
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(skills): move skills to canonical top-level skills/ with .claude symlink
Establish a single source of truth for agent skills. The real files now live in
the top-level skills/ directory (the publishing convention used by Vercel,
Supabase, and Cloudflare, and the tree Honcho's `npx skills add` distributes).
.claude/skills becomes a symlink to ../skills so Claude Code discovery keeps
working off the one tree — eliminating the parallel-copy sync burden.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: splitting context into references & verifying content is consistent.
* docs: fixing core language
* docs: fixing core language
* fix: language about observe_others
* chore: adding .agents folder for codex
* fix: add instructions.md into the mcp server & delete mcp skill in favor of including it in honcho-memory.
* chore: remove migrate docs (can be found on older versions)
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: document pgvector preinstall for least-privilege DB roles
Honcho issues CREATE EXTENSION IF NOT EXISTS vector before migrations
and again at server startup, both using the DB_CONNECTION_URI role. On
deployments where that role deliberately cannot create extensions
(managed Postgres, Kubernetes operators, NixOS), both statements fail
with a privilege error — IF NOT EXISTS does not save you, because
Postgres checks the privilege before checking for the extension.
Document preinstalling pgvector as a privileged role as the supported
path, and add a troubleshooting entry keyed to the exact error string.
Note that docker compose is unaffected, since the bundled database
service creates the extension via an initdb script.
Refs #614
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs: clarify why docker compose avoids the pgvector privilege error
The previous wording pinned the claim entirely on database/init.sql,
which only runs on first boot of an empty data volume. The load-bearing
reason is that the bundled stack connects as the postgres superuser, so
it can create the extension regardless of volume state. Name that first
and keep init.sql as the secondary reason.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(deriver): stop extraction examples teaching fabricated inferences
The EXAMPLES block in the minimal deriver prompt demonstrated two
inferences that its own output schema forbids:
- "I just had my 25th birthday last Saturday" -> "alice's birthday is
June 21st". A vague relative reference cannot yield a specific date;
this demonstrates inventing one.
- "I took my dog for a walk in NYC" -> "alice lives in NYC". Visiting a
place is not living there.
A third example invited "+ general knowledge" inference to produce a
deductive conclusion. The deriver has no channel for that output --
PromptRepresentation carries only `explicit`, described as "direct
quotes or clear paraphrases only, no interpretation or inference", and
deductive conclusions are produced by the Dreamer's DeductionSpecialist.
Replace all three with examples that stay inside the schema's contract.
The dog/NYC message is kept and shown extracting correctly, and a third
example shows that "lives in NYC" is valid when actually stated, so the
examples teach the boundary rather than just avoiding it.
Closes#626
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(deriver): keep the stated duration in the NYC residence example
The example dropped "six years" from "I've lived in NYC for six years",
which contradicts the prompt's own rule to extract all observations and
to contextualize each one. Emit both the residence fact and the duration.
Keeping "alice lives in NYC" alongside it is deliberate: that output is
the point of the example, contrasting with the preceding one where the
same conclusion is *not* supported by a single visit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix: resolve tiktoken encoding without constructing the embedding client
EmbeddingClient.encoding forced full client construction, which raises
'OpenAI API key is required' even though tiktoken needs no credentials.
The document dedup tie-break (src/crud/document.py) only needs .encoding
for token counting, so any test hitting that path fails in environments
without embedding keys — notably CI for pull requests from forks, where
repo secrets are unavailable (e.g. #908's test-python job failing on
tests/crud/test_document.py::test_duplicate_rejection_reinforces_existing).
Resolve the encoding from the configured model directly, falling back to
cl100k_base, and only reuse the underlying client's encoding when it has
already been constructed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: make embedding batch size configurable
Add optional max_batch_size to the embedding model config
(EMBEDDING_MODEL_CONFIG__MAX_BATCH_SIZE) to cap texts per request for
OpenAI-compatible providers with smaller limits than OpenAI's, such as
DashScope text-embedding-v4 (10) and Alibaba Bailian
qwen3.7-text-embedding (20). When unset, native provider defaults are
preserved (OpenAI 2048, Gemini 100).
Fixes#687.
* test(embedding): cover Gemini batching and config fallbacks per review
- Gemini transport now tested for configured batch splitting and the 100
default fallback
- OpenAI unset default (2048, single request) explicitly covered
- env-parsing test now asserts the value survives resolve_embedding_model_config
- docs: 100 is the client's conservative Gemini default, not a native limit
* test(embedding): assert provider batch-size defaults
---------
Co-authored-by: adavyas <adavyasharma@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
EmbeddingClient.encoding forced full client construction, which raises
'OpenAI API key is required' even though tiktoken needs no credentials.
The document dedup tie-break (src/crud/document.py) only needs .encoding
for token counting, so any test hitting that path fails in environments
without embedding keys — notably CI for pull requests from forks, where
repo secrets are unavailable (e.g. #908's test-python job failing on
tests/crud/test_document.py::test_duplicate_rejection_reinforces_existing).
Resolve the encoding from the configured model directly, falling back to
cl100k_base, and only reuse the underlying client's encoding when it has
already been constructed.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(llm): set HTTP timeout on Gemini clients (#785)
* fix(embedding): set HTTP timeout on Gemini embedding client (#785)
Same wedge-class failure as the LLM client: a stalled Gemini embedding
socket hangs the in-process reconciler, which shares the deriver worker's
uvloop event loop. Apply the same 10-minute timeout here, in lockstep
with src/llm/registry.py's _build_gemini_http_options.
* style(test): drop extra blank line in test_registry imports
* fix(llm): support per-request provider timeouts
* fix(llm): convert Gemini timeout to milliseconds
* fix(llm): validate Gemini HTTP options
* test(llm): type Anthropic stream context args
* test(llm): live per-request timeout coverage for all providers
Two live checks per provider: a generous timeout asserted at the SDK
call boundary, and a tight timeout that must abort well under the 600s
client default. Gemini's async transport can be aiohttp, so its tight
timeout surfaces as asyncio.TimeoutError rather than httpx.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* style(tests): drop extra blank line in anthropic backend test
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(llm): validate provider_params.timeout at config load
Move the timeout coercion into src.config as coerce_provider_timeout and
run it from a field validator on ModelOverrideSettings.provider_params, so
a bad value in config.toml/env fails at startup with the exact config path
instead of surfacing per-request as a retried 500. Good values normalize
to float seconds at load. The per-request guard in src.llm.backend now
delegates to the same coercion (wrapping ValueError in ValidationException)
and continues to cover extra_params passed programmatically.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: document provider_params.timeout load-time validation and gotchas
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(llm): address review nits on timeout plumbing
Apply eisene's review feedback:
- Rename PROVIDER_TIMEOUT_ERROR → PROVIDER_TIMEOUT_ERROR_TEXT
- Move request_timeout_from_extra_params from backend.py (pure
dataclasses) to request_builder.py (request assembly)
- Add comment explaining Gemini's ms timeout conversion
- Generalize _normalize_extra_params with _strip_none_params helper
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Aakash Kattelu <aakash@plasticlabs.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix: guard against top_k=0 reaching the vector store
Turbopuffer rejects top_k=0 with a 400 ('top_k must be between 1 and
10000'). The fix returns [] for a non-positive top_k, before the embedding call), and
floor the semantic budget at 1 so an explicitly requested search isn't
silently allocated zero.
* fix: comments
* fix: apply session scoping to all working-representation query paths
session_name was only applied to the recent-documents query in
RepresentationManager; the semantic and most-derived paths ignored it,
so limit_to_session leaked cross-session conclusions into perspectives.
- Thread a session allowlist (session_names) uniformly through all
three query paths; pushed down to pgvector and external vector stores
- Accept a list so the upcoming session-allowlist API reuses this path
- Fail closed on an empty allowlist (downstream stores drop empty IN
clauses, which would silently widen scope)
Fixes DEV-1994
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: bare-list membership sugar in the filter DSL
{"session_id": ["s1", "s2"]} is now shorthand for
{"session_id": {"in": [...]}} on regular columns, generically
(peer_id, etc.). JSONB metadata columns are excluded — a bare list
there keeps JSONB containment semantics, unchanged.
Previously a bare list on a regular column compiled to a type-mismatched
equality that matched nothing, so this is strictly additive.
Also translates the same shape in the turbopuffer/lancedb filter
builders, and fixes lancedb dropping empty IN clauses (fail-open) —
an empty membership list now emits an always-false condition.
Groundwork for DEV-1995 (session allowlist via the existing filters
DSL, no new API params)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: session allowlist on dialectic and representation via filters
Adds a constrained 'filters' body to peer.chat and /representation —
the same DSL search and conclusions already accept, supporting only the
session_id key (a session id, a bare list, or {"in": [...]}).
Unsupported keys and shapes are rejected with 422, never silently
ignored. Composes with session_id (must be included in the allowlist
when both are given). Capped at 1,000 sessions per request.
Enforcement is uniform at every recall chokepoint, fail-closed:
- dialectic prefetch + search_memory: conclusion recall restricted to
the allowlist; dream docs (session_name IS NULL) excluded
- message tools (search/grep/date-range/temporal/context/history):
strict intersection of allowlist and observer session membership
- get_reasoning_chain: unavailable under an allowlist (chains traverse
provenance across sessions and cannot be scoped without leaking)
- empty allowlist short-circuits to empty results everywhere
Auth: workspace keys pass the allowlist as-given; peer-scoped JWTs must
be a member of every allowlisted session (403 otherwise), mirroring the
existing single-session check.
Fixes DEV-1995
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: fail closed on empty session allowlist across all filter builders
Empty session allowlists relied solely on the early-return guard in
_get_working_representation_internal. The layers below it were
inconsistent, so a future direct caller (the DEV-1995 allowlist API)
would silently widen scope instead of failing closed:
- _build_filter_conditions used a truthiness check; an empty list was
treated like None and dropped the filter. Now uses `is not None`,
matching the recent/most-derived SQL paths.
- turbopuffer emitted a bare `In []` with undocumented (possibly
fail-open) semantics. Now emits an explicit always-false predicate,
mirroring lancedb's `1 = 0`.
Also extract the duplicated JSONB column tuple in filter.py to a
JSONB_COLUMNS constant.
Tests exercise each fail-closed guarantee at the layer it lives, rather
than masking it behind the early-return guard.
* fix: address tests
* fix(crud): fail closed when session_name is outside the allowlist
search/grep/history helpers scoped to a single session_name ignored the
session_names allowlist entirely — a caller could read a session the
allowlist forbids. The API routes guarded this with a 422, but the
dialectic tools call these CRUD functions directly and bypassed it.
Enforce it at the boundary: return [] when session_name is set and not in
the allowlist, across _semantic_search_messages (covers search_messages +
search_messages_temporal), grep_messages, get_messages_by_date_range,
get_recent_history, and get_observation_context.
Also rename the public param allowed_sessions -> session_names for
consistency with representation.py / chat.py / peers.py; the resolved
intersection keeps its distinct name allowed_session_names.
* fix: test
* fix(scopes): tighten and consolidate session allowlist per review
Addresses review feedback on the session allowlist (DEV-1995).
Behavior changes:
- Auth gate on peers.chat now uses active membership (left_at IS NULL)
via get_peer_session_names(active_only=True), matching the adjacent
is_peer_in_session check on options.session_id. Previously a peer that
had left a session was denied when naming it directly but permitted
when naming it in filters.session_id.
- Scoped conclusion recall is restricted to level == "explicit"
(ALLOWLIST_SAFE_LEVELS). Dream-derived conclusions are stamped with a
single session_name but synthesized across all sessions, so that stamp
can't be scoped on. Applied at all four recall paths. Unscoped recall
is unchanged. Follow-up to give conclusions an authoritative
source-session set is tracked in DEV-2201.
- The allowlist gate checks `is not None` rather than truthiness, so
filters={"session_id": []} reaches it instead of being skipped.
Refactors:
- New crud.message.resolve_session_scope replaces four near-identical
copies of the allowlist-membership intersection. Returns
(allowlist, deny) and never returns an empty list, so the None vs []
distinction that external stores fail open on lives in one tested
place. Takes db=None and opens its own short-lived session only when
distinction that external stores fail open on lives in one tested
place. Takes db=None and opens its own short-lived session only when
an observer lookup is needed, preserving external-lookup-first
ordering on the vector-store path.
- extract_session_allowlist takes must_include, collapsing the
session_id-in-allowlist check duplicated across both peer routes.
- DialecticAgent._select_tools dedupes the two toolset-selection blocks
and drops get_reasoning_chain under an allowlist, rather than paying
for the schema plus a wasted turn to return a refusal.
- Rename session_names -> session_allowlist across crud, agent tools,
dialectic and routes, to remove the one-character ambiguity with
session_name. Internal only; the public filters.session_id surface is
unchanged.
Docs:
- session_allowlist documented across all message and recall entry
points, including the None / [] / populated contract.
- session_name marked deprecated for scoping. Not removed and not
aliased: it also pins the query to one session, bypasses observer
scoping, and drives session-history injection into the dialectic
prompt, so it has no drop-in replacement.
- Note at the Document branch in utils/filter.py that the raw-key
fallback is load-bearing for session scoping.
Tests: 20 -> 39 in tests/test_session_allowlist.py, covering the
peer-scoped JWT gate (member, non-member, left-session, workspace-key
bypass, empty allowlist), the resolve_session_scope tri-state including
the no-DB-checkout path, must_include, and the level narrowing.
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix: redact Redis password from cache connection logs
The cache client logged the full Redis URL — including the password —
at INFO and WARNING levels on every connection attempt and failure.
This exposed the live Redis credential in stdout/container logs and any
downstream log aggregation.
Add _redact_cache_url() to mask the password component before logging.
URLs without a password are returned unchanged.
Closes#866
* fix: handle malformed URLs and IPv6 in _redact_cache_url
Address review feedback from VVoruganti and CodeRabbit:
- Wrap urlparse/urlunparse in try/except so malformed URLs (e.g. invalid
port) don't raise ValueError inside except blocks, which would crash
startup instead of degrading gracefully
- Preserve IPv6 brackets (e.g. [::1]) in reconstructed URLs
- Add Google-style Args/Returns docstring sections
- Add unit tests for password masking, no-password URLs, IPv6,
malformed inputs, and the invalid-port regression
* test: use real secrets in redaction test fixtures
Three fixtures were weakened by copy-paste mangling: literal '***'
placeholders instead of real passwords (assertions trivially true),
an unescaped '#' that truncated netloc parsing via the URL fragment,
and a no-password case that actually contained userinfo. Restore
inputs that genuinely exercise the masking paths.
* fix: never leak password through malformed-URL fallback
The catch-all fallback returned the original URL when parsing failed,
so a Redis URL with a password and an invalid port (typo, out-of-range)
was logged in clear text - the exact leak #866 exists to fix. Narrow
the handling: .port access gets its own try/except (invalid port is
omitted from the output; userinfo/hostname masking never raises), and
the outer fallback now returns a generic placeholder instead of the
raw input. Tightened the invalid-port test to assert the password is
absent and added out-of-range-port and unparseable-URL cases.
* fix: redact secrets in query params and scheme-less URLs
_redact_cache_url only masked userinfo, but a credential can reach the
URL through two other real configuration paths: redis-py accepts
?password= (all querystring options become client kwargs) and cashews
accepts ?secret= (its HMAC signing key) - and honcho's own default
CACHE.URL already uses a query param (?suppress=true), so this is the
expected configuration style. Separately, a URL missing its scheme
(':pass@host:6379/0') parses with an empty netloc, making the password
invisible to .password and echoing it back verbatim.
Mask sensitive query values in place on the raw query string (no
decode/re-encode, so non-secret params are preserved byte-for-byte)
and return the generic placeholder for @-carrying strings with no
parseable authority. Verified with a 20k-case randomized fuzz run in
addition to the unit tests: no functional credential reaches the
output.
Add a PEP 508 marker so lancedb is not installed on darwin/x86_64, wrap the
LanceDB vector store import in try/except for a clear config error, and
regenerate uv.lock.
Branch rebased onto upstream/main; prior src/utils/clients.py CI tweak is
obsolete because LLM wiring moved under src/llm/.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: apply session scoping to all working-representation query paths
session_name was only applied to the recent-documents query in
RepresentationManager; the semantic and most-derived paths ignored it,
so limit_to_session leaked cross-session conclusions into perspectives.
- Thread a session allowlist (session_names) uniformly through all
three query paths; pushed down to pgvector and external vector stores
- Accept a list so the upcoming session-allowlist API reuses this path
- Fail closed on an empty allowlist (downstream stores drop empty IN
clauses, which would silently widen scope)
Fixes DEV-1994
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: bare-list membership sugar in the filter DSL
{"session_id": ["s1", "s2"]} is now shorthand for
{"session_id": {"in": [...]}} on regular columns, generically
(peer_id, etc.). JSONB metadata columns are excluded — a bare list
there keeps JSONB containment semantics, unchanged.
Previously a bare list on a regular column compiled to a type-mismatched
equality that matched nothing, so this is strictly additive.
Also translates the same shape in the turbopuffer/lancedb filter
builders, and fixes lancedb dropping empty IN clauses (fail-open) —
an empty membership list now emits an always-false condition.
Groundwork for DEV-1995 (session allowlist via the existing filters
DSL, no new API params)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: fail closed on empty session allowlist across all filter builders
Empty session allowlists relied solely on the early-return guard in
_get_working_representation_internal. The layers below it were
inconsistent, so a future direct caller (the DEV-1995 allowlist API)
would silently widen scope instead of failing closed:
- _build_filter_conditions used a truthiness check; an empty list was
treated like None and dropped the filter. Now uses `is not None`,
matching the recent/most-derived SQL paths.
- turbopuffer emitted a bare `In []` with undocumented (possibly
fail-open) semantics. Now emits an explicit always-false predicate,
mirroring lancedb's `1 = 0`.
Also extract the duplicated JSONB column tuple in filter.py to a
JSONB_COLUMNS constant.
Tests exercise each fail-closed guarantee at the layer it lives, rather
than masking it behind the early-return guard.
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix: enforce explicit-document session purity in dedup/merge paths
Audit for DEV-2000 (Scopes RFC prerequisite): explicit-level documents must
stay session-pure so scope memory can be built by copying explicit documents
between collections. Two classes of violation were possible:
- Exact-content and semantic dedup in crud/document.py matched candidates
with no level or session scoping, so an explicit document could be
reinforced by — or soft-deleted in favor of — a same-content document from
a different session or a different level (silently merging cross-session
derivations into one row).
- The generic create_observations tool handler accepted level='explicit'
from agents with no message context (dreamer/dialectic), which would mint
session-less explicit documents.
Enforcement (refuse, never rewrite):
- create_documents refuses explicit documents with a null session_name
- exact dedup keys on (content, level, session-for-explicit); derived levels
keep cross-session consolidation
- is_rejected_duplicate scopes candidate search to the same level, and the
same session for explicit documents
- the create_observations tool rejects explicit-level input outside message
ingestion (deriver) context
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: add card_refresh dream type for event-driven peer-card updates
Adds a lightweight dream variant (DEV-2000, Scopes RFC prerequisite) that
runs ONLY the peer-card update — for event-driven refreshes such as scope
membership changes and cold starts:
- DreamType.CARD_REFRESH alongside OMNI; dispatched by process_dream to a
new run_card_refresh_dream orchestration
- CardRefreshSpecialist: restricted to get_recent_observations,
search_memory, and update_peer_card (no observation-mutating tools), with
a low tool-iteration cap of min(6, DREAM.MAX_TOOL_ITERATIONS)
- rebuild=True mode carried in the dream payload: the existing card is NOT
injected into the prompt and the specialist rebuilds it solely from
observations present in the collection (for use after removals)
- enqueue-able via the manual enqueue_dream path (bypasses volume gates);
the work-unit key already embeds the dream type so a card refresh never
collides with a pending omni dream. POST /v3/workspaces/{id}/schedule_dream
accepts dream_type=card_refresh plus the rebuild flag
- card refreshes never advance the omni dream guard pair
(last_dream_at / last_dream_document_count)
- shared PEER CARD prompt section extracted (verbatim) from
DeductionSpecialist for reuse; CallPurpose gains dream.card_refresh
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: fix tests
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Structured outputs for dialectic
* cleanup
* rename json_schema_to_pydantic to clarify it's not a general schema converter
* clean up schema DoS guards
* simplification and cleanup of schema conversion
* chore: ruff and pyproject toml
* chore: basedpyright cleanup in test
* fix: some needed unrelated test failures
* test(schema_conversion-and-anthropic-backend): expand test coverage
include table tests
* fix(llm): support combined tool calling and structured output across backends
- OpenAI: parse() 500s on non-strict function tools; route tool-carrying
structured requests through create() with an explicit json_schema
response_format (mirrors the streaming path)
- Anthropic: skip the '{' JSON prefill when tools are present so tool_use
blocks stay reachable; make the schema instruction conditional and rely
on parse + repair
- Gemini: native response_schema + function calling is rejected before
Gemini 3; with tools present, inject a schema instruction into the final
turn instead and rely on parse + repair
- All backends: tool-call turns carry no consumable content, so skip
structured-output parsing on them
Extracted from the dialectic structured-output branch (DEV-1652) so the
transport layer can land independently.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(live_llm): exercise combined tools + structured output per provider
Two-turn live flow per backend: a forced tool-call turn (structured
parsing must be skipped) followed by a replay turn that must return a
schema-conforming answer with tools still attached. Asserts the
provider-specific request shaping: no parse() for OpenAI (500s on
non-strict tools), no '{' prefill for Anthropic, no native
response_schema for Gemini.
Verified against live OpenAI (gpt-4.1, gpt-5, gpt-5.4, gpt-5.4-mini)
and Gemini (gemini-2.5-flash).
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(unified): dialectic chat with response_format schema under tool use
Adds response_format pass-through to the unified runner's chat query and
a test case that forces the dialectic tool loop (reasoning off + global
enumeration question) while requiring a schema-conforming JSON answer —
end-to-end coverage of the combined tools + structured output transport
path on whichever provider each level is configured with.
Verified locally against a full harness run (json_match assertions pass;
the llm_judge assertion additionally runs in CI where the Anthropic key
is available).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: some needed unrelated test failures
* ci: add label-triggered live LLM test workflow
Adding the run-live-llm label to a PR (or workflow_dispatch) runs
tests/live_llm/ against real provider APIs — the only place the
--live-llm suite runs in CI. Reuses the unified-tests environment and
its Secrets Manager staging-dotenv resolution for provider keys; runs
on ubuntu-latest (no Fly runner, no Docker — the suite only touches the
LLM backends). Pins LIVE_LLM_ANTHROPIC_45_PLUS_MODELS=claude-sonnet-4-5
since the Anthropic family has no default models and would otherwise
silently collect empty.
Opt-in by design: live model behavior is variable, so this is a signal,
not a required check.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci: run live LLM tests on main pushes touching the transport
Mirrors unified-tests' push trigger, scoped to paths that can affect
the live suite (src/llm/, config, the tests, deps, and the workflow
itself) so provider API calls aren't spent on unrelated changes.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci: disable auth in live LLM test environment
The staging dotenv sets AUTH_USE_AUTH=true without a usable JWT secret,
and src/config.py validates the pair at import time — the same reason
unified-tests overrides it. This suite never runs the API server.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(live_llm): fix gpt-5.4 reasoning_effort and gemini replay-turn flake
- test_live_openai: gpt-5.4 dropped 'minimal' from the reasoning_effort
vocabulary, so the gpt5 caching test 400'd — and the OpenAI backend's
BadRequestError terminal swallowed it into an empty CompletionResult.
Pick the effort per model generation.
- test_live_tools_structured_output: use tool_choice='auto' on the
replay turn, matching the production dialectic loop (which never
forces 'none') — NONE mode is what provoked gemini-2.5-flash's empty
candidates. Drop the temperature pin so retries actually resample,
and treat a repeat tool call as a retryable attempt.
Verified live: full suite green, gemini 4/4 consecutive passes.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci: fail live LLM run when no staging secret was loaded
If the latest-tag fetch fails and no second tag exists, the fallback
step is skipped rather than failed, and the job would proceed without
provider keys — every test then skips via require_provider_key and the
run goes green. Guard on both fetch outcomes so that path fails loudly.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(live-llm-tests-GHA): remove extra comments
* feat(structured-output): enable non-recursive schema references
* docs(structured-outputs): clean up new doc
* test(structured-output): fix caching refs memory leak, add tests
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(cli): add device code oauth login for honcho servers
* fix(cli): harden device-auth input validation and transport errors
Reject zero/negative auth-method choices instead of letting Python
negative indexing wrap to the tail of the options list, and wrap httpx
transport failures in the OAuth POST helpers as OAuthFlowError so
connection errors surface through existing caller handling rather than
escaping as an uncaught traceback.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat: add HONCHO_CONFIG_DIR env var for cli
* refactor(cli): address review nits on device-auth PR
- split `init` into manual-key and interactive helpers
- document that access_valid checks persisted expiry, not the token
- rename single-letter local in redacted()
- cover 500 alongside 404 in supports_device_login metadata probe
- add config edge-case tests: stale apiKey drop, garbage/string
accessExpiresAt, empty-env-var popping, refresh-rotation fallback,
missing-token access_valid
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(cli): stop deleting shared apiKey on device login; bind oauth grant to its host
apiKey is shared with sibling tools that read it from the same config
file, so device login must not remove it. Precedence flips to a live
OAuth token over apiKey; a dead grant now degrades to the saved key
with a warning instead of aborting. The oauth block records the host
it was minted against and is ignored (no use, no refresh) when
base_url points elsewhere, so a staging grant is never sent to prod.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* CI/CD Workflow file Added
* Updated Tag & SA Key
* update input Tag
* fix: pass workflow inputs to shell via env to prevent command injection
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Rajat Ahuja <rahuja445@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(llm): support combined tool calling and structured output across backends
- OpenAI: parse() 500s on non-strict function tools; route tool-carrying
structured requests through create() with an explicit json_schema
response_format (mirrors the streaming path)
- Anthropic: skip the '{' JSON prefill when tools are present so tool_use
blocks stay reachable; make the schema instruction conditional and rely
on parse + repair
- Gemini: native response_schema + function calling is rejected before
Gemini 3; with tools present, inject a schema instruction into the final
turn instead and rely on parse + repair
- All backends: tool-call turns carry no consumable content, so skip
structured-output parsing on them
Extracted from the dialectic structured-output branch (DEV-1652) so the
transport layer can land independently.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(live_llm): exercise combined tools + structured output per provider
Two-turn live flow per backend: a forced tool-call turn (structured
parsing must be skipped) followed by a replay turn that must return a
schema-conforming answer with tools still attached. Asserts the
provider-specific request shaping: no parse() for OpenAI (500s on
non-strict tools), no '{' prefill for Anthropic, no native
response_schema for Gemini.
Verified against live OpenAI (gpt-4.1, gpt-5, gpt-5.4, gpt-5.4-mini)
and Gemini (gemini-2.5-flash).
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: some needed unrelated test failures
* ci: add label-triggered live LLM test workflow
Adding the run-live-llm label to a PR (or workflow_dispatch) runs
tests/live_llm/ against real provider APIs — the only place the
--live-llm suite runs in CI. Reuses the unified-tests environment and
its Secrets Manager staging-dotenv resolution for provider keys; runs
on ubuntu-latest (no Fly runner, no Docker — the suite only touches the
LLM backends). Pins LIVE_LLM_ANTHROPIC_45_PLUS_MODELS=claude-sonnet-4-5
since the Anthropic family has no default models and would otherwise
silently collect empty.
Opt-in by design: live model behavior is variable, so this is a signal,
not a required check.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci: run live LLM tests on main pushes touching the transport
Mirrors unified-tests' push trigger, scoped to paths that can affect
the live suite (src/llm/, config, the tests, deps, and the workflow
itself) so provider API calls aren't spent on unrelated changes.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci: disable auth in live LLM test environment
The staging dotenv sets AUTH_USE_AUTH=true without a usable JWT secret,
and src/config.py validates the pair at import time — the same reason
unified-tests overrides it. This suite never runs the API server.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(live_llm): fix gpt-5.4 reasoning_effort and gemini replay-turn flake
- test_live_openai: gpt-5.4 dropped 'minimal' from the reasoning_effort
vocabulary, so the gpt5 caching test 400'd — and the OpenAI backend's
BadRequestError terminal swallowed it into an empty CompletionResult.
Pick the effort per model generation.
- test_live_tools_structured_output: use tool_choice='auto' on the
replay turn, matching the production dialectic loop (which never
forces 'none') — NONE mode is what provoked gemini-2.5-flash's empty
candidates. Drop the temperature pin so retries actually resample,
and treat a repeat tool call as a retryable attempt.
Verified live: full suite green, gemini 4/4 consecutive passes.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci: fail live LLM run when no staging secret was loaded
If the latest-tag fetch fails and no second tag exists, the fallback
step is skipped rather than failed, and the job would proceed without
provider keys — every test then skips via require_provider_key and the
run goes green. Guard on both fetch outcomes so that path fails loudly.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(live-llm-tests-GHA): remove extra comments
* ci(CODEOWNERS): introduce CODEOWNERS and gate GHA heavy test runs behind being a CODEOWNER
* ci(GHA-live-LLM-tests): consolidate common GHA steps
* test(test_live_openai): fix reasoning level adjustment for gpt-5
* test(live-llm-tests): temporary removal of gate to test the workflow
* test(live-llm-tests): revert removal of gate
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Split `REPRESENTATION_BATCH_MAX_TOKENS` into a "minimum work unit" setting on the producer side and a "maximum LLM tokens" setting on the consumer side
* feat(telemetry): CloudEvents + Langfuse tracing as projections over a captured LLM stream
Capture each LLM call once (CapturedLLMCall) and fan it out to multiple
exporters -- "one data model, two projections": a CloudEvents trace stream
(llm.call.traced / trace.content) and a Langfuse projection, both reconstructing
trace -> run -> step -> generation from the same source of truth.
- Capture seam (src/llm/capture.py): one canonicalization + content-addressed
hashing point, with an O(N) per-span memo so repeated context isn't re-hashed.
- Session correlation threaded telemetry -> captured call -> exporters,
namespaced only at the Langfuse export boundary.
- Span identity consolidated onto LLMTelemetryContext; dropped TRACE_ENDPOINT.
- Canonical generation/step names; dreamer branches nest under one dream trace;
tool calls become spans under their step.
- LANGFUSE_EXPORTER_MODE toggle ("exporter" default; "inline" kept one release
for side-by-side validation), centralized into computed settings predicates.
- Per-run/per-trace dedup registries (trace_session, langfuse_session) bounded
by an LRU so dedup and span grouping survive long-running workers.
- Embedding-call tracing; deterministic high-volume event sampling.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(telemetry): address trace-review findings (span/step_seq collisions, test, logging)
- Dreamer specialists mint a distinct span_id per execution (trace_id stays the
shared dream run_id), so their CloudEvents trace resource ids no longer collide
between deduction and induction.
- Tool-loop no-tool early-return streams the tail with the next ordinal
(iteration+2) instead of reusing the in-loop call's step_seq, avoiding a
colliding trace resource id; mirrors the synthesis path.
- Tighten test_clips_oversized_string to assert output stays within TRACE_MAX_BYTES.
- emit_trace logs the swallowed exception with exc_info for debuggability.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(telemetry): silence exporter-mode Langfuse warning + drop summarizer run_id placeholder
Two CloudEvents/Langfuse correctness fixes, independent of the trace viewer.
Langfuse exporter-mode gating: annotate_current_generation_io (and its two
executor.py call-site guards) were gated on LANGFUSE_PUBLIC_KEY instead of
langfuse_inline_enabled. In the default `exporter` mode they called
get_client().update_current_generation() with no active @observe span, logging
"No active span in current context" (~14 per dialectic run) and building
throwaway model_dump payloads on every LLM call. The LangfuseExporter projects
I/O from the captured stream, so these helpers must no-op in exporter mode.
Gated all three on langfuse_inline_enabled; added a regression test; fixed a
stale conditional_observe docstring.
Summarizer run_id placeholder: AgentToolSummaryCreatedEvent hardcoded
run_id="deriver"/iteration=0 because summarization is a single LLM call, not an
agentic run. That placeholder pollutes run_id grouping in the CloudEvents stream
(any consumer that groups by run_id sees a phantom "deriver" run). Made
run_id/iteration optional (None) and re-keyed get_resource_id on
message_id:summary_type (the real per-summary identity; run_id/iteration can no
longer identify it); bumped schema_version 2->3. Xatu ingestion stores only the
CloudEvent envelope, so the field/resource_id/version changes are transparent to it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs: update docstrings to be less verbose
* fix(telemetry): address PR review on captured-stream tracing
- embedding traces get a fresh span_id under parent_span_id=run_id, so
sibling embeddings in one run no longer share a span/idempotency key
- capture the provider finish_reason from stream chunks instead of
hardcoding "stop" on a successful drain
- gate the Langfuse exporter behind TELEMETRY.ENABLED (master switch) so
disabling telemetry sends no traces at all
- rename _emit_derived_content -> _emit_hashed_content
- inline the _emit_trace wrapper; drop unused trace_session.end_run
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: rename TELEMETRY_TRACE_PAYLOADS to TELEMETRY_TRACE_PAYLOADS_ENABLED
* fix(telemetry): capture provider tool calls in trace stream
The captured trace stream dropped assistant tool calls for openai/gemini:
build_captured_messages only read {role, content, tool_call_id}, but those
providers keep tool calls outside content (openai's tool_calls, gemini's
parts), so replayed tool-call turns landed as empty content and gemini lost
its text and tool results entirely. Anthropic (tool_use in content) was fine.
Normalize each input message per provider into a unified tool_calls
[{id, name, input}] field on CapturedMessage/TraceContentEvent, recovering
gemini text/results along the way, and fold tool_calls into
compute_content_hash so empty-content openai turns no longer collide in the
dedup store. langfuse_exporter._input now surfaces the calls.
Also fix a silent serialization drop: gemini thought_signature is bytes, so
model_dump(mode="json") on the traced event raised UnicodeDecodeError and
emit_trace swallowed it -- dropping the whole tool-calling iteration from the
trace stream (billing and Langfuse were unaffected). base64-encode the
signature on the telemetry path; replay keeps the raw bytes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(telemetry): type replay tool-call dict for bytes signature
thought_signature widened to str | bytes | None, but
_tool_call_result_to_dict's literal was inferred as
dict[str, str | dict[str, Any]], so the bytes assignment failed project-wide
basedpyright (the per-file pre-commit hook didn't catch it). Annotate the
dict as dict[str, Any]; the replay path keeps the raw bytes unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: remove 3 tests
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(conclusions): expose reasoning level + allow filtering by level
The `level` of a conclusion (explicit / deductive / inductive /
contradiction) was filterable server-side but stripped from the
`Conclusion` response and not surfaced in either SDK. This adds it
end-to-end so callers can list explicit-only ("not dreamed on")
conclusions without dropping to raw HTTP.
- api: add `level` to the Conclusion response schema
- python sdk: `ConclusionLevel` type, `level` on Conclusion/response,
`level=` kwarg on ConclusionScope.list() and the async variant
- ts sdk: `ConclusionLevel` type, `level` on Conclusion/response,
`level` option on list()
- tests: assert level is exposed; add level-filter list test
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(conclusions): use generic filters= on list() instead of level= kwarg
Match the documented SDK convention (peers/sessions/messages all take a
generic `filters` dict passed through to the same dynamic server-side
filter logic) instead of a one-off `level=` kwarg. `level` filtering now
works as `list(filters={"level": "explicit"})` alongside any other
supported filter/operator.
The `level` field on the Conclusion response (added in the previous
commit) is kept — it's still not otherwise returned by the API.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(conclusions): allow filtering by level on query() in py + ts SDKs
The branch's level-filter work exposed `filters=` on `list()` but left
`query()` (semantic search) hardcoding `{observer, observed}`, so callers
could filter the list endpoint by reasoning level but not semantic search —
asymmetric in both SDKs.
- Python: add keyword-only `filters` to `ConclusionScope.query` and
`ConclusionScopeAio.query`, merged over the scope's observer/observed.
- TypeScript: add optional `filters` arg to `ConclusionScope.query`,
mirroring the existing `list()` change.
The server `/conclusions/query` endpoint already honors filters in the body
(verified against production), so this is purely SDK surface parity.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(filters): document filtering conclusions by reasoning level
The using-filters page covered workspaces/peers/sessions/messages but not
conclusions. Add a "Filtering Conclusions" section showing level-based
filtering on both list() and query(), including the common "explicit only"
(exclude dream-derived) case and the in[deductive,inductive] inverse.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(conclusions): simplify filter merge to a single dict spread
Replace the merged_filters + if-block pattern in list()/query() (py sync,
aio, ts) with a single dict spread that layers the caller's filters over the
scope's observer/observed (and session). No behavior change — same merge
order (caller wins) — just less code.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(conclusions): reject scope-managed keys in SDK conclusion filters
The generic filters= argument on ConclusionScope.list()/query() spread
user-supplied filters last, so a stray observer/observed/session key
silently overrode the scope and returned data from a different peer
pair. Add a fail-loud guard in both the Python and TypeScript SDKs that
rejects scope-managed filter keys with a clear error, directing callers
to peer.conclusions / conclusions_of(target) and the session= parameter.
session_id remains a valid filter on query() (which has no dedicated
session parameter).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* feat: add exact content deduplication in document creation
* feat: add comment for index
* fix: harden times_derived logic across all callers to use max of inputs and existing + 1
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
The OpenAI backend passed tool_choice through raw while the Anthropic and
Gemini backends translate Honcho's canonical vocabulary to their native
form. On a mixed-provider fallback chain (e.g. Gemini primary -> OpenAI
backup), a canonical "any" reached OpenAI unchanged and was rejected as an
invalid param, since OpenAI only accepts none/auto/required.
Add a _convert_tool_choice to the OpenAI backend mirroring the others so a
single TOOL_CHOICE value resolves correctly regardless of which provider a
fallback lands on. "any"/"required" -> "required", auto/none pass through,
a tool-name string or {"name": ...} dict -> a function selection.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(llm): stop capturing live LLM clients in Langfuse generation spans
honcho_llm_call_inner is the @observe generation boundary, and default
auto-capture serialized every argument into the span input -- including
client_override (a live AsyncOpenAI/genai client) and selected_config
(which carries api_key). Auto-capture deep-copies the client into a
half-constructed object whose teardown raises:
- AsyncHttpxClientWrapper ... no attribute '_state' (OpenAI, stderr flood)
- BaseApiClient ... no attribute '_http_options' (Gemini, HONCHO-4HA)
and it leaked ModelConfig.api_key into traces.
Switch from auto-capture (denylist) to explicit annotation (allowlist):
disable capture_input/capture_output on the decorator and stamp curated,
serializable input (messages) and output (HonchoLLMCallResponse) via the
new annotate_current_generation_io helper. Full trace fidelity is
preserved; no client object or secret can reach a trace.
Fixes HONCHO-4HA
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(llm): track call tuning knobs as Langfuse model_parameters
Restore full trace fidelity after disabling @observe auto-capture: surface
every tuning knob (temperature, max_tokens, tools, reasoning effort, ...) on
the generation via model_parameters, sourced from the resolved effective
config instead of the raw function args.
Use a deny-list, not an allow-list: dump the whole ModelConfig and exclude
only secret-bearing fields (api_key, base_url, fallback, provider_params), so
new config knobs are traced automatically without keeping a hand-written list
in sync. The live client is never passed -- there is no useful trace
representation of it and serializing it is what triggered HONCHO-4HA.
Adds a deny-list test proving secrets never leak even when the config carries
a real api_key/base_url/provider_params (the production override-client path).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(llm): duplicate token usage to Langfuse + skip payload build when disabled
Mirror per-call token usage (input, output, prompt-cache read/creation) onto
the Langfuse generation via usage_details, so Langfuse renders native tokens
and cost in addition to the CloudEvents accounting.
Also guard both generation-annotation blocks behind settings.LANGFUSE_PUBLIC_KEY
so the model_dump-backed model_parameters payload (and the usage dict) are only
built when Langfuse is actually configured (addresses CodeRabbit: the annotate
helper no-ops when disabled, but the payload was still being constructed every
call).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: use git tags to fetch secrets for unified test
* fix: test failure
* fix: override AUTH_USE_AUTH and SENTRY_ENABLED
* fix: upload traces
* chore: rm run on PR
* fix: rm bucket from logs
* fix(agent_tools): strip display-format "id:" prefix from model-supplied observation IDs
Observations are presented to agents as [id:xxx], and models sometimes
copy the prefix verbatim despite tool-schema instructions to pass the
bare ID. This silently corrupts source_ids provenance on
create_observations_* (broken links stored in document metadata) and
breaks get_reasoning_chain lookups.
Normalize at both entry points. delete_observations is intentionally
not touched here since #746 already covers it.
Only the "id:" prefix is stripped: document IDs are nanoids whose
alphabet includes "-" and "_", so more aggressive cleanup could mangle
legitimate IDs.
Related to #719.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(agent_tools): strip whitespace remaining after "id:" prefix removal
Addresses CodeRabbit review: defends against "id: xxx" with a space
after the colon, and matches the docstring, which already promised
surrounding-whitespace stripping.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat: send OpenRouter app-attribution headers on OpenAI-compatible clients
Sets HTTP-Referer and X-Title on every AsyncOpenAI client constructed in
src/llm/registry.py (default, override-cached, and module-level CLIENTS) and
in the embedding client, so OpenRouter attributes Honcho's requests to the
"Honcho" app in its dashboard/analytics. Other OpenAI-compatible providers
ignore unrecognized headers, so this is safe to send unconditionally.
* fix: scope OpenRouter attribution headers to OpenRouter base URL only
Address review feedback on #805:
- Only inject attribution headers when the configured base_url starts
with https://openrouter.ai (via new _openrouter_headers() helper)
- Rename X-Title to X-Openrouter-Title per OpenRouter docs recommendation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor: drive default headers from base-URL map, drop embedding path
Replace the OpenRouter-specific _openrouter_headers helper with a generic
_DEFAULT_HEADERS_BY_BASE_URL prefix map + _default_headers_for lookup, so
OpenRouter always receives its attribution headers and another provider can be
added with a single map entry. Revert the embedding-client change (OpenRouter
has no embeddings endpoint, so that gate was dead code) and add a unit test for
the lookup helper.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: compact honcho logging
* fix: guard ms/s metric formatting against non-numeric values
Only apply float formatting when the metric value is numeric so a
str value with an ms/s unit falls through to a plain string instead
of raising. Applied to both the compact and rich log paths.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: add model config option for json_object mode
* fix: catch possible validation error from structured output
* fix(llm): harden structured_output_mode json_object path
Follow-up fixes to the json_object structured-output mode for
OpenAI-compatible providers without Structured Outputs support:
- runtime: carry structured_output_mode onto the per-attempt fallback
config (select_model_config_for_attempt dropped it, silently sending
json_schema to a provider that can't parse it)
- backend: return a graceful empty on a contentless json_object
response instead of raising, matching the json_schema path, and
preserve token usage by normalizing the response
- backend: narrow the parse-failure catch to BadRequestError only, so
transient JSONDecodeError/ValidationError propagate to retry/fallback
instead of being swallowed to empty on the first attempt
- config: reject structured_output_mode on non-openai transports
(silent no-op otherwise); trim docs to the deriver, the only
structured-output feature
- backend: validate clean JSON before repair, cache the schema
instruction, and share json_object setup between complete()/stream()
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(llm): consolidate structured-output repair, drop dead seam
Fold the OpenAI backend's three structured-output repair sites
(LengthFinishReasonError, parsed=None, json_object) into the one shared
_parse_or_repair_structured_content helper, gated by an empty_on_missing
flag: json_object returns a graceful empty on a contentless response so a
loose provider can't crash the call, while json_schema raises so the
retry/fallback chain engages.
Delete the dead execute_structured_output_call seam and its only
collaborators (attempt_structured_output_repair, StructuredOutputFailurePolicy)
— it was never called and its single-shot validate/repair/empty model
conflicts with the retry behavior in honcho_llm_call.
No behavior change. Adds tests covering the json_schema parse fallbacks
(repair, refusal passthrough, no-content raise).
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* telemetry: use session and user IDs in langfuse
* test: update old span test
* fix: disable langfuse in unit tests
* fix: add post-loop synthesis span
* refactor: address PR review feedback on langfuse tracing
- Consolidate track_name onto LLMTelemetryContext as the sole home;
remove the honcho_llm_call kwarg and update 4 callers to set it on
telemetry directly. Sentry ai_track now reads telemetry.track_name.
- Decouple escaped-stream self-stamping from run-context exit ordering:
stream_final_response now resets _in_agent_run explicitly around drain.
- Narrow langfuse_agent_step wrap in the tool loop — between-turn
bookkeeping (iteration_callback, choice switch, increment) lifted
outside the span so it scopes only the LLM call + tools.
- Reword test conftest comment to behavior-only language.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: switch langfuse spans to imperative handles
Replaces the context-manager-based langfuse_agent_run/step with imperative
LangfuseAgentRun/Step handles so the run span can outlive the function that
opens it. Streaming responses now own the run handle from construction and
close it after drain, stamping the accumulated streamed text as trace output
(previously blank). Multi-turn generations always stamp provider/model and
step metadata, fixing the regression where only the first turn was annotated.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(llm): record effective prompt-only input on run span
The run-level Langfuse span recorded the raw messages parameter, which is
None for prompt-only calls. Mirror execute_tool_loop's handling and record
the synthesized user message so the trace input isn't blank.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(llm): drop StreamingResponseWithMetadata.__anext__ to prevent span leak
The standalone __anext__ delegated straight to the inner stream, bypassing
the token-folding and Langfuse run-handle close that live only in the
__aiter__ generator. Any caller driving the wrapper via anext() instead of
`async for` would leak the run span and lose final-stream token accounting.
Latent today (all callers use `async for`), removed to close the footgun.
Add tests covering the run-handle drain path: full drain stamps the
accumulated streamed text as the span output and closes once; an abandoned
stream still closes via the finally rather than leaking.
* chore(llm): document intentional empty-body propagate_attributes block
The `with propagate_attributes(...): pass` stamps the active @observe trace
root via the context manager's __enter__ side effect; the empty body reads
as deletable dead code. Add a comment so it isn't removed. Addresses PR review.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(llm): restore api.py types after __anext__ removal
Dropping StreamingResponseWithMetadata.__anext__ made it stop satisfying
the AsyncIterator protocol, breaking the result annotation and the
isinstance narrowing in honcho_llm_call. Widen the tool-less result
annotation to include StreamingResponseWithMetadata and narrow positively
to HonchoLLMCallResponse before reading .content.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* perf(reconciler): only trace Sentry transactions when work is found
The reconciler enqueues sync_vectors every ~5 min per deriver instance.
process_item wrapped every dequeued reconciler task in a single
process_reconciler_task transaction, so idle cycles (the common case,
where the cycle finds no rows and exits immediately) still created and
sampled a transaction + profile, draining Sentry tracing/profiling quota.
Remove the top-level transaction and push tracing into the sync batch
helpers, starting a per-batch transaction only after rows are confirmed.
Idle cycles now emit zero transactions; busy sweeps emit one smaller
transaction per batch operation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(telemetry): drop infra/scrape transactions via a Sentry traces sampler
Sentry was sampling every transaction at a flat traces_sample_rate with no
sampler. The Prometheus /metrics scrape endpoint alone accounted for ~92% of
all traced transactions (and their profiles), with /openapi.json and the
deriver metrics server adding more pure noise.
Add a traces_sampler that returns 0.0 for infra/scrape endpoints (/metrics,
/health, /openapi.json, /docs, /redoc, and metrics/openapi transaction names)
and the configured rate for real traffic. Sampling here (vs
before_send_transaction) means dropped transactions are never recorded or
profiled and the decision propagates to child spans. Shared init covers both
the API server and the deriver worker.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Peer- and session-scoped JWTs were effectively workspace-scoped: auth() walked the route's declared scope and fell through to a workspace match, so a {w: ws-a, p: alice} token could act on any peer in ws-a.
* feat: peer keys can read sessions they belong to; require workspace on scoped keys
* fix: authorize JWTs by narrowest scope and gate member reads
Follow-up hardening on the narrowest-claim auth fix:
- Scope get_peer_config member-read to the caller's own peer; a session
member could previously read a co-member's per-session config.
- Enforce session membership on POST /peers/{id}/chat: the session_id
arrives in the body (invisible to require_auth), so a peer key could
read any session's injected message history. Check is_peer_in_session
in the handler before the dialectic runs.
- Consolidate the workspace-match check in auth() to a single hoisted
guard so no branch can silently re-open cross-workspace access.
- Normalize empty-string scope claims to None in verify_jwt so a blank
workspace can't satisfy the peer/session token-shape invariant.
- Extract scope_requires_workspace(), shared by verify_jwt and the keys
API so the creation-time guard and verification invariant can't drift.
route requires auth) and CLAUDE.md auth-scoping guidance.
- docs: describe narrow-scope key semantics in the platform reference.
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
The Dialectic section was the only LLM section in .env.template missing
MODEL_CONFIG__OVERRIDES__BASE_URL examples. Without them, users routing
to OpenAI-compatible providers (e.g. Siliconflow) weren't aware the
per-level override existed and fell back to the default OpenAI endpoint,
hitting AuthenticationError. The override already works; this just
enumerates it per reasoning level.
Fixes#818
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Some providers return output_tokens=None on certain completions (observed with
Gemini on tool-loop completions). HonchoLLMCallResponse types output_tokens as
int, so the None propagates into a Pydantic validation error that aborts the
call. In practice this surfaces in the Dreamer: a dream starts, deduction
succeeds, then induction crashes before inductive conclusions are persisted.
Coerce None -> 0 so token accounting degrades gracefully (under-counts rather
than crashing) for providers that omit output token counts.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(dedup): reinforce times_derived on duplicate detection
times_derived was never incremented: the reject-new branch dropped the
reinforcement and the new-wins branch reset the count to 1, so the column
stayed pinned at 1 for nearly every conclusion. With every value equal,
ORDER BY times_derived DESC resolved to arbitrary heap order (oldest rows
first), which froze stale conclusions to the front of injected context.
- reject-new: increment existing_doc.times_derived
- new-wins: carry existing count forward onto the replacement
- add created_at DESC tiebreaker to both most_derived queries
* test(dedup): guard times_derived reinforcement + recency tiebreak
Three regression tests, each fails on pre-fix code:
- most-derived ties break toward recency, not insertion order
- rejecting a duplicate reinforces the surviving doc
- a winning duplicate inherits the replaced doc's count + 1
* fix(dedup): atomic reinforcement increment + deterministic tiebreak
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* feat: defer embedding messages
* fix: rm gauges
* feat: embed messages immediately on create with reconciler fallback (#766)
Adds embed_messages_now background task so newly created messages are
searchable within seconds instead of waiting up to the reconciler
interval. Three-phase claim/lease → embed → persist never holds a DB
session across the embedding call; the reconciler remains the fallback
for failures and stragglers.
* fix: harden immediate-embed fast path and cover its error branches
Wrap embed_messages_now in a top-level try/except so a failure in the
claim or persist phase degrades to "reconciler will retry" instead of
escaping into the background-task runner; the rows stay pending+leased
and the reconciler heals them.
Add tests for the previously-uncovered branches: external-store-unavailable
persist path, the file-upload endpoint's embed scheduling, and direct unit
tests for the shared compute_chunk_positions / build_message_vector_record
helpers.
Document the semantic-search eventual-consistency window in search.mdx
(keyword matches are immediate; vector matches lag creation by seconds).
* fix: don't hold DB session across vector-store upserts
* fix: align semantic-search function to filter null rows
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* feat: implement read DB and fix queue stale cleanup
* fix: use read_db in internal methods
* fix: mention read db in the CLAUDE.md
* fix: make TRACING checkout hook autocommit-safe; sample cleanup-gate jitter once
The DB.TRACING checkout hook ran `SELECT set_config(...)` at pool checkout,
before the dialect applies the read engine's AUTOCOMMIT isolation level. That
statement autobegins a transaction, and psycopg then refuses to switch the
connection into AUTOCOMMIT ("can't change 'autocommit' now: connection in
transaction status INTRANS"), so every read_only session 500s under TRACING and
the INTRANS connection leaks back to poison later write checkouts. Run the hook
in autocommit and restore the prior mode so it never leaves an open transaction;
set_config(..., is_local=false) is session-scoped and survives the boundary.
Add a regression test (fails without the fix) covering read_only + TRACING.
Also sample the stale-cleanup gate's jittered interval once per attempt instead
of re-rolling it every poll, so the spacing is a fixed deadline per cycle rather
than a random walk (and is testable at non-zero jitter ratios).
* fix: reset request_context in TRACING checkout-hook test
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* feat: add generate_jwt.py script for creating scoped JWTs
Adds a CLI utility script for generating Honcho JWTs without needing
to call the /v1/keys API endpoint. Useful for local development and
bootstrapping admin tokens.
Features:
- --admin flag for full-access tokens
- --workspace / --peer / --session flags for scoped tokens
- --expires flag with human-friendly duration syntax (e.g. 5h, 30d, 1y)
- --print-only flag for scripting (outputs bare token)
Examples:
uv run python scripts/generate_jwt.py --admin
uv run python scripts/generate_jwt.py --admin --expires 24h
uv run python scripts/generate_jwt.py --workspace my-ws --expires 30d
uv run python scripts/generate_jwt.py --workspace my-ws --peer my-peer --expires 1y
* docs: document generate_jwt.py in README auth setup section
* fix: remove t='' override to preserve utc_now_iso default in JWTParams
Per CodeRabbit review: explicitly setting t="" bypasses JWTParams's
default utc_now_iso timestamp, causing tokens for the same scope to
become byte-identical. Omitting t lets the default apply, ensuring
each generated token is unique.
* fix: address JWT script review feedback
* fix: type, lint
---------
Co-authored-by: Rajat Ahuja <rahuja445@gmail.com>
* feat(config): make CORS allowed origins configurable via env
Replaces the hardcoded `origins` list in `src/main.py` with a new
`CORSSettings` block (env prefix `CORS_`), exposed as `settings.CORS.ORIGINS`.
Defaults match the prior hardcoded values, so self-hosted deployments behind
custom domains can now whitelist their frontend without editing source.
Documented in `.env.template` under a new CORS Settings section.
* docs(config): add docstring to CORSSettings
* refactor(config): inline CORS_ORIGINS into AppSettings
Drop the dedicated CORSSettings nested model and expose CORS_ORIGINS
directly on AppSettings. The CORS_ORIGINS env var keeps working as
before since AppSettings has no env prefix.
* fix(deriver): Remove connection retry logic and add jitter to polling interval
* chore(docs): Update changelog and document new configurations
* chore: increment version numbers
* feat(db): add connection retry, adaptive deriver polling, and pool metrics
Add resilience and visibility for DB connection handling under transaction-
pooler (Supavisor) saturation, where client-connection limits get exhausted
across many tenants.
- get_db/tracked_db now force an eager pool checkout with bounded exponential
backoff (tenacity), retrying SQLAlchemy TimeoutError + OperationalError so
transient pooler rejections degrade gracefully instead of 500ing. Toggle via
DB_CONNECTION_RETRY_ENABLED (+ delay/backoff knobs); ~10s default budget.
- Deriver polling backs off when idle or erroring (base -> max, x2 each cycle)
and snaps back to base on claimed work, cutting steady-state query load.
Toggle via DERIVER_POLLING_BACKOFF_ENABLED (+ max/multiplier).
- Add scrape-time db_pool_connections Prometheus gauge (checked_out/checked_in/
size/overflow, labeled api|deriver), registered in both the API lifespan and
the deriver metrics server.
- Make SqlalchemyIntegration explicit in both Sentry inits; wrap connection
acquisition in a db.pool.acquire span and capture live pool stats on
retry-exhaustion.
* feat(db): add acquisition counter and in-flight query gauge
Build on the pool-connection metrics with two signals that turn detection
into diagnosis under transaction-pooler saturation:
- db_connection_acquisitions{outcome=ok|retried|exhausted}: counts how often
connection checkout retries through pooler rejection — the alertable early
warning before requests start failing.
- db_queries_in_flight: statements actually executing on the wire (via
SQLAlchemy cursor-execute events, drift-proof across query errors). Pairs
with checked_out: the gap reveals connections held but parked (the "idle in
transaction during an external call" antipattern). Labeled namespace +
instance_type only; gated on METRICS.ENABLED for zero overhead when off.
Add DB-free unit tests for retry outcomes, polling backoff, and in-flight
gauge drift handling.
* fix: address CodeRabbit review on PR #758
- db: roll back the session on a retryable checkout failure before
retrying — a failed autobegin can leave it pending-rollback, making the
next db.connection() raise instead of re-checking-out cleanly. Cheap
Python-side cleanup when no connection was bound.
- metrics: guard DBPoolCollector.collect() so a pool-read/import hiccup
can't raise and abort the whole /metrics scrape (Prometheus drops ALL
metrics if any collector raises) — log and fall back to empty.
* fix(db): lazy retrying session + review fixes for connection backoff
Address Codex/CodeRabbit review on PR #758.
- Replace eager checkout with HonchoAsyncSession: a lazy AsyncSession that
checks out its connection (with retry) on the first DB-touching call, not at
construction. Request handlers doing non-DB work (embedding/file/LLM) before
their first query no longer pin a connection across it, while the API path
still gets checkout retry. Only the checkout is retried — the statement runs
once via super(), so writes are never duplicated. Tracing's set_config moves
into the same lazy acquire hook.
- Roll the session back on a retryable checkout failure before retrying, so a
failed autobegin can't leave it pending-rollback.
- Lower default POOL_TIMEOUT to 5s and validate it stays under the retry budget
for pooled (non-null) POOL_CLASS; update config.toml.example and v2/v3 docs.
- Clamp pool overflow gauge to >= 0 (was negative before the pool fills).
- Remove double-sleep in the deriver idle poll (true backoff cap, not 2x);
make in-flight instrumentation registration idempotent.
- Tests: HonchoAsyncSession lazy/idempotent acquire, statement-runs-once,
tracing, commit/rollback flag reset, get_db no-acquire-at-entry, polling-loop
single-sleep, and the POOL_TIMEOUT/retry-budget validator.
* fix(db): cover all DB-touching session methods; clear flag on close/reset
Address Codex follow-up review on PR #758 (polish, no behavior-critical bug).
- HonchoAsyncSession: wrap get/get_one/stream/stream_scalars/delete in addition
to execute/scalar/scalars/flush/merge/refresh/commit, so the "lazy checkout
with retry on first DB use" guarantee has no holes. connection() stays
unwrapped (acquire_connection_with_retry calls it — wrapping would recurse).
- Reset the acquired flag on close()/reset() too, so a session reused after
close/reset re-acquires (and re-wraps retry) on its next DB use.
- Fix stale comments: connection retry now applies lazily to the request path
via HonchoAsyncSession (config.py), and the FakeSession helper note.
- Tests: close/reset flag reset, and get/delete route through acquisition.
* feat: add new cloudevents for api routes
* fix: add total input tokens to RepresentationCompletedEvent
* feat(telemetry): inject honcho_version + emitter health metrics
* feat(telemetry): per-LLM-call event with try/finally emission + sampler
Adds LLMCallCompletedEvent (llm.call.completed) — fires once per provider hit
with full cost-attribution context: transport/provider_label, model, token
counts with cache breakdown, finish_reason, outcome (success or error),
is_final_attempt flag, retry/fallback state, duration, tool-call shape,
streaming flag, and agent correlation (run_id + iteration).
- src/telemetry/events/llm.py: new event class + CallPurpose closed enum
(deriver.representation, dialectic.answer, dream.deduction|induction,
summary.short|long). Resource id includes attempt so multi-attempt retries
in one iteration get distinct deterministic ids.
- src/telemetry/events/base.py: BaseEvent._volume_class ClassVar (default
"ground_truth"); the new event opts into "high_volume".
- src/config.py: TelemetrySettings.HIGH_VOLUME_SAMPLE_RATE (default 1.0).
- src/telemetry/emitter.py: deterministic sampler keyed on run_id (so an
entire agent trace is kept or dropped together). Aggregate envelopes
bypass the sampler. Sampled-out events increment the dedicated counter
separate from buffer_full/send_failed drops.
- src/llm/runtime.py: AttemptPlan gains attempt/retry_attempts/is_fallback
so the executor reads retry state without re-deriving it.
- src/llm/types.py: LLMTelemetryContext dataclass carrying workspace,
call_purpose, run_id, iteration, peer fields. Iteration is mutable so
the tool loop can set it per inner call.
- src/llm/executor.py: honcho_llm_call_inner wraps the backend call in
try/finally — emits on success AND on exception, with is_final_attempt
computed from AttemptPlan. Stream path emits a was_stream=True placeholder
(token totals deferred until streaming completion is wired through).
Telemetry failures swallowed.
- src/llm/api.py: threads telemetry kwarg through all 4 signatures into
both honcho_llm_call_inner and execute_tool_loop.
- src/llm/tool_loop.py: _telemetry_for_iteration helper copies the caller
context with iteration set per call — covers both the normal iteration
loop AND the max-iteration synthesis call (iteration N+1).
Tests cover success/error emission, sampler trace-coherence (same run_id →
same decision), volume_class enforcement, unknown call_purpose tolerance,
provider_label inference, and telemetry failure isolation. 378/378 pass.
* feat(telemetry): emit agent.iteration on every LLM response + synthesis
AgentIterationEvent was defined but never emitted on this branch. Phase 2
wires it up in execute_tool_loop so every LLM call inside an agentic loop
produces one event — including the no-tool terminating iteration and the
max-iteration synthesis call — and threads LLMTelemetryContext from dialectic
and dreamer specialists down through honcho_llm_call.
- src/telemetry/events/agent.py: AgentIterationEvent opts into
_volume_class="high_volume" so the Phase 1 sampler throttles it.
- src/llm/tool_loop.py: _emit_agent_iteration() helper fires once per
honcho_llm_call_inner response, BEFORE the no-tool early return so the
terminating iteration is counted. A second emission fires for the
max-iteration synthesis call BEFORE final_response is mutated with
cumulative totals (otherwise the per-iteration counts would double-count).
Emission is defensively skipped when telemetry context lacks run_id /
agent_type / parent_category / workspace_name; emit failures are swallowed.
- src/dreamer/specialists.py: BaseSpecialist.run passes LLMTelemetryContext
with parent_category="dream", agent_type=self.name, observer/observed,
call_purpose=f"dream.{self.name}".
- src/dialectic/core.py: _telemetry_context() builds a shared context for
both answer() and answer_stream(), using self._run_id (always set) +
workspace + observed peer.
Tests cover fresh-copy semantics, per-iteration vs terminating emission,
defensive skip cases, telemetry-failure isolation, and volume_class. 408/408
pass across telemetry + llm + utils + dreamer + dialectic.
* feat(telemetry): agent.tool.call.completed event + ToolResult metadata
Adds the missing generic per-tool-call event so read-only tools (search_*,
get_recent_history, get_observation_context, etc.) and the four existing
state-change tools all produce a telemetry record. Built on a new internal
ToolResult(content, metadata) contract so handlers can surface
search-specific fields (top_k/used_embedding/query_tokens/results_count)
to Phase 3 and create/delete counts to Phase 5's specialist rollups.
- src/telemetry/events/agent.py: AgentToolCallCompletedEvent at v1 with
_volume_class="high_volume". Resource id = {run_id}:{iteration}:{tool_call_seq}
so two calls to the same tool in one iteration don't collide
deterministic ids and get dedup-dropped downstream.
- src/utils/types.py: ToolResult dataclass; two new ContextVars
(_current_tool_call_seq + _last_tool_metadata) so tool_loop and the
execute_tool closure can communicate per-call telemetry without changing
the public Callable[[str, dict], Any] signature.
- src/utils/agent_tools.py: execute_tool times handlers, unwraps ToolResult,
publishes metadata, emits the event. Handlers updated to ToolResult
where useful: create/delete observations, update_peer_card, search_memory,
search_messages. Other handlers continue to return str.
- src/llm/tool_loop.py: set_current_tool_call_seq before each executor call;
read get_last_tool_metadata after and stash on all_tool_calls[i] for
Phase 5 rollups.
Tests cover ToolResult str-likeness, ContextVar round-trip, full-context
emission with search metadata, resource-id disambiguation, defensive skip
cases, telemetry isolation, truncation metadata, volume_class. 420/420 pass.
* feat(telemetry): RepresentationCompletedEvent v2 token breakdown + tool-less truncation
Bulks out the deriver's per-batch telemetry without bumping the event schema
version. New additive fields capture the full token breakdown (queued vs.
extra-context vs. scaffold), the cap configuration (batch_max_tokens,
max_input_tokens, was_flush_enabled), real cap-hit flags, and observer
fanout. `input_tokens` stays unchanged as the queued-message-tokens billing
key Xatu's Stripe meter reads.
The big enabler: src/llm/api.py now actually enforces max_input_tokens on
the tool-less LLM path. Before this, the deriver passed the kwarg but the
path silently dropped it — so the configured cap was advisory and
hit_input_token_cap couldn't be measured. Phase 4 wires truncation through
the same truncate_messages_to_fit helper the tool loop uses and surfaces
input_was_truncated on HonchoLLMCallResponse.
- src/telemetry/events/representation.py: 12 additive fields, schema_version
stays at 2.
- src/llm/types.py: input_was_truncated on HonchoLLMCallResponse.
- src/llm/api.py: tool-less path truncates messages before dispatch, flips
input_was_truncated on the response when clamping occurs. Split into
Literal[True]/Literal[False] branches for typecheck.
- src/deriver/queue_manager.py: QueueBatchResult dataclass replaces the
3-tuple return from get_queue_item_batch; carries hit_batch_token_cap
(computed from cumulative token sum vs cap), was_flush_enabled snapshot,
and batch_max_tokens. Worker loop unpacks + forwards.
- src/deriver/consumer.py: process_representation_batch gains the three
flag kwargs and forwards.
- src/deriver/deriver.py: derives the breakdown fields locally, populates
the new fields on emit, sources hit_input_token_cap from
response.input_was_truncated.
Tests cover schema stability, defaultable fields, input_tokens semantic
preservation, cap-hit flag round-trip, model_dump completeness, and
HonchoLLMCallResponse.input_was_truncated mutability. Existing
test_queue_processing.py tests updated for QueueBatchResult and mock
process_representation_batch signature. 479/479 pass.
* feat(telemetry): DreamRunEvent v2 scheduler reasons + DreamSpecialistEvent v2 rollups
Bumps both dream events to v2 with additive fields. DreamRunEvent gains
scheduler context (threshold_reason / delay_reason / documents_since_last_dream_at_schedule /
document_threshold / dream_type / enabled_types_count) threaded through the
dream queue payload — the two scheduler gates stay as separate fields rather
than collapsing into one trigger_reason, preserving the WHY-vs-WHEN
semantics. DreamSpecialistEvent gains denormalized rollups
(created_observation_count / deleted_observation_count / peer_card_updated /
search_tool_calls_count) sourced from Phase 3's ToolResult.metadata so the
counts reflect observation truth, not call truth.
- src/telemetry/events/dream.py: schema_version → 2 for both events; new
fields all defaultable so older producers still construct valid events.
- src/utils/queue_payload.py: DreamPayload + create_dream_payload accept
threshold_reason / delay_reason / documents_since_last_dream_at_schedule /
document_threshold.
- src/dreamer/dream_scheduler.py: check_and_schedule_dream computes the two
reasons at decision time and threads them through schedule_dream →
_delayed_dream → execute_dream → enqueue_dream.
- src/deriver/enqueue.py: create_dream_record / enqueue_dream gain the
kwargs and persist on the queue payload.
- src/dreamer/orchestrator.py: process_dream unpacks the payload; run_dream
accepts the kwargs and stamps them on DreamRunEvent.
- src/dreamer/specialists.py: BaseSpecialist.run walks response.tool_calls_made
and sums ToolResult.metadata.created_count / .deleted_count, sets
peer_card_updated, counts search-tool calls by name.
Tests cover schema_version bumps, defaultable Phase 5 fields,
threshold-vs-delay semantics, observation-vs-call-count rollup distinction,
and DreamPayload round-trip. Existing tests updated for the schema bump
and the new enqueue_dream kwargs. 488/488 pass.
* feat(telemetry): AgentToolSummaryCreatedEvent v2 token breakdown
Bumps schema_version to 2 and adds three additive breakdown fields so
analytics can answer "how much of a summary call's cost was the previous-
summary rollup vs. the new messages vs. the scaffold instructions".
- src/telemetry/events/agent.py: previous_summary_tokens, message_tokens,
prompt_scaffold_tokens added with sensible 0 defaults. input_tokens
retains its current semantic (provider-side LLM tokens) — the plan's
proposed `provider_input_tokens` was omitted because input_tokens
already serves that purpose and a duplicate would fork queries.
- src/utils/summarizer.py: emit now populates the three new fields from
values already in scope (messages_tokens, previous_summary_tokens,
prompt_tokens). Hoisted prompt_tokens calculation out of the
is_fallback conditional so both the save-summary path and the emit
share one binding — basedpyright couldn't prove the sibling-scope
binding was safe, and the compute is cheap + idempotent.
Tests cover schema bump, defaultable fields, input_tokens semantic
preservation, first-summary edge case, and breakdown round-trip.
493/493 pass.
* feat(telemetry): embedding.call.completed event + call-purpose ContextVar
Adds the final piece of cost-attribution telemetry: per-embedding-call
events covering every provider hit (single + batch + retry attempts).
Embedding calls are real provider spend that was invisible before this
phase; search-heavy paths (dialectic agentic) can produce more embedding
calls than LLM calls, so the new event participates in the shared
HIGH_VOLUME_SAMPLE_RATE.
- src/telemetry/events/llm.py: EmbeddingCallCompletedEvent at v1 with
_volume_class="high_volume". EmbeddingCallPurpose closed enum
(search_memory / search_messages / create_observations / vector_sync /
summary / message_create). Resource id = run:purpose:provider:model:input_count
so per-iteration calls in one agentic run don't collide.
- src/utils/types.py: _embedding_call_purpose ContextVar plus
@contextmanager wrapper. Nesting-safe via ContextVar.reset(token).
Callers wrap embedding-driving operations in
`with embedding_call_purpose("search_memory"): ...` — no changes to
the embedding client signature.
- src/embedding_client.py: _emit_embedding_call wraps each provider hit
with try/finally so success AND error paths emit. Errors propagate
unchanged. Each retry attempt of _process_batch emits its own event.
Unknown call_purpose slugs drop to None (validation against the enum
happens at emit time, not at context-manager-set time).
- src/utils/agent_tools.py: search_memory / search_messages /
search_messages_temporal / create_observations (batch + fallback) all
tag their embedding calls.
- src/crud/representation.py: save_representation tags with
CREATE_OBSERVATIONS; get_working_representation precompute tags with
SEARCH_MEMORY.
- src/crud/message.py: create_messages batch embed tags with
MESSAGE_CREATE; search_messages/temporal fallback tags with
SEARCH_MESSAGES.
Tests cover event shape, enum closure, ContextVar nesting/exception
cleanup, wrapper success+error emission, unknown-purpose graceful
fallback, telemetry-failure isolation. 550/550 pass across the full
telemetry+llm+utils+dreamer+dialectic+deriver+crud test set.
* chore: fix tests
* fix(telemetry): address review findings on stream events, context propagation, and cap detection
Five findings from a post-Phase-7 review (one resolved by the merge from
main, four addressed here):
- src/llm/executor.py: stream-path LLMCallCompletedEvent now fires AFTER
the stream is set up and drained (or on exception), with real duration
and accurate outcome. Previously the event was emitted before
execute_stream() ran and was always recorded as outcome="success" with
duration_ms=0, which silently masked stream-setup and stream-drain
failures. Wrapping the async generator in try/finally surfaces the real
outcome; token counts stay 0 because we still don't have them at stream
end (aggregate envelopes carry totals).
- src/deriver/deriver.py + src/utils/summarizer.py: deriver and summarizer
LLM calls now thread LLMTelemetryContext into honcho_llm_call. Before
this, the closed CallPurpose enum had DERIVER_REPRESENTATION /
SUMMARY_SHORT / SUMMARY_LONG slugs but those production call sites
didn't actually pass `telemetry=`, so their LLMCallCompletedEvents lost
workspace_name, parent_category, and call_purpose. summarizer threads
workspace_name through _create_and_save_summary → _create_summary →
create_short_summary / create_long_summary.
- src/utils/types.py + src/embedding_client.py: embedding_call_purpose
ctx manager now accepts workspace_name and run_id kwargs, backed by
two new ContextVars. EmbeddingCallCompletedEvent's publisher reads
both via get_embedding_workspace_name / get_embedding_run_id so
embedding events carry workspace and run correlation. All call sites
updated: search_memory / search_messages / search_messages_temporal /
_handle_create_observations_impl pass ctx.workspace_name +
ctx.run_id; create_observations standalone and create_messages pass
workspace_name; RepresentationManager.save_representation and
get_working_representation pass self.workspace_name.
- src/deriver/queue_manager.py: hit_batch_token_cap detection rewritten.
Previously summed kept-rows' token_count and checked against
batch_max_tokens, but the SQL filter `cumulative_token_count <= cap`
guarantees kept rows stay under the cap, so the flag almost never
fired. Now uses two follow-up queries: total token_count across the
included id range + EXISTS check for any session message past the
last-kept id. Both true → cap was actually binding.
(The fifth finding — deriver scaffold-token computation needing
estimate_deriver_prompt_tokens(custom_instructions) — was resolved by
the merge from main; the Phase 4 emit at src/deriver/deriver.py:283
already sources prompt_scaffold_tokens from the wrapped helper.)
567/567 telemetry+llm+utils+dreamer+dialectic+deriver+crud tests pass.
ruff + basedpyright clean.
* chore: ruff linting
* chore: clean AI generated comments references specs
* fix: address coderabbit changes
* fix(telemetry): address remaining PR review findings
Six findings from the PR 637 telemetry review batched into one commit.
- src/llm/executor.py + src/embedding_client.py: asyncio.CancelledError
now surfaces as outcome="cancelled" on both stream and sync paths,
distinct from "error". Client disconnects mid-stream and server
shutdowns are normal control flow and should not feed error-rate
alerting. LLMCallCompletedEvent and EmbeddingCallCompletedEvent
outcome Literal extended; docstrings + tests cover the new state.
- src/utils/types.py + src/llm/tool_loop.py: new iteration_scope()
context manager captures and resets the four per-tool-loop
ContextVars (_current_iteration, _current_tool_call_seq,
_current_provider_tool_call_id, _last_tool_metadata). Applied as a
typed decorator to execute_tool_loop so back-to-back loops in the
same asyncio Task (worker batches, tests) don't observe stale state.
- src/telemetry/events/api.py + src/routers/messages.py:
MessageCreatedEvent schema v1 → v2. Added required last_message_id
(nanoid public_id of the trailing message); get_resource_id now keys
on it instead of message_count, eliminating the collision case where
two same-size batches in the same session+source produced identical
event ids. message_count stays on the body for analytics.
- src/deriver/queue_manager.py: hit_batch_token_cap now computed from
the FINAL post-config-filter batch. Previously the flag used the
pre-filter messages_context[-1].id, which produced false positives
when _resolve_batch_configuration trimmed the trailing queue item —
telemetry reported a cap-hit when the actual returned batch was
short for unrelated reasons. Cap-detection block moved inside the
async with after the filter; no extra DB connection.
- src/config.py + src/telemetry/emitter.py: documented the
HIGH_VOLUME_SAMPLE_RATE orphan trade-off (rate<1.0 keeps aggregates
but drops children, so JOIN ON run_id queries see partial traces).
Behavior unchanged — rate defaults to 1.0.
- src/deriver/deriver.py: WARNING-level invariant logs when
response.input_tokens < messages_tokens (provider tokenization
drift) or prompt_scaffold_tokens <= 0 (estimator silent failure).
Best-effort — telemetry never bleeds into the deriver path
* fix(telemetry): stream retry, embed attempts, truncation, dedup
Address remaining audit findings on the cloudevents PR:
- Stream setup now runs inside the awaited honcho_llm_call_inner so
tenacity's retry wrapper in stream_final_response catches transient
setup failures (rate-limit, auth, network). Previously the returned
generator deferred execute_stream until first iteration — outside
the retry wrapper — crashing the request and bypassing telemetry.
- Embedding _emit_embedding_call gains an is_final_attempt parameter;
_process_batch threads the real retry index so dashboards stop
conflating one-shot, mid-retry, and exhausted-retry calls.
- _truncate_tool_output returns (text, original_chars, was_truncated)
and a new _maybe_truncated_result helper wraps in ToolResult when
truncation happens. Five handlers migrated. AgentToolCallCompletedEvent
fields was_truncated and result_chars_before_truncation are now
populated instead of always None/False.
- execute_tool_loop tracks any_iteration_truncated and stamps
input_was_truncated on the final response (both HonchoLLMCallResponse
and StreamingResponseWithMetadata). Dialectic now reports
hit_input_token_cap correctly.
- GetContextEvent.get_resource_id uses empty-string sentinel instead
of literal "none" so a peer named "none" can't collide with absent.
- generate_event_id folds honcho_version into the deterministic id so
same logical event from different deploys produces distinct ids.
* fix(telemetry): address audit findings across LLM/embed/event paths
Three rounds of telemetry audit findings, grouped by area:
Retry correctness
- Stream LLM setup now runs inside the awaited honcho_llm_call_inner so
tenacity's outer retry catches setup failures (Fix 1). Previously the
inner generator deferred execute_stream past the retry wrapper.
- stream_final_response bumps the per-retry attempt index via
dataclasses.replace so emitted events show [1, 2, 3] instead of
[1, 1, 1] (Fix 13).
- Embedding _emit_embedding_call takes is_final_attempt; _process_batch
threads the real retry index (Fix 2).
Token + cost reporting
- HonchoLLMCallResponse.hit_input_token_cap (renamed from
input_was_truncated) uses a token-based rule so single-message
over-cap inputs are correctly flagged — the deriver's prompt-only
path used to silently fly through. Propagated through tool_loop's
per-iteration check (Fix 4) and into RepresentationCompletedEvent.
- DialecticCompletedEvent gains hit_input_token_cap; output_tokens now
folds in the final-stream's cumulative usage via
StreamingResponseWithMetadata.__aiter__ (Fix 7).
Event emission completeness
- AgentToolCallCompletedEvent's was_truncated /
result_chars_before_truncation populated by _truncate_tool_output via
a new _maybe_truncated_result wrapper; 5 handlers migrated (Fix 3).
- DreamSpecialistEvent emits on failure with success=False + new
error_class field, via try/finally (Fix 11).
- DeletionCompletedEvent emits on failure paths via try/finally
(Fix 12).
- CleanupStaleItemsCompletedEvent.queue_items_cleaned populated from
deleted_count (Fix 8).
Embedding call attribution (Fix 9)
- embedding_call_purpose context manager accepts parent_category.
- 4 new EmbeddingCallPurpose enum values: DIALECTIC_PREFETCH,
SESSION_CONTEXT_SEARCH, PREFERENCE_EXTRACTION, GENERIC_DOCUMENT_SEARCH.
- Wrapped previously-unattributed sites: dialectic prefetch, session
context search, preference extraction, conclusions search, vector
sync (×2).
Deterministic event ID + dedup
- generate_event_id folds honcho_version into the hash so cross-deploy
events don't silently collide on ID (Fix 6).
- GetContextEvent resource_id uses empty-string sentinel instead of
"none" so a peer literally named "none" can't collide (Fix 5).
Queue batch cap detection (P2.1)
- hit_batch_token_cap keys on the pre-config-filter SQL boundary so the
"kept=900 of 1000 cap, next=300 excluded by cap" case reports True
while still avoiding the config-filter false positive.
Tool result metadata
- search_messages_temporal returns ToolResult with the same search_meta
shape as search_memory / search_messages (P2.3) — top_k,
used_embedding, embedding_query_count, query_tokens, results_count.
Tests: stream-setup retry, stream-retry attempt sequence, post-stream
output_tokens write-back, is_final_attempt matrix, truncation E2E,
tool-loop hit_input_token_cap propagation, honcho_version in event id,
GetContextEvent disambiguation, queue_items_cleaned round-trip.
* fix(telemetry): address audit findings across LLM/embed/event paths
Four rounds of telemetry audit findings (initial + 3 follow-ups), grouped
by area:
Retry correctness
- Stream LLM setup now runs inside the awaited honcho_llm_call_inner so
tenacity's outer retry catches setup failures (Fix 1). The inner
generator previously deferred execute_stream past the retry wrapper.
- stream_final_response bumps the per-retry attempt index via
dataclasses.replace so emitted events show [1, 2, 3] instead of
[1, 1, 1] (Fix 13).
- Embedding _emit_embedding_call takes is_final_attempt; _process_batch
threads the real retry index (Fix 2).
Token + cost reporting
- HonchoLLMCallResponse.hit_input_token_cap (renamed from
input_was_truncated) uses a token-based rule so single-message
over-cap inputs are correctly flagged — the deriver's prompt-only
path used to silently fly through. Propagated through tool_loop's
per-iteration check (Fix 4) and into RepresentationCompletedEvent.
- DialecticCompletedEvent gains hit_input_token_cap; output_tokens now
folds in the final-stream's cumulative usage via
StreamingResponseWithMetadata.__aiter__ (Fix 7).
Queue batch cap detection
- hit_batch_token_cap previously required total_in_range >= cap, which
produced false negatives whenever the kept range didn't fully exhaust
the budget. Replaced with a pre-config-filter SQL boundary check
(P2.1), then further refined to a queue-item boundary comparison
(Fix 14) so trailing-context trimming doesn't false-negative either.
Event emission completeness
- AgentToolCallCompletedEvent's was_truncated /
result_chars_before_truncation now populated by _truncate_tool_output
via _maybe_truncated_result; 5 handlers migrated (Fix 3).
- DreamSpecialistEvent emits on failure with success=False + new
error_class field, via try/finally (Fix 11). except BaseException
catches cancellations too (Fix 16).
- DeletionCompletedEvent emits on failure paths via try/finally
(Fix 12), and uses ValidationException for unsupported types per
project guideline (Fix 17).
- CleanupStaleItemsCompletedEvent.queue_items_cleaned populated from
deleted_count (Fix 8).
Embedding call attribution (Fix 9)
- embedding_call_purpose accepts parent_category.
- 4 new EmbeddingCallPurpose values: DIALECTIC_PREFETCH,
SESSION_CONTEXT_SEARCH, PREFERENCE_EXTRACTION, GENERIC_DOCUMENT_SEARCH.
- Wrapped previously-unattributed sites: dialectic prefetch, session
context search, preference extraction, conclusions search, vector
sync (×2).
Reconciler no longer holds DB session during embedding (Fix 15)
- _sync_documents and _sync_message_embeddings refactored into
three phases per CLAUDE.md guideline: fetch+detach in a small DB
scope, external embedding call without DB locks, writes in a fresh
short-lived DB scope. New _apply_*_sync helpers; orchestrators
expunge ORM objects before invoking. Vector store upsert + sync_state
updates stay in the apply phase together.
Deterministic event ID + dedup
- generate_event_id folds honcho_version into the hash so cross-deploy
events don't silently collide on ID (Fix 6).
- GetContextEvent resource_id uses empty-string sentinel instead of
"none" so a peer literally named "none" can't collide (Fix 5).
Tool result metadata
- search_messages_temporal returns ToolResult with the same search_meta
shape as search_memory / search_messages (P2.3).
- Dialectic.prefetched_conclusion_count uses Representation.len() so
inductive + contradiction observations count too (Fix 10).
* fix(telemetry): orchestrator emit + review feedback
Three more rounds of audit findings + inline PR review, grouped:
Orchestration / emit reliability
- run_dream wrapped in try/finally so DreamRunEvent always emits, even
on unexpected exceptions including CancelledError (`finally` still
runs while cancellation propagates). Specialist except clauses
broadened from SpecialistExecutionError (never raised in src/) to
Exception so provider/DB/tool failures are recorded with
deduction_success=False / induction_success=False instead of crashing
past the emit.
- BaseSpecialist.run() telemetry state initialization + try/finally
hoisted above the preflight phase (peer lookup, peer-card preload,
create_tool_executor, get_model_config, prompt construction) so
preflight failures emit DreamSpecialistEvent(success=False) instead
of being dropped on the floor.
- Reverted the Round-4 _sync_documents / _sync_message_embeddings
phase split. The split introduced a race: rows were released from
FOR UPDATE SKIP LOCKED before the embed call, allowing two workers
to claim and clobber the same batch. Long-held DB transaction
restored (pre-existing CLAUDE.md violation accepted as a deliberate
trade-off; proper fix requires a claim/in_flight migration tracked
separately).
Schema + naming (PR-internal — none of these have shipped)
- threshold_reason → trigger_reason on DreamRunEvent, DreamPayload, and
every emit/scheduler/router/test call site (~45 src + 21 test lines).
Name now accurately reflects the field's role across "manual",
"surprisal", and "document_threshold" values.
- MessageCreatedEvent reset to schema v1 (was internally bumped to v2
for last_message_id but never shipped at v1 — downstream sees it
for the first time at merge).
- DreamSpecialistEvent gains created_counts_by_level /
deleted_counts_by_level: dict[str, int] keyed on the closed
level taxonomy. Per-tool-call events use list[str] (≤10 items),
but specialist runs aggregate 20+ — dict keeps emissions compact.
- QueueBatchResult marked frozen=True.
Per-call embedding attribution
- Agent tool embedding_call_purpose wraps for search_memory,
search_messages, search_messages_temporal, create_observations now
driven embedding cost rolls up under the right workflow.
- create_observations() signature gains parent_category kwarg
(mirrors existing run_id pattern).
Manual dream scheduling
- Manual /schedule_dream route now passes trigger_reason="manual" and
delay_reason="immediate". Previously both arrived as null in
DreamRunEvent, breaking analytics joins.
Queue-batch SQL perf
- next_exists_check folded into the main CTE query via
bool_or(cumulative_token_count > batch_max_tokens) OVER () in a
nested subquery. Cap detection is now one roundtrip per batch
instead of two.
Code/doc cleanup
- representation.py docstring uses generic "downstream metering key"
language (was "Xatu's Stripe meter"). bench runner --base-url help
uses a generic example host (was "groudon.fly.dev"). Public-facing
code/docs shouldn't reference internal service names.
Tests added for: orchestrator failure-path DreamRunEvent emission,
specialists preflight try/finally coverage, manual-dream
trigger_reason/delay_reason round-trip, dict-rollup accumulation across
multiple tool calls in a specialist run, CTE-fold one-roundtrip
behavior. Full Python suite passes (1236).
* fix(telemetry): correctness + attribution + emitter robustness
- Dreamer iteration count: read response.iterations directly so
one-shot runs no longer report iterations=0 and tool-using runs
include the terminal/synthesis LLM call.
- RepresentationCompletedEvent.observer_count counts successful
saves, not attempts.
- search_memory empty-memory fallback reports the snippet count when
message context is returned (was always 0).
- Wire parent_category through every embedding emit path: message
create (api), save_representation (representation), per-observation
fallback (caller-supplied), and the peer/session context routes
(api). get_working_representation accepts parent_category and
embedding_purpose so the internal fallback embed lands in the same
analytics bucket as the route-level precompute even when the
precompute is suppressed.
- BatchItem carries token_count so _process_batch reuses chunk-prep
counts instead of re-encoding every chunk for the telemetry proxy.
- Drop vestigial EmbeddingCallCompletedEvent.batch_size (always ==
input_count).
- Emitter: release the lock during HTTP send so a failing endpoint's
retry+backoff (~36s worst case) doesn't block other flushers;
edge-trigger the 80%-capacity warning so sustained backpressure
doesn't flood logs; defer event_id generation past the high-volume
sampler for events with run_id so sampled-out children don't pay
the sha256; harden emit() against sync callers with no running
loop; track threshold-flush tasks so shutdown() drains in-flight
sends before closing the HTTP client.
* fix(telemetry): tool cancellation emit, nanoid run_ids, version unification
- execute_tool: wrap post-work in finally so AgentToolCallCompletedEvent
fires on CancelledError; explicit handler sets is_error/result_str
before re-raising.
- run_id: replace str(uuid.uuid4())[:8] with generate_nanoid() across
dialectic/dreamer/specialists; matches project-wide nanoid convention.
- Bump _schema_version on events touched by run_id widening:
DialecticCompletedEvent v1→v2 (also covers hit_input_token_cap field),
AgentIterationEvent v1→v2, AgentToolConclusionsCreatedEvent v1→v2,
AgentToolConclusionsDeletedEvent v2→v3, AgentToolPeerCardUpdatedEvent
v1→v2.
- Unify honcho_version: single HONCHO_VERSION constant in src/_version.py
read from pyproject.toml (importlib.metadata fallback). Drop
TELEMETRY.HONCHO_VERSION setting. Use the constant for the FastAPI app
version (no more hardcoded "3.0.6") and for emitter body injection.
- Delete 17 tautological per-event test_schema_version methods; the
parametrized contract test still enforces version >= 1 across all events.
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* feat(embedding): add dimensions_mode for OpenAI dimensions= forwarding
Add EMBEDDING_MODEL_CONFIG__DIMENSIONS_MODE (auto|always|never) controlling
whether the dimensions= parameter is forwarded on OpenAI embeddings.create
calls. auto (default) sends it when the operator explicitly set
EMBEDDING_VECTOR_DIMENSIONS and the configured model is not on the
known-rejecting allowlist (currently text-embedding-ada-002).
The provenance check (was VECTOR_DIMENSIONS explicitly set?) lives as
EmbeddingSettings.resolve_send_dimensions() because it needs access to
model_fields_set, which the standalone resolver does not have. The
resolved boolean is passed into _EmbeddingClient at construction time;
the client never inspects mode or provenance.
Also pins cloudevents <2.0 — 2.0.0 reorganized the package and dropped
cloudevents.conversion and cloudevents.http, which src/telemetry/emitter.py
imports. The original `>=1.12.0` constraint allowed the broken 2.0 resolve.
With the pin, the imports resolve cleanly and the basedpyright warning
cascade (37+ warnings about unknown types) disappears.
Drive-by cleanups (all unnecessary cast/ignore comments flagged by
basedpyright after the cloudevents downgrade):
- vector_store/lancedb.py, tests/conftest.py, and
tests/deriver/test_vector_reconciliation.py — drop dead pyright ignores
- sdks/python/src/honcho/http/{async_,}client.py — drop unnecessary
cast(datetime, ...) (parsedate_to_datetime already returns datetime)
- vector_store/turbopuffer.py — cast(Any, rows) for the upsert_rows
TypedDict that the SDK exposes but our row builder doesn't satisfy
- tests/test_datetime_parsing.py — ignore reportArgumentType on the
test that deliberately passes wrong types to assert raises
* feat(models): honor EMBEDDING_VECTOR_DIMENSIONS in pgvector columns
* feat(startup): atomic swap dim-vs-MIGRATED guard for runtime schema validator
Add src/startup/embedding_validator.py that introspects the actual pgvector
column dim at boot and refuses to start if it does not match
EMBEDDING_VECTOR_DIMENSIONS. Runs after the DB pool is up and before the
embedding client is constructed, in both src/main.py (FastAPI lifespan) and
src/deriver/__main__.py.
Implementation details:
- Schema-qualified pg_attribute join through pg_class/pg_namespace respects
DB.SCHEMA rather than relying on search_path
- Bounded retry (3 attempts, 1s backoff) for transient introspection failure,
then fail-closed with "could not validate embedding schema" — uncertainty
is not a green light to serve traffic
- External-store sampler (turbopuffer, lancedb) enumerates workspaces from
the application DB and probes their lazy-created namespaces; current
per-namespace probe is a no-op stub since the SDKs do not expose
uniform dim introspection — full enumeration is left to
`configure_embeddings --report` in Phase 3
Atomic guard swap: deletes the old dim-vs-MIGRATED config validator (which
forbade non-1536 pgvector unless MIGRATED=True) in the same commit as the
new runtime validator. There is no release window where non-1536 pgvector
can start unprotected. The 9 dual-write branches that use VECTOR_STORE.MIGRATED
remain untouched and load-bearing for legacy-tenant backend swaps.
VECTOR_STORE_DIMENSIONS deprecation: drop the "must match" raise; in
propagate_namespace, check model_fields_set and emit logger.warning +
DeprecationWarning (DeprecationWarning alone is filtered by Python's default
config and would not reach operators). Always overwrite with
EMBEDDING.VECTOR_DIMENSIONS regardless.
Test changes:
- tests/test_models_vector_dim.py: Phase 1's VECTOR_STORE_TYPE=lancedb +
MIGRATED=true escape hatches removed; the test now passes on plain
EMBEDDING_VECTOR_DIMENSIONS=768
- tests/llm/test_model_config.py: the two tests asserting the old guards
replaced with tests for the new deprecation + acceptance behavior
- tests/startup/test_embedding_validator.py: 10 new tests — dim assertion
logic (pass/mismatch/missing/unbounded/non-public-schema), fail-closed
retry budget, real-test-DB pass, real-DB ALTER-then-validate, deprecation
warning capture, non-1536 + pgvector + MIGRATED=false at config time
* feat(scripts): add configure_embeddings bootstrap CLI
Adds scripts/configure_embeddings.py alongside the other one-off scripts
(provision_db, migrate_db, generate_jwt_secret, etc.). Invoked as
`uv run python scripts/configure_embeddings.py` — same convention as the
existing scripts in that directory, including the sys.path shim that
lets src.* imports resolve when run directly.
Bootstrap step for self-hosted installs at a non-default
EMBEDDING_VECTOR_DIMENSIONS — runs between `alembic upgrade head` and
starting the API/deriver.
pgvector ALTER safety (single transaction):
- LOCK TABLE {schema}.documents, {schema}.message_embeddings IN ACCESS
EXCLUSIVE MODE — closes the TOCTOU window between population check
and ALTER
- COUNT(*) WHERE embedding IS NOT NULL on both tables; refuse with a
non-zero exit if either is populated (ALTER ... USING NULL would
silently wipe those vectors)
- Snapshot HNSW index DDL from pg_indexes; drop, ALTER, recreate from
the captured DDL so operator-set HNSW params (m, ef_construction)
survive the round trip
External vector stores (turbopuffer, lancedb) are never created or
modified — namespaces are per-workspace and lazy-created on first write.
The --report mode enumerates workspaces and collections from the
application DB, derives the expected namespaces via
get_vector_namespace(), and prints a per-namespace status table.
CLI modes (mutually exclusive):
- (default) interactive: print plan, prompt to confirm
- --dry-run: print plan and exit 0 without touching the DB
- --yes: apply without prompt
- --report: print external-store namespace inventory and exit
Also updates src/startup/embedding_validator.py error-message paths and
docs/v3/contributing/configuration.mdx invocations to point at the new
script location.
Tests cover plan no-op, plan needs-alter, plan raises on missing column,
ALTER + HNSW round-trip, refuse-when-populated (monkeypatched count to
avoid wiring the full workspace/peer/collection/document FK chain just
to land one vector row), and idempotency.
* docs: add changing-embeddings operations page
Document the supported way to change EMBEDDING_VECTOR_DIMENSIONS or
EMBEDDING_MODEL_CONFIG__MODEL on a Honcho deployment: provision a new
deployment at the desired configuration, replay source data out of
band, cut over at the application layer.
The page explains the asymmetry:
- Dimension is machine-enforced as immutable. The startup validator
introspects pg_attribute and crashes the API/deriver on mismatch.
- Model is operator-owned. There is no persistent metadata recording
which model produced each vector, so a same-dim model swap is
silently undetectable — flagged with a Warning callout.
Also documents the truncation edge case (text-embedding-3-large truncated to 1536 with EMBEDDING_VECTOR_DIMENSIONS left at default)
and the DIMENSIONS_MODE=always mitigation, plus a pointer that
storage-backend swap (VECTOR_STORE_MIGRATED + reconciler) is a distinct operation unaffected by this work.
Registers the page in docs/docs.json under the Self-Hosting nav group
and cross-links from configuration.mdx.
* fix(embedding): correct turbopuffer regex + tighten DIMENSIONS_MODE docs
- Turbopuffer attribute type for a vector column is `[N]f32` / `[N]f16` /
`[N]i8`, not `f32_vector(N)` as the earlier probe assumed. The earlier
regex returned None for the real SDK format, so existing Turbopuffer
namespaces would have been reported as "missing" instead of validated
for mismatch. Regex switched to `\[(\d+)\]` which is the
vendor-stable shape. Test cases rewritten to lock the actual format.
- docs/v3/contributing/configuration.mdx had a contradictory pair of
bullets: 223 said explicit 1536 makes `auto` forward dimensions=, 224
said `auto` would skip the parameter because 1536 is the default.
Operators reading both would (rightly) conclude they need `always`
even when `auto` would work. Rewrote both bullets so:
- `auto` is provenance-driven (explicit-set, not non-default-value).
- `always` is positioned as defense-in-depth for config layers that
might strip explicit default-valued envs, not the only path for
same-as-default truncation.
* fix(embedding): address PR #678 review comments
CodeRabbit + Rajat review feedback. All actionable items addressed
except two false-positives (responded on PR).
Bug fixes:
- deriver telemetry leak: validator was called outside try/finally so
shutdown_telemetry() did not run on validation failure. Moved inside.
- _emit_report printed "no effect with pgvector" unconditionally,
including from implicit post-apply calls. Added is_report_mode flag;
only print on explicit --report.
- LanceDB and Turbopuffer probes returned None when the namespace
existed but its schema was malformed (no vector field / unparseable
type string), silently bucketing real corruption as "missing"
(lazy-create) and letting it pass the startup validator. Now raise
VectorStoreError with actionable diagnostics; None remains valid only
for "namespace does not exist."
- Startup validator only sampled message namespaces; added a parallel
Collection-row sample so document namespaces are probed too, with the
same dim assertion. Mirrors the --report path.
Hygiene:
- StartupValidationError now subclasses HonchoException so existing
exception handlers recognize it. ValidationException is @final and
has 422 request-validation semantics that would be misleading here.
- scripts/configure_embeddings.py main() no longer spins up two event
loops. engine.dispose() moved into a try/finally inside _async_main
so cleanup runs in the same loop as the pipeline.
- Replaced hand-rolled retry loop with tenacity.AsyncRetrying; same
fail-closed semantics, less code, before_sleep_log for visibility.
- Added _validate_identifier() defense-in-depth: DB.SCHEMA and HNSW
index names are regex-checked against [A-Za-z_][A-Za-z0-9_]* before
SQL interpolation. Operator config + DB catalog are not user input
under the current threat model, but the constraint is cheap to gate.
Test + docs:
- test_app_settings_accepts_non_1536_with_any_vector_store_configuration
now actually exercises turbopuffer (was missing); supplies a dummy
TURBOPUFFER_API_KEY to satisfy the model_validator.
- changing-embeddings.mdx: hyphenated "out-of-band" per reviewer style.
* fix: modify conftest to fix ci
* fix: ci tests for typescript server
* docs(readme): repositioning pass + staleness fixes (P0-P4 audit)
Restructure README to match dual audience (AI-tool users + product
developers) per Vineeth's audit. No content deleted - long internal
sections collapsed under `<details>` for scannability.
Staleness fixes:
- Replace 404'd doc links (.../tutorial/SDK, /api-reference/introduction)
with verified replacements under /v3/documentation/reference/sdk
and /v3/api-reference/introduction
- Fix Python quickstart to pass api_key (managed default api.honcho.dev
would 401 otherwise)
- Drop hardcoded `gpt-4` model reference; read OPENAI_MODEL from env
- Replace archived Dialectic blog link with current Chat Endpoint docs
- Drop M3-Macbook-specific note; minor grammar ("deriver's" -> "derivers")
- Replace TL;DR Python-only example with side-by-side Python + TypeScript
framed around the "Honcho Loop" (store / reason / query / inject)
New sections:
- Start Here: three-path table (AI tools / building product / self-host)
- The Honcho Loop: operation model before code
- What Honcho Gives You: API-at-a-glance table
- Integrations: verified install commands for Claude Code (plugin + raw
MCP), OpenCode, OpenClaw, Hermes
- Honcho vs RAG: stubbed with TODO; copy deferred to marketing
- SDKs section with clearer Python/TypeScript landing pointers
Restructured:
- Core Concepts moved above Architecture; Collections/Documents reframed
as internal mechanism (Conclusions is the public surface)
- Storage / Reasoning / Retrieving deep-dive wrapped in <details>
- Local Development, Pre-commit hooks, Fly deployment, full config
matrix wrapped in <details>
Known follow-up (not in this branch): SDK docs at docs.honcho.dev and
PyPI PKG-INFO advertise `HONCHO_BASE_URL`, but the actual SDK code
(sdks/python/src/honcho/client.py:234, sdks/typescript/src/client.ts:154)
reads `HONCHO_URL`. README aligned with code; docs + PKG-INFO need
separate fix.
* docs(readme): restore "stateful agents" in opening sentence
Plastic Labs' canonical positioning uses "stateful agents" across
materials, and the original README opened with "for building stateful
agents." The repositioning pass in d6d60435 dropped the term entirely
(now zero occurrences) by following Vineeth's suggested opening copy
verbatim - but his audit's executive summary explicitly praised the
"stateful agents" positioning and didn't ask to remove it. Restoring
it in the bolded thesis sentence.
* docs(readme): drop self-referential "observations" in Conclusions bullet
The Conclusions definition shouldn't define itself in terms of
"observations." Per Plastic's positioning, "conclusions" is the
documentation-facing name for what the Deriver produces;
"observations" remains the internal code symbol. The README's
two remaining "observations" references (inside the <details>
Internal storage block and the Storage primitives block) are
explicit code-internal framing and stay.
* docs(readme): restore content dropped without audit instruction
Self-audit against Vineeth's audit found seven items I'd dropped that weren't in the audit's instructions to drop: outcome-marketing line, Contents TOC (audit said rename, not remove), multi-repo prose, org-onboarding detail, peer-paradigm feature bullets, Architecture "Key Features" bullets, and Learn More pointers. Also fixes two residual "Dialectic API" → "Chat Endpoint" mentions the original P0 sweep missed.
* docs(readme): add "Why Honcho" capability table + agent-skill onboarding
Closes the two gaps flagged in the freshness/repositioning audit: adds Vineeth's recommended "Why Honcho" capability table between Start Here and The Honcho Loop, and adds the `npx skills add plastic-labs/honcho` + `/honcho-integration` agent-skill path as a subsection of Integrations (verified against current docs).
* docs: split contributor-only sections out of README; trust auth for local postgres
- Move pre-commit hooks setup from README to CONTRIBUTING.md (pure
contributor content; the README still links to it).
- Move Fly.io deployment notes from README to the self-hosting docs.
- Wrap remaining <details>/<summary> blocks with markdownlint
disable/enable to clear pre-existing MD033/MD001 failures.
- Add POSTGRES_HOST_AUTH_METHOD=trust to the example compose template
with an inline warning, so host-side tests and tooling can connect
without supplying a password.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: (docs) update docs and evals urls and split pre-commit into contributing docs
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: refresh CLAUDE.md to match current architecture
Bring the project CLAUDE.md in line with the codebase as of main:
- API prefix /v1 -> /v3; add Conclusions + Webhooks to route listing;
note that Collections/Documents are now partially exposed via the
Conclusions API
- New Runtime Architecture section describing the API server / deriver
worker split and the in-process Reconciler scheduler
- Rewrite Agent Architecture:
* Deriver - "minimal" single-LLM-call architecture (no agentic tool
loop); current entry point and prompts
* Dialectic - actual DIALECTIC_TOOLS list, 5 reasoning tiers,
DIALECTIC_TOOLS_MINIMAL for the minimal level
* Dreamer - orchestrator + DeductionSpecialist + InductionSpecialist,
surprisal-based prioritization, reasoning trees
* Summarizer documented as a distinct agent
- Refresh project structure tree: add cache/, llm/ (+backends/),
reconciler/, telemetry/, vector_store/; correct schemas/ (now a
directory); drop nonexistent dialectic/agent/, deriver/agent/,
dreamer/agent.py + dreamer.py
- Architectural decisions: fill in the missing #2 (Peer Paradigm); add
hybrid search (FTS + vector), pluggable external vector stores,
composite-FK multi-tenancy, dialectic reasoning tiers
* docs(CLAUDE.md): rename "observations" -> "conclusions" in agent prose
Per Plastic's positioning, "conclusions" is the documentation-facing
term for what the Deriver produces; "observations" remains the
internal code-symbol vocabulary (`create_observations`,
`get_observation_context`, etc.). Swap conceptual prose, preserve
all backticked code references.
- New terminology callout at the top of Agent Architecture so the
mapping is explicit for coding agents reading this file
- Deriver/Dreamer prose: observations -> conclusions for the abstract
noun; specialist tool lists keep their `get_recent_observations`,
`create_observations_deductive`, etc. unchanged
- Reasoning-trees bullet rephrased to "each conclusion links to its
premises and downstream conclusions" (the original "observations
link to premises/conclusions" becomes recursive after the swap;
rephrasing makes intent clearer)
- Tree comment for surprisal.py updated to prose-style "conclusion
prioritization"
Companion to a3fa16ef on kass/readme-refresh, which made the same
swap in the README's Conclusions definition.
* docs(CLAUDE.md): align "Dialectic API" with README's "Chat Endpoint" rename
Two spots framed the public surface as "Dialectic API" — preserve the code-agent name (Dialectic) while matching the documentation-facing "Chat Endpoint" we standardized on in the README.
* chore: nits in CLAUDE.md
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
The previous mutual-exclusion check compared --test-dir against its
default string literal, so passing --test-file together with an
explicit --test-dir tests/unified/test_cases silently bypassed the
check. Replace with argparse.add_mutually_exclusive_group() and apply
the default path post-parse so the bare invocation still works.
Add "Use the Skill" section recommending `npx skills add plastic-labs/vercel-ai-sdk`
with the manual symlink approach as a collapsed alternative.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* docs(integrations): add @honcho-ai/vercel-ai-sdk guide
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(integrations): rewrite Vercel AI SDK guide as cookbook style (DEV-1485)
Reshapes the guide to cookbook formula, adds Full Script section, fixes
maxSteps → stopWhen for ai-sdk v5, renames package, and prunes stale notes.
See PR for full decision log.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(integrations): lead Vercel AI SDK verification with direct-inspection check
- Restructure Verifying section: direct inspection (token delta + dashboard) is now step 1 so readers isolate Honcho's contribution before grading model behavior
- Behavioral tests (first turn, multi-turn, cross-session, tool calling) follow as steps 2-5
- Note `result.toolCalls` as the way to confirm which Honcho tool fired (tool names don't appear in `result.text`)
- Signpost the Full Script from Complete Example so the two snippets read as a staircase, not a duplicate
Addresses review comments on PR #635.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(tests): satisfy basedpyright in test_representation_manager
The save-representation tests added in #615 were structurally correct but
failed strict typing in two places. Static Analysis has been red on main
since the merge.
- `mock_save.await_args` is `_Call | None`; assert it's not None before
reading `.kwargs` / `.args` so basedpyright can narrow the type
- `SimpleNamespace(...)` passed as `message_level_configuration` is an
intentional duck-typed mock (only `.dream.enabled` is read by
`save_representation`), so opt out at the call site with
`# pyright: ignore[reportArgumentType]` rather than constructing a
full `ResolvedConfiguration` (matches the existing `reportPrivateUsage`
ignore pattern in this file)
No runtime behavior changes; `uv run basedpyright` is now clean
project-wide.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(tests): pad timestamp windows in test_messages for clock skew
Three timestamp tests captured `before_request` / `after_request` with
`datetime.now(UTC)` on the host and asserted the server's `created_at`
fell within. Under Docker, the Postgres container's clock can skew tens
of ms from the macOS host, flipping the assertion intermittently under
parallel pytest load.
Pad each window by 1 second on both sides — wide enough to absorb
realistic skew, narrow enough that the test still proves the timestamp
is server-current.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(integrations): tighten Verifying section after end-to-end smoke
Smoke-tested all five verification steps against a fresh Sonnet 4.6 + Honcho integration. Three findings, all reflected here:
- Cross-session recall (#4): added Note about DERIVER_REPRESENTATION_BATCH_MAX_TOKENS=1024 — short warmups don't accumulate enough content to flush observations, so cross-session recall returns empty even on a working integration.
- Tool calling prompt (#5): replaced the honcho_chat patterns prompt with a verbatim-retrieval honcho_search prompt. Sonnet skips honcho_chat when middleware-injected context already answers; verbatim retrieval forces a fire.
- Tool inspection (#5): replaced result.toolCalls reference with result.steps[i].toolCalls + flatMap snippet. Top-level toolCalls is empty in multi-step calls (stopWhen: stepCountIs(N)) — the fires are nested inside steps.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(integrations): make Step 4 cross-session test durable via honcho_search
Replace the prose-recall test ("Based on what we've talked about, what do you know about me?") with a forced honcho_search call. Prose recall depended on the model getting deriver-built representation/peer-card in its system prompt, which is gated behind DERIVER_REPRESENTATION_BATCH_MAX_TOKENS=1024 — short tutorial-length conversations don't trigger it, producing false negatives on a working integration.
honcho_search hits message embeddings, which are computed synchronously at message persist time (src/crud/message.py:262-276), so peer-scoped retrieval works regardless of how short the prior session was. Also folds the result.steps[i].toolCalls inspection snippet from the old Step 5 into Step 4 — same prompt, no need for two sections.
Drops Step 5 entirely.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* fix(dreamer): threshold and time-guard semantics
Finding 2: filter count_stmt on documents.level == 'explicit' in
check_and_schedule_dream. Dreamer-created levels (deductive, inductive,
contradiction) are consolidation output, not input, and would otherwise
inflate the threshold count and create a feedback loop.
Finding 3 (code-level): relocate last_dream_at write from enqueue_dream
(enqueue.py) to process_dream (orchestrator.py), inside the
'if result is not None' block. Duplicate enqueues can no longer reset
the 8-hour time guard clock. Failed/never-run dreams don't advance it.
Success criteria: lenient (any non-null DreamResult counts). Pending
Vineeth confirmation — will adjust to strict/middle if requested.
Tests pending in follow-up commits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(dreamer): threshold filter + last_dream_at relocation regression tests
Tests for Finding 2 and Finding 3 (code-level):
- TestThresholdFilter (tests/dreamer/test_dream_scheduler.py):
* Mixed levels below explicit threshold: 30 explicit + 40 deductive
+ 10 inductive → no trigger (core regression, buggy count would trigger)
* Explicit-only at threshold: 60 explicit → triggers
* Contradiction excluded: 100 contradiction + 10 explicit → no trigger
(confirms positive == "explicit" filter excludes all dreamer output)
- TestEnqueueDreamMetadataShape (tests/deriver/test_enqueue_dream.py):
* AsyncMock-patched update_collection_internal_metadata verifies
enqueue writes last_dream_document_count but NOT last_dream_at
- TestLastDreamAtCompletionWrite (tests/dreamer/test_dreamer_integration.py):
* Happy path: run_dream returns DreamResult → last_dream_at written
* Failure path: run_dream returns None → last_dream_at absent
* Exception path: run_dream raises → last_dream_at absent,
process_dream swallows exception (queue-processed semantics preserved)
Docstring on check_and_schedule_dream tightened: "document threshold"
-> "explicit-observation threshold" to reflect filter semantics.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(dreamer): preserve last_dream_document_count in completion write
CodeRabbit caught this: update_collection_internal_metadata uses a
top-level JSONB `||` merge, so passing {"dream": {"last_dream_at": ...}}
replaces the entire "dream" subkey and drops last_dream_document_count
that was written by enqueue_dream.
Symptom: after every completed dream, the baseline drops to 0. Next
check_and_schedule_dream reads documents_since_last_dream as
current_count - 0 = current_count, so any collection with >= 50
explicit observations can re-trigger immediately once the 8h guard
expires, even with no new raw material.
Fix: read-modify-write. Fetch current collection, merge last_dream_at
into the existing "dream" dict, write the merged dict back. Preserves
sibling keys (current: last_dream_document_count; future-proof for
telemetry fields that might land in PR 4).
Regression test added to tests/dreamer/test_dreamer_integration.py:
pre-seeds {"dream": {"last_dream_document_count": 42}}, runs
process_dream, asserts both last_dream_at is written AND
last_dream_document_count == 42 is preserved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(dreamer): address CodeRabbit feedback on b89997c
- enqueue.py: read-modify-write preserves last_dream_at when writing baseline
- dream_scheduler.py: explicit-level filter on execute_dream count query
- test fixture: pin DOCUMENT_THRESHOLD and ENABLED_TYPES for stability
- integration test: timezone-aware assertion on last_dream_at
Regression test added for enqueue sibling-drop (symmetric to c8fe40a).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(dreamer): session lookup symmetry + row lock on dream metadata RMW
- dream_scheduler.py: explicit-level filter on execute_dream session lookup
(baseline and session pick must agree on the same document set)
- crud.collection.get_collection: optional with_for_update flag for callers
that need serialized read-modify-write on internal_metadata
- enqueue.py + orchestrator.py: pass with_for_update=True on the RMW reads
to close the TOCTOU between concurrent enqueue and completion writes
Follow-up filed for jsonb_set-based nested updates (docs/factory/backlog/).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(dreamer): explicit-only count on manual schedule_dream route
The third caller of enqueue_dream — POST /workspaces/{id}/schedule_dream —
was passing an all-levels document count as the baseline, breaking symmetry
with check_and_schedule_dream and execute_dream after Loop 2's filter fixes.
Filter the manual route's count to match.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(dreamer): document explicit-only invariant on enqueue_dream.document_count
Loop 3 follow-up on d76627a. The parameter's semantic tightened across Loop
2 (check_and_schedule_dream, execute_dream) and Loop 3 (schedule_dream route)
to "explicit-level count, used as the baseline," but the signature still read
"Current document count for metadata update." The next caller would have no
way to know from the function contract.
Docstring now spells out: (1) the value is explicit-only, (2) it's written
as last_dream_document_count, (3) it's the baseline that
check_and_schedule_dream subtracts from to compute
documents_since_last_dream, (4) passing a count that includes non-explicit
levels (deductive, inductive, contradiction) inflates the baseline and
suppresses the next scheduled dream.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(dreamer): rename current_document_count → current_explicit_count
Loop 3 follow-up on d4e10e3. After Loop 2's filter landed, the local in
check_and_schedule_dream held an explicit-only count but was still named
current_document_count — asymmetric with execute_dream's current_explicit_count
(line 201) and contradicting the filter on line 269 that produces the value.
Pure rename: three occurrences (definition at 271, subtraction at 274, log
extra key at 282). No test references. Naming-as-invariant alignment with
d76627a (query filters), d4e10e3 (parameter docstring), and Loop 1's local
rename in execute_dream.
The persisted JSONB key last_dream_document_count is the one remaining
drift-layer; filed as plastic-claudebook backlog item for a separate PR
with an intentional migration path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(dreamer): atomic guard-pair write + in-flight stampede defense
Loop 4 response to Vineeth's CHANGES_REQUESTED on PR #573.
The pre-Loop-4 enqueue-time write of last_dream_document_count was serving
double duty: rate limiter AND stampede latch. By arming the 8h guard the
moment a dream entered the pipeline, it implicitly blocked a second dream
from being scheduled during the in-flight window. Loop 3 relocated the
last_dream_at write to completion without moving its sibling baseline,
splitting the semantic pair and exposing the latch role that had lived
only in Vineeth's head.
Invariant (now pinned to check_and_schedule_dream's docstring): from the
moment a dream is scheduled until it completes or fails, no second dream
may be enqueued for the same (workspace, observer, observed) — and the
baseline count advances only when consolidation actually happened.
Changes:
- enqueue_dream: remove the last_dream_document_count write entirely and
drop the document_count parameter. enqueue no longer touches dream
metadata; the implicit stampede latch is replaced by an explicit
queue-backed defense.
- process_dream: extend the existing row-locked RMW to write both guard
fields atomically. Current explicit-doc count is recomputed inside the
locked block (not carried on DreamPayload) so the pair reflects the
actual consolidation moment.
- check_and_schedule_dream: query QueueItem for pending dreams on this
collection's work_unit_keys (mirrors uq_queue_dream_pending_work_unit_key)
before arming a timer. Uses queue state as source of truth rather than
reflecting it into metadata.
- Tests: two new coherence tests under TestGuardPairCoherence —
test_pending_queue_item_blocks_second_schedule walks the stampede timeline,
test_silent_failure_allows_retry_on_same_corpus verifies failed dreams
don't consume the baseline. Existing tests updated to the new contract.
* chore(dreamer): trim comment slop from loop-4 atomic pair work
Compress three verbose comments added in d24958d — the invariant itself
is captured in check_and_schedule_dream's docstring, so the inline
narrative restates what the code already says.
- dream_scheduler.py defense C block: 5 lines → 2
- orchestrator.py atomic pair write: 4 lines → 1
- enqueue.py docstring paragraph: 5 lines → 2
Net: +5/-14. Follows Eri's eef27be precedent on sillytavern-honcho PR #7.
---------
Co-authored-by: lilyplasticlabs <lily@plasticlabs.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(deriver): ignore blank observations before embedding
* Address PR review on observation normalization
* Harden mock await arg access in tests
* Unify blank observation filtering across tool paths
* Move soft-delete query test back to fixture class
The Surprisal module passes `{"level": levels}` directly to
`get_all_documents()`, but `apply_filter()` expects operator syntax:
`{"level": {"in": levels}}`.
Without the `in` operator, the filter is silently ignored, causing
`_fetch_level_observations()` to return 0 results. This makes the
entire Surprisal phase of the Dream cycle a no-op.
Fixes#559
- New Group Chats subsection: documents per-character peer routing
(each group member gets their own peer, not a collapsed group-<id>
peer) and lazy peer registration for characters joining mid-chat.
- Session Naming: documents the freeze-on-first-assign invariant
(changing the naming mode doesn't reroute existing chats) and
the Reset button for explicit session rollover.
- Tool table: add honcho_save_conclusion — prior fix undercounted
(2 -> 3 tools). The extension registers all three.
The plugin also writes to a root-level `sessions` map (ST dir → last
Honcho session ID), not only to `hosts.sillytavern.*`. The earlier
phrasing overstated the isolation claim.
- Add Prerequisites section with SillyTavern install link + Node >= 18
requirement (was buried in Next Steps; users hit install step with no
awareness ST needed to exist first).
- Expand restart step into a callout: restart required for server-plugin
reload, not for client-side edits.
- Configure step now documents the three editable inputs (API key,
Workspace ID, Your peer name) and where each saves.
- Fix 'three-cubes icon' -> 'puzzle piece icon'.
- Installer step list fleshed out: 6 steps (was 4), including config.yaml
bootstrap and enableServerPlugins flip. Dropped the false claim that
the plugin seeds a minimal ~/.honcho/config.json on first run.
- Global Config section rewritten: resolution order now generalized to
apiKey / workspace / peerName (was apiKey-only); documents panel
write-back to hosts.sillytavern.*; dropped aiPeer references (it's a
telemetry-only field, not user-facing).
- Add a Disable / Enable global config subsection covering the opt-out
toggle and the Inherit / Push local / Cancel diff dialog.
- Troubleshooting: two new rows (stale peer name on new chat, cancelled
diff dialog).
- Clarify installer step 4 — the plugin seeds config.json if absent
- 'Puzzle piece' -> 'three-cubes' for the Extensions icon (current ST UI)
- API key step notes the UI-overrides-config precedence explicitly
- 'Honcho workspace ID' -> 'default Honcho workspace ID (configurable)'
- Add Note after Context-modes table — Context only is session-scoped
and returns empty until enough messages accumulate; Reasoning is the
better default for fresh peers
- Next Steps gains two cards: Install SillyTavern (upstream docs) and
the Claude Code setup skill (skills/setup/SKILL.md)
Follow-ups tracked separately — tool rename (observation -> conclusion,
matching the /conclusion endpoint), architecture Excalidraw.
The MCP Worker hardcoded https://api.honcho.dev for every request, forcing
anyone running a self-hosted Honcho instance to patch the source before
deploying their own Worker alongside it.
Route the baseUrl through the Worker env so operators can set
HONCHO_API_URL (via .dev.vars for local development or wrangler secret for
deployed Workers) and point the Worker at their instance. The variable is
intentionally not exposed as a request header: that would let public
clients steer traffic to internal URLs, which is a latency and security
regression.
When HONCHO_API_URL is unset, the Worker falls back to
https://api.honcho.dev, so existing deployments are unaffected.
Closes#508
* feat: adding honcho-cli package
* feat: adding more support for command-level flags, also including workarounds for getting raw SDK info
* feat: adding peer config
* feat: adding setup commands
* chore: setting up package dependencies for cli
* feat: promote init/doctor to top-level + polish wizard
* feat: make init --yes fall back to existing config
* chore: updating documentation
* chore: updating tagline
* feat: structurally updating recomended settings for CLI
* fix: style
* fix: removing redundant describe method
* fix: delete key generation commands and fixing session ID
* fix: removing defaults and changing config write path.
* chore: pagnating conclusions
* chore: require workspace
* fix: polish command surfaces — scoping, validation, perf, consistency
* chore: removing session message
* fix: CLI output shape, destructive-confirm previews, skip needless round-trips
* chore: CLI polish — peer inspect config, drop dead helper, doc/help consistency
* chore: update readme
* chore: updating tests
* chore: doc updates
* fix: config command
* chore: unused code
* fix: doctor command
* fix: removing quiet tag and fixing session key ordering
* fix: config commands and session id command
* fix: removing message_count
* fix: branding circular dependency
* fix: refactor lazy imports to use common.py correctly.
* fix: removing all lazy imports
* chore: cr fixes
* fix: config, env, flag setup
* chore: updating skill
* feat: adding workspace, session, and message create
* fix: init now supports local honcho
* chore: cr
* feat(cli): CLI surface polish — reasoning flag, peer-scoped messages, help sync
Add --reasoning/-r to peer chat (minimal..max), -p peer filter to
message list with newest-first ordering, and a curated welcome panel
with getting-started/memory/commands sections.
Sync the welcome panel and group help strings with the actual
registered commands — drop phantom 'session clone', add the 4 missing
peer commands and 7 missing session commands, fix conclusion/message/
workspace group docstrings that claimed commands that don't exist.
* feat(cli): themed, unified help system with pattern/example
Replace the hand-rolled welcome with a layered system:
- Theme typer.rich_utils (dim borders, brand color) so every --help
inherits the voice.
- HonchoTyperGroup subclass renders a curated 3-panel welcome
(getting started / memory / commands) with recipes Typer can't
auto-generate.
- Unify the front door: bare 'honcho', 'honcho --help', and
'honcho help' all render the same welcome via one code path;
sub-groups and leaf commands still get Typer's themed renderer.
- Replace Click's 'Usage: …' line with pattern/example rows at every
sub-group and leaf command, so the help voice stays consistent from
top to leaves.
* refactor(cli): address review — typed exceptions, chmod 600, tighter redaction, class-based help, tests
- Replace module-level monkey-patch of TyperGroup/TyperCommand.get_usage
with HonchoTyperGroup applied via cls= on every sub-Typer. Lives in
a new _help.py module to avoid circular imports. No longer leaks
behavior changes into other Typer users in the same process.
- _test_connection dispatches on the SDK's typed exceptions
(AuthenticationError, ConnectionError, TimeoutError, APIError)
instead of substring-matching error messages.
- Config.save() now chmods ~/.honcho/config.json to 0o600 after write
so the plaintext API key isn't world-readable on multi-user hosts.
- Tighten api_key redaction to '***<last4>' (was 'header...last4'),
matching setup._redact for consistency. Short keys fully masked.
- Add test_validation.py covering safe IDs, unsafe chars, path
traversal, and empty input. Update test_config.py redaction cases
and add 0o600 permission assertion. Fix stale patch paths in
test_commands.py that pointed at honcho_cli.main instead of the
command modules where get_client is actually imported.
* feat(cli): add options panel to welcome menu
Append a fourth panel listing the global flags (-w/-p/-s, --json,
--version, --help) with their env-var counterparts. Discoverable
from bare 'honcho' without needing to hunt for --help.
* chore(cli): drop --version from welcome options panel
* feat(cli): add pixel-honcho icon to banner
Prepend a 13-char ASCII rendering of honcho-pixel.svg to the HONCHO
wordmark. Uses Unicode half-blocks to pack 12 pixel rows into 6 text
rows, faithfully preserving the SVG outline (two eye dots, mouth slit,
tapering foot). Appears in bare 'honcho', 'honcho --help', 'honcho
--version', and 'honcho init'.
* fix: polish Honcho CLI wolcome panel and error messages
* fix: honcho workspace inspect speed
* chore: minor fix to session pagination
* fix: removing NDJSON output
* chore: consolidating honcho CLI's dula argv grammar onto Pattern A (command-first)
* chore: clean up imports
* fix: four `-s` consistency fixes applied
* chore: minor changes to memory rows
* fix: changing package name to honcho-cli
* fix: removing pixel face
---------
Co-authored-by: Erosika <eri@plasticlabs.ai>
* fix: Add JSON repair for truncated LLM responses across all providers and Gemini thinking budget support
LengthFinishReasonError from OpenAI-compatible providers (custom, openai, groq) was crashing the deriver
with 14k+ occurrences in production. The vLLM path already had repair logic but it was gated on
provider=="vllm", unreachable when routing through litellm as a custom provider.
- Extract shared _repair_response_model_json() helper for all providers
- Catch LengthFinishReasonError in OpenAI/custom parse() path and repair truncated JSON
- Add repair fallback to Anthropic and Gemini response_model paths
- Add repair fallback to Groq response_model path
- Pass thinking_budget_tokens to Gemini 2.5 models via thinking_config
- Add 14 tests covering repair paths for all providers and Gemini thinking budget
Fixes HONCHO-YC
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: live llm integration tests
* feat: Consistent Model Config Protocol
* fix: migrate the remaining app callers off the legacy llm_settings path
* fix: Docs and regression tests
* fix: refactor llm runtime path to model-config-only API
* fix: refactor config to nested model-config source of truth
* fix: refactor llm streaming and tool dispatch through backends
* fix: cut over llm config to nested model_config only
* fix: collapse vllm and custom into openai_compatible transport
* feat: refactor llm config to explicit transports and bare model ids
* feat: (embed) Add configurability for embedding model
* fix: tests for embedding provider
* fix: Address Review Comments
* fix: (llm) remove Groq backend and per-vendor base URLs
* chore: move llm tests
* fix: (llm) address review findings — config regressions, backend bugs, dead code
* fix: address backend end silly errors
* chore: (docs) update configuration and self-hosting guides
* chore: fix tests
* fix: address code rabbit comments
* fix: add validation to the dream settings
* fix: further address code rabbit comments
* fix: Address Code Rabbit Comments
* fix: Another round of code rabbit
* fix: Address Code Rabbit Nits
* fix: tests
* refactor: rename thinking validator to reflect transport scope
_validate_anthropic_thinking_minimum only enforces the >=1024 rule for
Anthropic and no-ops for other transports, so the name was misleading
now that it's shared across ConfiguredModelSettings, FallbackModelSettings,
and ModelConfig. Renamed to _validate_thinking_constraints with a docstring
clarifying per-transport behavior. No logic change.
* fix(config): drop transport-specific thinking params when env override changes transport
_fill_defaults_for_nested_field previously preserved the default MODEL_CONFIG's
thinking_budget_tokens/thinking_effort across a transport override. This leaked
Gemini-family defaults (e.g. thinking_budget_tokens=1024) into OpenAI-transport
overrides, and the OpenAI backend then correctly rejected the unsupported param
at call time (OpenAI uses reasoning.effort, not a token budget).
The helper now strips thinking_budget_tokens and thinking_effort from the
default dict when the env override supplies a transport different from the
default's. Explicit thinking params in the override are preserved.
* fix(config): apply thinking-param strip to dialectic level merge too
DialecticSettings._merge_level_defaults does its own inline MODEL_CONFIG
merge (parallel to _fill_defaults_for_nested_field), so the previous fix
missed dialectic-level overrides. E.g. flipping
DIALECTIC_LEVELS__minimal__MODEL_CONFIG__TRANSPORT from gemini (default)
to openai still leaked the default thinking_budget_tokens=0 into the
openai config, which the OpenAI backend then rejected at call time.
The level-merge path now applies the same 'strip transport-specific
thinking params when transport changes' rule as the generic helper.
Added a regression test exercising the merge validator directly.
* refactor(llm): wire ModelConfig knobs through, prune clients.py migration leftovers
Three connected fixes to finish carving the LLM stack out of src/utils/clients.py
and into src/llm/:
1. Propagate ModelConfig tuning knobs into backend calls.
honcho_llm_call_inner built extra_params from only {json_mode, verbosity},
silently dropping top_p, top_k, frequency_penalty, presence_penalty, seed,
and operator-supplied provider_params from any ModelConfig. Thread the
selected config through ProviderSelection and merge
build_config_extra_params(selected_config) into extra_params; per-call
kwargs still win over provider_params defaults. Makes
_build_config_extra_params public as build_config_extra_params so
clients.py and request_builder.py share one translation. Adds
TestModelConfigExtraParamsPropagation covering OpenAI/Anthropic knob
propagation, provider_params passthrough, and per-call override
precedence.
2. Drop dead extract_openai_* duplicates in clients.py.
extract_openai_reasoning_content, extract_openai_reasoning_details, and
extract_openai_cache_tokens had no callers outside their own definitions
— the live implementations live in src/llm/backends/openai.py. -103
lines from clients.py.
3. Unify on ModelTransport, delete SupportedProviders.
The "google" vs "gemini" split forced a _provider_for_model_config
translation shim in two places. Replace all SupportedProviders usages
with ModelTransport, rename CLIENTS["google"] → CLIENTS["gemini"],
update provider branches + LLMError labels + reasoning-trace entries
accordingly. Trace JSONL now writes "provider": "gemini" instead of
"google" — consistent with the broader env-var rename cutover.
Also tidies up pre-existing basedpyright findings in tests/llm/test_model_config.py
(pydantic before-validator dict inputs + descriptor-proxy call).
ruff: clean. basedpyright: 0 errors, 0 warnings. Tests: 153/153 pass across
tests/utils/test_clients.py, tests/utils/test_length_finish_reason.py,
tests/llm/, tests/dialectic/, tests/deriver/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(llm): finish the src/utils/clients.py → src/llm/ migration
honcho_llm_call_inner now delegates to request_builder.execute_completion
and execute_stream instead of re-implementing backend call scaffolding
inline. The new _effective_config_for_call helper carries per-call kwargs
(temperature, stop_seqs, thinking_budget_tokens, reasoning_effort) onto
the selected ModelConfig — or synthesizes a minimal config for the
test-only callers that pass provider+model directly. max_output_tokens
is zeroed on the effective config to preserve the current
"per-call max_tokens wins" semantic; honoring ModelConfig.max_output_tokens
is a separable correctness concern.
Side effect of routing through the new path: ConfiguredModelSettings'
thinking_budget_tokens validator now fires on synthesized configs.
test_anthropic_thinking_budget was asserting that a sub-1024 budget
propagated to Anthropic — bumped to 1024 to match what Anthropic actually
accepts.
Unified client construction. Promoted the cached client factories in
src/llm/__init__.py (get_anthropic_client, get_openai_client,
get_gemini_client, get_{anthropic,openai,gemini}_override_client) to
public API and added them to __all__. Promoted
credentials._default_transport_api_key → default_transport_api_key.
Deleted the duplicate _build_client and _default_credentials_for_provider
from clients.py; _client_for_model_config now falls through to the
public factories. CLIENTS dict and _get_backend_for_provider stay as the
mockable seam for the ~50 patch.dict(CLIENTS, {...}) test call sites.
Wired operator-configurable Gemini cached-content reuse end-to-end.
PromptCachePolicy moved from src/llm/caching.py into src/config.py so
ModelConfig can reference it as a field without a circular import;
caching.py re-exports the name for existing imports. Added
cache_policy: PromptCachePolicy | None on ConfiguredModelSettings,
FallbackModelSettings, ResolvedFallbackConfig, and ModelConfig.
resolve_model_config, _resolve_fallback_config, and
_select_model_config_for_attempt copy the field through.
honcho_llm_call_inner passes effective_config.cache_policy into
execute_completion / execute_stream, so operators opt in via
e.g. DERIVER_MODEL_CONFIG__CACHE_POLICY__MODE=gemini_cached_content
and the selection actually fires instead of sitting on a dead path.
New regression test test_cache_policy_reaches_gemini_backend asserts the
PromptCachePolicy object reaches the Gemini backend's extra_params.
ruff + basedpyright: clean. Tests: 154/154 pass across
tests/utils/test_clients.py, tests/utils/test_length_finish_reason.py,
tests/llm/, tests/dialectic/, tests/deriver/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(llm): move all LLM orchestration into src/llm/ and delete clients.py
The 1624-line src/utils/clients.py has been carved up into focused modules
under src/llm/ and deleted. There is now one golden path for LLM
orchestration and no dual entrypoint.
New module layout:
src/llm/
__init__.py thin stable re-export surface
api.py public honcho_llm_call with retry + fallback + tool
loop delegation
executor.py honcho_llm_call_inner (single-call executor); bridges
to request_builder.execute_completion / execute_stream
tool_loop.py execute_tool_loop + stream_final_response, plus
assistant-tool-message and tool-result formatting
runtime.py AttemptPlan dataclass (replaces the loose
ProviderSelection NamedTuple), effective_config_for_call,
plan_attempt, per-retry temperature bump, attempt
ContextVar
registry.py single owner of CLIENTS dict + cached default and
override SDK-client factories + backend/history-adapter
selection + high-level get_backend(config)
conversation.py count_message_tokens, tool-aware message grouping,
truncate_messages_to_fit
types.py HonchoLLMCallResponse, HonchoLLMCallStreamChunk,
StreamingResponseWithMetadata, IterationData,
IterationCallback, ReasoningEffortType, VerbosityType,
ProviderClient
request_builder.py low-level request assembly (ModelConfig → backend
complete/stream); no longer owns credential resolution
credentials.py default_transport_api_key, resolve_credentials
caching.py gemini_cache_store; re-exports PromptCachePolicy
from src.config
backend.py Protocol + normalized result types
history_adapters.py provider-specific assistant/tool message shapes
structured_output.py
backends/ AnthropicBackend, OpenAIBackend, GeminiBackend
handle_streaming_response had no production callers; it is deleted. The
three tests that used it now drive honcho_llm_call_inner(stream=True,
client_override=...) directly, which exercises the same code path the
public API uses.
Dead credential passthrough removed. The ProviderBackend Protocol and
all three concrete backends no longer accept api_key / api_base — those
are baked into the underlying SDK client at registry construction time
and were being del'd everywhere they appeared. request_builder also
stops resolving and forwarding them.
Client construction is unified. The cached default-client factories
(get_anthropic_client, get_openai_client, get_gemini_client) and override
factories (get_*_override_client) are promoted to public API; the
module-level CLIENTS dict populates from them and remains the
patch.dict(CLIENTS, {...}) mocking seam tests rely on. Old duplicate
helpers (_build_client, _default_credentials_for_provider) are gone.
default_transport_api_key is promoted to public.
Application imports now come from src.llm (dreamer, dialectic, deriver,
summarizer, telemetry-adjacent tests). No code imports from
src.utils.clients anywhere in the repo.
ruff: clean. basedpyright: 0 errors, 0 warnings. Tests: 1013/1013 pass
across the entire non-infra test suite (excluding tests/unified,
tests/bench, tests/live_llm, tests/alembic).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(llm): sanitize tool schemas for Gemini's function_declarations validator
Gemini's native-transport function-declarations validator accepts a narrow
subset of JSON-Schema / OpenAPI: type, format, description, nullable, enum,
properties, required, items, minItems, maxItems, minimum, maximum, title.
Anything else — additionalProperties, allOf, if/then/else, $ref, anyOf,
oneOf, $defs, patternProperties — triggers an INVALID_ARGUMENT 400 at call
time.
Our agent tool schemas in src/utils/agent_tools.py use several of those
(additionalProperties: false, allOf + if/then conditionals) because they
were authored for OpenAI strict-mode + Anthropic, which need the richer
vocabulary. GeminiBackend._convert_tools was passing them straight through.
Add _sanitize_schema(): walks the parameters tree and drops unsupported
keywords while preserving semantics for the keywords that hold user data
(properties maps field-name → sub-schema; required / enum are lists of
literals; items is a single sub-schema). Other backends are untouched and
continue to receive the full strict schemas.
Regression tests:
- test_gemini_sanitize_schema_strips_unsupported_keywords: confirms
additionalProperties, allOf + if/then, and $defs are stripped at nested
levels while legitimate fields survive.
- test_gemini_convert_tools_sanitizes_parameters_schema: end-to-end
_convert_tools output has no forbidden keys.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: fix tool calling syntax for gemini
* refactor(llm): normalize defaults, widen OpenAI reasoning-model routing
* chore: fix test
* fix(llm): address post-migration review feedback
* fix(llm): gemini robustness + dreamer specialist ergonomics
* chore: addres review comments
* chore: (docs) unrelease changelog addition
* chore: (docs) merge commit changes
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Erosika <eri@plasticlabs.ai>
The HEALTHCHECK directive probes an HTTP endpoint that only the API
serves. The deriver service reuses this image but is a background queue
worker with no HTTP server — the probe can never succeed, so Docker
permanently marks the deriver container as unhealthy.
Remove the HEALTHCHECK from the shared image. Service-level health
checks belong in each service's own configuration (e.g. Kubernetes
readiness/liveness probes on the API Deployment only).
Closes#521
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: further remove extraneous transactions
* fix: (search) use 2 phase function to reduce un-needed transaction
* fix: refactor agent search to perform external operations before making a transaction
* fix: reduce scope of queue manager transaction
* fix: (bench) add concurrency to test bench
* fix: address review findings for search dedup, webhook idempotency, and bench throttling
* Fix Leakage in non-session-scoped chat call (#526)
* fix: (search) reduce scope for peer based searches
* fix: tests
* fix: (test) address coderabbit comment
* fix: drop db param from deliver_webhook
---------
Co-authored-by: Rajat Ahuja <rahuja445@gmail.com>
* fix: Inconsistencies in Docs, health endpoint, troubleshooting guide
* fix: (docs) maintain consistency on postgres db name
* chore: (docs) update v2 contributing docs with updates db paths
* docs: overhaul self-hosting docs for provider-agnostic setup
- .env.template: lead with provider options (custom, vllm, google,
anthropic, openai, groq) instead of baking in vendor-specific keys.
All provider/model settings commented out so server fails fast until
configured. Separate endpoint config from per-feature provider+model
from tuning knobs.
- docker-compose.yml.example: fix healthcheck -d honcho -> -d postgres
to match POSTGRES_DB=postgres.
- config.toml.example: reorder and document LLM key section with
OpenRouter and vLLM examples.
- self-hosting.mdx: replace multi-vendor key table with provider options
table. Add examples for OpenRouter, vLLM/Ollama, and direct vendor
keys. Remove duplicated key lists from Docker/manual setup sections.
- configuration.mdx: replace scattered provider docs with provider types
table. Fix Docker Compose snippet to match actual compose file. Note
code defaults as fallback, not recommended path.
- troubleshooting.mdx: add alternative provider issues section (custom
provider config, model name format, Docker localhost, structured
output failures).
* docs: add Docker build troubleshooting for permission errors
- Document BuildKit requirement (RUN --mount syntax)
- AppArmor/SELinux blocking Docker builds on Linux
- Volume mount UID mismatch between host and container app user
- Note in self-hosting docs that Docker path builds from source
* docs: reframe self-hosting as contributor/dev path, point to cloud service
* Revert "docs: reframe self-hosting as contributor/dev path, point to cloud service"
This reverts commit 3e766eb1a9.
* docs: add production compose, model guidance, thinking budget docs
- Add docker-compose.prod.yml for VM/server deployment: no source
mounts, restart policies, 127.0.0.1-bound ports, cache enabled
- Add model tier guidance and community quick-start link to self-hosting
- Document THINKING_BUDGET_TOKENS gotcha for non-Anthropic providers
- Add reverse proxy examples (Caddy + nginx) to production section
- Add backup/restore commands to production considerations
* docs: simplify self-hosting to single provider, restructure config guide
Self-hosting page now defaults to one OpenAI-compatible endpoint
with one model for all features. Moved model tiers, alternative
providers, and per-feature tuning into the configuration guide.
Eliminated duplicate config priority sections, dev/prod split,
and redundant TOML examples.
* docs: merge compose files, restore provider/model to feature sections in .env.template
Single docker-compose.yml.example with dev sections commented out.
Moved PROVIDER and MODEL back alongside each feature in .env.template
so settings stay colocated with their module. Updated self-hosting
docs to reference single compose file.
* fix: broken anchor links, redundant migration step, minor inconsistencies
Fix 4 broken internal links (#llm-provider-setup, #llm-api-keys,
#which-api-keys-do-i-need, #alternative-providers) to point to
correct headings. Remove redundant Docker migration step (entrypoint
already runs alembic). Fix cache URL missing ?suppress=true in
reference config. Fix uv install command to use official method.
* docs: env template ready to use, simplify self-hosting flow
.env.template now has provider/model lines uncommented with
placeholder values — user just sets endpoint, key, and model name.
Thinking budgets default to 0 for non-Anthropic providers.
Self-hosting page: removed 30-line env var wall, LLM setup now
points to the template. Merged duplicate verify sections.
Removed api_key from SDK examples (auth off by default).
* docs: reorder next steps, configuration guide first
* fix: default embedding provider to openrouter for single-endpoint setup
Without this, embeddings default to openai which requires a separate
LLM_OPENAI_API_KEY. Setting to openrouter routes embeddings through
the same OpenAI-compatible endpoint as everything else.
* fix: review issues — hermes page, thinking budget, production wording
Hermes integration page: replaced inline Docker/manual setup with
link to self-hosting guide, added elkimek community link. Removed
old env var names (OPENAI_API_KEY without LLM_ prefix).
Troubleshooting: removed "or 1" from thinking budget guidance.
Self-hosting: softened "production-ready" to "production-oriented"
since auth is disabled by default.
* docs: model examples in template, expanded LLM setup, better verify flow
.env.template: added "e.g. google/gemini-2.5-flash" hints next to
model placeholders so users know the expected format.
Self-hosting: expanded LLM Setup to show the 3 things users need to
set (endpoint, key, model name) with find-replace tip. Added build
time note, deriver log check, and real smoke test (create workspace)
to verify section. Health check now notes it doesn't verify DB/LLM.
* fix: smoke test uses v3 API path, not v1
* docs: clarify deriver metrics port vs Prometheus host port
* fix: remove deprecated memoryMode from hermes config example
* docs: update hermes page to match current memory provider config
Updated config to match hermes-agent docs: removed apiKey (not needed
for self-hosted), added hermes memory setup CLI command, added config
fields table (recallMode, writeFrequency, sessionStrategy, etc.).
Better verification tests: store-and-recall across sessions, direct
tool calling test. Links to upstream hermes docs for full field list.
* fix: invalid THINKING_BUDGET_TOKENS=0 and missing docker/ in image
Comment out THINKING_BUDGET_TOKENS=0 in .env.template — deriver,
summary, and dream validators require gt=0. Dialectic levels also
commented out since non-thinking models don't need the override.
Add COPY for docker/ directory in Dockerfile so entrypoint.sh is
available when docker-compose.yml.example references it.
* chore: Additional troubleshooting step
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
- Fix tool names to match actual plugin code (honcho_context,
honcho_search_conclusions, honcho_search_messages, honcho_ask)
- Add link to OpenClaw Honcho Memory docs (docs.openclaw.ai)
- Add OpenClaw Memory Docs card in Next Steps
- Fix QMD setup: remove manual collection commands, link to OpenClaw QMD docs
* feat: retry on more httpx exceptions
* fix: Add retry parity to typescript and update docs
* chore: (skills) update skills to match latest state of the sdk
* chore: (docs) update stale sdk code
* chore: (docs) clean up inconsistencies in docs
* chore: Rebuild Package
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* fix: dialectic held connection
* fix: (agent) pre-compute embeddings for agent tools
* fix: (tests) refactor tests to use smaller test db connections
* fix: Embedding client to branch depending on vector store
* fix: reflect dedup-skipped observations in created counts and isolate DB sessions in extract_preferences
* fix: (tests) update tests to match changes
* fix: expunge docs + don't pass in db to query_documents
---------
Co-authored-by: Rajat Ahuja <rahuja445@gmail.com>
* fix: adding strict checking and updating readme
* chore: changelog and version
* fix: Add strict validation to all Python and TypeScript classes
* fix: Address Code Rabit Comments
* fix: duplicate searchQuery param in typescript session.context()
* feat: add created_at, is_active fields, and get_message method
* feat: Add pagintion params to sdk
* fix: Remove lazy initalization behavior from sdks
* fix: Address File Upload Validation, add compatibility shims, address review comments
* chore: Docs updates
* fix: Convert session config from API format in Peer.sessions()
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: Pass all args to Session constructor in Peer.sessions()
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: Preserve createdAt in Peer.refresh(), pass all data in session.peers()
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: changelog for ts
* fix: Review Comments
* fix: Add createAt and to peers call
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
RepresentationManager._query_documents_recent() and
._query_documents_most_derived() do not filter soft-deleted documents,
unlike every other document query function in the codebase. This causes
the deriver's working representation to include documents that are being
garbage-collected.
Refs #444
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: comprehensive overview guide with all recent integrations
- Add all recent integrations: Gmail, Granola, OpenClaw, Agent Zero, Hermes
- Organize by use case: Quick Start, Agent Frameworks, Platform Integrations, Community Agents, Communication Platforms, Data Import
- Include feature comparison matrix with support levels
- Add clear categorization by setup time and complexity
- Structure follows OpenClaw documentation format with clear sections
* docs: update overview guide to match dev-1372-new format
- Use simpler, cleaner organization with focused sections
- Group by: AI Assistants, Platform Connectors, Agent Frameworks, Migrations
- Match icon and formatting style from dev-1372-new branch
- Include all recent integrations: Gmail, Granola, Hermes, Agent Zero, OpenClaw
* docs: clean up Platform Connectors section description
Remove redundant description text for cleaner presentation
* fix: add observability to docker compose + get docker compose into a usable state
* fix: init.sql location
* fix(docker): use built venv at runtime
---------
Co-authored-by: adavyas <adavyasharma@gmail.com>
New integration guide for the sillytavern-honcho extension covering
install, global config, context architecture, enrichment modes, and
troubleshooting. Added to v3 integrations nav.
* docs: replace Chatbots section with Tutorials, move Reachy Mini into it
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: restructure nav — move modeling-data to Core Concepts, file-uploads to Advanced, merge Migrations into Tutorials
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: final moving
* docs: minor changes
* docs: adding community integrations
* chore: cr updates
* fix: Rename patterns page and nitpicks on organization
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
description: Integrate Honcho memory and social cognition into existing Python or TypeScript codebases. Use when adding Honcho SDK, setting up peers, configuring sessions, implementing the dialectic chat endpoint for AI agents, or wiring Honcho into bot frameworks (nanobot, openclaw, picoclaw, etc).
Honcho is an open source memory library for building stateful agents. It works with any model, framework, or architecture. You send Honcho the messages from your conversations, and custom reasoning models process them in the background — extracting premises, drawing conclusions, and building rich representations of each participant over time. Your agent can then query those representations on-demand ("What does this user care about?", "How technical is this person?") and get grounded, reasoned answers.
The key mental model: **Peers** are any participant — human or AI. Both are represented the same way. Observation settings (`observe_me`, `observe_others`) control which peers Honcho reasons about. Typically you want Honcho to model your users (`observe_me=True`) but not your AI assistant (`observe_me=False`). **Sessions** scope conversations between peers. **Messages** are the raw data you feed in — Honcho reasons about them asynchronously and stores the results as the peer's **representation**. No messages means no reasoning means no memory.
Your agent accesses this memory through `peer.chat(query)` (ask a natural language question, get a reasoned answer), `session.context()` (get formatted conversation history + representations), or both.
## Integration Workflow
Follow these phases in order:
### Phase 1: Codebase Exploration
Before asking the user anything, explore the codebase to understand:
1. **Language & Framework**: Is this Python or TypeScript? What frameworks are used (FastAPI, Express, Next.js, etc.)?
3. **Entity structure**: Identify users, agents, bots, or other entities that interact
4. **Session/conversation handling**: How does the app currently manage conversations?
5. **Message flow**: Where are messages sent/received? What's the request/response cycle?
Use Glob and Grep to find:
- `**/*.py` or `**/*.ts` files with "openai", "anthropic", "llm", "chat", "message"
- User/session models or types
- API routes handling chat or conversation endpoints
> **Bot framework detected?** If the codebase is built around an agent loop, tool registry, session manager, and message bus (e.g., nanobot, openclaw, picoclaw), read `{baseDir}/references/bot-frameworks.md` for framework-specific integration guidance and check `{baseDir}/references/bot-frameworks/<framework>/` for concrete reference implementations.
### Phase 2: Interview (REQUIRED)
After exploring the codebase, use the **AskUserQuestion** tool to clarify integration requirements. Ask these questions (adapt based on what you learned in Phase 1):
#### Question Set 1 - Entities & Peers
Ask about which entities should be Honcho peers:
- header: "Peers"
- question: "Which entities should Honcho track and build representations for?"
- options based on what you found (e.g., "End users only", "Users + AI assistant", "Users + multiple AI agents", "All participants including third-party services")
- Include a follow-up if they have multiple AI agents: should any AI peers be observed?
#### Question Set 2 - Integration Pattern
Ask how they want to use Honcho context:
- header: "Pattern"
- question: "How should your AI access Honcho's user context?"
- options:
- "Tool call (Recommended)" - "Agent queries Honcho on-demand via function calling"
- "Pre-fetch" - "Fetch user context before each LLM call with predefined queries"
- "context()" - "Include conversation history and representations in prompt"
- "Multiple patterns" - "Combine approaches for different use cases"
#### Question Set 3 - Session Structure
Ask about conversation structure:
- header: "Sessions"
- question: "How should conversations map to Honcho sessions?"
- options based on their app (e.g., "One session per chat thread", "One session per user", "Multiple users per session (group chat)", "Custom session logic")
#### Question Set 4 - Specific Queries (if using pre-fetch pattern)
If they chose pre-fetch, ask what context matters:
- header: "Context"
- question: "What user context should be fetched for the AI?"
response = await peer.chat("What does this user prefer?")
```
Match the client to the framework — check whether the codebase uses `async def` handlers or sync `def` handlers and choose accordingly. The rest of this skill shows sync Python examples; swap to `.aio` equivalents for async codebases.
## Core Integration Patterns
### 1. Initialize with a Single Workspace
Use ONE workspace for your entire application. The workspace name should reflect your app/product.
**Python:**
```python
from honcho import Honcho
import os
# Sync client (Flask, Django, scripts)
honcho = Honcho(
workspace_id="your-app-name",
api_key=os.environ["HONCHO_API_KEY"],
environment="production"
)
# Async client (FastAPI, Starlette) — use honcho.aio for all operations
# honcho.aio.peer(), honcho.aio.session(), etc.
```
**TypeScript:**
```typescript
import { Honcho } from '@honcho-ai/sdk';
// All methods are async by default
const honcho = new Honcho({
workspaceId: "your-app-name",
apiKey: process.env.HONCHO_API_KEY,
environment: "production"
});
```
### 2. Create Peers for ALL Entities
Create peers for **every entity** in your business logic - users AND AI assistants.
**Python:**
```python
from honcho import PeerConfig
# Human users
user = honcho.peer("user-123")
# AI assistants - set observe_me=False so Honcho doesn't model the AI
"description": "Query Honcho to retrieve relevant context about the user based on their history and preferences. Use this when you need to understand the user's background, preferences, past interactions, or goals.",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "A natural language question about the user, e.g. 'What are this user's main goals?' or 'What communication style does this user prefer?'"
description: Migrates Honcho Python SDK code from v1.6.0 to v2.0.0. Use when upgrading honcho package, fixing breaking changes after upgrade, or when errors mention AsyncHoncho, observations, Representation class, .core property, or get_config methods.
---
# Honcho Python SDK Migration (v1.6.0 → v2.0.0)
## Overview
This skill migrates code from `honcho` Python SDK v1.6.0 to v2.0.0 (required for Honcho 3.0.0+).
**Key breaking changes:**
- `AsyncHoncho`/`AsyncPeer`/`AsyncSession` removed → use `.aio` accessor
- "Observation" → "Conclusion" terminology
- `Representation` class removed (returns `str` now)
description: Migrates Honcho TypeScript SDK code from v1.6.0 to v2.0.0. Use when upgrading @honcho-ai/sdk, fixing breaking changes after upgrade, or when errors mention removed APIs like .core, getConfig, observations, or snake_case properties.
// pollDeriverStatus() has no replacement - see note below
```
**Important:** `pollDeriverStatus()` and its polling pattern have been removed entirely. Do not rely on the queue ever being empty. The queue is a continuous processing system—new messages may arrive at any time, and waiting for "completion" is not a valid pattern. If your code previously polled for queue completion, redesign it to work without that assumption.
description:Conclusions, representations, or dialectic answers are wrong, noisy, missing, or low-quality — not a hard crash.
title:"[Quality] "
labels:["quality"]
body:
- type:markdown
attributes:
value:|
Use this when Honcho runs without erroring, but **memory formation or recall quality** is off (bad conclusions, missed facts, weak chat answers, polluted representations, etc.).
For crashes, 5xxs, auth failures, or incorrect API mechanics, use the **Bug report** template instead.
**Donot paste production user content, full peer representations, or secrets.** Redact or invent a minimal synthetic example.
- type:dropdown
id:deploy_mode
attributes:
label:Deploy mode
options:
- Managed (api.honcho.dev / app.honcho.dev)
- Self-hosted
- Unsure
validations:
required:true
- type:input
id:version
attributes:
label:Honcho version
description:Server image tag or release, and SDK version if you use one. Write "managed" if you are not self-hosting.
placeholder:e.g. server v2.4.1, honcho-ai 2.1.0
validations:
required:true
- type:textarea
id:description
attributes:
label:What is wrong with the quality?
description:Describe the failure mode (noise, omission, contradiction, staleness, over/under-generalization, etc.).
placeholder:After ingesting messages about X, Honcho concludes Y / chat answers Z…
validations:
required:true
- type:textarea
id:repro
attributes:
label:Minimal scenario
description:>
Smallest synthetic message sequence or setup that triggers the issue.
Prefer invented names/facts over real user data. Include observer/observed
peer setup if relevant (self vs cross-peer).
placeholder:|
1. Peers:alice (user), bot (agent); session S
2. Messages ingested:…
3. Query / conclusion listing shows:…
validations:
required:true
- type:textarea
id:config
attributes:
label:Relevant config
description:>
Custom instructions, provider/model, deriver/dream settings, or workspace/peer
config that affects reasoning. Redact secrets.
placeholder:|
Provider/model:…
Custom instructions:(summary or redacted)
Other:…
validations:
required:false
- type:textarea
id:evidence
attributes:
label:Evidence
description:Redacted conclusion text, chat excerpts, or counts that show the failure. No production PII.
validations:
required:false
- type:textarea
id:context
attributes:
label:Additional context
description:Frequency, scale (message/conclusion counts), related issues, workarounds.
description:Add Honcho to an app store, agent framework, plugin marketplace, or other third-party surface — or improve an existing integration.
title:"[Integration] "
labels:["integration"]
body:
- type:markdown
attributes:
value:|
Use this when you want Honcho available in (or better supported by) an external product surface — app stores, agent frameworks, plugin marketplaces, IDE extensions, MCP clients, etc.
For core API/SDK product features that are not about a third-party surface, use the **Feature request** template instead.
- type:dropdown
id:request_kind
attributes:
label:What kind of request is this?
options:
- New integration / listing (Honcho is not there yet)
- Improve an existing integration
- Official plugin / extension
- Marketplace or app-store listing
- Docs / guide for integrating with a specific tool
- Other
validations:
required:true
- type:input
id:target
attributes:
label:Target product or platform
description:Name of the app, framework, marketplace, or tool.
placeholder:e.g. Claude Code, Cursor, CrewAI, OpenClaw, VS Code Marketplace…
validations:
required:true
- type:input
id:target_url
attributes:
label:Link (if any)
description:Docs, marketplace page, repo, or product URL.
placeholder:https://…
validations:
required:false
- type:textarea
id:why
attributes:
label:Why does this matter?
description:Who would use it, and what does the integration unlock?
validations:
required:true
- type:textarea
id:shape
attributes:
label:What should the integration look like?
description:>
e.g. one-click install, MCP server listing, native memory backend,
SDK recipe, plugin with slash commands, env-var setup, etc.
Link to prior art or a sketch if you have one.
validations:
required:false
- type:textarea
id:context
attributes:
label:Additional context
description:Related issues, community demand, constraints, offers to help build it.
`The linked ${issues.length===1?'issue is':'issues are'} not open with the `+
`\`${REQUIRED_LABEL}\` label: ${detail}.`,
};
}
functionnoticeBody({owner,repo,reason}){
return[
MARKER,
'Thanks for the contribution. This pull request does not clear our issue gate yet.',
'',
`**${reason}**`,
'',
`Every pull request to Honcho needs to be linked to an open issue carrying the \`${REQUIRED_LABEL}\` label. We do this so the review queue only holds work we have already agreed should be built — it means nobody spends time on a change we cannot merge.`,
'',
'To get this moving:',
'',
`1. Find or open an issue describing the change. [Approved issues are here](https://github.com/${owner}/${repo}/issues?q=is%3Aissue+is%3Aopen+label%3A${REQUIRED_LABEL}).`,
`2. Make the case for it in [Discord](${DISCORD}) — maintainers are most active there, and it is by far the fastest route to a decision.`,
`3. Once the issue has the label, link it: put \`Fixes #<number>\` in this pull request's description, or use **Development** in the sidebar.`,
'',
`**This will close automatically in ${GRACE_HOURS} hours if it is still unlinked.** Nothing is lost if that happens — link the issue, reopen, and it goes into the review queue.`,
'',
`See [CONTRIBUTING.md](https://github.com/${owner}/${repo}/blob/main/CONTRIBUTING.md) for the full process. If you think this is wrong, say so here and a maintainer will take a look.`,
core.info(`#${pr.number}: ${Math.round(GRACE_HOURS-hours)}h of grace left`);
continue;
}
awaitact(`#${pr.number}: closing — notified ${Math.round(hours)}h ago, still failing`,close(pr,
`Closing this: ${GRACE_HOURS} hours have passed and the gate is still not clear. This is not a judgement on the code. Link an approved issue and reopen — it goes straight into the review queue.`,
));
}
// Stale drafts. The gate skips drafts entirely, so they never carry the label;
// this pass keys off inactivity and applies the shared exemptions itself.
awaitact(`#${pr.number}: closing stale draft — ${Math.round(days)}d without activity`,close(pr,
`Closing this draft after ${DRAFT_STALE_DAYS} days without activity, to keep the pull request list readable. Reopen whenever you pick it back up — nothing here is lost.`,
@ -5,6 +5,313 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/)
and this project adheres to [Semantic Versioning](http://semver.org/).
## [3.1.1] - 2026-09-02
### Changed
- Server `requires-python` is `>=3.13`, matching the production image. Self-hosters on 3.10–3.12 need to upgrade; SDK and CLI floors are unchanged (#1090)
### Fixed
- Concurrent `create_documents` writers to the same collection deadlocked on `times_derived` reinforcement UPDATEs issued in batch order; the error was swallowed per-document, the batch was lost, and the queue item was marked processed. Writers now lock target rows with `SELECT ... ORDER BY id FOR UPDATE` before applying, abort the batch on `SQLAlchemyError` instead of continuing through a dead session, and retry transient errors (deadlock, serialization failure, lock/statement timeout, lost connection) up to `MAX_RETRYABLE_ATTEMPTS` instead of burning the item (#1033)
- Scope backfill no longer embeds, writes, and syncs every planned copy at once. A 14k-document session is ~580MB of vectors; several concurrent backfills OOM-killed the deriver at its 1000Mi limit and crash-looped because the work units never completed. Phases 2–4 now run per chunk of 500 specs, reload source embeddings per chunk, and drop them once synced. Membership is locked across chunk writes so a concurrent leave cannot commit between the check and the inserts (#1104)
- Model-generated observations with NUL bytes (`\u0000`) no longer fail the exact-content dedup pre-fetch with a Postgres `DataError` that dropped the whole observer batch. Ingress already stripped NUL from user content; the deriver now strips it so stored text matches embedded text. All-NUL content is dropped rather than stored empty (#1095)
- `search_messages` no longer forwards `top_k=0` to Turbopuffer (which requires 1..10000). Zero/negative limits short-circuit to empty results; tool limits are floored at 1. The documents path was already guarded (#970); this closes the message path (#1084)
- OpenAI-compatible tool-call turns with `content=null` keep null through history replay instead of being coerced to `""`. Providers that bind reasoning state to the exact assistant message shape were breaking on the empty string. Tool-less null still becomes `""` (#1064)
- The production image now ships `pyproject.toml` in the runtime stage, so the service reports its real version instead of `unknown` in OpenAPI and telemetry (#1074)
## [3.1.0] - 2026-08-25
### Added
- Scopes: a named grouping of sessions that acts as a visibility boundary on recall, implemented as a facade over an observer peer (`scope.{name}` with `{"kind": "scope"}`). Developers manage them exclusively through `/v3/workspaces/{workspace_id}/scopes` (create-or-get, list, get, add/list/remove session membership) and an optional `scopes` field on session create — never through the observer/observed mechanics. Scope peers cannot author messages, cannot be a chat or representation `target`, are excluded from `peers.list` by default (`PeerGet.kind` = `"scope"` / `"all"` switches the view), and are rejected on the generic session-peer routes. Workspace-level key required; peer- and session-scoped keys get 401. Legacy peers occupying a reserved `scope.` name without the kind flag are refused with 409, never adopted (#884)
- `scope` read option on chat, representation, session context, and workspace search. A single scope swaps the observer to the backing scope peer so conclusion recall, peer cards, and message tools stay inside that scope's membership. A list of scopes takes the union of member sessions (capped at `MAX_SESSION_ALLOWLIST_ENTRIES`) and executes via the session-allowlist path. Empty scopes fail closed. `scope` is mutually exclusive with `filters` and `session_id`. Workspace- or admin-level key required (403 otherwise). Scope peers are also rejected as `peer_target` / `peer_perspective` on session context and as the path peer or `target` on `GET /peers/{id}/context` (#897)
- Scope backfill-by-copy and removal reconciliation. Adding a session that already has messages copies its explicit-level documents into the scope's collections (no LLM re-derivation; idempotent via `copied_from`). Removing a session soft-deletes those copies and fail-closed cascades to derived documents whose `source_ids` intersect anything removed, then enqueues a `card_refresh` dream with `rebuild=True` plus an omni dream. `GET /v3/workspaces/{workspace_id}/scopes/{scope_id}/status` reports per-session backfill state (`pending` / `completed` / `failed`, plus `docs_copied`) (#904)
- Workspace-level chat at `POST /v3/workspaces/{workspace_id}/chat`: agentic dialectic over the whole workspace instead of a single (observer, observed) pair. Prefetches workspace stats and the top active peers' self cards, then searches pair-scoped memory with `[observer->observed]` attribution. Supports `session_id`, `scope`, `reasoning_level`, `response_format`, and SSE streaming (#931)
- MCP workspace discovery: tools accept `workspace_id`, the worker honors an optional `X-Honcho-Workspace-ID` connection header, and `list_workspace` / `create_workspace` tools let clients pick or create a workspace instead of relying on the SDK default (#1020)
- MCP `search` also queries conclusions in parallel with messages when `peer_id` is given, returning `{messages, conclusions}`. The conclusions leg degrades to `[]` on error so search never gets worse than before (#974)
- Prometheus metrics for physical DB connections, visible even under `DB_POOL_CLASS=null`: `db_connections_open` (gauge) and `db_connections_established` (counter), hooked to SQLAlchemy connection-lifecycle events and registered on both the API and the deriver (#1055)
- Bounded-label Prometheus series are zero-initialized at process start so an absent series means a broken scrape rather than "nothing happened" (#927)
### Changed
- Workspace and pair chat system prompts now describe Honcho, peers, and the harness on their own terms, and render only the tools the request actually offers. The pair prompt no longer advertises a write tool that is not in the loadout (#1066)
- Deriver idle polling backoff is longer and no longer reset by periodic reconciler work, so downstream connection pools can cull idle DB connections (#1015)
- LLM provider SDKs are lazy-loaded so idle API and deriver processes no longer pay for every provider at import time (#1011)
- Production image is a multi-stage build: LanceDB/PyArrow move behind an optional `lancedb` extra (`INSTALL_LANCEDB=true` to restore them), FastAPI's unused cloud CLI is dropped, and the venv is copied into the runtime image with final ownership so Docker does not double the layer. Default unpacked image is about 663 MB (was 1.7 GB) (#1014)
- Redis Cluster cache keys hash-tag the namespace so one deployment's keys land on a single shard instead of opening a connection to every node. No behaviour change on a non-cluster backend; existing keys age out by TTL (#1058)
- Deriver extraction prompt no longer leaks its own few-shot examples into extracted conclusions (#1028)
### Fixed
- Observer-scoped `get_observation_context` no longer materializes every session the observer has ever joined into a `session_name IN (...)` list (twice in one statement). Past ~32k sessions that hit psycopg's bind-parameter ceiling and 500'd. The observer half is now a correlated `EXISTS` over `session_peers`, two bind parameters regardless of membership size (#1065)
- Re-adding an already-active session peer no longer advances `joined_at`, so `peer_perspective` search keeps messages from the original join. Genuine leave-and-rejoin still starts a new window (#1059)
- Transient embedding-provider errors (for example an OpenAI-compatible 200 with empty `data: []`) were relabeled as token-limit errors. Only genuine oversize input raises `EmbeddingTokenLimitError`; other provider errors propagate unchanged (#791)
- The filter DSL now fails closed with a 422 instead of a 500 on bad shapes, coerces operands by column type (so `{"session_id": {"ne": "abc"}}` is a string inequality rather than "invalid numeric"), and treats `NOT` / `ne` as null-safe (`IS NOT TRUE` / `IS DISTINCT FROM`) so negation no longer drops rows whose field is unset. Closed-set columns like `level` reject unknown values. Session-allowlist entries must be well-formed ids (`*` is 422, not a silent widen) (#947)
- `ne` on JSONB metadata keys is null-safe: a missing key is not equal to the compared value, so `{"metadata": {"foo": {"ne": "bar"}}}` includes rows where `foo` is unset (#1036)
- Oversized texts in `simple_batch_embed` are truncated to the embedding token cap instead of failing the whole batch. Representation processing reports failed observer saves in `RepresentationCompletedEvent` and raises when every observer save fails (#1019)
- Assistant `reasoning_content` (DeepSeek / some OpenRouter models) is preserved across tool-loop turns. Previously the tool loop dropped thinking content before building the next assistant history message, so continuation requests failed. `reasoning_details` still takes precedence when both are present (#1034)
- `create_observations` now honors `DERIVER_DEDUPLICATE` instead of hardcoding `deduplicate=True`, matching the representation write path (#1018)
- `provider_params.timeout` is forwarded to the OpenAI-compatible embedding client, not just the LLM client (#1024)
- Conclusions semantic-search validation errors name the field and the constraint instead of returning a generic 422 (#960)
- OpenAI-compatible embedding calls request `encoding_format=float` so providers that default to base64 do not break pgvector inserts (#938)
- Gemini batch embedding works for `gemini-embedding-2*` models, which rejected the previous request shape (#745)
- MCP OAuth with no advertised scopes no longer defaults to read-only (which 403'd chat and search POSTs). Protected-resource metadata advertises read and write (#1004)
## [3.0.12] - 2026-08-10
### Added
- Session allowlist on the Dialectic and representation via a constrained `filters` body on `POST /peers/{peer_id}/chat` and `/representation`, supporting only the `session_id` key (a session id, a bare list, or `{"in": [...]}`). Unsupported keys and shapes are rejected with 422 rather than silently ignored, it composes with `session_id` (which must be included in the allowlist when both are given), and it is capped at 1,000 sessions per request. Enforcement is uniform and fail-closed at every recall chokepoint: scoped conclusion recall is restricted to `level == "explicit"` (dream-derived conclusions carry a single `session_name` but are synthesized across all sessions, so that stamp can't be scoped on), `get_reasoning_chain` is unavailable under an allowlist, and an empty allowlist short-circuits to empty results everywhere. Workspace keys pass the allowlist as-given; peer-scoped JWTs must be an active member of every allowlisted session (401 otherwise) (#882)
- Bare-list membership sugar in the filter DSL: `{"session_id": ["s1", "s2"]}` is now shorthand for `{"session_id": {"in": [...]}}` on regular columns generically. JSONB metadata columns are excluded and keep containment semantics. Strictly additive, since a bare list on a regular column previously compiled to a type-mismatched equality that matched nothing (#881)
- Optional structured outputs on the Dialectic: `response_format` (a JSON Schema with root type `object`) on peer chat makes `content` a JSON string conforming to that schema. Only a conservative subset of JSON Schema is supported, with DoS guards and non-recursive `$ref` support (#896)
- Combined tool calling and structured output in the LLM transport layer, with per-backend request shaping: OpenAI routes tool-carrying structured requests through `create()` with an explicit `json_schema` response format (`parse()` 500s on non-strict function tools), Anthropic skips the `{` JSON prefill when tools are present so `tool_use` blocks stay reachable, and Gemini injects a schema instruction into the final turn instead of using native `response_schema` (rejected alongside function calling before Gemini 3). All backends skip structured-output parsing on tool-call turns, which carry no consumable content (#907)
- `card_refresh` dream type: a lightweight dream that runs only the peer-card update, for event-driven refreshes such as membership changes and cold starts. Handled by a new `CardRefreshSpecialist` restricted to `get_recent_observations`, `search_memory`, and `update_peer_card` (no observation-mutating tools) with a tool-iteration cap of `min(6, DREAM.MAX_TOOL_ITERATIONS)`. `POST /v3/workspaces/{workspace_id}/schedule_dream` accepts `dream_type=card_refresh` plus a `rebuild` flag, which omits the existing card from the prompt so the specialist rebuilds it solely from observations present in the collection. Card refreshes never advance the omni dream guard pair (`last_dream_at` / `last_dream_document_count`) (#883)
- Full-fidelity LLM trace stream, with Langfuse as one projection over it: each call is captured once (`CapturedLLMCall`) and fanned out to a CloudEvents trace stream (`llm.call.traced` / `trace.content`) and a Langfuse exporter, both reconstructing trace → run → step → generation from the same source of truth. Adds `TELEMETRY_TRACE_PAYLOADS_ENABLED` (default `false`), `TELEMETRY_TRACE_MAX_BYTES` (default 262144, per-message cap with oversized content clipped), `TELEMETRY_TRACE_PURPOSES` (JSON list of `CallPurpose` values; empty means all), and `LANGFUSE_EXPORTER_MODE` (`exporter` by default; `inline` is kept for one release for side-by-side validation). Embedding calls are traced, dreamer branches nest under one dream trace, tool calls become spans under their step, and high-volume events are sampled deterministically. `TRACE_ENDPOINT` is dropped (#845)
- Redis Cluster support via `CACHE_CLUSTER` (for example GCP Memorystore for Redis Cluster), alongside a new `CACHE_LOCK_WAIT_CHECK_INTERVAL_SECONDS` (#905)
- `EMBEDDING_MODEL_CONFIG__MAX_BATCH_SIZE` caps texts per embedding request for OpenAI-compatible providers with smaller limits than OpenAI's, such as DashScope `text-embedding-v4` (10) and Alibaba Bailian `qwen3.7-text-embedding` (20). When unset, native provider defaults are preserved (OpenAI 2048, Gemini 100) (#983)
- Per-request provider timeouts via `provider_params.timeout` on any model config, validated at config load so a bad value fails at startup with the exact config path instead of surfacing per-request as a retried 500. Good values normalize to float seconds; Gemini's is converted to milliseconds (#832)
- `RepresentationCompletedEvent` now reports deduplication counts: `exact_dup_in_batch_count`, `exact_dup_existing_count`, `semantic_dup_rejected_count`, and `semantic_dup_replaced_count` (#910)
- OAuth discovery for MCP clients: the MCP worker serves `/.well-known/oauth-protected-resource` (RFC 9728) without auth so clients can discover the authorization server, and a 401 now carries `WWW-Authenticate: Bearer resource_metadata="..."` (exposed cross-origin) to start the flow (#923)
- Prometheus metrics for the immediate-embed fast path: tasks shed because `EMBEDDING_MAX_PENDING_EMBED_TASKS` was reached, and the current in-flight task count (#892)
- Docs: a detailed system architecture diagram, a Codex integration guide (#879), a structured-outputs page (#896), a section on filtering conclusions by reasoning level (#851), a health-check endpoint reference, and SDK updates (#867)
### Changed
- **Breaking config change:**`DERIVER_REPRESENTATION_BATCH_MAX_TOKENS` is split into two settings that were previously conflated — `DERIVER_REPRESENTATION_BATCH_WORK_UNIT_TARGET_TOKENS` (default 512), the producer-side minimum a work unit accumulates before the deriver claims it, where `0` disables the gate; and `DERIVER_REPRESENTATION_BATCH_TARGET_INPUT_TOKENS` (default 1024), the consumer-side maximum context-window tokens per deriver LLM call. Deployments setting the old name must migrate (#889)
- The immediate-embed fast path now applies backpressure: `EMBEDDING_MAX_PENDING_EMBED_TASKS` (default 50) caps in-flight embed tasks, and once saturated, message creation skips the fast path entirely and the reconciler embeds on its next cycle. `0` disables the fast path (#892)
- Explicit-level documents are now kept session-pure, so memory can be built by copying explicit documents between collections. Enforcement refuses rather than rewrites: `create_documents` rejects explicit documents with a null `session_name`, exact dedup keys on (content, level, session-for-explicit), semantic dedup scopes candidate search to the same level and — for explicit documents — the same session, and the generic `create_observations` tool rejects `level='explicit'` outside message-ingestion (deriver) context. Derived levels keep cross-session consolidation (#883)
- Sentry's `before_send` filter is centralized as `default_before_send` in `src/telemetry/sentry.py` instead of living only in the API's `main.py`, so the deriver gets the same non-actionable-exception filtering. All Sentry events also carry a `namespace` tag for correlation (#934, #870)
- The minimal deriver's extraction examples no longer teach inferences its own output schema forbids. The `EXAMPLES` block demonstrated deriving a specific birthday from "I just had my 25th birthday last Saturday", deriving residence from a single visit ("I took my dog for a walk in NYC" → "alice lives in NYC"), and a "+ general knowledge" deductive output the deriver has no channel for. The replacements stay inside the schema's contract and teach the boundary: the dog/NYC message is kept and shown extracting correctly, and a separate example shows "lives in NYC" is valid when actually stated (#985)
- Dreamer specialists are instructed not to output summaries (#894)
- `session_name` is deprecated for scoping in favor of the session allowlist. It is not removed and not aliased: it also pins the query to one session, bypasses observer scoping, and drives session-history injection into the dialectic prompt, so it has no drop-in replacement (#882)
- The MCP worker no longer requires the `X-Honcho-User-Name` or `X-Honcho-Assistant-Name` headers (#923)
### Fixed
- Session scoping was applied to only one of the three working-representation query paths: `session_name` reached the recent-documents query, but the semantic and most-derived paths ignored it, so `limit_to_session` leaked cross-session conclusions into perspectives. The allowlist is now threaded uniformly through all three paths and pushed down to pgvector and external vector stores (#881)
- Empty membership lists failed open in the vector-store filter builders, silently widening scope: LanceDB dropped empty `IN` clauses and Turbopuffer emitted a bare `In []` with undocumented semantics. Both now emit an explicit always-false predicate, and `_build_filter_conditions` checks `is not None` rather than truthiness so an empty list is no longer treated like `None` (#881, #882)
- Session-scoped CRUD helpers ignored the session allowlist entirely, so a caller could read a session the allowlist forbids. The API routes guarded this with a 422, but the dialectic tools call these CRUD functions directly and bypassed it. `_semantic_search_messages` (covering `search_messages` and `search_messages_temporal`), `grep_messages`, `get_messages_by_date_range`, `get_recent_history`, and `get_observation_context` now return `[]` when `session_name` is set and outside the allowlist (#882)
- The cache client logged the full Redis URL — including the password — at INFO and WARNING on every connection attempt and failure, exposing the live credential in container logs and downstream aggregation. Credentials are now redacted across userinfo, the `?password=` (redis-py) and `?secret=` (cashews) query params, scheme-less URLs whose password is invisible to `.port`/`.password` parsing, and malformed URLs, whose fallback previously echoed the raw input verbatim (#869)
- A `top_k` of `0` reached the vector store, where Turbopuffer rejects it with a 400 (`top_k must be between 1 and 10000`). A non-positive `top_k` now returns `[]` before the embedding call, and the semantic budget floors at 1 so an explicitly requested search isn't silently allocated zero (#970)
- Gemini clients had no HTTP timeout, so a stalled socket wedged the deriver worker's uvloop event loop, which the in-process reconciler shares. A 10-minute timeout is now set on both the Gemini LLM client and the Gemini embedding client (#903)
- Dreamer conclusions were dated to ingestion time rather than their latest source observation, and their timestamps are now normalized (#890)
- Langfuse I/O annotation was gated on `LANGFUSE_PUBLIC_KEY` instead of `langfuse_inline_enabled`, so in the default `exporter` mode it called `update_current_generation()` with no active span — logging "No active span in current context" roughly 14 times per dialectic run and building throwaway `model_dump` payloads on every LLM call. Separately, `AgentToolSummaryCreatedEvent` hardcoded `run_id="deriver"` / `iteration=0`, polluting `run_id` grouping in the CloudEvents stream with a phantom run; both fields are now optional and the resource id is keyed on `message_id:summary_type` (schema_version 2 → 3) (#845)
- Assistant tool calls were dropped from the captured trace stream for OpenAI and Gemini: `build_captured_messages` read only `{role, content, tool_call_id}`, but those providers keep tool calls outside `content`, so replayed tool-call turns landed as empty content and Gemini lost its text and tool results entirely. Tool calls are now normalized per provider into a unified `tool_calls` field and folded into the content hash. Gemini's `thought_signature` is bytes, so `model_dump(mode="json")` raised `UnicodeDecodeError` inside `emit_trace`, silently dropping whole tool-calling iterations from the trace stream (billing and Langfuse were unaffected); it is now base64-encoded on the telemetry path while replay keeps the raw bytes (#845)
- `EmbeddingClient.encoding` forced full client construction, raising "OpenAI API key is required" even though tiktoken needs no credentials. The document dedup tie-break only needs `.encoding` for token counting, so any test hitting that path failed in environments without embedding keys — notably CI for pull requests from forks. The encoding is now resolved from the configured model directly, falling back to `cl100k_base`, and the underlying client's encoding is reused only when it has already been constructed (#955)
- The Docker build failed under Podman because the uv build inputs weren't copied (#878)
- LanceDB was installed on macOS Intel, where it doesn't work. A PEP 508 marker excludes `darwin/x86_64` and the LanceDB vector-store import is wrapped so a misconfiguration surfaces as a clear config error (#496)
- Prompt checks requiring the literal token "json" for `json_object` mode are now satisfied in lowercase (#887)
- Reverted an unintended `RepresentationCompletedEvent` schema-version increment
- Documented preinstalling pgvector as a privileged role for deployments where the `DB_CONNECTION_URI` role deliberately cannot create extensions (managed Postgres, Kubernetes operators, NixOS). `CREATE EXTENSION IF NOT EXISTS vector` does not help there, because Postgres checks the privilege before checking whether the extension exists. Docker Compose is unaffected, since the bundled stack connects as the `postgres` superuser (#984)
## [3.0.11] - 2026-06-24
### Added
- `api_request_duration_seconds` Prometheus histogram tracking per-route request latency, labeled by method and endpoint (#837)
- LLM `provider_params` passthroughs (`extra_body` / `extra_headers` / `extra_query`) are now forwarded to the underlying provider transport across all backends, with shape validation that rejects non-mapping values (#821)
- `structured_output_mode` model-config option to use `json_object` mode for OpenAI-compatible providers that lack native Structured Outputs support (used by the deriver) (#820)
- OpenRouter app-attribution headers (`HTTP-Referer` / `X-Openrouter-Title`) are now sent on OpenAI-compatible clients when the configured base URL is OpenRouter, so requests are attributed to "Honcho" in OpenRouter's dashboard (#805)
- Langfuse traces are now tagged with user and session IDs for easier trace filtering (#814)
- `DERIVER_REPRESENTATION_BATCH_MAX_AGE_SECONDS` (default 1800s) lets sub-threshold representation work units flush once their oldest unprocessed queue item ages out. Set it to `0` to keep the legacy behavior where sub-threshold tails wait indefinitely unless `DERIVER_FLUSH_ENABLED=true` (#826)
- Conclusion responses now include a `level` field (`explicit`, `deductive`, `inductive`, `contradiction`); list/query endpoints support filtering by `level` via `filters`, with reserved filter keys protected from being overridden by user-supplied filters (#851)
### Changed
- Peer-scoped JWTs now get read-only access to the sessions their peer is an active member of (session context, summaries, peers, their own per-session config, search, and message reads). Session-scoped JWTs remain confined to their session and cannot reach peer routes (#679)
- Compacted Honcho's log output, with guarded ms/s metric formatting that falls back to a plain string for non-numeric values (#836)
- Sentry now drops noisy infra/scrape transactions: the reconciler opens a transaction only once a batch has rows (idle cycles emit none), and a `traces_sampler` returns `0.0` for `/metrics`, `/health`, `/openapi.json`, `/docs`, `/redoc`, and the deriver metrics server. `SENTRY.TRACES_SAMPLE_RATE` still governs real traffic (#834)
### Fixed
- Peer- and session-scoped JWTs were effectively workspace-scoped: authorization walked the route's declared scope and fell through to a workspace match, so a `{w, p: alice}` token could act on any peer in the workspace. JWTs are now authorized by their narrowest claim and never widen to workspace access (#679)
- The keys API now rejects creating a peer- or session-scoped key without a workspace. Such keys were minted successfully but failed verification on every request (#679)
- Agent-supplied observation IDs carrying the display-format `id:` prefix are now normalized (prefix and trailing whitespace stripped) before `source_ids` are stored and on `get_reasoning_chain` lookups, fixing corrupted provenance links and broken reasoning-chain traversal (#795)
- Fixed a `create_tree` keyword-argument mismatch in the Dreamer's surprisal tree construction (#749)
- Providers that omit output-token counts (observed with Gemini on tool-loop completions) returned `output_tokens=None`, which raised a Pydantic validation error that aborted the call and crashed the Dreamer's induction phase before inductive conclusions were persisted. `None` is now coerced to `0` so token accounting degrades gracefully (#809)
- Document creation now performs exact (case-insensitive, whitespace-trimmed) content deduplication before the existing semantic dedup step: exact duplicates within a batch collapse to a single insert, and an exact match against a live document reinforces it (atomic `times_derived` increment) instead of creating a new row (#861)
- The OpenAI backend passed `tool_choice` through raw while the Anthropic and Gemini backends translate Honcho's canonical vocabulary to their native form, so on a mixed-provider fallback chain (for example Gemini primary → OpenAI backup) a canonical `"any"` reached OpenAI unchanged and was rejected as an invalid param. The OpenAI backend now converts it, mirroring the others: `any`/`required` → `required`, `auto`/`none` pass through, and a tool-name string or `{"name": ...}` dict becomes a function selection (#850)
- Langfuse `@observe` auto-capture serialized every argument of `honcho_llm_call_inner` into the generation span input, including `client_override` (a live `AsyncOpenAI`/`genai` client) and `selected_config` (which carries `api_key`). Auto-capture deep-copied the client into a half-constructed object whose teardown raised (`AsyncHttpxClientWrapper ... no attribute '_state'` on OpenAI, flooding stderr; `BaseApiClient ... no attribute '_http_options'` on Gemini), and it leaked `ModelConfig.api_key` into traces. Capture is now an explicit allowlist: `capture_input`/`capture_output` are disabled and curated, serializable input and output are stamped instead, with tuning knobs surfaced as `model_parameters` via a secret-bearing denylist and per-call token usage mirrored as `usage_details` (#849)
## [3.0.10] - 2026-06-15
### Added
- Messages are now embedded via a background task rather than blocking API request
- Read-only DB session mode (`get_read_db` / `tracked_db(..., read_only=True)`) so reads don't hold a transaction open across the work
- `CORS_ORIGINS` env var to configure CORS allowed origins without editing source; defaults match the prior hardcoded list, so self-hosted deployments behind custom domains can whitelist their frontend (#697)
- `scripts/generate_jwt.py` — utility for minting scoped or admin Honcho JWTs (`--admin`, `--workspace`/`--peer`/`--session`, `--expires` with human-friendly durations, `--print-only`) without calling the keys API (#757)
- `STALE_WORK_UNIT_CLEANUP_INTERVAL_SECONDS` (default 60s) — minimum jittered spacing between deriver stale-work-unit cleanup runs, so cleanup no longer runs on every seconds-scale poll (`0.0` keeps the legacy every-poll behavior) (#773)
### Changed
- Optimized the deriver and dreamer prompt cache prefixes to improve prompt-cache hit rates (#806)
### Fixed
- `times_derived` is now properly reinforced when a duplicate conclusion is detected. It had been pinned at 1 for nearly every conclusion (the reject-new branch dropped the increment and the new-wins branch reset the count to 1), so `ORDER BY times_derived DESC` fell back to arbitrary heap order and froze stale conclusions to the front of injected context. Reinforcement is now an atomic increment and both most-derived queries gained a `created_at DESC` recency tiebreaker (#768)
- Webhook creation now correctly rejects private/internal IP addresses (#793)
## [3.0.9] - 2026-06-02
### Changed
- Connection acquisition is now a single attempt with no server-side retry, on a vanilla `AsyncSession`. A new `DB_CONNECT_TIMEOUT_SECONDS` (default 2s) bounds the attempt so a saturated or unreachable pooler fails fast instead of holding a client connection open to re-knock. A saturated DB now surfaces to the caller — the API returns an error and the deriver backs off and retries on a later poll — which lets the pooler drain rather than amplifying saturation.
### Added
- Deriver poll jitter so instances that start together don't poll in lockstep: `DERIVER_POLLING_STARTUP_JITTER_SECONDS` (random delay before the first poll, default 30s) and `DERIVER_POLLING_JITTER_RATIO` (±fraction applied to every poll sleep, default 0.5). Both disable at `0.0`; the underlying backoff schedule is unchanged.
### Removed
- Reverted the connection-checkout retry and `HonchoAsyncSession` custom session introduced in 3.0.8. Removed the `DB_CONNECTION_RETRY_ENABLED` / `DB_CONNECTION_RETRY_MAX_DELAY_SECONDS` / `DB_CONNECTION_RETRY_BACKOFF_INITIAL_SECONDS` / `DB_CONNECTION_RETRY_BACKOFF_MAX_SECONDS` settings, the `db_connection_acquisitions{outcome=...}` Prometheus counter, and the `db.pool.acquire` Sentry span. Alerting built on `db_connection_acquisitions` should migrate to `db_pool_connections` / `db_queries_in_flight`.
## [3.0.8] - 2026-06-01
### Added
- Connection-checkout retry with bounded exponential backoff (tenacity) on `get_db`/`tracked_db`: transient transaction-pooler (Supavisor) rejections — SQLAlchemy `TimeoutError` and `OperationalError` — now retry with backoff instead of surfacing as 500s under client-connection saturation. Gated by
`DB_CONNECTION_RETRY_ENABLED` with configurable delay/backoff knobs; ~10s default budget (#758)
- `HonchoAsyncSession` — a lazy `AsyncSession` that checks out its pooled connection (with retry) on the first DB-touching call rather than at construction. Request handlers doing non-DB work (embedding, file, LLM) before their first query no longer pin a pooler connection across it. Only the checkout is retried;
the statement still runs exactly once, so writes are never duplicated (#758)
- Adaptive deriver queue polling: the poll interval backs off when the queue is idle or erroring (base → max, doubling each cycle) and snaps back to base the moment work is claimed, cutting steady-state query load against the DB. Gated by `DERIVER_POLLING_BACKOFF_ENABLED` with configurable max/multiplier (#758)
- New Prometheus `db_pool_connections` gauge (checked_out / checked_in / size / overflow), labeled `api`|`deriver`, registered in both the API lifespan and the deriver metrics server (#758)
- New Prometheus `db_connection_acquisitions{outcome=ok|retried|exhausted}` counter — the alertable early-warning signal that connection checkouts are retrying through pooler rejection, before requests start failing (#758)
- New Prometheus `db_queries_in_flight` gauge — statements actually executing on the wire (via SQLAlchemy cursor-execute events). Paired with `checked_out`, the gap reveals connections held but parked (the "idle in transaction during an external call" antipattern). Gated on `METRICS.ENABLED` for zero overhead when
off (#758)
- Explicit `SqlalchemyIntegration` in both the API and deriver Sentry inits; connection acquisition wrapped in a `db.pool.acquire` span with live pool stats captured on retry exhaustion (#758)
### Changed
- Default `POOL_TIMEOUT` lowered to 5s, with validation that it stays under the connection-retry budget when a pooled (non-null) `POOL_CLASS` is configured; `config.toml.example` and the v2/v3 configuration docs updated to match (#758)
- `HonchoAsyncSession` wraps every DB-touching session method (execute / scalar / scalars / flush / merge / refresh / commit / get / get_one / stream / stream_scalars / delete) so the lazy-checkout-with-retry guarantee has no holes; the acquired flag resets on `close()`/`reset()` so a reused session re-acquires on
next use (#758)
### Fixed
- Roll the session back on a retryable checkout failure before retrying — a failed autobegin could otherwise leave it pending-rollback, making the next connection attempt raise instead of cleanly re-checking-out (#758)
- Guard `DBPoolCollector.collect()` so a pool-read/import hiccup can't raise and abort the entire `/metrics` scrape (Prometheus drops all metrics if any collector raises) (#758)
- Clamp the pool overflow gauge to ≥ 0 (it could report negative before the pool fills) (#758)
- Removed a double-sleep in the deriver idle poll so the backoff cap is a true cap rather than 2× (#758)
## [3.0.7] - 2026-05-21
### Added
- New `src/llm/` module as the single owner of provider runtime: clients, backends, history adapters, tool loop, request builder, credentials, and caching policy (#459)
- `AttemptPlan` dataclass captures per-retry provider selection (client, model, reasoning_effort, thinking_budget_tokens, selected_config) and pins it across stream-final retries so streaming doesn't bounce back to primary after the tool loop has settled on fallback (#459)
- Gemini JSON-schema sanitizer for `function_declarations` — strips keywords Gemini's validator rejects (`additionalProperties`, `allOf`, etc.) while preserving semantics for all other backends (#459)
- Dreamer specialists derive `effective_max_tokens` from `model_config.max_output_tokens` with a per-specialist default fallback (#459)
- New cloudevent `LLMCallCompletedEvent` (`llm.call.completed`) fires once per provider hit with full cost-attribution context: transport/provider_label, model, token counts with cache breakdown, finish_reason, outcome, `is_final_attempt`, retry/fallback state, duration, tool-call shape, streaming flag, and agent correlation (`run_id` + iteration). Includes a `CallPurpose` closed enum (`deriver.representation`, `dialectic.answer`, `dream.deduction|induction`, `summary.short|long`) (#637)
- `RepresentationCompletedEvent` now carries `total_input_tokens` for full-trace cost attribution (#637)
- Per-emitter `honcho_version` injection on all CloudEvents plus emitter health metrics (#637)
- `TelemetrySettings.HIGH_VOLUME_SAMPLE_RATE` (default 1.0) — deterministic per-`run_id` sampler so an entire agent trace is kept or dropped together; aggregate envelopes bypass the sampler (#637)
- Deriver custom instructions: per-workspace/peer guidance threaded into the deriver prompt with a `MAX_CUSTOM_INSTRUCTIONS_TOKENS` budget (default 2000); deriver `MAX_INPUT_TOKENS` raised 23000 → 25000 to make room (#609)
- Configurable embedding dimensions: `EMBEDDING_MODEL_CONFIG__DIMENSIONS_MODE` (`auto`/`always`/`never`) controls whether the OpenAI `dimensions=` parameter is forwarded; `auto` (default) sends it when the operator explicitly set `EMBEDDING_VECTOR_DIMENSIONS` and the model is not on the known-rejecting allowlist (#678)
- New `honcho-cli` package — Python CLI for inspecting and managing peers, sessions, and configuration against a Honcho deployment (#424)
- `HONCHO_API_URL` env var support in the MCP Worker, enabling self-hosted Honcho deployments to point the Worker at their own instance instead of `https://api.honcho.dev` (#575)
- API ID `max_length` increased from 100 to 512 across `WorkspaceCreate`, `PeerCreate`, and `SessionCreate` to align the API contract with the underlying DB schema (#684)
- All LLM orchestration moved out of `src/utils/clients.py` into `src/llm/` with modules split by responsibility (api, executor, tool_loop, runtime, registry, conversation, request_builder, credentials, caching, backends, history_adapters) (#459)
- Default `ModelConfig` factories (deriver, summary, dreamer specialists, dialectic levels) normalized to `openai/gpt-5.4-mini` with no extra parameters set by default; operators add transport/thinking overrides explicitly (#459)
- OpenAI reasoning-model routing widened via `_uses_max_completion_tokens` heuristic covering `gpt-5.x` and `o1/o3/o4` — these models receive `max_completion_tokens` instead of `max_tokens` (#459)
- Override client factories switched from unbounded `@cache` to `@lru_cache(maxsize=128)` for predictable memory growth on long-running processes (#459)
- `get_backend` now delegates to `client_for_model_config`, so the live-test path and production path share one missing-API-key validation (#459)
- Blocked Gemini responses (`SAFETY`, `RECITATION`, `PROHIBITED_CONTENT`, `BLOCKLIST`) raise `LLMError` in the streaming path too (previously only the non-streaming path), ensuring retry/fallback logic fires uniformly (#459)
- Transport-change env overrides now strip transport-specific thinking params (thinking_budget_tokens vs. reasoning_effort) during config merge, including at the dialectic-level merge, so switching from Anthropic → OpenAI doesn't leave orphaned Anthropic-only params that the OpenAI backend would reject (#459)
- `max_tool_iterations` out-of-range inputs now raise `ValidationException` instead of being silently clamped (#459)
- Public API schemas (`WorkspaceCreate`, `PeerCreate`, `SessionCreate`) and SDK validation (`api_types.py`, `validation.ts`) accept IDs up to 512 chars (was 100) (#684)
- Peer card prompts reframed as stable identity markers (replaces the prior "biographical/profile facts" language). Induction specialist is now opted out of peer card writes (`can_update_peer_card = False`) so only deduction touches the card (#686)
- Vector store queries no longer fetch embedding vectors — only document metadata is returned, reducing payload size and DB load (pgvector, lancedb, turbopuffer) (#682)
- Langfuse trace metadata now includes `namespace`, `model`, and `provider` so traces can be filtered by deployment slice (#565)
- Deriver: model-aware tokenizer (replaces the previously hardcoded encoding) and explicit guard on empty message content (#647)
- Dialectic level defaults now merge correctly with per-level overrides in `src/config` (#656)
- Default dialectic tool choice switched from forced/required to `auto` (#630)
- Vector sync given a substantial retry budget to tolerate transient embedding provider outages (#604)
- `AgentToolConclusionsDeletedEvent` payload now carries `levels` for parity with the rest of the conclusion event surface (#612)
- Turbopuffer vector store: `InternalServerError` caught and surfaced as a warning rather than a hard failure; unused `upsert_with_retry` and `VectorUpsertResult` removed; explicit silent and explicit-error paths for vector DB server errors (#561)
- Troubleshooting docs updated to reflect nested-env-var form for per-component thinking-budget overrides (#459)
- README refresh (#681)
- CLAUDE.md refreshed against the current `src/` layout (#680)
### Fixed
- Fallback `ModelConfig` temperature and `thinking_budget_tokens` reach the backend on the final retry — previously the primary's values were pre-populated into caller kwargs early and clobbered fallback values via `effective_config_for_call(update=...)` (#459)
- Stream-final retries pin to the `AttemptPlan` that succeeded rather than re-running provider selection through the outer `current_attempt` ContextVar (which could roll streaming back to primary after the tool loop had already switched to fallback) (#459)
- OpenAI structured-output calls continue to use `chat.completions.parse()` with strict schema enforcement, while tool-calling paths use `chat.completions.create()` without `strict:True` for broader proxy compatibility (OpenRouter, vLLM, Ollama) (#459)
- Gemini `cached_content` reuse keys now include `system_instruction` and `tool_config` so cache hits don't cross configurations that differ only in those fields (#459)
- Removed strict parameter validation for thinking params on Anthropic and OpenAI transports — was rejecting valid per-transport configs (#686)
- `reverse` query parameter is now honored on the v3 workspace list (`POST /v3/workspaces/list`), peer list (`POST /v3/workspaces/{workspace_id}/peers/list`), workspace-scoped session list (`POST /v3/workspaces/{workspace_id}/sessions/list`), and peer-scoped session list (`POST /v3/workspaces/{workspace_id}/peers/{peer_id}/sessions`). Honcho SDKs at 2.1.0+ were already sending `reverse=true` for these routes but the server silently ignored it. Ties on `created_at` now fall back to the internal nanoid `id` so ordering remains stable across pages (#685)
- LLM client factories now receive `base_url` from `LLMSettings` for default providers — previously the override path honored `base_url` but the default path didn't, so operators pointing at OpenAI-compatible proxies via `LLM__OPENAI_BASE_URL` were ignored (#643, fixes #641)
- Internal N+1 query in dialectic agent tool execution — collapsed per-iteration DB lookups into a single fetch (#652)
- Dreamer threshold and time-guard semantics: `check_and_schedule_dream` count filter now includes only `documents.level == 'explicit'` (dreamer-created levels are output, not input, and were inflating the threshold and creating a feedback loop); `last_dream_at` write relocated from `enqueue_dream` into `process_dream` so duplicate enqueues or failed runs no longer reset the 8-hour time guard (#573)
- Deriver: blank observations are filtered out before embedding (previously triggered noisy embedding calls and persisted empty rows); blank-observation filtering unified across tool paths (#615)
- Surprisal module: filter for level observations changed from `{"level": levels}` to `{"level": {"in": levels}}` — `apply_filter()` requires operator syntax, so the prior call silently returned 0 results and made the entire Surprisal phase of the Dream cycle a no-op (#581, fixes #559)
- Removed stale `stop_sequences` from tests (#607)
- Embedding client: `embed()` now wraps single-string input in an array, restoring compatibility with OpenAI-compatible third-party providers that reject scalar input (#586)
- Docker Compose: deriver service startup gated on the API service healthcheck (prevents races where the deriver starts before the API has run migrations) (#689)
- Docker image: `HEALTHCHECK` directive removed from the shared base image — it probed an HTTP endpoint only the API serves, permanently marking deriver containers as unhealthy. Service-level health checks now belong in each service's own configuration (k8s readiness/liveness probes on the API Deployment only) (#530)
- `tests/unified`: `--test-dir`/`--test-file` arguments now use an argparse mutually-exclusive group instead of manual validation (#650)
- CrewAI example updated for the latest CrewAI protocol (#631)
### Removed
- `src/utils/clients.py` deleted; its responsibilities are split across `src/llm/registry.py`, `src/llm/credentials.py`, and the backend-specific modules (#459)
- `HEALTHCHECK` directive removed from the shared Docker image (#530)
## [3.0.6] - 2026-04-10
### Changed
- Tightened transaction scopes across search, agent tools, queue manager, and webhook delivery to minimize DB connection hold time during external operations (#525)
- Search operations refactored to two-phase pattern — external work (embeddings, LLM calls) completes before opening a transaction (#525)
- Agent tool executor performs external operations before acquiring DB sessions (#525)
- Queue manager transaction scope reduced to only the critical section (#525)
- Webhook delivery no longer holds a DB session parameter (#525)
### Fixed
- Session leakage in non-session-scoped dialectic chat calls (#526)
### Added
- Health check endpoint (`/health`) for container orchestration and load balancer probes (#510)
## [3.0.5] - 2026-04-03
### Fixed
- explicit rollback on all transactions to force connection closed
## [3.0.4] - 2026-04-02
### Added
- JSONB metadata validation enforces 100 key limit and max depth of 5 (#419)
### Changed
- Schemas refactored from single `schemas.py` into `schemas/api.py`, `schemas/configuration.py`, and `schemas/internal.py` with backwards-compatible re-exports (#419)
### Fixed
- Missing `deleted_at` filter on `RepresentationManager._query_documents_recent()` and `._query_documents_most_derived()` allowed soft-deleted documents to leak into the deriver's working representation (#456)
- `CleanupStaleItemsCompletedEvent` emitted spuriously when no queue item was actually deleted (#454)
- Memory leak: `_observation_locks` switched to `WeakValueDictionary` to prevent unbounded growth (#419)
- SQL injection in `dependencies.py`: parameterized `set_config` calls to prevent injection via request context (#419)
- NUL byte crashes: string inputs (message content, queries, peer cards) now stripped at schema level (#419)
- Filter recursion depth capped at 5 to prevent stack overflow (#419)
- Dedup-skipped observations now correctly reflected in created counts (#477)
- External vector store support for message search — routes queries through configured external vector store with oversampling and
deduplication to handle chunked embeddings (#479)
- Dialectic agent no longer holds a DB connection during LLM calls — embeddings are pre-computed before tool execution, DB sessions isolated in `extract_preferences`, `query_documents` no longer accepts a DB session parameter (#477)
## [3.0.3] - 2026-02-25
### Added
@ -454,7 +761,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
### Changed
- `/list` endpoints to not require a request body
- `metamessage_type` to `label` with backwards compatability
- `metamessage_type` to `label` with backwards compatibility
- Database Provisioning to rely on alembic
- Database Session Manager to explicitly rollback transactions before closing
the connection
@ -628,7 +935,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
- Authentication Middleware now implemented using built-in FastAPI Security
module
- Get by name routes for users and collections now include "name" in slug
- Python SDK moved to separate [respository](https://github.com/plastic-labs/honcho-python)
- Python SDK moved to separate [repository](https://github.com/plastic-labs/honcho-python)
### Fixed
@ -699,7 +1006,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
### Changed
- session_data is now metadata
- session_data is a JSON field used python `dict` for compatability
- session_data is a JSON field used python `dict` for compatibility
@ -10,7 +10,7 @@ Honcho is an infrastructure layer for building AI agents with memory and social
- Imbuing agents with a sense of identity
- Personalizing user experiences through understanding user psychology
- Providing a Dialectic API that injects personal context just-in-time
- Providing a Chat Endpoint (the Dialectic agent) that injects personal context just-in-time
- Supporting development of LLM-powered applications that adapt to end users
- Enabling multi-peer sessions where multiple participants (users or agents) can interact
@ -32,23 +32,25 @@ Honcho uses a peer-based model where both users and agents are represented as "p
- **Peer** (formerly User): Any participant in the system (human or AI)
- **Session**: A conversation context that can involve multiple peers
- **Message**: Data units that can represent communication between peers OR arbitrary data ingested by a peer to enhance its global representation
- **Collections & Documents**: Internal vector storage for peer representations (not exposed via API)
- **Collections & Documents**: Internal vector storage for peer representations.Collections are keyed by `(observer, observed)` peer pairs. Collections/Documents are not directly exposed via API, but the observations stored within them are exposed as **Conclusions** (see `/v3/.../conclusions` endpoints).
## Architecture Overview
### API Structure
All API routes follow the pattern: `/v1/{resource}/{id}/{action}`
All API routes follow the pattern: `/v3/{resource}/{id}/{action}`. Most "list/search" endpoints are `POST` so they can accept rich filter bodies.
- Provides bespoke responses informed by the representation
- Integrates long-term facts from vector storage
@ -114,128 +116,170 @@ cd sdks/typescript && bun run tsc --noEmit
- Explicit error handling with appropriate exception types
- Docstrings: Use Google style docstrings
- **Never hold a DB session during external calls** (LLM, embedding, HTTP). If a function needs both a DB session and an external call result, compute the external result first and pass it as a parameter. This avoids tying up DB connections during slow network I/O. Use `tracked_db` for short-lived, DB-only operations; pass a shared session when multiple DB-only calls can reuse one connection.
- **Never write through a read-only session** (`tracked_db(..., read_only=True)`, `get_read_db`, `ReadSessionLocal`). These run in AUTOCOMMIT mode with no transaction: writes are NOT blocked by the database — they silently commit immediately, and `begin_nested()` savepoints break. There is no runtime guard; this is enforced by convention only. Use `read_only=True` strictly for SELECT-only windows; anything that mutates (including get-or-create paths) must use a regular write session.
#### Multi-row locking and deadlocks
Tables written concurrently by more than one worker — `documents` (deriver, dreamer, scope backfill/removal, reconciler) and `queue` (every deriver replica) — deadlock when two writers touch an overlapping row set in different orders. Rules:
- **A multi-row `SELECT ... FOR UPDATE` MUST carry an explicit `ORDER BY <pk>`.** Without it Postgres locks in scan order, which differs per plan, so two writers with overlapping sets can cycle. `_apply_document_row_updates` in `src/crud/document.py` is the reference implementation.
- **`WHERE id IN (...)` does NOT impose an order**, so sorting the Python list is a no-op — the list order is discarded and the planner picks `Bitmap Heap Scan` (ctid order), `Index Scan` (id order), or `Seq Scan` per invocation. Deterministic ordering requires either a preceding `SELECT ... ORDER BY id FOR UPDATE` or `WHERE id IN (SELECT id ... ORDER BY id FOR UPDATE)`.
- **`Document.id` is a random nanoid** (`models.py`), so id order is uncorrelated with physical order — an unordered predicate `UPDATE`/`DELETE` is roughly a coin flip against an id-ordered locker per row pair, not a rare edge case. (`QueueItem.id` is an integer identity, so there id order is also chronological.)
- **Prefer no lock at all.** A single `UPDATE ... WHERE <predicate>` acquires row locks as it writes and has no separate lock phase to get wrong. Reach for `FOR UPDATE` only when a value must be read, computed in Python, and written back — that read-modify-write is the only reason `_apply_document_row_updates` locks (it replaced a server-side `func.greatest()`), and `populate_existing=True` is required with it so the identity map doesn't serve a stale pre-lock value. Server-side expressions (`func.greatest`, the JSONB `-` operator) avoid the lock entirely; see `_clear_work_unit_retry_attempts` in `src/deriver/queue_manager.py`.
- `FOR UPDATE SKIP LOCKED` (the reconciler's claim pattern) never waits, so it cannot be a deadlock partner — but holding those locks across an external call still stalls other writers. See the "never hold a DB session during external calls" rule above.
#### Auth scoping
- **`allow_member_read=True` (in `require_auth(...)`) is read-only — NEVER set it on a route that mutates state.** It lets a peer-scoped key reach a session route when its peer is an active member of the session, so on a mutating route it would hand any session member write access (message injection, config mutation, deletion). HTTP method is not a reliable read/write signal here (some read routes use POST for a richer body), so this is enforced by an explicit allowlist in `tests/routes/test_auth_route_policy.py` — adding the flag to a new route fails that test until you consciously add the route to `EXPECTED_MEMBER_READ_ROUTES`, and you must never add a mutating method there.
- **When a member-read route is keyed by another sub-resource** (e.g. `peers/{peer_id}/config`), the handler must additionally confirm a peer-scoped caller only reads its OWN resource (`jwt_params.p == peer_id`, else raise `AuthenticationException`). Membership grants session access, not access to a co-member's data. See `get_peer_config` in `src/routers/sessions.py`.
### Runtime Architecture
Honcho runs as two cooperating processes that share a Postgres database and Redis cache:
- **API server** (`uv run fastapi dev src/main.py`) — handles HTTP, enqueues background work, returns immediately. Hosts the **Dialectic** agent inline (synchronous tool loop during chat requests).
- **Deriver worker** (`uv run python -m src.deriver`) — long-running queue consumer (uvloop). Runs the **Deriver**, **Summarizer**, and **Dreamer** off the queue. Can run multiple instances (`DERIVER_WORKERS`). Also hosts an in-process **Reconciler scheduler** (`src/reconciler/`) that periodically embeds messages with `sync_state='pending'` in `MessageEmbedding` and cleans up stale queue items — embedding generation is decoupled from message creation by design.
### Agent Architecture
Honcho uses three specialized LLM agents that work together to form memories and answer queries:
Honcho uses several specialized LLM agents. They share tool definitions and the LLM client abstraction in `src/utils/agent_tools.py` + `src/llm/`.
#### 1. Deriver Agent (`src/deriver/agent/`)
> **Terminology:** what users see as **conclusions** (the public API surface and the term we use in documentation) is called **observations** in code symbols — `create_observations`, `delete_observations`, `get_observation_context`, etc. Doc prose below uses "conclusions"; references to actual code symbols stay as "observations."
**Role**: Memory formation through content ingestion
#### 1. Deriver (`src/deriver/`)
The Deriver processes incoming messages and extracts observations about peers.
**Role**: Memory formation through content ingestion.
- **Trigger**: Messages created via API are enqueued for background processing
The Deriver processes batches of incoming messages and extracts conclusions about peers. The current architecture is "minimal deriver" — a **single LLM call** per batch using structured output, not an agentic tool loop. This trades flexibility for cost and predictability.
#### 2. Dialectic Agent (`src/dialectic/agent/`)
- **Trigger**: Messages enqueued by `src/deriver/enqueue.py` on message create; consumed by `src/deriver/queue_manager.py` → `consumer.process_item()` → `deriver.process_representation_tasks_batch()`.
- **Output**: Explicit conclusions (direct facts) and deductive conclusions (inferences) saved to `(observer, observed)` collections.
- **Custom instructions**: per-workspace/peer guidance can be threaded into the prompt via reasoning configuration; `DERIVER__MAX_CUSTOM_INSTRUCTIONS_TOKENS` caps the addition (default 2000) and `DERIVER__MAX_INPUT_TOKENS` defaults to 25000 to make room.
**Role**: Analysis and recall for answering queries
#### 2. Dialectic (`src/dialectic/`)
The Dialectic answers questions about peers by strategically gathering context from memory.
**Role**: Analysis and recall for answering queries.
- **Trigger**: API call to `/peers/{peer_id}/chat` with `agentic=true`
The Dialectic answers questions about peers by strategically gathering context from memory. It is the only tool-using agent on the synchronous request path — it loops over `DIALECTIC_TOOLS` until it has enough context to answer. (The Dreamer specialists also use tools, but run off the queue.)
#### 3. Dreamer Agent (`src/dreamer/agent.py`)
- **Trigger**: API call to `POST /v3/.../peers/{peer_id}/chat`.
- **Tools** (see `DIALECTIC_TOOLS` in `src/utils/agent_tools.py`): `search_memory`, `search_messages`, `get_observation_context`, `grep_messages`, `get_messages_by_date_range`, `search_messages_temporal`, `get_reasoning_chain`. At the `minimal` reasoning level, a reduced set (`DIALECTIC_TOOLS_MINIMAL`) is used: just `search_memory` + `search_messages`.
- **Reasoning levels**: 5 tiers — `minimal`, `low`, `medium`, `high`, `max` — each with its own model config (see `DialecticLevelSettings` in `src/config.py`).
- **Output**: Natural language response grounded in gathered context. Supports SSE streaming.
The Dreamer is an orchestrated multi-specialist system that runs during scheduled "dreams" to consolidate conclusions and build reasoning trees.
- **Trigger**: Scheduled via `DreamScheduler` (`src/dreamer/dream_scheduler.py`) or explicit dream task on the queue.
- **Strategy**: Surprisal-based prioritization (`src/dreamer/surprisal.py`) selects which conclusions to expand. The orchestrator (`orchestrator.run_dream`) runs two specialist phases:
2. **InductionSpecialist** — produces inductive conclusions from explicit + deductive conclusions. Tools: same discovery set + `create_observations_inductive`, `update_peer_card`.
- **Reasoning trees** (`src/dreamer/trees/`, migration `f1a2b3c4d5e6_add_reasoning_tree_columns`): each conclusion links to its premises and downstream conclusions, enabling `get_reasoning_chain` traversal at recall time.
- **Entry point**: `src/dreamer/orchestrator.py` → `process_dream()` (the package-level export from `src/dreamer/__init__.py`), which wraps `run_dream()`.
- **Trigger**: Runs as part of the queue pipeline alongside representation tasks.
- **Tiers**: short summary every `SUMMARY_MESSAGES_PER_SHORT_SUMMARY` messages (default 20); long summary every `SUMMARY_MESSAGES_PER_LONG_SUMMARY` (default 60). Token caps configurable via `SUMMARY_MAX_TOKENS_SHORT` / `SUMMARY_MAX_TOKENS_LONG`.
#### Shared Agent Infrastructure
All agents share common infrastructure in `src/utils/agent_tools.py`:
- **Tool definitions**: Unified tool schemas used by all agents
- **LLM subsystem** (`src/llm/`): provider-agnostic `honcho_llm_call()`. Backends in `src/llm/backends/` (`anthropic.py`, `gemini.py`, `openai.py`). Includes prompt caching (`caching.py`), structured output (`structured_output.py`), tool loop (`tool_loop.py`), history adapters for cross-provider message formats, and a model registry. Per-retry provider selection is pinned via an `AttemptPlan` so stream-final retries don't bounce back to primary after the tool loop has settled on fallback.
- **Per-agent model config**: each agent has its own `MODEL_CONFIG` in `src/config.py` with fallback chains (see `ConfiguredModelSettings`, `FallbackModelSettings`).
- **Telemetry**: cloudevents in `src/telemetry/events/` cover API routes, dialectic, dream, deletion, reconciliation, representation, and per-call LLM accounting (`llm.py` — `LLMCallCompletedEvent` fires once per provider hit with full cost-attribution context). High-volume events are sampled deterministically per `run_id` via `TelemetrySettings.HIGH_VOLUME_SAMPLE_RATE`.
- **Prometheus metrics** (`src/telemetry/prometheus/`): every metric carries a `namespace` label and every recorder is fail-soft (a metrics error never propagates into a request or a worker loop). Counter children with a *bounded* label domain are zero-initialized per process at startup — `initialize_bounded_metrics(instance_type=...)`, called from the `src/main.py` lifespan (`api`) and `src/deriver/__main__.py` (`deriver`) — so an absent series means a broken scrape rather than "nothing happened". Two consequences worth knowing before touching telemetry:
- **Adding a `BaseEvent` subclass requires adding its `_event_type` to `ALL_EVENT_TYPES`** in `src/telemetry/events/__init__.py` (and to `HIGH_VOLUME_EVENT_TYPES` if `_volume_class == "high_volume"`). Enforced by the drift guards in `tests/telemetry/test_metric_zero_init.py`, which assert set-equality against the discovered subclasses.
- **A service-wide, non-additive gauge must be refreshed by every replica on its own timer**, and aggregated with `max()`/`avg()`, never `sum()`. `message_embeddings_pending` is the example: it reports a DB-global count, so it is driven from `ReconcilerScheduler._scheduler_loop` (runs on all replicas) rather than from the work-unit-deduped reconciliation cycle — otherwise, combined with the zero-init, every replica that never won the work unit would export a confident permanent `0`.
### Project Structure
```
src/
├── main.py # FastAPI app setup with middleware and exception handlers
├── models.py # SQLAlchemy ORM models with proper type annotations
├── schemas.py # Pydantic validation schemas for API
├── config.py # Configuration management
├── db.py # Database connection and session management
│ ├── representation.py # Representation formatting (distinct from crud/representation.py)
│ ├── search.py, filter.py, formatting.py
│ ├── tokens.py # tiktoken-based counting
│ ├── work_unit.py, queue_payload.py
│ ├── config_helpers.py, json_parser.py, files.py
│ └── types.py
└── webhooks/ # Webhook delivery
├── events.py
└── webhook_delivery.py
```
- Tests in pytest with fixtures in tests/conftest.py
- Use environment variables via python-dotenv (.env)
- Tests in pytest with fixtures in tests/conftest.py; subdirs mirror src/ (`tests/deriver/`, `tests/dialectic/`, etc.) plus `tests/bench/` (perf benchmarks), `tests/integration/`, `tests/live_llm/` (gated by `--live-llm`), and `tests/unified/` (the unified runner).
- Use environment variables via python-dotenv (.env). Config precedence: env > .env > config.toml > defaults.
### Database Design
@ -248,12 +292,18 @@ src/
### Key Architectural Decisions
1. **Multi-Peer Sessions**: Sessions can have multiple participants with different observation settings
3. **Background Processing**: Async queue system for expensive operations
4. **Provider Abstraction**: Model client supports multiple LLM providers
5. **Scoped Authentication**: JWTs can be scoped to workspace, peer, or session level
6. **Batch Operations**: Support for bulk message creation (up to 100 messages)
7. **Session History**: Two-tier summarization (short every 20 messages, long every 60)
1. **Peer Paradigm**: humans and AI agents are unified as "Peers"; many-to-many with Sessions. Internal vector storage (Collections/Documents) is keyed by `(observer, observed)` peer pairs — the same mechanism powers self-representation (`observer == observed`) and cross-peer modeling.
2. **Multi-Peer Sessions**: Sessions can have multiple participants with different observation settings.
3. **API server / worker split**: API enqueues, deriver worker process consumes. Never block HTTP on LLM work. The Reconciler runs as an in-process scheduler inside the deriver, handling async embedding sync and queue cleanup.
4. **"Minimal" deriver**: memory formation is a single structured-output LLM call per batch, not an agentic tool loop. Predictable cost, lower latency. The Dialectic is the one true tool-using agent.
5. **Provider-agnostic LLM layer** (`src/llm/`): all model calls go through `honcho_llm_call()`. Backends (`anthropic`, `gemini`, `openai`) sit behind a registry; per-agent `MODEL_CONFIG` with fallback chains is resolved at call time.
6. **Dialectic reasoning tiers**: 5 levels (`minimal` → `max`); each level has its own model config and tool set (`minimal` uses a reduced toolset).
7. **Hybrid search**: Postgres FTS (GIN index on `to_tsvector('english', content)`) + vector similarity (HNSW on `MessageEmbedding.embedding`). `MessageEmbedding` is a separate table from `Message` with its own `sync_state` so embedding is decoupled from message creation.
8. **Pluggable external vector stores**: defaults to pgvector inline; can swap to turbopuffer or lancedb (`VECTOR_STORE_*` config; `src/vector_store/`).
9. **Composite-FK multi-tenancy**: `workspace_name` participates in nearly every composite FK. Cross-workspace data leakage is structurally impossible at the schema level.
10. **Scoped Authentication**: JWTs can be scoped to workspace, peer, or session level.
11. **Batch Operations**: Bulk message creation up to 100 messages per request.
12. **Session History**: Two-tier summarization — short every `SUMMARY_MESSAGES_PER_SHORT_SUMMARY` (default 20), long every `SUMMARY_MESSAGES_PER_LONG_SUMMARY` (default 60).
Thank you for your interest in contributing to Honcho! This guide outlines the process for contributing to the project and our development conventions.
<!-- This file is mirrored at docs/v3/contributing/guidelines.mdx. Update both. -->
## Getting Started
Thanks for your interest in contributing. This guide covers how work gets accepted, how
Honcho is put together, and what a mergeable pull request looks like.
Before you start contributing, please:
Honcho is a small team maintaining a project that gets more proposals than we can review.
The rules below exist so that the work you do has somewhere to land — not to keep you out.
1. **Set up your development environment** - Follow the [Local Development guide](./README.md#local-development) in the README to get Honcho running locally.
## Contents
2. **Join our community** - Feel free to join us in our [Discord](http://discord.gg/plasticlabs) to discuss your changes, get help, or ask questions.
- [Before you write code](#before-you-write-code)
- [What gets prioritized](#what-gets-prioritized)
- [If you're an agent](#if-youre-an-agent)
- [How Honcho works](#how-honcho-works)
- [Where to change what](#where-to-change-what)
- [Local setup](#local-setup)
- [Making the change](#making-the-change)
- [Opening the pull request](#opening-the-pull-request)
- [Reporting bugs and requesting features](#reporting-bugs-and-requesting-features)
- [Security](#security)
- [License](#license)
3. **Review existing issues** - Check the [issues tab](https://github.com/plastic-labs/honcho/issues) to see what's already being worked on or to find something to contribute to.
## Before you write code
## Contribution Workflow
**Every pull request needs an issue, and that issue needs the `maintainer-approved` label.**
### 1. Fork and Clone
A pull request that is not linked to an approved issue gets labelled
`needs-approved-issue`, with a comment explaining why. You then have 72 hours to link one
before it is closed automatically. Reopening costs nothing once the link is in place. This
is automated. We do this because an unreviewable backlog helps nobody: a PR against an
unapproved issue is work you did that we may not be able to merge, no matter how good it
| Does | Serves HTTP, enqueues background work, returns immediately | Consumes the queue: Deriver, Summarizer, Dreamer, Reconciler |
| Hosts | The Dialectic agent, inline on the request path | Everything else |
The split is the load-bearing design decision: **an HTTP request never blocks on LLM work**,
with the single exception of the Dialectic chat endpoint, which is synchronous by nature.
If you are adding something slow, it belongs in the worker.
The deriver is a separate process. If messages go in and nothing ever comes out, the usual
cause is that nobody started it.
### The path of a message
Worth tracing once, because it crosses most of the codebase:
1. `POST /v3/workspaces/{w}/sessions/{s}/messages` lands in `src/routers/messages.py`.
2. The row is written, then `enqueue()` in `src/deriver/enqueue.py` creates `queue_item`
rows — one set of work per observing peer.
3. `src/deriver/queue_manager.py` polls the queue, claiming work units so that messages in a
session are processed in order.
4. `process_item()` in `src/deriver/consumer.py` dispatches on task type — representation,
summary, deletion, reconciliation.
5. For a representation task, `process_representation_tasks_batch()` in
`src/deriver/deriver.py` makes **one structured-output LLM call for the whole batch** and
writes the resulting conclusions into the collection keyed by the
`(observer, observed)` peer pair.
6. Later, `src/dialectic/` reads those conclusions back at recall time to answer a chat
request.
Embedding is deliberately *not* on this path. `MessageEmbedding` rows are written with
`sync_state='pending'` and embedded asynchronously by the Reconciler
(`src/reconciler/sync_vectors.py`), which runs on a scheduler inside the deriver process.
### The four agents
They share tool definitions in `src/utils/agent_tools.py` and the provider-agnostic LLM
client in `src/llm/`. Each has its own `MODEL_CONFIG` with a fallback chain in
`src/config.py`.
| Agent | Where | Shape |
| --- | --- | --- |
| **Deriver** | `src/deriver/` | A single structured-output call per message batch. Not a tool loop — this is a deliberate cost and latency tradeoff. |
| **Dialectic** | `src/dialectic/` | The one tool-using agent on the request path. Loops over tools until it can answer. Five reasoning tiers from `minimal` to `max`, each with its own model and tool set. |
| **Dreamer** | `src/dreamer/` | Off-queue consolidation. Two specialist phases (deduction, then induction) that build reasoning trees over existing conclusions. |
| **Summarizer** | `src/utils/summarizer.py` | Direct LLM call, no tools. Two tiers — short and long summaries at different message counts. |
Prompts live in `src/deriver/prompts.py`, `src/dialectic/prompts.py`, and
`src/dreamer/specialists.py`.
### A note on naming
What the public API and documentation call **conclusions** are called **observations**
throughout the code — `create_observations`, `get_observation_context`, and so on. Likewise
**collections** and **documents** are internal storage concepts that are not exposed
directly through the API. Do not rename across that boundary in a drive-by change; the
public and internal vocabularies are being reconciled deliberately.
## Where to change what
| I want to change... | Start here |
| --- | --- |
| An HTTP endpoint | `src/routers/` — one module per resource |
| A database query | `src/crud/` — mirrors the router layout |
| The database schema | `src/models.py`, plus a migration in `migrations/versions/` |
| A configuration value | `src/config.py`, and add it to `config.toml.example` and `.env.template` |
| A tool an agent can call | `src/utils/agent_tools.py` — definitions plus the per-agent tool lists |
| A prompt | `src/deriver/prompts.py`, `src/dialectic/prompts.py`, `src/dreamer/specialists.py` |
| Embeddings or vector storage | `src/embedding_client.py`, `src/vector_store/` |
| Telemetry or metrics | `src/telemetry/` — see the notes in `CLAUDE.md` before adding an event type |
| Authentication and scoping | `src/security.py`, `src/dependencies.py` |
| The Python or TypeScript SDK | `sdks/python/`, `sdks/typescript/` |
| The CLI | `honcho-cli/` |
| The MCP server | `mcp/` |
| Public documentation | `docs/v3/` — Mintlify; nav lives in `docs/docs.json` |
Tests in `tests/` mirror `src/`. `CLAUDE.md` at the repo root has more detail on house
conventions, and is worth skimming even if you are not using an agent.
## Local setup
To run a personal instance, install the CLI (`uv tool install honcho-cli`) and then run `honcho start --setup` (Docker + an LLM provider key — not the Honcho API key from `honcho init`) — [CLI in the README](./README.md#cli).
To **develop this repo**, clone it and:
```bash
uv sync # create the venv and install dependencies
uv run alembic upgrade head # apply migrations
```
Run both processes, in separate terminals:
```bash
uv run fastapi dev src/main.py # API server, reloads on change
uv run python -m src.deriver # background worker
```
Everything Python goes through `uv run`. Redis is optional for local development; without it
caching is simply disabled.
### Running without a model provider
`src/mock_provider/` is a deterministic, OpenAI-compatible endpoint, so you can run the full
stack with no provider account, no API key, and no spend. It answers `/v1/chat/completions`
and `/v1/embeddings` with obviously-synthetic content derived from the request, and the same
request always produces the same response. Run it from the standard image or the repo:
```bash
uv run fastapi run --host 0.0.0.0 --port 8106 src/mock_provider/main.py
```
Then point Honcho at it. All three variables are required:
```bash
export LLM_OPENAI_API_KEY=any-non-empty-string # only truthiness is checked
Honcho is an open source memory library with a managed service for building stateful
agents. Use it with any model, framework, or architecture. It enables agents to build
and maintain state about any entity--users, agents, groups, ideas, and more. And because
it's a continual learning system, it understands entities that change over time. Using
Honcho as your memory system will earn your agents higher retention, more trust, and
help you build data moats to out-compete incumbents.
**Honcho is memory infrastructure for building stateful agents that understand changing people, agents, groups, projects, and ideas over time.**
> Honcho has defined the Pareto Frontier of Agent Memory. Watch the [video](https://x.com/honchodotdev/status/2002090546521911703?s=20), check out our [evals page](https://evals.honcho.dev/), and read the [blog post](https://blog.plasticlabs.ai/research/Benchmarking-Honcho) for more detail.
Store messages and events, let Honcho reason in the background, then query peer representations, session context, search results, or natural-language insights from any model or framework. Use it managed at [api.honcho.dev](https://api.honcho.dev), run a local stack with [`honcho start`](#cli), or self-host the FastAPI server yourself.
## TL;DR - Getting Started
Using Honcho as your memory system will earn your agents higher retention, more trust, and help you build data moats to out-compete incumbents.
With Honcho you can easily setup your application's workflow, save your
interaction history, and leverage the reasoning it does to inform the behavior of
your agents
> Honcho has defined the Pareto Frontier of Agent Memory. Watch the [video](https://x.com/honchodotdev/status/2002090546521911703?s=20), check out our [evals page](https://honcho.dev/evals/), and read the [blog post](https://blog.plasticlabs.ai/research/Benchmarking-Honcho) for more detail.
> Typescript examples are available in our [docs](https://docs.honcho.dev).
## Contents
1. Install the SDK
```bash
# Python
pip install honcho-ai
uv add honcho-ai
poetry add honcho-ai
```
2. Setup your `Workspace`, `Peers`, `Session`, and send `Messages`
```python
from honcho import Honcho
# 1. Initialize your Honcho client
honcho = Honcho(workspace_id="my-app-testing")
# 2. Initialize peers
alice = honcho.peer("alice")
tutor = honcho.peer("tutor")
# 3. Create a session and add messages
session = honcho.session("session-1")
# Adding messages from a peer will automatically add them to the session
session.add_messages(
[
alice.message("Hey there — can you help me with my math homework?"),
tutor.message("Absolutely. Send me your first problem!"),
]
)
```
3. Leverage reasoning from Honcho to inform your agent's behavior
```python
### 1. Use the chat endpoint to ask questions about your users in natural language
response = alice.chat("What learning styles does the user respond to best?")
### 2. Use session context to continue a conversation with an LLM
- [Retrieving Data & Insights](#retrieving-data--insights)
- [SDKs](#sdks)
- [Learn More](#learn-more)
- [Contributing](#contributing)
- [License](#license)
The Honcho project is split between several repositories with this one hosting
the core service logic. This is implemented as a FastAPI server/API to store
data about an application's state.
The Honcho project is split between several repositories, with this one hosting the core service logic — implemented as a FastAPI server. Client SDKs for Python and TypeScript live in the [`sdks/`](./sdks) directory. The [`honcho-cli`](./honcho-cli) package lives here too.
There are also client SDKs implemented in the `sdks/` directory with support
[https://app.honcho.dev](https://app.honcho.dev) and get started with $100 free credits. When you sign up you'll be prompted to
join an organization which will have a dedicated instance of Honcho.
1. **Store** conversations, events, documents, or tool traces as messages on a session.
2. **Reason** — Honcho processes the queue in the background and updates peer representations.
3. **Query** — ask Honcho for context, search results, peer representations, or a natural-language answer.
4. **Inject** — drop the result into any LLM call or agent framework.
Provision API keys and change your base url to point to
[https://api.honcho.dev](https://api.honcho.dev)
Concretely: workspaces hold peers, peers participate in sessions, messages live on sessions, and Honcho builds a per-peer representation that you query through the [Chat Endpoint](https://honcho.dev/docs/v3/documentation/features/chat) or directly.
Additionally, Honcho can be self-hosted for testing and evaluation purposes. See
the [Local Development](#local-development) section below for details on how to set up a local
version of Honcho.
## Quickstart
## Local Development
Get an API key at [app.honcho.dev](https://app.honcho.dev) — when you sign up you'll be prompted to join an organization, which gets its own dedicated Honcho instance and $100 free credits. Or install the CLI and run [`honcho start --setup`](#cli), then point the SDK at `http://localhost:8000`.
Below is a guide on setting up a local environment for running the Honcho
Server.
### Python
> This guide was made using a M3 Macbook Pro. For any compatibility issues
> on different platforms, please raise an Issue.
```bash
pip install honcho-ai
# or: uv add honcho-ai
# or: poetry add honcho-ai
```
### Prerequisites and Dependencies
```python
import os
from honcho import Honcho
# Managed service uses api.honcho.dev by default. For self-hosted, pass
# base_url="http://localhost:8000" or set HONCHO_URL.
honcho = Honcho(
workspace_id="my-app-testing",
api_key=os.environ["HONCHO_API_KEY"],
)
# 1. Store: peers and messages on a session
alice = honcho.peer("alice")
tutor = honcho.peer("tutor")
session = honcho.session("session-1")
session.add_messages([
alice.message("Hey there — can you help me with my math homework?"),
tutor.message("Absolutely. Send me your first problem!"),
])
# 2. Reason: happens asynchronously in the background.
# 3. Query: ask Honcho what it knows, or pull prompt-ready context.
answer = alice.chat("What learning styles does the user respond to best?")
> **Note:** background reasoning is asynchronous. Newly-added messages may take a moment to be reflected in chat/representation responses; for low-latency reads, use the [`representation`](https://honcho.dev/docs/v3/documentation/features/representation) endpoint.
See the full [SDK Reference](https://honcho.dev/docs/v3/documentation/reference/sdk) and [API Reference](https://honcho.dev/docs/v3/api-reference/introduction).
## Integrations
Honcho ships a first-party memory plugin for every major coding agent. They all read the same
`~/.honcho/config.json`, so one key configures all of them — and pointing two at the same `workspace`
Get a key at [app.honcho.dev](https://app.honcho.dev), then `honcho init` (or `uv tool install honcho-cli && honcho init`) writes it to `~/.honcho/config.json` once for every integration.
### Claude Code
Two ways, depending on how deep you want to go:
**Plugin (richer integration — recommended for Claude Code users):**
Windows (PowerShell): `irm https://raw.githubusercontent.com/plastic-labs/cursor-honcho/main/install.ps1 | iex`. The installer wires global hooks and MCP config. Details: [cursor-honcho](https://github.com/plastic-labs/cursor-honcho).
A native Cordis plugin. It injects memory into the system prompt and captures new information from the session event feed. The model gets three tools — honcho_search, honcho_chat, and honcho_remember — and you can run /honcho to check status.
`openclaw honcho setup` prompts for your API key, writes the config, and optionally migrates legacy `MEMORY.md` / `USER.md` / `IDENTITY.md` files into Honcho (non-destructive — originals are never deleted). Details: [OpenClaw guide](https://honcho.dev/docs/v3/guides/integrations/openclaw) · [repo](https://github.com/plastic-labs/openclaw-honcho).
### Hermes
```bash
hermes memory setup # select "honcho", point at api.honcho.dev or your local server
For wiring the Honcho SDK into an existing application, install the integration skill — it explores your codebase, asks about integration preferences, generates the SDK setup, and verifies it works:
```bash
npx skills add plastic-labs/honcho
```
Then invoke `/honcho-integration` in Claude Code (or `/honcho-dev:integrate` via the plugin marketplace). The same command also installs the memory skills — `honcho-memory` (concepts: the recall/record loop, session and peer strategy, plus how to connect and drive an MCP-connected Honcho) and `honcho-cli` (inspecting a deployment, or running a local stack with `honcho start`). Details: [agentic development guide](https://honcho.dev/docs/v3/documentation/introduction/vibecoding).
### Other MCP clients
The same `claude mcp add` form (or its client-specific equivalent) works in any MCP-compatible client. See [MCP guide](https://honcho.dev/docs/v3/guides/integrations/mcp).
## CLI
[`honcho-cli`](https://pypi.org/project/honcho-cli/) inspects a Honcho deployment from the terminal, or runs a personal local stack with Docker.
```bash
uv tool install honcho-cli
honcho init # Honcho API key or browser login + server URL
`honcho init` authenticates the CLI against a Honcho server. `honcho start --setup` is a separate step: it writes the LLM provider key the local deriver needs and starts API + deriver + Postgres + Redis.
Full commands and local-stack details: [CLI reference](https://honcho.dev/docs/v3/documentation/reference/cli) · [`honcho-cli/README.md`](./honcho-cli/README.md). To develop the server from source, see [Self-hosting](#self-hosting).
## Core Concepts
Honcho organises everything around **peers** — humans and AI agents alike are first-class entities. The peer model enables:
- Multi-participant sessions with mixed human and AI agents
- Configurable observation settings (which peers observe which others)
- Flexible identity management for all participants
- Support for complex multi-agent interactions
Peers exchange messages within sessions; Honcho reasons over those messages to build a representation of each peer that you can query.
- **Workspace** (formerly App): top-level container; isolates data between use cases.
- **Peer** (formerly User): any participant — human user or AI agent.
- **Session**: a conversation context; many-to-many with peers.
- **Scope**: a named grouping of sessions that bounds recall (chat, representation, search) to those members.
- **Message**: an atomic data unit (peer-to-peer communication or ingested document chunk).
What you query out of Honcho:
- **Conclusions** — what Honcho has extracted about a peer (deductive and inductive). Exposed via the [conclusions API](https://honcho.dev/docs/v3/api-reference/introduction).
- **Representations** — static, low-latency snapshots of what Honcho knows about a peer (optionally session-scoped).
<summary>Internal storage (Collections & Documents)</summary>
Internally, Honcho stores peer-related observations in **collections** of vector-embedded **documents**. Collections are keyed by `(observer, observed)` peer pairs — the same mechanism powers self-representation (`observer == observed`) and cross-peer modelling (peer X's understanding of peer Y). These primitives are not exposed directly; the Conclusions API is the public surface.
</details>
<!-- markdownlint-enable MD033 -->
<!-- TODO(vineeth/marketing): write the "Honcho vs RAG / vector DB / memory-only" comparison.
Audit recommendation referenced; copy intentionally deferred to avoid inventing
positioning claims unsupported by primary sources. -->
## Benchmarks & Evals
Honcho's evals span LongMemEval, LoCoMo, and other long-conversation benchmarks. See the [evals page](https://honcho.dev/evals/), the [research blog post](https://blog.plasticlabs.ai/research/Benchmarking-Honcho), and the [Pareto-frontier announcement video](https://x.com/honchodotdev/status/2002090546521911703?s=20) for methodology and reproducible results.
## Self-hosting
Honcho is open source under AGPL-3.0. To **run** a personal instance, install the CLI (`uv tool install honcho-cli`) and then [`honcho start --setup`](#cli). The paths below are for building from source, contributing, or deploying without the CLI.
With the database set up and environment variables configured, run the migrations
@ -287,208 +510,41 @@ In a separate terminal, run:
uv run python -m src.deriver
```
The deriver generates representation, summaries, peer cards, and manages dreaming tasks. You can increase the number of deriver's to improve runtime efficiency.
The deriver generates representations, summaries, peer cards, and manages dreaming tasks. You can increase the number of derivers to improve runtime efficiency.
### Pre-commit Hooks
</details>
<!-- markdownlint-enable MD033 -->
Honcho uses pre-commit hooks to ensure code quality and consistency across the project. These hooks automatically run checks on your code before each commit, including linting, formatting, type checking, and security scans.
#### Installation
To set up pre-commit hooks in your development environment:
1. **Install pre-commit using uv**
```bash
uv add --dev pre-commit
```
2. **Install the pre-commit hooks**
```bash
uv run pre-commit install \
--hook-type pre-commit \
--hook-type commit-msg \
--hook-type pre-push
```
This will install hooks for `pre-commit`, `commit-msg`, and `pre-push` stages.
You can run the hooks manually on all files without making a commit:
```bash
uv run pre-commit run --all-files
```
Or run specific hooks:
```bash
uv run pre-commit run ruff --all-files
uv run pre-commit run basedpyright --all-files
```
### Docker
As mentioned earlier a `docker-compose` template is included for running Honcho.
As an alternative to running Honcho locally it can also be run with the compose
template.
The docker-compose template is set to use an environment file called `.env`.
You can also copy the `.env.template` and fill with the appropriate values.
Copy the template and update the appropriate environment variables before
launching the service:
```bash
cd honcho
cp .env.template .env
# update the file with openai key and other wanted environment variables
cp docker-compose.yml.example docker-compose.yml
docker compose up
```
### Deploy on Fly
The API can also be deployed on fly.io. Follow the [Fly.io
Docs](https://fly.io/docs/getting-started/) to setup your environment and the
`flyctl`.
A sample `fly.toml` is included for convenience.
> Note: The fly.toml does not include launching a Postgres database. This must
> be configured separately
Once `flyctl` is set up use the following commands to launch the application:
```bash
cd honcho
flyctl launch --no-deploy # Follow the prompts and edit as you see fit
cat .env | flyctl secrets import # Load in your secrets
flyctl deploy # Deploy with appropriate environment variables
```
Contributors: see [`CONTRIBUTING.md`](./CONTRIBUTING.md) for pre-commit setup. Deploying to Fly.io: see [Self-hosting docs → Deploying on Fly.io](https://honcho.dev/docs/v3/contributing/self-hosting#deploying-on-fly-io).
## Configuration
Honcho uses a flexible configuration system that supports both TOML files and environment variables. Configuration values are loaded in the following priority order (highest to lowest):
Honcho uses a flexible configuration system that supports both TOML files and environment variables. Configuration values are loaded in priority order: **environment variables > `.env` file > `config.toml` > defaults**.
1. Environment variables
2. `.env` file (for local development)
3. `config.toml` file
4. Default values
### Using config.toml
Copy the example configuration file to get started:
Copy the example file to get started:
```bash
cp config.toml.example config.toml
```
Then modify the values as needed. The TOML file is organized into sections:
The file is organized by subsystem — `[app]`, `[db]`, `[auth]`, `[cache]`, `[llm]`, `[deriver]`, `[dialectic]`, `[summary]`, `[dream]`, `[peer_card]`, `[webhook]`, `[metrics]`, `[telemetry]`, `[vector_store]`, and `[sentry]`. Any value can be overridden by an environment variable named `{SECTION}_{KEY}`, using `__` for nesting (`DIALECTIC_LEVELS__low__MODEL_CONFIG__MODEL`), or just `{KEY}` for app-level settings.
The application will use the production connection URI while keeping the pool size from config.toml.
See the [configuration reference](https://honcho.dev/docs/v3/contributing/configuration) for every available option, and [`.env.template`](./.env.template) for an annotated list of environment variables.
## Architecture
The functionality of Honcho can be split into two different services: Storage
and Insights.
Honcho splits into two services: **Storage** (workspaces, peers, sessions, scopes, messages, internal collections) and **Insights** (reasoning, conclusions, representations, summaries, the chat endpoint). Storage is synchronous via the API; Insights is asynchronous via a background queue consumed by the deriver worker process.
### Peer Paradigm
**Key features:**
Honcho uses an entity-centric model where both users and agents are represented as "[peers](https://blog.plasticlabs.ai/blog/Beyond-the-User-Assistant-Paradigm;-Introducing-Peers)". This unified approach enables:
- **Rich Reasoning System** — multiple implementation methods that extract conclusions from interactions and build comprehensive representations of peers
- **Chat Endpoint** — reasoning-informed responses that integrate conclusions with current context
- **Background Processing** — asynchronous processing pipeline for expensive operations like representation updates and session summarization
- **Multi-Provider Support** — configurable LLM providers for different use cases
- Multi-participant sessions with mixed human and AI agents
- Configurable observation settings (which peers observe which others)
- Flexible identity management for all participants
- Support for complex multi-agent interactions
#### Key Features
- **Rich Reasoning System**: Multiple implementation methods that extract conclusions from interactions and build comprehensive representations of peers
- **Chat API**: Provides reasoning-informed responses that integrate conclusions with current context
- **Background Processing**: Asynchronous processing pipeline for expensive operations like representation updates and session summarization
- **Multi-Provider Support**: Configurable LLM providers for different use cases
### Storage
<!-- markdownlint-disable MD033 MD001 -->
<details>
<summary>Storage primitives in detail</summary>
Honcho contains several different primitives used for storing application and
peer data. This data is used for managing conversations, modeling peer
@ -503,23 +559,22 @@ Below is a mapping of the different primitives and their relationships.
Workspaces
├── Peers ←──────────────────┐
│ ├── Sessions │
│ └── Collections │
│ └── Documents │
│ └── (internal collections, keyed by observer/observed peer pair)
│ │
├── Scopes ←─────────────────┤ (many-to-many with sessions)
│ │
└── Sessions ←───────────────┤ (many-to-many)
└── Sessions ←───────────────┤ (many-to-many with peers)
├── Peers ───────────────┘
└── Messages (session-level)
```
**Relationship Details:**
- A **Workspace** contains multiple **Peers**
- **Peers** and **Sessions** have a many-to-many relationship (peers can participate in multiple sessions, sessions can have multiple peers)
- **Messages** can exist at two levels:
- **Session-level**: Communication between peers within a session
- **Collections** belong to specific **Peers**
- **Documents** are stored within **Collections**
- A **Workspace** contains multiple **Peers** and **Scopes**.
- **Peers** and **Sessions** have a many-to-many relationship (peers can participate in multiple sessions, sessions can have multiple peers).
- **Scopes** and **Sessions** have a many-to-many relationship (a session can belong to several scopes; a scope groups many sessions).
- **Messages** belong to a session and are labelled by their source peer.
- **Internal collections** of vector-embedded **documents** are keyed by `(observer, observed)` peer pairs. They are not directly exposed via the API; the observations stored in them are exposed as **Conclusions**.
Users familiar with APIs such as the OpenAI Assistants API will be familiar with
much of the mapping here.
@ -533,7 +588,7 @@ isolate data between use cases and provide multi-tenant capabilities.
#### Peers
Within a `Workspace` everything revolves around a `Peer`. The `Peer` object
represents any participant in the system - whether human users or AI agents.
represents any participant in the system — whether human users or AI agents.
This unified model enables complex multi-participant interactions.
#### Sessions
@ -541,48 +596,64 @@ This unified model enables complex multi-participant interactions.
The `Session` object represents a set of interactions between `Peers` within a
`Workspace`. Other applications may refer to this as a thread or conversation.
Sessions can involve multiple peers with configurable observation settings.
A session can optionally join one or more **Scopes** at creation, or later via
the scopes API.
#### Scopes
A `Scope` is a named grouping of sessions inside a `Workspace`. It is a
visibility boundary on recall: chat, representation, session context, and
workspace search answered through a scope see only what happened in that
scope's member sessions. The underlying peers keep their unified
representations across everything they have participated in.
Developers manage scopes through the scopes API (`honcho.scope(...)` /
`honcho.scopes()`) and an optional `scopes` field on session create — not
through observer/observed configuration. Adding a session that already has
messages copies its existing explicit conclusions into the scope (no
re-derivation); removing one reconciles those copies back out. Query
backfill progress with the scope `status` endpoint.
A single scope name answers from that scope's collection and card. A list of
scopes restricts recall to the union of their member sessions. Empty scopes
fail closed. `scope` is mutually exclusive with `session` / `filters` on the
same read.
#### Messages
The `Message` represents an atomic data unit that can exist at two levels:
The `Message` represents an atomic data unit that exists at the session level:
communication between peers within a session context. All messages are labelled
by their source peer and can be processed asynchronously to update their
representations. This flexible design allows for both conversational interactions
and broader data ingestion for personality modelling.
- **Session-level Messages**: Communication between peers within a session context
</details>
<!-- markdownlint-enable MD033 MD001 -->
All messages are labeled by their source peer and can be processed
asynchronously to update their representations. This flexible design allows for
both conversational interactions and broader data ingestion for personality
modeling.
#### Collections
At a high level a `Collection` is a named group of `Documents`. Developers
familiar with RAG based applications will be familiar with these. `Collections`
store vector embedded data that developers and agents can retrieve against using
functions like cosine similarity.
Collections are also used internally by Honcho while creating representations of peers.
#### Documents
As stated before a `Document` is vector embedded data stored in a `Collection`.
### Reasoning
<!-- markdownlint-disable MD033 -->
<details>
<summary>Reasoning pipeline</summary>
The reasoning functionality of Honcho is built on top of the Storage service. As
`Messages` and `Sessions` are created for `Peers`, Honcho will asynchronously
reason about peer psychology to derive facts about them and store them
in reserved `Collections`.
in reserved internal collections.
A high level summary of the pipeline is as follows:
1. Messages are created via the API
2. Derivation Tasks are enqueued for background processing including:
- `representation`: To update representations of `Peers`
4. Results are stored internally and surfaced via the Conclusions API, Representations, Peer Cards, and the Chat Endpoint.
### Retrieving Data & Insights
</details>
<!-- markdownlint-enable MD033 -->
<!-- markdownlint-disable MD033 MD001 -->
<details>
<summary>Retrieving data and insights</summary>
Honcho exposes several different ways to retrieve data from the system to best
serve the needs of any given application.
@ -606,36 +677,42 @@ the results.
#### Chat API
The flagship interface for using these insights is through
the [`Chat` Endpoint](https://blog.plasticlabs.ai/archive/ARCHIVED;-Introducing-Honcho's-Dialectic-API).
The flagship interface for using these insights is the [Chat Endpoint](https://honcho.dev/docs/v3/documentation/features/chat) (`POST /peers/{peer_id}/chat`). It takes natural-language requests to get data about a peer and returns reasoning-grounded responses. Examples:
This is a regular API endpoint (`/peers/{peer_id}/chat`) that takes natural language requests to get data
about the `Peer`. This robust design lets us use this single endpoint for all
cases where extra personalization or information about the `Peer` is necessary.
A developer's application can treat Honcho as an oracle to the `Peer` and
consult it when necessary. Some examples of how to leverage the Dialectic
API include:
- Asking Honcho for a generic or specific insight about the `Peer`
- Asking Honcho to hydrate a prompt with data about the `Peer`s behavior
- Asking Honcho for a 2nd opinion or approach about how to respond to the Peer
- Getting personalized responses that incorporate long-term facts and context
- Asking Honcho for a generic or specific insight about the peer.
- Asking Honcho to hydrate a prompt with data about the peer's behaviour.
- Asking Honcho for a second opinion on how to respond.
- Getting personalised responses that incorporate long-term facts and context.
#### Representations
For low-latency use cases,
Honcho provides access to a `representation` endpoint that
returns a static document with insights about a `Peer` in the context of a
particular session.
For low-latency use cases, Honcho provides access to a `representation` endpoint that returns a static document with insights about a peer in the context of a particular session. Use this to quickly add context to a prompt without having to wait for an LLM response.
Use this to quickly add context to a prompt without having to wait for an LLM
response.
</details>
<!-- markdownlint-enable MD033 MD001 -->
## SDKs
- **Python** — [`honcho-ai`](https://pypi.org/project/honcho-ai/) on PyPI · source in [`sdks/python/`](./sdks/python)
- **TypeScript** — [`@honcho-ai/sdk`](https://www.npmjs.com/package/@honcho-ai/sdk) on npm · source in [`sdks/typescript/`](./sdks/typescript)
- **CLI** — [`honcho-cli`](https://pypi.org/project/honcho-cli/) on PyPI · source in [`honcho-cli/`](./honcho-cli) · [CLI reference](https://honcho.dev/docs/v3/documentation/reference/cli)
SDKs are versioned independently of the server. Current SDK versions track each other; the server badge above reflects the deployed server version.
See the [SDK Reference](https://honcho.dev/docs/v3/documentation/reference/sdk) for full API surface, the [API Reference](https://honcho.dev/docs/v3/api-reference/introduction) for the raw HTTP API, and per-SDK example folders for runnable demos.
## Learn More
- [Developer documentation](https://honcho.dev/docs/) — full API surface, guides, integrations.
- [CLI reference](https://honcho.dev/docs/v3/documentation/reference/cli) — local stack, inspect/debug commands, scripting.
- [Plastic Labs blog](https://blog.plasticlabs.ai/) — design philosophy and history of the project.
## Contributing
We welcome contributions to Honcho! Please read our [Contributing Guide](./CONTRIBUTING.md) for details on our development process, coding conventions, and how to submit pull requests.
We welcome contributions to Honcho. One thing to know before you start: **pull requests must be linked to an issue carrying the `maintainer-approved` label**, or they are closed automatically. [Browse the approved queue](https://github.com/plastic-labs/honcho/issues?q=is%3Aissue+is%3Aopen+label%3Amaintainer-approved), or make your case in [Discord](http://discord.gg/honcho) — that is where maintainers are most active.
See [CONTRIBUTING.md](./CONTRIBUTING.md) for the full process, an architecture walkthrough, and a map of where to change what. For vulnerabilities, see [SECURITY.md](./SECURITY.md) — note that Honcho does not operate a bug bounty.
## License
Honcho is licensed under the AGPL-3.0 License. Learn more at the [License file](./LICENSE)
Honcho is licensed under the AGPL-3.0 License. Learn more at the [License file](./LICENSE).
The `main` branch of this repo maps to the latest canary version of Honcho. To see which versions are supported please refer to the git tags in the repo or the [compatibility guide](https://honcho.dev/docs/changelog/compatibility-guide).
## Reporting a Vulnerability
Do not open a public issue for a suspected vulnerability. Report it privately through one of:
1. **[GitHub Private Vulnerability Reporting](https://github.com/plastic-labs/honcho/security/advisories/new)** — preferred; it keeps the report, our replies, and any fix coordinated in one place.
2. Email [support@honcho.dev](mailto:support@honcho.dev) with `[SECURITY]` in the subject.
Include as much of the following as you have:
- **Version** — a git commit SHA, or the release tag you are running
- **Deployment** — self-hosted or the managed service at `api.honcho.dev`
- **Affected component** — API, deriver, dialectic, auth/JWT, an SDK, or the managed offering
- **Reproduction** — the exact steps, requests, or script that trigger it
- **Proof of concept** — the smallest thing that demonstrates the issue actually works
- **Impact** — what an attacker gains, and what they need to already have to get it
- **How you found it** — manual review, fuzzing, a scanner, or model-assisted analysis
Reports with a working proof of concept get looked at first. A report that only describes a
theoretical problem is much slower for us to act on, because we have to build the repro
ourselves before we can confirm anything.
Honcho stores conversational data and peer representations. **Do not attach production user
content, API keys, or JWTs** to a report — if we need a sample, we will ask for a redacted
one.
## Testing
Test against an instance you operate. Do not run security testing against `api.honcho.dev`
or against any Honcho deployment that is not yours — self-hosting is a first-class path and
takes a few minutes to set up — install the CLI (`uv tool install honcho-cli`) then run `honcho start --setup` (Docker + an LLM provider key), or see [Self-hosting](./README.md#self-hosting).
## What to Expect
We will acknowledge your report and tell you whether we consider it in scope. If it is, we
will let you know when a fix ships.
We do not commit to a response SLA, we do not coordinate CVE assignment on request, and we
do not operate a disclosure timeline you can hold us to. This is a small team.
## Out of Scope
The following are not treated as vulnerabilities. Reports consisting only of these will be
closed without a detailed response:
- Automated scanner output with no working proof of concept
- Model-generated findings that have not been verified by a human against a running instance
- Missing security headers or TLS configuration with no demonstrated exploit
- Rate limiting, or resource exhaustion with no demonstrated impact beyond your own instance
- Vulnerabilities in dependencies with no demonstrated exploit path through Honcho
- Configuration weaknesses that require an already-compromised host, or that come from
deliberately insecure settings (for example running with `AUTH_USE_AUTH=false`, which is
the documented local-development default and is not intended for a public deployment)
- Social engineering, phishing, and physical access
For ordinary bugs, memory or recall quality problems, and feature requests, use the
@ -27,7 +27,296 @@ Welcome to the Honcho changelog! This section documents all notable changes to t
### Honcho API and SDK Changelogs
<Tabs>
<Tab title="Honcho API">
<Update label="v3.0.3 (Current)">
<Update label="v3.1.1 (Current)">
### Changed
- Server `requires-python` is `>=3.13`, matching the production image. Self-hosters on 3.10–3.12 need to upgrade; SDK and CLI floors are unchanged (#1090)
### Fixed
- Concurrent `create_documents` writers to the same collection deadlocked on `times_derived` reinforcement UPDATEs issued in batch order; the error was swallowed per-document, the batch was lost, and the queue item was marked processed. Writers now lock target rows with `SELECT ... ORDER BY id FOR UPDATE` before applying, abort the batch on `SQLAlchemyError` instead of continuing through a dead session, and retry transient errors (deadlock, serialization failure, lock/statement timeout, lost connection) up to `MAX_RETRYABLE_ATTEMPTS` instead of burning the item (#1033)
- Scope backfill no longer embeds, writes, and syncs every planned copy at once. A 14k-document session is ~580MB of vectors; several concurrent backfills OOM-killed the deriver at its 1000Mi limit and crash-looped because the work units never completed. Phases 2–4 now run per chunk of 500 specs, reload source embeddings per chunk, and drop them once synced. Membership is locked across chunk writes so a concurrent leave cannot commit between the check and the inserts (#1104)
- Model-generated observations with NUL bytes (`\u0000`) no longer fail the exact-content dedup pre-fetch with a Postgres `DataError` that dropped the whole observer batch. Ingress already stripped NUL from user content; the deriver now strips it so stored text matches embedded text. All-NUL content is dropped rather than stored empty (#1095)
- `search_messages` no longer forwards `top_k=0` to Turbopuffer (which requires 1..10000). Zero/negative limits short-circuit to empty results; tool limits are floored at 1. The documents path was already guarded (#970); this closes the message path (#1084)
- OpenAI-compatible tool-call turns with `content=null` keep null through history replay instead of being coerced to `""`. Providers that bind reasoning state to the exact assistant message shape were breaking on the empty string. Tool-less null still becomes `""` (#1064)
- The production image now ships `pyproject.toml` in the runtime stage, so the service reports its real version instead of `unknown` in OpenAPI and telemetry (#1074)
</Update>
<Update label="v3.1.0">
### Added
- Scopes: a named grouping of sessions that acts as a visibility boundary on recall, implemented as a facade over an observer peer (`scope.{name}` with `{"kind": "scope"}`). Developers manage them exclusively through `/v3/workspaces/{workspace_id}/scopes` (create-or-get, list, get, add/list/remove session membership) and an optional `scopes` field on session create — never through the observer/observed mechanics. Scope peers cannot author messages, cannot be a chat or representation `target`, are excluded from `peers.list` by default (`PeerGet.kind` = `"scope"` / `"all"` switches the view), and are rejected on the generic session-peer routes. Workspace-level key required; peer- and session-scoped keys get 401. Legacy peers occupying a reserved `scope.` name without the kind flag are refused with 409, never adopted (#884)
- `scope` read option on chat, representation, session context, and workspace search. A single scope swaps the observer to the backing scope peer so conclusion recall, peer cards, and message tools stay inside that scope's membership. A list of scopes takes the union of member sessions (capped at `MAX_SESSION_ALLOWLIST_ENTRIES`) and executes via the session-allowlist path. Empty scopes fail closed. `scope` is mutually exclusive with `filters` and `session_id`. Workspace- or admin-level key required (403 otherwise). Scope peers are also rejected as `peer_target` / `peer_perspective` on session context and as the path peer or `target` on `GET /peers/{id}/context` (#897)
- Scope backfill-by-copy and removal reconciliation. Adding a session that already has messages copies its explicit-level documents into the scope's collections (no LLM re-derivation; idempotent via `copied_from`). Removing a session soft-deletes those copies and fail-closed cascades to derived documents whose `source_ids` intersect anything removed, then enqueues a `card_refresh` dream with `rebuild=True` plus an omni dream. `GET /v3/workspaces/{workspace_id}/scopes/{scope_id}/status` reports per-session backfill state (`pending` / `completed` / `failed`, plus `docs_copied`) (#904)
- Workspace-level chat at `POST /v3/workspaces/{workspace_id}/chat`: agentic dialectic over the whole workspace instead of a single (observer, observed) pair. Prefetches workspace stats and the top active peers' self cards, then searches pair-scoped memory with `[observer->observed]` attribution. Supports `session_id`, `scope`, `reasoning_level`, `response_format`, and SSE streaming (#931)
- MCP workspace discovery: tools accept `workspace_id`, the worker honors an optional `X-Honcho-Workspace-ID` connection header, and `list_workspace` / `create_workspace` tools let clients pick or create a workspace instead of relying on the SDK default (#1020)
- MCP `search` also queries conclusions in parallel with messages when `peer_id` is given, returning `{messages, conclusions}`. The conclusions leg degrades to `[]` on error so search never gets worse than before (#974)
- Prometheus metrics for physical DB connections, visible even under `DB_POOL_CLASS=null`: `db_connections_open` (gauge) and `db_connections_established` (counter), hooked to SQLAlchemy connection-lifecycle events and registered on both the API and the deriver (#1055)
- Bounded-label Prometheus series are zero-initialized at process start so an absent series means a broken scrape rather than "nothing happened" (#927)
### Changed
- Workspace and pair chat system prompts now describe Honcho, peers, and the harness on their own terms, and render only the tools the request actually offers. The pair prompt no longer advertises a write tool that is not in the loadout (#1066)
- Deriver idle polling backoff is longer and no longer reset by periodic reconciler work, so downstream connection pools can cull idle DB connections (#1015)
- LLM provider SDKs are lazy-loaded so idle API and deriver processes no longer pay for every provider at import time (#1011)
- Production image is a multi-stage build: LanceDB/PyArrow move behind an optional `lancedb` extra (`INSTALL_LANCEDB=true` to restore them), FastAPI's unused cloud CLI is dropped, and the venv is copied into the runtime image with final ownership so Docker does not double the layer. Default unpacked image is about 663 MB (was 1.7 GB) (#1014)
- Redis Cluster cache keys hash-tag the namespace so one deployment's keys land on a single shard instead of opening a connection to every node. No behaviour change on a non-cluster backend; existing keys age out by TTL (#1058)
- Deriver extraction prompt no longer leaks its own few-shot examples into extracted conclusions (#1028)
### Fixed
- Observer-scoped `get_observation_context` no longer materializes every session the observer has ever joined into a `session_name IN (...)` list (twice in one statement). Past ~32k sessions that hit psycopg's bind-parameter ceiling and 500'd. The observer half is now a correlated `EXISTS` over `session_peers`, two bind parameters regardless of membership size (#1065)
- Re-adding an already-active session peer no longer advances `joined_at`, so `peer_perspective` search keeps messages from the original join. Genuine leave-and-rejoin still starts a new window (#1059)
- Transient embedding-provider errors (for example an OpenAI-compatible 200 with empty `data: []`) were relabeled as token-limit errors. Only genuine oversize input raises `EmbeddingTokenLimitError`; other provider errors propagate unchanged (#791)
- The filter DSL now fails closed with a 422 instead of a 500 on bad shapes, coerces operands by column type (so `{"session_id": {"ne": "abc"}}` is a string inequality rather than "invalid numeric"), and treats `NOT` / `ne` as null-safe (`IS NOT TRUE` / `IS DISTINCT FROM`) so negation no longer drops rows whose field is unset. Closed-set columns like `level` reject unknown values. Session-allowlist entries must be well-formed ids (`*` is 422, not a silent widen) (#947)
- `ne` on JSONB metadata keys is null-safe: a missing key is not equal to the compared value, so `{"metadata": {"foo": {"ne": "bar"}}}` includes rows where `foo` is unset (#1036)
- Oversized texts in `simple_batch_embed` are truncated to the embedding token cap instead of failing the whole batch. Representation processing reports failed observer saves in `RepresentationCompletedEvent` and raises when every observer save fails (#1019)
- Assistant `reasoning_content` (DeepSeek / some OpenRouter models) is preserved across tool-loop turns. Previously the tool loop dropped thinking content before building the next assistant history message, so continuation requests failed. `reasoning_details` still takes precedence when both are present (#1034)
- `create_observations` now honors `DERIVER_DEDUPLICATE` instead of hardcoding `deduplicate=True`, matching the representation write path (#1018)
- `provider_params.timeout` is forwarded to the OpenAI-compatible embedding client, not just the LLM client (#1024)
- Conclusions semantic-search validation errors name the field and the constraint instead of returning a generic 422 (#960)
- OpenAI-compatible embedding calls request `encoding_format=float` so providers that default to base64 do not break pgvector inserts (#938)
- Gemini batch embedding works for `gemini-embedding-2*` models, which rejected the previous request shape (#745)
- MCP OAuth with no advertised scopes no longer defaults to read-only (which 403'd chat and search POSTs). Protected-resource metadata advertises read and write (#1004)
</Update>
<Update label="v3.0.12">
### Added
- Session allowlist on the Dialectic and representation via a constrained `filters` body on `POST /peers/{peer_id}/chat` and `/representation`, supporting only the `session_id` key (a session id, a bare list, or `{"in": [...]}`). Unsupported keys and shapes are rejected with 422 rather than silently ignored, it composes with `session_id` (which must be included in the allowlist when both are given), and it is capped at 1,000 sessions per request. Enforcement is uniform and fail-closed at every recall chokepoint: scoped conclusion recall is restricted to `level == "explicit"` (dream-derived conclusions carry a single `session_name` but are synthesized across all sessions, so that stamp can't be scoped on), `get_reasoning_chain` is unavailable under an allowlist, and an empty allowlist short-circuits to empty results everywhere. Workspace keys pass the allowlist as-given; peer-scoped JWTs must be an active member of every allowlisted session (401 otherwise) (#882)
- Bare-list membership sugar in the filter DSL: `{"session_id": ["s1", "s2"]}` is now shorthand for `{"session_id": {"in": [...]}}` on regular columns generically. JSONB metadata columns are excluded and keep containment semantics. Strictly additive, since a bare list on a regular column previously compiled to a type-mismatched equality that matched nothing (#881)
- Optional structured outputs on the Dialectic: `response_format` (a JSON Schema with root type `object`) on peer chat makes `content` a JSON string conforming to that schema. Only a conservative subset of JSON Schema is supported, with DoS guards and non-recursive `$ref` support (#896)
- Combined tool calling and structured output in the LLM transport layer, with per-backend request shaping: OpenAI routes tool-carrying structured requests through `create()` with an explicit `json_schema` response format (`parse()` 500s on non-strict function tools), Anthropic skips the `{` JSON prefill when tools are present so `tool_use` blocks stay reachable, and Gemini injects a schema instruction into the final turn instead of using native `response_schema` (rejected alongside function calling before Gemini 3). All backends skip structured-output parsing on tool-call turns, which carry no consumable content (#907)
- `card_refresh` dream type: a lightweight dream that runs only the peer-card update, for event-driven refreshes such as membership changes and cold starts. Handled by a new `CardRefreshSpecialist` restricted to `get_recent_observations`, `search_memory`, and `update_peer_card` (no observation-mutating tools) with a tool-iteration cap of `min(6, DREAM.MAX_TOOL_ITERATIONS)`. `POST /v3/workspaces/{workspace_id}/schedule_dream` accepts `dream_type=card_refresh` plus a `rebuild` flag, which omits the existing card from the prompt so the specialist rebuilds it solely from observations present in the collection. Card refreshes never advance the omni dream guard pair (`last_dream_at` / `last_dream_document_count`) (#883)
- Full-fidelity LLM trace stream, with Langfuse as one projection over it: each call is captured once (`CapturedLLMCall`) and fanned out to a CloudEvents trace stream (`llm.call.traced` / `trace.content`) and a Langfuse exporter, both reconstructing trace → run → step → generation from the same source of truth. Adds `TELEMETRY_TRACE_PAYLOADS_ENABLED` (default `false`), `TELEMETRY_TRACE_MAX_BYTES` (default 262144, per-message cap with oversized content clipped), `TELEMETRY_TRACE_PURPOSES` (JSON list of `CallPurpose` values; empty means all), and `LANGFUSE_EXPORTER_MODE` (`exporter` by default; `inline` is kept for one release for side-by-side validation). Embedding calls are traced, dreamer branches nest under one dream trace, tool calls become spans under their step, and high-volume events are sampled deterministically. `TRACE_ENDPOINT` is dropped (#845)
- Redis Cluster support via `CACHE_CLUSTER` (for example GCP Memorystore for Redis Cluster), alongside a new `CACHE_LOCK_WAIT_CHECK_INTERVAL_SECONDS` (#905)
- `EMBEDDING_MODEL_CONFIG__MAX_BATCH_SIZE` caps texts per embedding request for OpenAI-compatible providers with smaller limits than OpenAI's, such as DashScope `text-embedding-v4` (10) and Alibaba Bailian `qwen3.7-text-embedding` (20). When unset, native provider defaults are preserved (OpenAI 2048, Gemini 100) (#983)
- Per-request provider timeouts via `provider_params.timeout` on any model config, validated at config load so a bad value fails at startup with the exact config path instead of surfacing per-request as a retried 500. Good values normalize to float seconds; Gemini's is converted to milliseconds (#832)
- `RepresentationCompletedEvent` now reports deduplication counts: `exact_dup_in_batch_count`, `exact_dup_existing_count`, `semantic_dup_rejected_count`, and `semantic_dup_replaced_count` (#910)
- OAuth discovery for MCP clients: the MCP worker serves `/.well-known/oauth-protected-resource` (RFC 9728) without auth so clients can discover the authorization server, and a 401 now carries `WWW-Authenticate: Bearer resource_metadata="..."` (exposed cross-origin) to start the flow (#923)
- Prometheus metrics for the immediate-embed fast path: tasks shed because `EMBEDDING_MAX_PENDING_EMBED_TASKS` was reached, and the current in-flight task count (#892)
- Docs: a detailed system architecture diagram, a Codex integration guide (#879), a structured-outputs page (#896), a section on filtering conclusions by reasoning level (#851), a health-check endpoint reference, and SDK updates (#867)
### Changed
- **Breaking config change:** `DERIVER_REPRESENTATION_BATCH_MAX_TOKENS` is split into two settings that were previously conflated — `DERIVER_REPRESENTATION_BATCH_WORK_UNIT_TARGET_TOKENS` (default 512), the producer-side minimum a work unit accumulates before the deriver claims it, where `0` disables the gate; and `DERIVER_REPRESENTATION_BATCH_TARGET_INPUT_TOKENS` (default 1024), the consumer-side maximum context-window tokens per deriver LLM call. Deployments setting the old name must migrate (#889)
- The immediate-embed fast path now applies backpressure: `EMBEDDING_MAX_PENDING_EMBED_TASKS` (default 50) caps in-flight embed tasks, and once saturated, message creation skips the fast path entirely and the reconciler embeds on its next cycle. `0` disables the fast path (#892)
- Explicit-level documents are now kept session-pure, so memory can be built by copying explicit documents between collections. Enforcement refuses rather than rewrites: `create_documents` rejects explicit documents with a null `session_name`, exact dedup keys on (content, level, session-for-explicit), semantic dedup scopes candidate search to the same level and — for explicit documents — the same session, and the generic `create_observations` tool rejects `level='explicit'` outside message-ingestion (deriver) context. Derived levels keep cross-session consolidation (#883)
- Sentry's `before_send` filter is centralized as `default_before_send` in `src/telemetry/sentry.py` instead of living only in the API's `main.py`, so the deriver gets the same non-actionable-exception filtering. All Sentry events also carry a `namespace` tag for correlation (#934, #870)
- The minimal deriver's extraction examples no longer teach inferences its own output schema forbids. The `EXAMPLES` block demonstrated deriving a specific birthday from "I just had my 25th birthday last Saturday", deriving residence from a single visit ("I took my dog for a walk in NYC" → "alice lives in NYC"), and a "+ general knowledge" deductive output the deriver has no channel for. The replacements stay inside the schema's contract and teach the boundary: the dog/NYC message is kept and shown extracting correctly, and a separate example shows "lives in NYC" is valid when actually stated (#985)
- Dreamer specialists are instructed not to output summaries (#894)
- `session_name` is deprecated for scoping in favor of the session allowlist. It is not removed and not aliased: it also pins the query to one session, bypasses observer scoping, and drives session-history injection into the dialectic prompt, so it has no drop-in replacement (#882)
- The MCP worker no longer requires the `X-Honcho-User-Name` or `X-Honcho-Assistant-Name` headers (#923)
### Fixed
- Session scoping was applied to only one of the three working-representation query paths: `session_name` reached the recent-documents query, but the semantic and most-derived paths ignored it, so `limit_to_session` leaked cross-session conclusions into perspectives. The allowlist is now threaded uniformly through all three paths and pushed down to pgvector and external vector stores (#881)
- Empty membership lists failed open in the vector-store filter builders, silently widening scope: LanceDB dropped empty `IN` clauses and Turbopuffer emitted a bare `In []` with undocumented semantics. Both now emit an explicit always-false predicate, and `_build_filter_conditions` checks `is not None` rather than truthiness so an empty list is no longer treated like `None` (#881, #882)
- Session-scoped CRUD helpers ignored the session allowlist entirely, so a caller could read a session the allowlist forbids. The API routes guarded this with a 422, but the dialectic tools call these CRUD functions directly and bypassed it. `_semantic_search_messages` (covering `search_messages` and `search_messages_temporal`), `grep_messages`, `get_messages_by_date_range`, `get_recent_history`, and `get_observation_context` now return `[]` when `session_name` is set and outside the allowlist (#882)
- The cache client logged the full Redis URL — including the password — at INFO and WARNING on every connection attempt and failure, exposing the live credential in container logs and downstream aggregation. Credentials are now redacted across userinfo, the `?password=` (redis-py) and `?secret=` (cashews) query params, scheme-less URLs whose password is invisible to `.port`/`.password` parsing, and malformed URLs, whose fallback previously echoed the raw input verbatim (#869)
- A `top_k` of `0` reached the vector store, where Turbopuffer rejects it with a 400 (`top_k must be between 1 and 10000`). A non-positive `top_k` now returns `[]` before the embedding call, and the semantic budget floors at 1 so an explicitly requested search isn't silently allocated zero (#970)
- Gemini clients had no HTTP timeout, so a stalled socket wedged the deriver worker's uvloop event loop, which the in-process reconciler shares. A 10-minute timeout is now set on both the Gemini LLM client and the Gemini embedding client (#903)
- Dreamer conclusions were dated to ingestion time rather than their latest source observation, and their timestamps are now normalized (#890)
- Langfuse I/O annotation was gated on `LANGFUSE_PUBLIC_KEY` instead of `langfuse_inline_enabled`, so in the default `exporter` mode it called `update_current_generation()` with no active span — logging "No active span in current context" roughly 14 times per dialectic run and building throwaway `model_dump` payloads on every LLM call. Separately, `AgentToolSummaryCreatedEvent` hardcoded `run_id="deriver"` / `iteration=0`, polluting `run_id` grouping in the CloudEvents stream with a phantom run; both fields are now optional and the resource id is keyed on `message_id:summary_type` (schema_version 2 → 3) (#845)
- Assistant tool calls were dropped from the captured trace stream for OpenAI and Gemini: `build_captured_messages` read only `{role, content, tool_call_id}`, but those providers keep tool calls outside `content`, so replayed tool-call turns landed as empty content and Gemini lost its text and tool results entirely. Tool calls are now normalized per provider into a unified `tool_calls` field and folded into the content hash. Gemini's `thought_signature` is bytes, so `model_dump(mode="json")` raised `UnicodeDecodeError` inside `emit_trace`, silently dropping whole tool-calling iterations from the trace stream (billing and Langfuse were unaffected); it is now base64-encoded on the telemetry path while replay keeps the raw bytes (#845)
- `EmbeddingClient.encoding` forced full client construction, raising "OpenAI API key is required" even though tiktoken needs no credentials. The document dedup tie-break only needs `.encoding` for token counting, so any test hitting that path failed in environments without embedding keys — notably CI for pull requests from forks. The encoding is now resolved from the configured model directly, falling back to `cl100k_base`, and the underlying client's encoding is reused only when it has already been constructed (#955)
- The Docker build failed under Podman because the uv build inputs weren't copied (#878)
- LanceDB was installed on macOS Intel, where it doesn't work. A PEP 508 marker excludes `darwin/x86_64` and the LanceDB vector-store import is wrapped so a misconfiguration surfaces as a clear config error (#496)
- Prompt checks requiring the literal token "json" for `json_object` mode are now satisfied in lowercase (#887)
- Reverted an unintended `RepresentationCompletedEvent` schema-version increment
- Documented preinstalling pgvector as a privileged role for deployments where the `DB_CONNECTION_URI` role deliberately cannot create extensions (managed Postgres, Kubernetes operators, NixOS). `CREATE EXTENSION IF NOT EXISTS vector` does not help there, because Postgres checks the privilege before checking whether the extension exists. Docker Compose is unaffected, since the bundled stack connects as the `postgres` superuser (#984)
</Update>
<Update label="v3.0.11">
### Added
- `api_request_duration_seconds` Prometheus histogram tracking per-route request latency, labeled by method and endpoint (#837)
- LLM `provider_params` passthroughs (`extra_body` / `extra_headers` / `extra_query`) are now forwarded to the underlying provider transport across all backends, with shape validation that rejects non-mapping values (#821)
- `structured_output_mode` model-config option to use `json_object` mode for OpenAI-compatible providers that lack native Structured Outputs support (used by the deriver) (#820)
- OpenRouter app-attribution headers (`HTTP-Referer` / `X-Openrouter-Title`) are now sent on OpenAI-compatible clients when the configured base URL is OpenRouter, so requests are attributed to "Honcho" in OpenRouter's dashboard (#805)
- Langfuse traces are now tagged with user and session IDs for easier trace filtering (#814)
- `DERIVER_REPRESENTATION_BATCH_MAX_AGE_SECONDS` (default 1800s) lets sub-threshold representation work units flush once their oldest unprocessed queue item ages out. Set it to `0` to keep the legacy behavior where sub-threshold tails wait indefinitely unless `DERIVER_FLUSH_ENABLED=true` (#826)
- Conclusion responses now include a `level` field (`explicit`, `deductive`, `inductive`, `contradiction`); list/query endpoints support filtering by `level` via `filters`, with reserved filter keys protected from being overridden by user-supplied filters (#851)
### Changed
- Peer-scoped JWTs now get read-only access to the sessions their peer is an active member of (session context, summaries, peers, their own per-session config, search, and message reads). Session-scoped JWTs remain confined to their session and cannot reach peer routes (#679)
- Compacted Honcho's log output, with guarded ms/s metric formatting that falls back to a plain string for non-numeric values (#836)
- Sentry now drops noisy infra/scrape transactions: the reconciler opens a transaction only once a batch has rows (idle cycles emit none), and a `traces_sampler` returns `0.0` for `/metrics`, `/health`, `/openapi.json`, `/docs`, `/redoc`, and the deriver metrics server. `SENTRY.TRACES_SAMPLE_RATE` still governs real traffic (#834)
### Fixed
- Peer- and session-scoped JWTs were effectively workspace-scoped: authorization walked the route's declared scope and fell through to a workspace match, so a `{w, p: alice}` token could act on any peer in the workspace. JWTs are now authorized by their narrowest claim and never widen to workspace access (#679)
- The keys API now rejects creating a peer- or session-scoped key without a workspace. Such keys were minted successfully but failed verification on every request (#679)
- Agent-supplied observation IDs carrying the display-format `id:` prefix are now normalized (prefix and trailing whitespace stripped) before `source_ids` are stored and on `get_reasoning_chain` lookups, fixing corrupted provenance links and broken reasoning-chain traversal (#795)
- Fixed a `create_tree` keyword-argument mismatch in the Dreamer's surprisal tree construction (#749)
- Providers that omit output-token counts (observed with Gemini on tool-loop completions) returned `output_tokens=None`, which raised a Pydantic validation error that aborted the call and crashed the Dreamer's induction phase before inductive conclusions were persisted. `None` is now coerced to `0` so token accounting degrades gracefully (#809)
- Document creation now performs exact (case-insensitive, whitespace-trimmed) content deduplication before the existing semantic dedup step: exact duplicates within a batch collapse to a single insert, and an exact match against a live document reinforces it (atomic `times_derived` increment) instead of creating a new row (#861)
- The OpenAI backend passed `tool_choice` through raw while the Anthropic and Gemini backends translate Honcho's canonical vocabulary to their native form, so on a mixed-provider fallback chain (for example Gemini primary → OpenAI backup) a canonical `"any"` reached OpenAI unchanged and was rejected as an invalid param. The OpenAI backend now converts it, mirroring the others: `any`/`required` → `required`, `auto`/`none` pass through, and a tool-name string or `{"name": ...}` dict becomes a function selection (#850)
- Langfuse `@observe` auto-capture serialized every argument of `honcho_llm_call_inner` into the generation span input, including `client_override` (a live `AsyncOpenAI`/`genai` client) and `selected_config` (which carries `api_key`). Auto-capture deep-copied the client into a half-constructed object whose teardown raised (`AsyncHttpxClientWrapper ... no attribute '_state'` on OpenAI, flooding stderr; `BaseApiClient ... no attribute '_http_options'` on Gemini), and it leaked `ModelConfig.api_key` into traces. Capture is now an explicit allowlist: `capture_input`/`capture_output` are disabled and curated, serializable input and output are stamped instead, with tuning knobs surfaced as `model_parameters` via a secret-bearing denylist and per-call token usage mirrored as `usage_details` (#849)
</Update>
<Update label="v3.0.10">
### Added
- Messages are now embedded via a background task rather than blocking API request
- Read-only DB session mode (`get_read_db` / `tracked_db(..., read_only=True)`) so reads don't hold a transaction open across the work
- `CORS_ORIGINS` env var to configure CORS allowed origins without editing source; defaults match the prior hardcoded list, so self-hosted deployments behind custom domains can whitelist their frontend (#697)
- `scripts/generate_jwt.py` — utility for minting scoped or admin Honcho JWTs (`--admin`, `--workspace`/`--peer`/`--session`, `--expires` with human-friendly durations, `--print-only`) without calling the keys API (#757)
- `STALE_WORK_UNIT_CLEANUP_INTERVAL_SECONDS` (default 60s) — minimum jittered spacing between deriver stale-work-unit cleanup runs, so cleanup no longer runs on every seconds-scale poll (`0.0` keeps the legacy every-poll behavior) (#773)
### Changed
- Optimized the deriver and dreamer prompt cache prefixes to improve prompt-cache hit rates (#806)
### Fixed
- `times_derived` is now properly reinforced when a duplicate conclusion is detected. It had been pinned at 1 for nearly every conclusion (the reject-new branch dropped the increment and the new-wins branch reset the count to 1), so `ORDER BY times_derived DESC` fell back to arbitrary heap order and froze stale conclusions to the front of injected context. Reinforcement is now an atomic increment and both most-derived queries gained a `created_at DESC` recency tiebreaker (#768)
- Webhook creation now correctly rejects private/internal IP addresses (#793)
</Update>
<Update label="v3.0.9">
### Changed
- Connection acquisition is now a single attempt with no server-side retry, on a vanilla `AsyncSession`. A new `DB_CONNECT_TIMEOUT_SECONDS` (default 2s) bounds the attempt so a saturated or unreachable pooler fails fast instead of holding a client connection open to re-knock. A saturated DB now surfaces to the caller — the API returns an error and the deriver backs off and retries on a later poll — which lets the pooler drain rather than amplifying saturation.
### Added
- Deriver poll jitter so instances that start together don't poll in lockstep: `DERIVER_POLLING_STARTUP_JITTER_SECONDS` (random delay before the first poll, default 30s) and `DERIVER_POLLING_JITTER_RATIO` (±fraction applied to every poll sleep, default 0.5). Both disable at `0.0`; the underlying backoff schedule is unchanged.
### Removed
- Reverted the connection-checkout retry and `HonchoAsyncSession` custom session introduced in 3.0.8. Removed the `DB_CONNECTION_RETRY_ENABLED` / `DB_CONNECTION_RETRY_MAX_DELAY_SECONDS` / `DB_CONNECTION_RETRY_BACKOFF_INITIAL_SECONDS` / `DB_CONNECTION_RETRY_BACKOFF_MAX_SECONDS` settings, the `db_connection_acquisitions{outcome=...}` Prometheus counter, and the `db.pool.acquire` Sentry span. Alerting built on `db_connection_acquisitions` should migrate to `db_pool_connections` / `db_queries_in_flight`.
</Update>
<Update label="v3.0.8">
### Added
- Connection-checkout retry with bounded exponential backoff (tenacity) on `get_db`/`tracked_db`: transient transaction-pooler (Supavisor) rejections — SQLAlchemy `TimeoutError` and `OperationalError` — now retry with backoff instead of surfacing as 500s under client-connection saturation. Gated by
`DB_CONNECTION_RETRY_ENABLED` with configurable delay/backoff knobs; ~10s default budget (#758)
- `HonchoAsyncSession` — a lazy `AsyncSession` that checks out its pooled connection (with retry) on the first DB-touching call rather than at construction. Request handlers doing non-DB work (embedding, file, LLM) before their first query no longer pin a pooler connection across it. Only the checkout is retried;
the statement still runs exactly once, so writes are never duplicated (#758)
- Adaptive deriver queue polling: the poll interval backs off when the queue is idle or erroring (base → max, doubling each cycle) and snaps back to base the moment work is claimed, cutting steady-state query load against the DB. Gated by `DERIVER_POLLING_BACKOFF_ENABLED` with configurable max/multiplier (#758)
- New Prometheus `db_pool_connections` gauge (checked_out / checked_in / size / overflow), labeled `api`|`deriver`, registered in both the API lifespan and the deriver metrics server (#758)
- New Prometheus `db_connection_acquisitions{outcome=ok|retried|exhausted}` counter — the alertable early-warning signal that connection checkouts are retrying through pooler rejection, before requests start failing (#758)
- New Prometheus `db_queries_in_flight` gauge — statements actually executing on the wire (via SQLAlchemy cursor-execute events). Paired with `checked_out`, the gap reveals connections held but parked (the "idle in transaction during an external call" antipattern). Gated on `METRICS.ENABLED` for zero overhead when
off (#758)
- Explicit `SqlalchemyIntegration` in both the API and deriver Sentry inits; connection acquisition wrapped in a `db.pool.acquire` span with live pool stats captured on retry exhaustion (#758)
### Changed
- Default `POOL_TIMEOUT` lowered to 5s, with validation that it stays under the connection-retry budget when a pooled (non-null) `POOL_CLASS` is configured; `config.toml.example` and the v2/v3 configuration docs updated to match (#758)
- `HonchoAsyncSession` wraps every DB-touching session method (execute / scalar / scalars / flush / merge / refresh / commit / get / get_one / stream / stream_scalars / delete) so the lazy-checkout-with-retry guarantee has no holes; the acquired flag resets on `close()`/`reset()` so a reused session re-acquires on
next use (#758)
### Fixed
- Roll the session back on a retryable checkout failure before retrying — a failed autobegin could otherwise leave it pending-rollback, making the next connection attempt raise instead of cleanly re-checking-out (#758)
- Guard `DBPoolCollector.collect()` so a pool-read/import hiccup can't raise and abort the entire `/metrics` scrape (Prometheus drops all metrics if any collector raises) (#758)
- Clamp the pool overflow gauge to ≥ 0 (it could report negative before the pool fills) (#758)
- Removed a double-sleep in the deriver idle poll so the backoff cap is a true cap rather than 2× (#758)
</Update>
<Update label="v3.0.7">
### Added
- New `src/llm/` package as the single owner of provider runtime: clients, backends, history adapters, tool loop, request builder, credentials, and caching policy (#459)
- New cloudevent `LLMCallCompletedEvent` (`llm.call.completed`) fires once per provider hit with full cost-attribution context: transport/provider_label, model, token counts with cache breakdown, finish_reason, outcome, retry/fallback state, duration, tool-call shape, streaming flag, and agent correlation (`run_id` + iteration) (#637)
- `RepresentationCompletedEvent` now carries `total_input_tokens` for full-trace cost attribution; per-emitter `honcho_version` injection; deterministic per-`run_id` high-volume sampler via `TelemetrySettings.HIGH_VOLUME_SAMPLE_RATE` (#637)
- Deriver custom instructions: per-workspace/peer guidance threaded into the deriver prompt with a `MAX_CUSTOM_INSTRUCTIONS_TOKENS` budget (default 2000); deriver `MAX_INPUT_TOKENS` raised 23000 → 25000 (#609)
- New `honcho-cli` package — Python CLI for inspecting and managing peers, sessions, and configuration against a Honcho deployment (#424)
- `HONCHO_API_URL` env var support in the MCP Worker for self-hosted deployments (#575)
- API ID `max_length` increased from 100 to 512 across `WorkspaceCreate`, `PeerCreate`, and `SessionCreate` to align with the DB schema (#684)
- `AttemptPlan` dataclass pins per-retry provider selection across stream-final retries so streaming doesn't bounce back to primary after the tool loop has settled on fallback (#459)
- Gemini JSON-schema sanitizer for `function_declarations` — strips keywords Gemini's validator rejects while preserving semantics for other backends (#459)
### Changed
- All LLM orchestration moved out of `src/utils/clients.py` into `src/llm/` with modules split by responsibility (#459)
- Default `ModelConfig` factories (deriver, summary, dreamer specialists, dialectic levels) normalized with no extra parameters set by default; operators add transport/thinking overrides explicitly (#459)
- OpenAI reasoning-model routing widened to cover `gpt-5.x` and `o1/o3/o4` — these models receive `max_completion_tokens` instead of `max_tokens` (#459)
- Peer card prompts reframed as stable identity markers; induction specialist now opts out of peer card writes so only deduction touches the card (#686)
- Vector store queries no longer fetch embedding vectors — only document metadata is returned, reducing payload size and DB load (pgvector, lancedb, turbopuffer) (#682)
- Langfuse trace metadata now includes `namespace`, `model`, and `provider` so traces can be filtered by deployment slice (#565)
- Deriver: model-aware tokenizer (replaces the previously hardcoded encoding) and explicit guard on empty message content (#647)
- Dialectic level defaults now merge correctly with per-level overrides (#656)
- Default dialectic tool choice switched to `auto` (#630)
- Vector sync given a substantial retry budget to tolerate transient embedding provider outages (#604)
- `AgentToolConclusionsDeletedEvent` payload now carries `levels` (#612)
- Turbopuffer: `InternalServerError` caught and surfaced as a warning rather than a hard failure; vector store sync errors downgraded to warnings (#561)
### Fixed
- `reverse` query parameter is now honored on the v3 workspace list, peer list, workspace-scoped session list, and peer-scoped session list. Honcho SDKs at 2.1.0+ were already sending `reverse=true` for these routes but the server silently ignored it. Ties on `created_at` now fall back to the internal nanoid `id` for stable ordering across pages (#685)
- LLM client factories now receive `base_url` from `LLMSettings` for default providers — operators pointing at OpenAI-compatible proxies via `LLM__OPENAI_BASE_URL` were previously ignored on the default path (#643, fixes #641)
- Internal N+1 query in dialectic agent tool execution — collapsed per-iteration DB lookups into a single fetch (#652)
- Dreamer threshold and time-guard semantics: count filter now includes only `documents.level == 'explicit'` (was inflating threshold via dreamer-created levels and creating a feedback loop); `last_dream_at` write relocated from enqueue to process so duplicate enqueues or failed runs no longer reset the 8-hour time guard (#573)
- Deriver: blank observations are filtered out before embedding (previously triggered noisy embedding calls and persisted empty rows) (#615)
- Surprisal module: filter format corrected from `{"level": levels}` to `{"level": {"in": levels}}` — the prior call silently returned 0 results and made the entire Surprisal phase of the Dream cycle a no-op (#581, fixes #559)
- Embedding client: `embed()` now wraps single-string input in an array, restoring compatibility with OpenAI-compatible third-party providers that reject scalar input (#586)
- Docker Compose: deriver service startup gated on the API service healthcheck — prevents races where the deriver starts before the API has run migrations (#689)
- Docker image: `HEALTHCHECK` directive removed from the shared base image; service-level health checks now belong in each service's own configuration (#530)
- Removed strict parameter validation for thinking params on Anthropic and OpenAI transports — was rejecting valid per-transport configs (#686)
- Stream-final retries pin to the `AttemptPlan` that succeeded rather than re-running provider selection through the outer `current_attempt` ContextVar (#459)
- Gemini `cached_content` reuse keys now include `system_instruction` and `tool_config` so cache hits don't cross configurations (#459)
- CrewAI example updated for the latest CrewAI protocol (#631)
### Removed
- `src/utils/clients.py` deleted; its responsibilities are split across `src/llm/registry.py`, `src/llm/credentials.py`, and the backend-specific modules (#459)
- `HEALTHCHECK` directive from the shared Docker image (#530)
</Update>
<Update label="v3.0.6">
### Changed
- Tightened transaction scopes across search, agent tools, queue manager, and webhook delivery to minimize DB connection hold time during external operations (#525)
- Search operations refactored to two-phase pattern — external work (embeddings, LLM calls) completes before opening a transaction (#525)
- Agent tool executor performs external operations before acquiring DB sessions (#525)
- Queue manager transaction scope reduced to only the critical section (#525)
- Webhook delivery no longer holds a DB session parameter (#525)
### Fixed
- Session leakage in non-session-scoped dialectic chat calls (#526)
### Added
- Health check endpoint (`/health`) for container orchestration and load balancer probes (#510)
</Update>
<Update label="v3.0.5">
### Fixed
- explicit rollback on all transactions to force connection closed
</Update>
<Update label="v3.0.4">
### Added
- JSONB metadata validation enforces 100 key limit and max depth of 5 (#419)
### Changed
- Schemas refactored from single `schemas.py` into `schemas/api.py`, `schemas/configuration.py`, and `schemas/internal.py` with backwards-compatible re-exports (#419)
### Fixed
- Missing `deleted_at` filter on `RepresentationManager._query_documents_recent()` and `._query_documents_most_derived()` allowed soft-deleted documents to leak into the deriver's working representation (#456)
- `CleanupStaleItemsCompletedEvent` emitted spuriously when no queue item was actually deleted (#454)
- Memory leak: `_observation_locks` switched to `WeakValueDictionary` to prevent unbounded growth (#419)
- SQL injection in `dependencies.py`: parameterized `set_config` calls to prevent injection via request context (#419)
- NUL byte crashes: string inputs (message content, queries, peer cards) now stripped at schema level (#419)
- Filter recursion depth capped at 5 to prevent stack overflow (#419)
- Dedup-skipped observations now correctly reflected in created counts (#477)
- External vector store support for message search — routes queries through configured external vector store with oversampling and
deduplication to handle chunked embeddings (#479)
- Dialectic agent no longer holds a DB connection during LLM calls — embeddings are pre-computed before tool execution, DB sessions isolated in `extract_preferences`, `query_documents` no longer accepts a DB session parameter (#477)
</Update>
<Update label="v3.0.3">
### Added
- Consolidated session context into a single DB session with 40/60 token budget allocation between summary and messages
@ -477,7 +766,7 @@ Welcome to the Honcho changelog! This section documents all notable changes to t
### Changed
- `/list` endpoints to not require a request body
- `metamessage_type` to `label` with backwards compatability
- `metamessage_type` to `label` with backwards compatibility
- Database Provisioning to rely on alembic
- Database Session Manager to explicitly rollback transactions before closing
the connection
@ -511,7 +800,72 @@ Welcome to the Honcho changelog! This section documents all notable changes to t
<Tab title="Python SDK">
[Python SDK](https://pypi.org/project/honcho-ai/)
<Update label="v2.0.1 (Current)">
<Update label="v2.4.0">
### Added
- Scopes: `Honcho.scope()` / `HonchoAio.scope()` get-or-create a named visibility boundary, `Honcho.scopes()` lists them, and a `Scope` object adds/removes sessions, lists membership, and reads backfill `status()`. `Honcho.session(..., scopes=[...])` joins a new session to scopes at creation. Requires a Honcho server with the matching API support (Honcho v3.1.0+).
- `scope` option on `Peer.chat()` / `chat_stream()`, representation, session context, and workspace search. A single scope answers from that scope's collection and card; a list of scopes restricts recall to the union of their member sessions (explicit-only). Mutually exclusive with `session` / `sessions` / `filters`.
- Workspace-level chat: `Honcho.chat()` / `HonchoAio.chat()` and `chat_stream()` ask a question across every peer in the workspace, with the same `session`, `scope`, `reasoning_level`, and `response_format` options as `Peer.chat()`. Requires a Honcho server with the matching API support (Honcho v3.1.0+).
### Changed
- `ConclusionScope` is renamed to `ConclusionsView`. The old name remains as a deprecated alias for one more minor version. "Scope" now means a named set of sessions (`Scope`); these objects are views over one observer/observed pair.
</Update>
<Update label="v2.3.0">
### Added
- `response_format` on `Peer.chat()` / `PeerAio.chat()` and `Peer.chat_stream()` / `PeerAio.chat_stream()`, for constraining a dialectic answer to a schema. Pass a Pydantic model class to get a validated instance back (parsed via `model_validate_json`), or a raw JSON Schema dict to get the JSON string as-is. Overloads type the return precisely, so a model class narrows to that model and a dict narrows to `str`. On the streaming variants, chunks stay raw text that accumulates to a JSON string — parse it after the stream completes. Requires a Honcho server with the matching API support (Honcho v3.0.12+).
- `response_format` field on `DialecticParams`.
</Update>
<Update label="v2.2.0">
### Added
- `ConclusionLevel` type (`explicit`, `deductive`, `inductive`, `contradiction`) and a `level` field on `Conclusion`, exposing the reasoning level the server already tracked but previously stripped from responses.
- `filters` parameter on `ConclusionScope.list()` and `ConclusionScope.query()` (sync and async), passed through to the same dynamic server-side filter logic as `peers()`/`sessions()`/`messages()`. Filter explicit-only conclusions with `filters={"level": "explicit"}`, or by any other supported field/operator. Requires a Honcho server with the matching API support (Honcho v3.0.11+).
### Fixed
- Scope-managed filter keys (`observer`, `observed`, `session`) are now rejected with a clear `ValueError` if passed in `filters`, instead of silently overriding the scope and returning conclusions from a different peer pair. Use `peer.conclusions` / `conclusions_of(target)` and the `session=` parameter instead. `session_id` remains a valid filter on `query()`.
</Update>
<Update label="v2.1.2">
### Added
- `page`, `size`, and `reverse` pagination parameters on `Honcho.workspaces()` and `HonchoAio.workspaces()`, closing the gap from 2.1.0 which added these to other list methods but not to `workspaces()`. Honoring `reverse` on the workspace/peer/session list routes also requires a Honcho server with the matching API fix; older servers silently ignore the parameter.
- `peers` parameter on `Honcho.session()` and `HonchoAio.session()` — attach peers to a session at creation time instead of needing a follow-up `session.add_peers()` call. Accepts the same shapes as `Session.add_peers` (peer ID string, `Peer` object, list of either, or tuples with `SessionPeerConfig`).
### Changed
- `WorkspaceCreateParams`, `PeerCreateParams`, and `SessionCreateParams` now accept IDs up to 512 characters (was 100), matching the server-side schema change in Honcho v3.0.7.
</Update>
<Update label="v2.1.1">
### Fixed
- Broadened HTTP retry logic to cover `httpx.NetworkError` and `httpx.RemoteProtocolError` in addition to `httpx.TimeoutException` and `httpx.ConnectError`, improving resilience against transient network failures
</Update>
<Update label="v2.1.0">
### Added
- `created_at` property on `Peer` and `Session` objects
- `is_active` property on `Session` objects
- `get_message(message_id)` method on `Session` (sync and async) to fetch a single message by ID
- `page`, `size`, and `reverse` pagination parameters on all list methods
### Changed
- **Breaking**: `peer()` and `session()` now always make a get-or-create API call — no more lazy initialization
- Response configuration models now tolerate unknown fields from newer servers for forward compatibility
### Fixed
- Sync and async `Session.get_metadata()`, `get_configuration()`, and `refresh()` now refresh cached `created_at` and `is_active` values along with metadata and configuration
- `honcho.__version__` now derives from package metadata, with a source-checkout fallback, so it stays aligned with released package versions
</Update>
<Update label="v2.0.2">
### Changed
- All input models now reject unknown fields via strict Pydantic validation (`extra="forbid"`). Previously, misspelled or extraneous fields were silently ignored. Now a `ValidationError` is raised with the unrecognized field name.
</Update>
<Update label="v2.0.1">
### Added
- `set_peer_card` method
@ -625,7 +979,98 @@ Welcome to the Honcho changelog! This section documents all notable changes to t
- Scopes: `honcho.scope()` get-or-creates a named visibility boundary, `honcho.scopes()` lists them, and a `Scope` object adds/removes sessions, lists membership, and reads backfill `status()`. `honcho.session({ scopes: [...] })` joins a new session to scopes at creation. Requires a Honcho server with the matching API support (Honcho v3.1.0+).
- `scope` option on `peer.chat()` / `chatStream()`, representation, session context, and workspace search. A single scope answers from that scope's collection and card; a list of scopes restricts recall to the union of their member sessions (explicit-only). Mutually exclusive with `session` / `sessions` / `filters`.
- Workspace-level chat: `honcho.chat()` / `honcho.chatStream()` ask a question across every peer in the workspace, with the same `session`, `scope`, `reasoningLevel`, and `responseFormat` options as `peer.chat()`. Requires a Honcho server with the matching API support (Honcho v3.1.0+).
### Changed
- `ConclusionScope` is renamed to `ConclusionsView`. The old name remains as a deprecated alias for one more minor version. "Scope" now means a named set of sessions (`Scope`); these objects are views over one observer/observed pair.
</Update>
<Update label="v2.3.0">
### Added
- `responseFormat` option on `peer.chat()` and `peer.chatStream()`, for constraining a dialectic answer to a schema. Pass a Zod schema to get a parsed, validated result back, or a raw JSON Schema object to get the JSON string as-is. Overloads type the return precisely, so a Zod schema narrows to its inferred type and a plain object narrows to `string`. On `chatStream()`, chunks stay raw text that accumulates to a JSON string — parse it after the stream completes. Requires a Honcho server with the matching API support (Honcho v3.0.12+).
</Update>
<Update label="v2.2.0">
### Added
- `ConclusionLevel` type (`explicit`, `deductive`, `inductive`, `contradiction`) and a `level` field on `Conclusion`, exposing the reasoning level the server already tracked but previously stripped from responses.
- `filters` option on `conclusions.list()` and `conclusions.query()`, passed through to the same dynamic server-side filter logic as the other list endpoints. Filter explicit-only conclusions with `{ filters: { level: 'explicit' } }`, or by any other supported field/operator. Requires a Honcho server with the matching API support (Honcho v3.0.11+).
### Fixed
- Scope-managed filter keys (`observer`, `observed`, `session`) are now rejected with a clear error if passed in `filters`, instead of silently overriding the scope and returning conclusions from a different peer pair. Use `peer.conclusions` / `peer.conclusionsOf(target)` and the dedicated `session` option instead. `session_id` remains a valid filter on `query()`.
</Update>
<Update label="v2.1.2">
### Added
- `peers` option on `Honcho.session()` — attach peers to a session at creation time instead of needing a follow-up `session.addPeers()` call. Accepts the same `PeerAddition` shape as `session.addPeers()` (peer ID strings, `Peer` objects, arrays of either, or a record with per-peer `observe_me`/`observe_others` config).
### Changed
- ID validation in `validation.ts` now accepts workspace, peer, and session IDs up to 512 characters (was 100), matching the server-side schema change in Honcho v3.0.7.
### Fixed
- `Honcho.workspaces()` now actually forwards the `reverse` option to the server. The 2.1.0 changelog listed `workspaces()` among the list methods that gained `reverse`, but `client.ts` was missing the field on the params type and request builder, so the option was silently dropped. Honoring `reverse` on the workspace/peer/session list routes also requires a Honcho server with the matching API fix; older servers silently ignore the parameter.
</Update>
<Update label="v2.1.1">
### Fixed
- Broadened fetch error retry logic to catch all `TypeError` network failures (connection resets, DNS errors, etc.) instead of only those with `'fetch'` in the message, improving resilience across runtimes (Node, Bun, browsers)
</Update>
<Update label="v2.1.0">
### Added
- `createdAt` property on `Peer` and `Session` wrapper objects
- `isActive` property on `Session` wrapper objects
- `getMessage(messageId)` method on `Session` to fetch a single message by ID
- `Peer.representation()`, `Session.representation()`, and `Session.context()` now accept `Message` objects for `searchQuery`
- `page`, `size`, and `reverse` pagination controls on all list methods
### Changed
- **Breaking**: `searchQuery` removed from top-level `context()` options — use `representationOptions.searchQuery` instead:
- List methods (`peers()`, `sessions()`, `messages()`, `workspaces()`) support both the new options object and the legacy raw-filter form
- Representation search options now accept strings and content-like objects, including `Message` instances, while rejecting whitespace-only or invalid runtime inputs
- **Breaking**: `peer()` and `session()` now always make a get-or-create API call — no more lazy initialization. If you relied on constructing SDK objects without triggering a network request, note that every `peer()` and `session()` call now hits the API:
```typescript
// Before (v2.0.x) — no API call
const session = honcho.session("my-session");
// After (v2.1.0) — makes a get-or-create API call
- Response configuration models now tolerate unknown fields from newer servers for forward compatibility
- Moved `@types/node` from `dependencies` to `devDependencies`
### Fixed
- `uploadFile()` now rejects unsupported top-level binary/object inputs and only validates inputs the serializer can actually upload
- `uploadFile()` now serializes message configuration using API field names, matching `addMessages()`
- Session fetch methods now refresh cached `createdAt` and `isActive` values alongside metadata and configuration
</Update>
<Update label="v2.0.2">
### Changed
- Client constructor now rejects unknown options via `.strict()` Zod validation. Previously, misspelled options (e.g., `baseUrl` instead of `baseURL`) were silently ignored, causing the SDK to fall back to defaults. Now a `ZodError` is thrown with the unrecognized key name.
- All input schemas now use `.strict()` validation to reject unknown fields.
- `FileUploadSchema.configuration` now uses `MessageConfigurationSchema` instead of open record type.
### Fixed
- README example used `baseUrl` instead of `baseURL`.
</Update>
<Update label="v2.0.1">
### Added
- `setPeerCard` method
@ -738,6 +1183,54 @@ Welcome to the Honcho changelog! This section documents all notable changes to t
- A TTY notice when a newer `honcho-cli` is on PyPI (`uv tool upgrade honcho-cli`). Skipped in JSON mode; disable with `HONCHO_NO_UPDATE_CHECK`
### Fixed
- `--setup` for openai-compatible writes `EMBEDDING_MODEL_CONFIG__OVERRIDES__BASE_URL` into the profile `.env` alongside `LLM_OPENAI_BASE_URL` (#1068)
- `--setup` API key prompts echo `*` per character so a paste is visibly received instead of a blank getpass field
</Update>
<Update label="v0.1.3">
### Added
- `honcho start`, `honcho stop`, and `honcho status` — run a personal Honcho stack in Docker (API, deriver, Postgres, Redis). Profiles live under `~/.honcho/profiles/`. First start pins `ghcr.io/plastic-labs/honcho:latest` by digest and copies the image `config.toml`. Optional `--setup basic` / `--setup advanced` wizard writes LLM overrides to `.env` (#1029)
- `honcho session view` — session transcript table (`--last N`, `--page N --size M`, `--all`, `--reverse`, `--ids`, peer filter via `-p`). Content is shown verbatim, timestamps are normalized to UTC, and the command is read-only: unlike the other session commands it never get-or-creates the session (#1006)
### Fixed
- `honcho message list --last N` no longer stops at the first page of 50 — it walks pages to fill the requested window (#1006)
</Update>
<Update label="v0.1.2">
### Added
- Device-code OAuth login for managed Honcho servers. `honcho init` now offers browser-based login (RFC 8628 device authorization grant) when the host advertises the device grant in its OAuth authorization-server metadata; tokens are persisted to `~/.honcho/config.json` and auto-refreshed (#891)
- `HONCHO_CONFIG_DIR` environment variable for pointing the CLI at an alternate config directory (#891)
### Changed
- An OAuth grant now records the host it was minted against and is ignored — neither used nor refreshed — when `base_url` points elsewhere, so a staging grant is never sent to production. A live OAuth token takes precedence over a stored `apiKey`, and a dead grant degrades to the saved key with a warning instead of aborting. Device login no longer deletes the shared `apiKey`, which sibling tools read from the same config file (#891)
</Update>
<Update label="v0.1.1">
### Fixed
- Declare `click` as an explicit dependency. The CLI imported `click` directly but relied on it being pulled in transitively, so installs without it on the path could fail at runtime (#787)
</Update>
<Update label="v0.1.0">
### Added
- Initial release of `honcho-cli` — a terminal for inspecting and managing a Honcho deployment (#424)
- `workspace`, `peer`, `session`, `message`, `conclusion`, and `config` command groups for managing resources against any Honcho server
- `init` onboarding flow that prompts for and persists connection settings, with flag/env-var pre-seeding for non-interactive use
- Per-command flags, environment variables, and a config file for pointing the CLI at different servers (local, self-hosted, or hosted)
- Rich terminal output and an agent-usage mode for scripting against the CLI
- Documentation and an agent skill for the CLI (#589)
</Update>
</Tab>
</Tabs>
## Getting Help
@ -745,4 +1238,4 @@ Welcome to the Honcho changelog! This section documents all notable changes to t
If you encounter issues using the Honcho API or its SDKs:
1. Open an issue on [GitHub](https://github.com/plastic-labs/honcho/issues)
2. Join our [Discord community](http://discord.gg/plasticlabs) for support
2. Join our [Discord community](http://discord.gg/honcho) for support
@ -59,4 +59,4 @@ Finally, Claude needs instructions on how to use Honcho. The Desktop app doesn't
<Note>Be sure to update the \<app_name\> and \<user_name\> variables in the instructions.txt file.</Note>
Claude should then query for insights before responding and write your messages to storage! If you come up with more creative ways to get Claude to manage its own memory with Honcho, feel free to [let us know](https://discord.gg/plasticlabs) or make a PR on this [repo](https://github.com/plastic-labs/honcho-mcp/tree/main)!
Claude should then query for insights before responding and write your messages to storage! If you come up with more creative ways to get Claude to manage its own memory with Honcho, feel free to [let us know](https://discord.gg/honcho) or make a PR on this [repo](https://github.com/plastic-labs/honcho-mcp/tree/main)!
Thank you for your interest in contributing to Honcho! This guide outlines the process for contributing to the project and our development conventions.
## Before you write code
**Every pull request needs an issue, and that issue needs the `maintainer-approved` label.**
A pull request that is not linked to an approved issue gets labelled `needs-approved-issue`, with a comment explaining why. You then have 72 hours to link one before it is closed automatically. Reopening costs nothing once the link is in place. This is automated. We do this because an unreviewable backlog helps nobody: a PR against an unapproved issue is work you did that we may not be able to merge, no matter how good it is.
So, in order:
1. **Find approved work.** Browse [issues labelled `maintainer-approved`](https://github.com/plastic-labs/honcho/issues?q=is%3Aissue+is%3Aopen+label%3Amaintainer-approved). That label is the queue of things we have agreed should be built. Anything in it is fair game — comment on the issue to claim it.
2. **Or open an issue and get it approved.** Use the [issue templates](https://github.com/plastic-labs/honcho/issues/new/choose). Maintainers triage and apply the label.
3. **If you feel strongly about an issue, come to [Discord](https://discord.gg/honcho).** This is the fastest path by a wide margin. Maintainers are more active there than in the issue tracker, and a five-minute conversation about what you want to build usually resolves whether it fits before either side spends real time on it.
4. **Then open the PR** and link the issue — either `Fixes #123` in the description, or **Development → link an issue** in the sidebar. Both work.
Small exceptions we will not be pedantic about: fixing a typo, a broken link, or an obviously wrong code sample. Open the PR, explain it in one line, and we will sort out the issue linkage.
## Getting Started
Before you start contributing, please:
1. **Set up your development environment** - Follow the [Local Development guide](https://github.com/plastic-labs/honcho/blob/main/CONTRIBUTING.md#local-development) in the Honcho repository to get Honcho running locally.
2. **Join our community** - Feel free to join us in our [Discord](http://discord.gg/plasticlabs) to discuss your changes, get help, or ask questions.
2. **Join our community** - Feel free to join us in our [Discord](https://discord.gg/honcho) to discuss your changes, get help, or ask questions.
3. **Review existing issues** - Check the [issues tab](https://github.com/plastic-labs/honcho/issues) to see what's already being worked on or to find something to contribute to.
- Any relevant issue numbers (use "Closes #123" to auto-close issues)
- A link to the approved issue — `Fixes #123` in the description, or **Development → link an issue** in the sidebar. This is required; see [Before you write code](#before-you-write-code).
1. **Automated checks** - Your PR will run through automated checks including tests and linting
1. **Automated checks** - Your PR will run through automated checks including tests, linting, and the issue gate
2. **Project maintainer review** - A project maintainer will review your code for:
- Code quality and adherence to standards
- Functionality and correctness
@ -153,17 +171,21 @@ We welcome various types of contributions:
When reporting bugs or requesting features:
1. Check if the issue already exists
2. Use the appropriate issue template
2. Use the appropriate [issue template](https://github.com/plastic-labs/honcho/issues/new/choose) (bug, memory/recall quality, feature, integration, or documentation)
3. Provide clear reproduction steps for bugs
4. Include relevant environment information
4. Include relevant environment information (managed vs self-hosted, server version, SDK)
5. Be specific about expected vs actual behavior
6. Redact secrets, JWTs, and production user content
- **Security issues** - Report **privately** only — see [`SECURITY.md`](https://github.com/plastic-labs/honcho/blob/main/SECURITY.md) (GitHub Private Vulnerability Reporting or email). Do not open a public issue.
@ -353,6 +353,20 @@ import { Honcho } from "@honcho-ai/sdk";
```
</CodeGroup>
### Observation and Peer Join Order
Reasoning tasks are scheduled at the time a message is created, based on which peers are in the session **at that moment**. Honcho does not retroactively schedule reasoning for peers that join later.
This means:
- If Peer C joins a session **after** messages from Peer A and Peer B have already been sent, Peer C will **not** receive reasoning tasks for those earlier messages—even if Peer C has `observe_others` enabled.
- Peer C will only begin observing new messages sent after they join the session.
- Similarly, if a peer leaves a session, they stop being included as an observer for any messages sent after their departure.
<Warning>
There is no retroactive reasoning. If your application needs an observer peer to reason about prior conversation history, add the peer to the session **before** messages are sent. Alternatively use the .chat() endpoint to include the conversation history in the agent's context, regardless of if they were reasoned against or not
For the full filter syntax including logical operators (AND, OR, NOT), comparison operators, and metadata filtering, see the [Using Filters](https://docs.honcho.dev/v2/documentation/core-concepts/features/using-filters) documentation.
For the full filter syntax including logical operators (AND, OR, NOT), comparison operators, and metadata filtering, see the [Using Filters](https://honcho.dev/docs/v2/documentation/core-concepts/features/using-filters) documentation.
<Note>
For comprehensive details about CrewAI's memory system, see the [official CrewAI Memory documentation](https://docs.crewai.com/en/concepts/memory).
@ -70,4 +70,4 @@ You may customize your assistant name and/or workspace ID. Both are optional.
4. Finally, Claude needs instructions on how to use Honcho. The Desktop app doesn't allow you to add system prompts directly, but you can create a project and paste these [instructions](https://raw.githubusercontent.com/plastic-labs/honcho/refs/heads/main/mcp/instructions.md) into the "Project Instructions" field.
Claude should then query for insights before responding and write your messages to storage! If you come up with more creative ways to get Claude to manage its own memory with Honcho, feel free to [let us know](https://discord.gg/plasticlabs) or make a PR on this [repo](https://github.com/plastic-labs/honcho/tree/main/mcp)!
Claude should then query for insights before responding and write your messages to storage! If you come up with more creative ways to get Claude to manage its own memory with Honcho, feel free to [let us know](https://discord.gg/honcho) or make a PR on this [repo](https://github.com/plastic-labs/honcho/tree/main/mcp)!
description: "How to switch embedding dimension or model on a Honcho deployment"
icon: "rotate"
---
## Short answer: you can't, in place.
The embedding dimension is **machine-enforced** as immutable for the life of a deployment. The embedding model is **operator-owned** as immutable by contract. The supported way to change either is:
1. Stand up a new deployment at the desired configuration.
2. Replay or re-embed your data into it out-of-band.
3. Cut traffic over to the new deployment.
The rest of this page explains why, and what the safety boundaries actually are.
## Why dimension is enforced and model is not
On boot, both the API (`src/main.py` lifespan) and the deriver (`src/deriver/__main__.py`) run the validator in `src/startup/embedding_validator.py`. It does a schema-qualified `pg_attribute` lookup against `documents.embedding` and `message_embeddings.embedding`, decodes the declared `atttypmod`, and compares it to `EMBEDDING_VECTOR_DIMENSIONS`. A mismatch crashes the process with an actionable error before any HTTP route is served or any queue task is processed.
There is no equivalent check for the model. The pgvector column does not record what model produced the vectors inside it, and this design intentionally avoids adding new persistent metadata fields. The runtime has no way to detect that you swapped `text-embedding-3-small` for a different model that emits the same dimension.
That last point is a real footgun:
<Warning>
Changing `EMBEDDING_MODEL_CONFIG__MODEL` to a different model at the **same dimension** (for example `text-embedding-3-small@1536` → `text-embedding-3-large` truncated to 1536) will silently succeed. New writes will use the new model; existing rows still hold vectors from the old model; recall quality will degrade with no startup or runtime warning.
Treat model identity as a contract you own. If you need to change it, follow the destroy + rebuild path below.
</Warning>
## Recipe: changing dim or model
Concretely, for either a dim change or a model change:
1. **Provision the new deployment** with the target environment.
uv run python scripts/configure_embeddings.py --dry-run
uv run python scripts/configure_embeddings.py --yes
```
2. **Replay your source data** (messages, documents, ingested content) into the new deployment via your normal application path. Honcho's existing message-creation API will re-derive embeddings using the new configuration. There is no in-place re-embedding tool — that would be a separate spec covering atomicity, cost-per-token, and dialectic-during-migration semantics.
3. **Cut over** at your application layer (DNS, load balancer, feature flag — whatever you use). The old deployment can stay running until you are confident in the new one; this design does not require an atomic switch.
The startup validator on the new deployment will refuse to start if step 1's `configure_embeddings.py` did not run, so a misconfiguration cannot quietly write wrong-dim vectors into the new schema.
## Edge case: truncation at the default dimension
If you are using `text-embedding-3-large` but truncating to 1536 (the default), be aware that `EMBEDDING_MODEL_CONFIG__DIMENSIONS_MODE=auto` will **not** forward `dimensions=` to the API — `auto` interprets the default as "operator did not opt into a non-default dim." The provider will return native 3072, the response-dim validator will reject it, and the request will fail.
For this case, either set `EMBEDDING_VECTOR_DIMENSIONS=1536` explicitly (so `auto` knows the operator opted in), or set `EMBEDDING_MODEL_CONFIG__DIMENSIONS_MODE=always`.
## Backend swap (turbopuffer ↔ lancedb ↔ pgvector) is a different operation
Switching the *storage backend* at constant dim/model — for example moving from pgvector to Turbopuffer — is supported via `src/reconciler/sync_vectors.py` and `VECTOR_STORE_MIGRATED`. That flow is unchanged by the embedding-pipeline work and is documented separately. It is **not** the destroy + rebuild path described above.
Thank you for your interest in contributing to Honcho! This guide outlines the process for contributing to the project and our development conventions.
{/* This file mirrors CONTRIBUTING.md in the repo root. Update both. */}
## Getting Started
Thanks for your interest in contributing. This guide covers how work gets accepted, how
Honcho is put together, and what a mergeable pull request looks like.
Before you start contributing, please:
Honcho is a small team maintaining a project that gets more proposals than we can review.
The rules below exist so that the work you do has somewhere to land — not to keep you out.
1. **Set up your development environment** - Follow the [Local Development guide](https://github.com/plastic-labs/honcho/blob/main/CONTRIBUTING.md#local-development) in the Honcho repository to get Honcho running locally.
## Before you write code
2. **Join our community** - Feel free to join us in our [Discord](http://discord.gg/plasticlabs) to discuss your changes, get help, or ask questions.
**Every pull request needs an issue, and that issue needs the `maintainer-approved` label.**
3. **Review existing issues** - Check the [issues tab](https://github.com/plastic-labs/honcho/issues) to see what's already being worked on or to find something to contribute to.
A pull request that is not linked to an approved issue gets labelled
`needs-approved-issue`, with a comment explaining why. You then have 72 hours to link one
before it is closed automatically. Reopening costs nothing once the link is in place. This
is automated. We do this because an unreviewable backlog helps nobody: a PR against an
unapproved issue is work you did that we may not be able to merge, no matter how good it
| Does | Serves HTTP, enqueues background work, returns immediately | Consumes the queue: Deriver, Summarizer, Dreamer, Reconciler |
| Hosts | The Dialectic agent, inline on the request path | Everything else |
The split is the load-bearing design decision: **an HTTP request never blocks on LLM work**,
with the single exception of the Dialectic chat endpoint, which is synchronous by nature.
If you are adding something slow, it belongs in the worker.
The deriver is a separate process. If messages go in and nothing ever comes out, the usual
cause is that nobody started it.
### The path of a message
Worth tracing once, because it crosses most of the codebase:
1. `POST /v3/workspaces/{w}/sessions/{s}/messages` lands in `src/routers/messages.py`.
2. The row is written, then `enqueue()` in `src/deriver/enqueue.py` creates `queue_item`
rows — one set of work per observing peer.
3. `src/deriver/queue_manager.py` polls the queue, claiming work units so that messages in a
session are processed in order.
4. `process_item()` in `src/deriver/consumer.py` dispatches on task type — representation,
summary, deletion, reconciliation.
5. For a representation task, `process_representation_tasks_batch()` in
`src/deriver/deriver.py` makes **one structured-output LLM call for the whole batch** and
writes the resulting conclusions into the collection keyed by the
`(observer, observed)` peer pair.
6. Later, `src/dialectic/` reads those conclusions back at recall time to answer a chat
request.
Embedding is deliberately *not* on this path. `MessageEmbedding` rows are written with
`sync_state='pending'` and embedded asynchronously by the Reconciler
(`src/reconciler/sync_vectors.py`), which runs on a scheduler inside the deriver process.
### The four agents
They share tool definitions in `src/utils/agent_tools.py` and the provider-agnostic LLM
client in `src/llm/`. Each has its own `MODEL_CONFIG` with a fallback chain in
`src/config.py`.
| Agent | Where | Shape |
| --- | --- | --- |
| **Deriver** | `src/deriver/` | A single structured-output call per message batch. Not a tool loop — this is a deliberate cost and latency tradeoff. |
| **Dialectic** | `src/dialectic/` | The one tool-using agent on the request path. Loops over tools until it can answer. Five reasoning tiers from `minimal` to `max`, each with its own model and tool set. |
| **Dreamer** | `src/dreamer/` | Off-queue consolidation. Two specialist phases (deduction, then induction) that build reasoning trees over existing conclusions. |
| **Summarizer** | `src/utils/summarizer.py` | Direct LLM call, no tools. Two tiers — short and long summaries at different message counts. |
Prompts live in `src/deriver/prompts.py`, `src/dialectic/prompts.py`, and
`src/dreamer/specialists.py`.
### A note on naming
What the public API and documentation call **conclusions** are called **observations**
throughout the code — `create_observations`, `get_observation_context`, and so on. Likewise
**collections** and **documents** are internal storage concepts that are not exposed
directly through the API. Do not rename across that boundary in a drive-by change; the
public and internal vocabularies are being reconciled deliberately.
## Where to change what
| I want to change... | Start here |
| --- | --- |
| An HTTP endpoint | `src/routers/` — one module per resource |
| A database query | `src/crud/` — mirrors the router layout |
| The database schema | `src/models.py`, plus a migration in `migrations/versions/` |
| A configuration value | `src/config.py`, and add it to `config.toml.example` and `.env.template` |
| A tool an agent can call | `src/utils/agent_tools.py` — definitions plus the per-agent tool lists |
| A prompt | `src/deriver/prompts.py`, `src/dialectic/prompts.py`, `src/dreamer/specialists.py` |
This guide helps you set up a local environment to run Honcho for development, testing, or self-hosting.
**Just want a running instance?** `uv tool install honcho-cli` only installs the `honcho` command. Then run [`honcho start --setup`](/v3/documentation/reference/cli#local-stack) (Docker + an LLM provider key) — that pulls a published image and starts API, deriver, Postgres, and Redis. The rest of this page is for building from source, contributing, or deploying without the CLI.
## Overview
By the end of this guide, you'll have:
@ -20,9 +22,9 @@ By the end of this guide, you'll have:
Before you begin, ensure you have the following installed:
- **Git** - [Download from git-scm.com](https://git-scm.com/downloads)
- **Docker** (optional) - [Download from docker.com](https://www.docker.com/products/docker-desktop/)
- **Docker** - required for the CLI local stack and the compose-from-source path; not needed for a fully manual setup. [Download from docker.com](https://www.docker.com/products/docker-desktop/)
### Database Options
You'll need a PostgreSQL database with the pgvector extension. Choose one:
@ -32,9 +34,51 @@ You'll need a PostgreSQL database with the pgvector extension. Choose one:
- **Railway** - Simple cloud PostgreSQL hosting
- **Your own PostgreSQL server**
## Docker Setup (Recommended)
## LLM Setup
The easiest way to get started is using Docker Compose, which handles both the database and Honcho server.
Honcho uses LLMs for memory extraction, summarization, dialectic chat, and dreaming. The server will **fail to start** without a provider configured.
If you keep the built-in defaults, you only need one API key: all text-generation features default to `openai / gpt-5.4-mini`, and embeddings default to `openai / text-embedding-3-small`. Any OpenAI-compatible endpoint works too — OpenRouter, Together, Fireworks, Ollama, vLLM, or LiteLLM. Models must support tool calling (function calling).
After copying `.env.template` to `.env`, the default setup is:
```bash
# Required for the built-in defaults
LLM_OPENAI_API_KEY=sk-...
```
If you want a different model or an OpenAI-compatible proxy, uncomment and edit the relevant `*_MODEL_CONFIG__TRANSPORT`, `*_MODEL_CONFIG__MODEL`, and `*_MODEL_CONFIG__OVERRIDES__BASE_URL` lines in the Deriver, Dialectic, Summary, and Dream sections. For example:
For recommended model tiers per feature, using multiple providers, or direct vendor API keys, see the [Configuration Guide](./configuration#llm-configuration).
</Info>
## Personal local stack (CLI)
Recommended if you want Honcho running locally without cloning this repo or building an image. Install the CLI, then run the setup wizard:
```bash
uv tool install honcho-cli
honcho start --setup basic # prompts for LLM provider + key, then starts Docker
```
`honcho start` pulls `ghcr.io/plastic-labs/honcho:latest`, pins that digest, and starts API + deriver + Postgres + Redis. Stack files live under `~/.honcho/profiles/local/`. It does **not** rewrite `environmentUrl` in `~/.honcho/config.json` (that file is shared with plugins). Talk to the stack with `HONCHO_BASE_URL=http://127.0.0.1:8000`, or run `honcho init --base-url http://127.0.0.1:8000` to persist local as the CLI default.
See the [CLI reference](/v3/documentation/reference/cli#local-stack) for `--setup`, profiles, `--image`, ports, `honcho status` / `stop`, and pointing the CLI at local.
## From source (Docker Compose)
Docker Compose in this repo handles the database, Redis, and Honcho server. The compose file **builds the image from source** so you can develop against local code. A pre-built image is published at `ghcr.io/plastic-labs/honcho:latest` (what `honcho start` uses); it is not on Docker Hub. Building from source requires Docker with BuildKit enabled — see [Troubleshooting](./troubleshooting#docker-build-fails-with-permission-errors) if the build fails.
The compose file is production-oriented by default (ports bound to `127.0.0.1`, restart policies, caching enabled). For development, uncomment the source mounts and monitoring services inside the file.
### 1. Clone the Repository
@ -51,45 +95,39 @@ Copy the example environment file and configure it:
cp .env.template .env
```
Edit `.env` and set your API keys (if using LLM features):
```bash
# Optional API keys (required for LLM features)
OPENAI_API_KEY=your-openai-api-key
ANTHROPIC_API_KEY=your-anthropic-api-key
# Database will be created automatically by Docker
Edit `.env` and configure your LLM provider — see [LLM Setup](#llm-setup) above. The database connection is set in the compose file. Auth is disabled by default (`AUTH_USE_AUTH=false`).
### 3. Start the Services
```bash
# Copy the example docker-compose file
cp docker-compose.yml.example docker-compose.yml
# Start PostgreSQL and Honcho
docker compose up -d
docker compose up -d --build
```
### 4. Verify It's Working
The first build takes a few minutes (compiling from source). Subsequent starts are fast.
Check that both services are running:
The default image does not include LanceDB. To use `VECTOR_STORE_TYPE=lancedb`, build with `INSTALL_LANCEDB=true docker compose up -d --build`.
This starts four services: **api** (port 8000), **deriver** (background worker), **database** (PostgreSQL with pgvector, port 5432), and **redis** (port 6379). All ports are bound to `127.0.0.1`. Redis caching is enabled by default.
For development, uncomment the source mount and monitoring sections inside `docker-compose.yml` to enable live reload, Prometheus, and Grafana.
### 4. Verify
Migrations run automatically on startup.
```bash
# Check all containers are running
docker compose ps
```
Test the Honcho API:
```bash
# Health check (confirms the process is up)
curl http://localhost:8000/health
# Check the deriver is processing (look for "polling" or "processing" in logs)
docker compose logs deriver --tail 20
```
You should see a response indicating the service is healthy.
For a full end-to-end test, see [Verify Your Setup](#verify-your-setup) below.
## Manual Setup
@ -134,29 +172,31 @@ Download from [postgresql.org](https://www.postgresql.org/download/windows/)
```bash
docker run --name honcho-db \
-e POSTGRES_DB=honcho \
-e POSTGRES_USER=postgres \
-e POSTGRES_PASSWORD=postgres \
-p 5432:5432 \
-d pgvector/pgvector:pg15
```
### 3. Create Database and Enable Extensions
### 3. Enable Extensions
Connect to PostgreSQL and set up the database:
Connect to PostgreSQL and enable pgvector:
```bash
# Connect to PostgreSQL
psql -U postgres
# Create database and enable extensions
CREATE DATABASE honcho;
\c honcho
# Enable the pgvector extension on the default database
CREATE EXTENSION IF NOT EXISTS vector;
CREATE EXTENSION IF NOT EXISTS pg_trgm;
\q
```
<Note>
**Least-privilege database roles.** Honcho runs `CREATE EXTENSION IF NOT EXISTS vector` before migrations and again at startup, so if the role in `DB_CONNECTION_URI` can't create extensions, both fail with a privilege error — `IF NOT EXISTS` doesn't help, because Postgres checks the privilege first.
Run the statement above once as a superuser (or `rds_superuser`) and the application role needs no extension privileges. See [Troubleshooting](/v3/contributing/troubleshooting) if you hit this.
</Note>
### 4. Configure Environment
Create a `.env` file with your settings:
@ -165,17 +205,10 @@ Create a `.env` file with your settings:
cp .env.template .env
```
Edit `.env` with your configuration:
Edit `.env` — configure your LLM provider (see [LLM Setup](#llm-setup) above) and set the database connection:
The server will be available at `http://localhost:8000`.
### 7. Start the Background Worker (Deriver)
In a **separate terminal**, start the deriver background worker:
```bash
uv run python -m src.deriver
```
The deriver is essential for Honcho's core functionality. It processes incoming messages to extract observations, build peer representations, generate session summaries, and run dream consolidation. Without it, messages will be stored but no memory or reasoning will occur.
## Cloud Database Setup
If you prefer to use a managed PostgreSQL service:
@ -206,7 +249,6 @@ If you prefer to use a managed PostgreSQL service:
2. **Enable pgvector extension** in the SQL editor:
```sql
CREATE EXTENSION IF NOT EXISTS vector;
CREATE EXTENSION IF NOT EXISTS pg_trgm;
```
3. **Get your connection string** from Settings > Database
4. **Update your `.env` file** with the connection string
@ -227,23 +269,38 @@ Once your Honcho server is running, verify everything is working:
```bash
curl http://localhost:8000/health
# {"status":"ok"}
```
### 2. API Documentation
Note: `/health` only confirms the process is running. It does not check database or LLM connectivity.
### 2. Smoke Test (database + API)
This confirms the database connection, migrations, and API are all working:
```bash
# Create a workspace
curl -s -X POST http://localhost:8000/v3/workspaces \
-H "Content-Type: application/json" \
-d '{"name": "test"}' | python3 -m json.tool
```
If you get back a workspace object with an `id`, your database is connected and migrations ran correctly.
### 3. API Documentation
Visit `http://localhost:8000/docs` to see the interactive API documentation.
### 3. Test with SDK
Create a simple test script:
### 4. Test with SDK
```python
from honcho import Honcho
# Connect to your local instance
client = Honcho(base_url="http://localhost:8000")
client = Honcho(
base_url="http://localhost:8000",
workspace_id="test"
)
# Create a test peer
peer = client.peer("test-user")
print(f"Created peer: {peer.id}")
```
@ -259,8 +316,7 @@ Now that Honcho is running locally, you can connect your applications:
from honcho import Honcho
client = Honcho(
base_url="http://localhost:8000", # Your local instance
api_key="your-api-key" # If auth is enabled
base_url="http://localhost:8000",
)
```
@ -269,56 +325,125 @@ client = Honcho(
import { Honcho } from '@honcho-ai/sdk';
const client = new Honcho({
baseUrl: 'http://localhost:8000', // Your local instance
apiKey: 'your-api-key' // If auth is enabled
baseUrl: 'http://localhost:8000',
});
```
### Next Steps
- **Configure Honcho**: Visit the [Configuration Guide](./configuration) for model tiers, provider options, and tuning
- **Use the CLI**: install with `uv tool install honcho-cli`, then [`honcho start --setup`](/v3/documentation/reference/cli#local-stack) for a local stack; inspect with `honcho workspace inspect` / `honcho doctor`
- **Explore the API**: Check out the [API Reference](../api-reference/introduction)
- **Try the SDKs**: See our [guides](../guides) for examples
- **Configure Honcho**: Visit the [Configuration Guide](./configuration) for detailed settings
- **Join the community**: [Discord](https://discord.gg/plasticlabs)
- **Join the community**: [Discord](https://discord.gg/honcho)
## Troubleshooting
### Common Issues
Running into issues? See the [Troubleshooting Guide](./troubleshooting) for detailed solutions to common problems including:
**Database Connection Errors**
- Ensure PostgreSQL is running
- Verify the connection string format: `postgresql+psycopg://...`
- Check that pgvector extension is installed
- Startup failures (missing API keys, database issues)
- Runtime errors ("An unexpected error occurred" on every request)
- Deriver not processing messages
- Database connection and migration issues
- Docker and Redis problems
- CLI local stack (`honcho start`) — missing LLM key, health timeout, still talking to api.honcho.dev
**API Key Issues**
- Verify your OpenAI and Anthropic API keys are valid
- Check that the keys have sufficient credits/quota
**Port Already in Use**
- Pass a different port to FastAPI or stop other services using port 8000
**Docker Issues**
- Ensure Docker is running
- Check container logs: `docker compose logs`
- Restart containers: `docker compose down && docker compose up -d`
**Migration Errors**
- Ensure the database exists and pgvector is enabled
- Check database permissions
- Run migrations manually: `uv run alembic upgrade head`
- Secure your database with strong credentials and restrict network access
- The production compose binds PostgreSQL and Redis to `127.0.0.1` only — they are not accessible from the network
### Scaling the Deriver
- Increase `DERIVER_WORKERS` (default: 1) for higher message throughput
- You can also run multiple deriver processes across machines — they coordinate via the database queue
- Monitor deriver logs for processing backlog
### Scaling the API
Set `API_WORKERS` to run multiple API server processes in the Docker container. It defaults to `1`, preserving the existing single-process behavior.
When connection pooling is enabled (`DB_POOL_CLASS` is not `null`), each API process creates its own SQLAlchemy connection pool. Keep the combined capacity below the PostgreSQL connection limit:
With the default pooled settings (`10 + 20`), each API worker can open up to 30 connections. For example, `API_WORKERS=3` allows up to 90 API connections. Leave additional headroom for the deriver, migrations, administration, and monitoring.
When `DB_POOL_CLASS=null`, SQLAlchemy uses `NullPool`; `DB_POOL_SIZE` and `DB_MAX_OVERFLOW` do not apply, and connections are opened and closed per use.
### Caching
- The production compose enables Redis caching by default (`CACHE_ENABLED=true`)
- For the development compose, enable manually: `CACHE_ENABLED=true`
- Configure `CACHE_URL` to point to your Redis instance (or use a managed Redis service)
### Database Migrations
- Always run `uv run alembic upgrade head` after updating Honcho before starting the server
- Check current migration status with `uv run alembic current`
### LLM Providers
- Ensure your API keys are configured (see [LLM Setup](#llm-setup))
- For alternative providers or per-feature model overrides, see the [Configuration Guide](./configuration#llm-configuration)
### Monitoring
- Enable Prometheus metrics with `METRICS_ENABLED=true`. The API exposes `/metrics` on port 8000, the deriver on port 9090 (internal to its container — not published to the host by default).
- Enable Sentry error tracking with `SENTRY_ENABLED=true`
- The development compose includes Prometheus (host port 9090) and Grafana (host port 3000) for scraping and dashboards. Uncomment those services to enable them.
The API can be deployed on [Fly.io](https://fly.io). Follow the [Fly.io getting-started docs](https://fly.io/docs/getting-started/) to set up your account and install `flyctl`. A sample `fly.toml` is included in the repo for convenience.
<Note>
The included `fly.toml` does not provision a PostgreSQL database. Stand one up separately (Fly Postgres, Supabase, Neon, or another managed provider) and set `DB_CONNECTION_URI` to point at it.
</Note>
Once `flyctl` is set up, from the repo root:
```bash
flyctl launch --no-deploy # follow the prompts and edit as you see fit
description: 'Common issues and solutions when self-hosting Honcho'
icon: 'wrench'
---
This page covers common issues you may encounter when self-hosting Honcho, what causes them, and how to fix them.
## Startup Failures
### Server won't start: "Missing client for ..."
```
ValueError: Missing client for Deriver: google
```
**Cause:** The server validates at startup that all configured LLM providers have API keys. If a provider is referenced in your configuration but the corresponding API key isn't set, the server refuses to start.
**Fix:** Set the API keys for your configured providers. With default configuration, you need:
```bash
LLM_GEMINI_API_KEY=... # Used by deriver, summary, dialectic minimal/low
LLM_ANTHROPIC_API_KEY=... # Used by dialectic medium/high/max, dream
LLM_OPENAI_API_KEY=... # Used by embeddings (when EMBED_MESSAGES=true)
```
See the [LLM Setup](/v3/contributing/self-hosting#llm-setup) section for provider configuration. You can change which providers are used in your `.env` or `config.toml` (see [Configuration Guide](./configuration#llm-configuration)).
### Server won't start: "JWT_SECRET must be set"
```
ValueError: JWT_SECRET must be set if USE_AUTH is true
```
**Cause:** You enabled authentication (`AUTH_USE_AUTH=true`) but didn't provide a JWT secret.
**Fix:** Generate a secret and set it:
```bash
python scripts/generate_jwt_secret.py
# Then set the output as:
AUTH_JWT_SECRET=<generated_secret>
```
Or disable authentication for local development: `AUTH_USE_AUTH=false`
## Runtime Errors
### API returns "An unexpected error occurred" on every request
**Cause:** This is almost always a database issue. The health endpoint (`/health`) will return `{"status": "ok"}` even when the database is unreachable because it doesn't check the database connection. The actual error appears in the server logs.
**Common causes and fixes:**
1. **Database is unreachable** — Check that PostgreSQL is running and the `DB_CONNECTION_URI` is correct
2. **Migrations haven't been run** — The server starts successfully without tables, but every API call will fail. Run:
```bash
uv run alembic upgrade head
```
In Docker:
```bash
docker compose exec api uv run alembic upgrade head
```
3. **pgvector extension not installed** — The `vector` extension must be enabled in your database:
```sql
CREATE EXTENSION IF NOT EXISTS vector;
```
**How to diagnose:** Check the server logs for the actual error. Look for:
2. **Check model names match the provider's format.** OpenRouter uses `vendor/model` format (e.g., `anthropic/claude-haiku-4-5`), not the raw model ID.
3. **Ensure your model supports tool calling.** The deriver, dialectic, and dream agents require tool use. Check the provider's model page for tool calling support.
4. **Check server logs for the actual error.** API errors from the upstream provider will appear in Honcho's logs with the HTTP status code and message body.
### vLLM / Ollama not responding
1. **Verify the model server is running** and accessible from the Honcho process (or container):
```bash
curl http://localhost:8000/v1/models # vLLM
curl http://localhost:11434/v1/models # Ollama
```
2. **In Docker**, `localhost` inside a container doesn't reach the host. Use `host.docker.internal` (macOS/Windows) or the host's network IP:
3. **Structured output failures** — vLLM's structured output support is limited to certain response formats. If you see JSON parsing errors, check the deriver/dream logs for the raw response. See [Deriver produces no observations](#deriver-produces-no-observations) below.
### Deriver produces no observations
If messages are processed (the queue drains, no errors in logs) but peers never accumulate observations — and you're using an OpenAI-compatible provider — the likely cause is that the provider doesn't support OpenAI Structured Outputs (`json_schema`). The OpenAI backend requests `json_schema` by default; providers like **Z.AI GLM** and some **Ollama/vLLM** deployments either reject it or silently ignore it and return prose, which the deriver can't parse into observations.
**Fix:** set `STRUCTURED_OUTPUT_MODE=json_object` on the deriver's model config to request loose JSON mode, which injects the schema into the prompt instead:
This is a per-model-config setting on the OpenAI transport; set it on whichever features use the affected provider (e.g. `DREAM_DEDUCTION_MODEL_CONFIG__STRUCTURED_OUTPUT_MODE`).
### Thinking budget errors with non-Anthropic providers
If you see errors like `thinking budget not supported`, `invalid parameter`, or silent failures where agents produce no output, one of your per-component `*_MODEL_CONFIG__THINKING_BUDGET_TOKENS` overrides is likely set to a value > 0 with a provider that doesn't support Anthropic-style extended thinking. The built-in defaults do not set thinking budgets, so this only applies if you added those overrides yourself.
**Fix:** Set `*_MODEL_CONFIG__THINKING_BUDGET_TOKENS=0` for every component when using models that don't support thinking:
### "permission denied to create extension \"vector\""
**Cause:** The role in `DB_CONNECTION_URI` is not a superuser and lacks privileges to create extensions. Honcho issues `CREATE EXTENSION IF NOT EXISTS vector` both before running migrations and again at server startup, so this fails in both places. `IF NOT EXISTS` does not help — Postgres checks the privilege before checking whether the extension exists.
**Fix:** Preinstall pgvector once with a privileged role (superuser, or `rds_superuser` on RDS) in the Honcho database:
```sql
CREATE EXTENSION IF NOT EXISTS vector;
```
The `IF NOT EXISTS` calls then short-circuit and the application role needs no extension privileges. This is the supported path for deployments where the platform, not the application, owns extension management.
You won't hit this with `docker compose` — the bundled stack connects as `postgres`, a superuser, and its database service also creates the extension from `database/init.sql` on first boot. It comes up on managed Postgres, Kubernetes operators, and other setups where you bring your own database and its role is deliberately not a superuser.
## Cache & Redis
### Redis is optional
Redis is used for caching when `CACHE_ENABLED=true` (default: `false`). If Redis is unreachable, Honcho **gracefully falls back to in-memory caching** and logs a warning. This means:
- The server and deriver will still start and function normally
- Performance may be reduced under high load without Redis
- You do not need Redis for local development or testing
### Redis connection issues
If you see Redis connection warnings in logs but `CACHE_ENABLED=false`, they can be safely ignored. If you want caching:
```bash
# Start Redis via Docker
docker run -d -p 6379:6379 redis:latest
# Configure Honcho
CACHE_ENABLED=true
CACHE_URL=redis://localhost:6379/0
```
## Docker Issues
### Docker build fails with permission errors
The Honcho Dockerfile uses BuildKit mount syntax and creates a non-root `app` user. Common build failures:
**1. BuildKit not enabled**
The Dockerfile uses `RUN --mount=type=cache` which requires Docker BuildKit. If you see syntax errors during build:
```bash
# Ensure BuildKit is enabled
DOCKER_BUILDKIT=1 docker compose build
```
Or add to your Docker daemon config (`/etc/docker/daemon.json`):
```json
{ "features": { "buildkit": true } }
```
**2. Permission denied during build or at runtime (Linux)**
On Linux, AppArmor or SELinux can block Docker build operations and volume mounts. Symptoms include permission denied errors during `COPY`, `RUN`, or when the container tries to access mounted volumes.
```bash
# Check if AppArmor is blocking Docker
sudo aa-status | grep docker
# Temporarily test without AppArmor (for diagnosis only)
docker compose down
sudo aa-remove-unknown
docker compose up -d
```
For SELinux, add `:z` to volume mounts in `docker-compose.yml`:
```yaml
volumes:
- .:/app:z
```
**3. Volume mount UID mismatch**
The Dockerfile creates a non-root `app` user, but `docker-compose.yml.example` mounts `.:/app` which overlays the container filesystem with host-owned files. The `app` user inside the container may not have permission to read them.
If you see permission errors at runtime (not build time), you can either:
- Run without the source mount (remove `- .:/app` from volumes — the image already contains the code)
- Or fix ownership: `sudo chown -R 100:101 .` (matches the `app` user inside the container)
4. Ensure migrations ran: `docker compose exec api uv run alembic upgrade head`
### Port conflicts
If port 8000 is already in use:
```bash
# Check what's using the port
lsof -i :8000
# Or change the port mapping in docker-compose.yml
ports:
- "8001:8000" # Map to a different host port
```
### Rebuilding after code changes
```bash
docker compose build --no-cache
docker compose up -d
```
## CLI local stack (`honcho start`)
The CLI local stack lives under `~/.honcho/profiles/` (default profile `local`) and uses the published image `ghcr.io/plastic-labs/honcho:latest`. Logs: `docker compose -p honcho-local logs`. Full flags: [CLI reference](/v3/documentation/reference/cli#local-stack).
### `MISSING_LLM_KEY`
**Cause:** No provider key in the environment or the profile `.env`.
**Fix:** Export `LLM_OPENAI_API_KEY`, `LLM_ANTHROPIC_API_KEY`, or `LLM_GEMINI_API_KEY`, or run `honcho start --setup basic` in a TTY (not with `--json`).
### Timed out waiting for `/health`
**Cause:** The API container started but did not become ready within `--timeout` (default 180s).
**Fix:** Check `docker compose -p honcho-local logs api` and `... logs deriver`. Increase `--timeout`. Confirm Docker is running. A first-time GHCR pull happens *before* this wait (during image pin) — if that step hung, look at Docker pull logs instead.
### CLI still talks to `api.honcho.dev`
**Cause:** `honcho start` does not rewrite `environmentUrl` in `~/.honcho/config.json`.
**Fix:** Prefix commands with `HONCHO_BASE_URL=http://127.0.0.1:8000`, or run `honcho init --base-url http://127.0.0.1:8000` to persist local as the default. `honcho status` prints the one-shot hint.
### Port already in use
The CLI remaps 8000/5432/6379 automatically unless you pinned them with `--api-port` / `--db-port` / `--redis-port`. Pass those flags if you need a specific host port.
## Getting Help
If your issue isn't covered here:
- **Check the logs** — most issues are diagnosed from server or deriver logs (`docker compose logs` for from-source compose; `docker compose -p honcho-local logs` for `honcho start`)
@ -34,7 +34,7 @@ Honcho has a hierarchical data model centered around the entities below.
Workspaces are the top-level containers in Honcho. They provide complete isolation between different applications or environments, essentially serving as a namespace to keep different workloads separate. You might use separate workspaces for development, staging, and production environments, or to isolate different product lines. They also enable multi-tenant SaaS applications where each customer gets their own isolated workspace with complete data separation.
Authentication is scoped to the workspace level, and configuration settings can be applied workspace-wide to control behavior across all peers and sessions within that workspace.
Authentication is issued at the workspace level, and configuration settings can be applied workspace-wide to control behavior across all peers and sessions within that workspace.
---
@ -50,12 +50,14 @@ You can use peers for any entity that persists over time--individual users in ch
### <Icon icon="message" /> Sessions
Sessions represent interaction threads or contexts between peers. A session can involve multiple peers and provides temporal boundaries for when a set of interactions starts and ends. This lets you scope context and memory to specific interactions while still maintaining longer-term peer representations that span sessions.
Sessions represent interaction threads or contexts between peers. A session can involve multiple peers and provides temporal boundaries for when a set of interactions starts and ends. This lets you confine context and memory to specific interactions while still maintaining longer-term peer representations that span sessions.
Use sessions to scope things like support tickets, meeting transcripts, learning sessions, or conversations. You can also use single-peer sessions as a way to import external data--create a session with just one peer and structure emails, documents, or files as messages to enrich that peer's representation.
Use sessions for things like support tickets, meeting transcripts, learning sessions, or conversations. You can also use single-peer sessions as a way to import external data--create a session with just one peer and structure emails, documents, or files as messages to enrich that peer's representation.
Session-level configuration gives you fine-grained control over perspective-taking behavior. You can configure whether a peer should form representations of other peers in the session, and whether other peers should form representations of them.
Sessions are also the unit of visibility: when one peer's history spans contexts that shouldn't inform each other, you can group sessions into named [scopes](/v3/documentation/features/advanced/scopes) that bound recall to just those sessions.
---
### <Icon icon="envelope" /> Messages
@ -76,6 +78,20 @@ When you need context from Honcho, you query through the "Chat" endpoint or "Get
The diagram above shows how agents write messages to Honcho, which triggers reasoning that updates peer representations. Agents can then query representations to get additional context for their next response. Black arrows represent read/write of regular data (messages, storage), while red arrows represent read/write of reasoned-over data (logic, peer representations).
### Under the Hood: Write, Reasoning, and Query Paths
Honcho runs as two cooperating processes: an **API server** that handles requests and enqueues background work, and a **worker** that consumes that work off the queue. This split is what keeps the write path fast--your request never waits on an LLM call.
**Write path (synchronous).** A message is stored and a reasoning task is enqueued in the same request; the API returns immediately. Nothing about the reasoning that follows blocks the caller.
**Deriver + Summarizer (async, per-message).** The worker picks up queued tasks. The Deriver reads new messages and extracts conclusions about the peer--explicit statements and direct deductions. In parallel, the Summarizer periodically rolls up recent messages into short- and long-form session summaries. Both run per-message rather than on a schedule.
**Dreamer (periodic).** On a schedule (or triggered on demand), the Dreamer revisits existing conclusions to consolidate and deepen them: removing redundant or stale ones, drawing inductive conclusions across patterns that span multiple messages, and updating peer cards--compact biographical summaries of a peer. This is where memory gets richer over time, not just larger.
**Query path (Dialectic).** A `chat()` call spawns a Dialectic agent that answers your question by exploring memory--searching conclusions semantically, pulling supporting messages, and tracing a conclusion back to the premises it was drawn from--before synthesizing a grounded answer. This all happens inline during the request, since answering well is worth the latency that write-path reasoning is designed to avoid.
## Configuration & Extensibility
Honcho is designed to be flexible. Settings cascade hierarchically from workspace to peer to session, so you can set defaults at the workspace level and override them for specific peers or sessions. Feature flags let you enable or disable reasoning modes, perspective tracking, and other capabilities. You can bring your own LLM provider--OpenAI, Anthropic, or custom endpoints--and metadata fields let you extend any primitive with custom JSON data. Batch operations let you create up to 100 messages in a single API call for efficient bulk ingestion.
description: "Design your workspace, peers, and sessions for common application patterns"
icon: "cubes"
---
<Info>
This page covers **how to structure** workspaces, peers, and sessions for real applications. For the conceptual model behind them, start with [Architecture](/v3/documentation/core-concepts/architecture).
Ready to add Honcho to your codebase? The **`/honcho-integration` skill** applies these patterns for you — it explores your code, asks how your peers and sessions should map to your app, and wires in the Honcho SDK. Run it in any coding agent that supports skills (Claude Code, Cursor, and others).
</Info>
## Quick Reference
**Workspaces isolate, peers persist, and sessions bound the active context.**
| Decision | Recommendation |
|----------|---------------|
| How many workspaces? | One workspace per application, tool, tenant, or collaboration boundary. Split workspaces only when you need hard isolation between products, customers, environments, or agents. |
| When should agents share a workspace? | When agents collaborate over the same product, project, team, user, customer, or game state. Separate them when they should not see or influence each other's memory. |
| Who should be a peer? | Any persistent participant whose messages should be attributed or reasoned about: users, agents, assistants, NPCs, students, or customers. Use one peer for the same entity across sessions and platforms. |
| How should I divide sessions? | Match each session to the active interaction: per-conversation, per-channel, per-task run, per-project, per-import, or other bounded context. Reuse a session when local context should keep accumulating. |
| How does cross-session reasoning work? | Session memory stays local to one session. Peer representations accumulate across every session where the peer is included, and `session.context()` becomes cross-session when you include a peer target. |
| Should I set `observe_me: false`? | Yes, for deterministic peers Honcho does not need to model, like bots or tool agents. Still save their messages so other peers have session context. Keep it enabled for users and evolving agents. |
| Do I need `observe_others`? | Only when a peer needs its own perspective on another participant, such as in games, multi-agent systems, or parent/subagent workflows. |
| When do I need a scope? | When one peer's history spans contexts that must not leak into each other's recall — but you still want one workspace and one unified peer. Group the confidential sessions into a [scope](/v3/documentation/features/advanced/scopes) and pass it at query time. |
| Perspectives or scopes? | `observe_others` gives a *participant* its own view of another peer. A scope bounds recall to *where things were said*, for a reader that isn't a participant. If the reader is in the session, use perspectives; if you're fencing off a set of sessions, use a scope. |
## Workspace Design
A workspace is a hard isolation boundary. **Default to one workspace per application,** and split only at a real privacy, compliance, or product boundary (e.g. per-tenant SaaS, or a tool that needs intentionally isolated memory). Agents that collaborate over the same product, user, or game state belong in the *same* workspace so each can retrieve what the others produced.
If what you actually need is "this part of a peer's history shouldn't inform that assistant," don't split the workspace — that severs the peer's identity too. Use a [scope](/v3/documentation/features/advanced/scopes) instead: the peer stays whole, and recall through the scope sees only its member sessions.
Honcho plugins default to one workspace *per host* (`hermes`, `claude_code`, `cursor`, `opencode`). To unify memory across them, point each at the same workspace — see [Unified Memory Setup](/v3/guides/recipes/unified-memory-setup).
<Info>
The SDK creates a workspace called `default` when no `workspace_id` is specified.
</Info>
---
## Peer Design
Give each real-world entity **one** stable peer ID and reuse it everywhere — splitting one entity across `user-web`, `user-discord`, and `user-slack` builds three separate representations. Prefix IDs by source for multi-channel apps (`discord_491827364`), and if a peer goes by multiple names, store the aliases in its peer card with `set_card()` / `setCard()`.
<Tip>
For unified context across Honcho plugins, set the same user peer ID (`peerName`) everywhere — that shared ID is what connects memory across Claude Code, Cursor, OpenCode, and your own app. See [Unified Memory Setup](/v3/guides/recipes/unified-memory-setup).
</Tip>
---
## Session Design
Sessions define the temporal boundaries of an interaction. Where you draw those boundaries affects how summaries are generated and how context is retrieved.
**Common session patterns**
| Pattern | Session covers | Example |
|---------|-------------------|---------|
| Per-conversation | Each new chat thread | ChatGPT or Claude Code style UI where each thread is a session |
| Per-channel | A persistent channel or room | Discord channel, Slack thread |
| Per-interaction | A bounded task or encounter | A support ticket, a game encounter |
| Per-project | A persistent work area | Coding agent memory for one repository |
| Per-import | A batch of external data | Importing emails or documents for a single peer |
Create a **new** session when context resets (new conversation, new day, new topic); **reuse** one when context should keep accumulating (ongoing channel, persistent thread).
**How cross-session reasoning works**
- **Session memory** is local to an interaction — summaries and recent-message context describe only what happened there.
- **Peer memory** (representations) accumulates reasoning across every session the peer is part of.
So you can start a session fresh or pull in a peer's long-term memory. [`session.context()`](/v3/documentation/features/get-context) returns the current session's summary and recent messages; add a [peer target](/v3/documentation/features/get-context#peer-representation-in-context) to fold in that peer's cross-session history.
---
## Choosing an Isolation Boundary
Honcho gives you three boundaries at different strengths. Pick the weakest one that solves your problem:
| Boundary | Strength | Use when |
|----------|----------|----------|
| **Workspace** | Hard isolation — nothing crosses, including the peer itself | Different products, tenants, or environments |
| **[Scope](/v3/documentation/features/advanced/scopes)** | Recall boundary — one peer, but queries through the scope see only its sessions | One peer's contexts must not leak into each other (clinical vs. billing, per-reseller support) |
| **Session allowlist** (`sessions=[...]`) | Ad-hoc recall restriction, decided per request | The session set varies per query, or you need a quick boundary without provisioning anything |
Two things scopes are **not**:
- **Not authorization.** A workspace key reads any session, scoped or not. A scope constrains queries that name it; it doesn't protect data from queries that don't.
- **Not topic filtering.** Scopes bound recall by *where something was said*, not what it's about. A therapy detail mentioned in a billing session lands in the billing scope. If you might ever need a scope boundary, align your session boundaries with your confidentiality boundaries from the start — the session is the unit scopes can enforce.
---
## Common Mistakes
- **Splitting one identity across peer IDs** -- If the same user is `alice`, `alice-discord`, and `alice-cursor`, Honcho builds separate representations. Use one stable peer ID when you want unified memory.
- **Too many tiny sessions** -- Summaries and recent messages are local to one session. Splitting a continuous conversation across many sessions fragments that local context. Reuse a session when context should flow continuously.
- **Separating agents that should collaborate** -- If agents need shared product, customer, or team context, put them in the same workspace. Separate workspaces are hard isolation boundaries.
- **Leaving `observe_me` on for assistants** -- Wastes reasoning compute on a peer you control. Deterministic behavior doesn't need to be modeled.
- **Turning on `observe_others` everywhere** -- Directional representations are powerful, but they add complexity. Use them when peers need distinct perspectives, not just because a session has multiple peers.
- **A scope per reader** -- Scopes should map to real confidentiality boundaries, not to consumers. If every assistant gets its own scope, you've rebuilt workspace fragmentation inside one workspace, and each projection reasons over a thin slice. Fewer, boundary-shaped scopes; many readers can share one.
- **Treating scopes as access control** -- A scope bounds *recall*, not *access*. Enforce who may query what in your application layer; use scopes to keep the answers themselves from drawing on out-of-bounds sessions.
- **Forgetting `peer_target` on session context** -- `session.context()` defaults to the active session's summary and recent messages, which are local to that session. It becomes cross-session only through adding a peer_target which includes the peer representation.
- **Blocking on processing** -- Messages are processed asynchronously in the background. Don't poll or wait for reasoning to complete before continuing your application flow.
@ -54,6 +54,8 @@ The explicit reasoning model ([Neuromancer XR](https://blog.plasticlabs.ai/resea
The reasoning that Honcho does is something we're constantly iterating and improving on. Our goal is simple--provide the richest, most relevant context in the fastest, cheapest way possible in order to simulate statefulness in whatever setting you need.
Two components produce this logic: the **Deriver** extracts explicit and deductive conclusions from incoming messages as they arrive, and the **Dreamer** periodically revisits stored conclusions to consolidate them and draw inductive conclusions across patterns spanning multiple messages. See [Architecture](/v3/documentation/core-concepts/architecture) for how these fit into the request/background split.
## How It Works
When you write messages to Honcho, they're stored immediately and enqueued for background processing. Reasoning asynchronously ensures fast writes while still providing rich reasoning capabilities. Messages are stored immediately without blocking, and session-based queues maintain chronological consistency so reasoning tasks affecting the same peer representation are always processed in order.
@ -64,26 +66,16 @@ The reasoning outputs--conclusions, summaries, peer cards--are stored as part of
The diagram above shows how agents write messages to Honcho, which triggers reasoning that updates peer representations. Agents can then query representations to get additional context for their next response.
### Token Batching
Rather than running inference on every individual message, Honcho accumulates messages in the queue and processes them as a batch once the total token count of pending messages for a given peer representation crosses a threshold--roughly **1,000 tokens** at the current batch size. This keeps ingestion costs down, since Honcho charges based on reasoning passes, and ensures each pass has a meaningful amount of context to work with. At ~1,000 tokens the batch comfortably fits in the context window of any modern LLM, so no content is lost.
If a user sends several short messages in a row (e.g., "yes", "ok", "sounds good"), those messages sit in the queue until enough content has accumulated. Once the threshold is met, the full batch is processed together in a single reasoning call.
<Note>
This batching only applies to **representation** tasks (conclusion extraction). Summary and dream tasks have their own scheduling logic and are not subject to the token threshold.
</Note>
## Balances & Design Choices
Off-the-shelf LLMs can perform formal logical reasoning, but they aren't optimized for it. Honcho uses custom models trained specifically for logical rigor (following formal reasoning rules rather than plausible-sounding text), structured output (consistent JSON schema with premises and conclusions), and efficiency (smaller, faster models tuned for this specific task). This allows Honcho to reason more reliably and at lower cost than general-purpose frontier LLMs.
The approach balances quality with practical constraints. Custom models are smaller and cheaper to run, scaffolded conclusions are more token-efficient than raw conversation history, and we batch where appropriate to optimize update frequency.
The approach balances quality with practical constraints. Custom models are smaller and cheaper to run, and scaffolded conclusions are more token-efficient than raw conversation history.
Honcho's reasoning capabilities are actively being improved. Current areas of development include enhanced inductive and abductive reasoning, multi-hop and temporal reasoning, and expanded file types and modalities. The system is designed to be extensible--new reasoning capabilities can be added without breaking existing functionality.
<Note>
If you find that the data you're uploading to Honcho isn't being reasoned over to your liking, we'd love to improve it for you and ingest your data for free--reach out via [Discord](https://discord.gg/plasticlabs) or [email](mailto:support@plasticlabs.ai)!
If you find that the data you're uploading to Honcho isn't being reasoned over to your liking, we'd love to improve it for you and ingest your data for free--reach out via [Discord](https://discord.gg/honcho) or [email](mailto:support@plasticlabs.ai)!
description: 'Advanced configuration and querying for representations'
title: 'Directional Representations'
description: 'How peers build and query representations of other peers'
icon: 'circle'
---
@ -22,7 +22,7 @@ You can retrieve a subset of conclusions from a peer's representation using `rep
alice_rep = session.representation("alice")
# Or via chat
response = alice.chat("What are Alice's main interests?", session_id=session.id)
response = alice.chat("What are Alice's main interests?", session=session.id)
```
This is sufficient for most applications—Honcho reasons over every message written to the peer, storing conclusions that any part of your system can retrieve.
@ -80,7 +80,8 @@ The `target` parameter controls which representation you retrieve:
<CodeGroup>
```python Python
from honcho import Honcho, SessionPeerConfig
from honcho import Honcho
from honcho.api_types import SessionPeerConfig
honcho = Honcho()
session = honcho.session("game-session")
@ -159,13 +160,13 @@ The `target` parameter also works with the chat endpoint:
# Query using conclusions from Honcho's representation (across all sessions)
honcho_answer = alice.chat(
"What did Bob say about breakfast?",
session_id=session.id
session=session.id
)
# Query using conclusions from Alice's representation of Bob (from Alice's sessions only)
alice_answer = alice.chat(
"What did Bob say about breakfast?",
session_id=session.id,
session=session.id,
target="bob"
)
```
@ -174,13 +175,13 @@ alice_answer = alice.chat(
// Query using conclusions from Honcho's representation (across all sessions)
const honchoAnswer = await alice.chat(
"What did Bob say about breakfast?",
{ sessionId: session.id }
{ session: session.id }
);
// Query using conclusions from Alice's representation of Bob (from Alice's sessions only)
const aliceAnswer = await alice.chat(
"What did Bob say about breakfast?",
{ sessionId: session.id, target: "bob" }
{ session: session.id, target: "bob" }
);
```
</CodeGroup>
@ -213,7 +214,7 @@ Most applications don't need directional representations. Start with the default
Under the hood, Honcho stores representations as (observer, observed) pairs in internal collections:
- **Collection**: A unique (observer, observed, workspace) tuple containing documents
- **Documents**: Individual conclusions and artifacts (deductive, inductive, abductive conclusions, summaries, peer cards) with session scoping
- **Documents**: Individual conclusions and artifacts (deductive, inductive, abductive conclusions, summaries, peer cards) with per-session filtering
When you retrieve with `target`, Honcho fetches documents from the specific (observer, observed) collection. When you retrieve without `target`, it fetches from the (peer, peer) collection—the peer's self-representation.
@ -224,7 +225,7 @@ This architecture enables:
## Semantic Search Parameters
Both `representation()` and `chat()` support semantic filtering to retrieve a subset of relevant conclusions. You can optionally filter by session to retrieve only conclusions from specific session context:
Both `representation()` and `chat()` support semantic filtering to retrieve a subset of relevant conclusions. You can optionally filter by session — pass `session` to restrict to a single session, or use the REST-only [session allowlist](/v3/documentation/features/advanced/using-filters#scoping-recall-to-sessions) to restrict to a set of sessions:
| Parameter | Type | Description |
|-----------|------|-------------|
@ -264,7 +265,21 @@ Directional representations update automatically through the reasoning pipeline
2. The message sender has `observe_me=true` (or session-level equivalent)
3. Other peers in the session have `observe_others=true`
The pipeline respects scoping—Honcho's representations reason over messages across all sessions, while directional representations only reason over messages from sessions where the observer was an active participant.
The pipeline respects these boundaries—Honcho's representations reason over messages across all sessions, while directional representations only reason over messages from sessions where the observer was an active participant.
### Peer Join Order Matters
Reasoning tasks are scheduled at the time a message is created, based on which peers are in the session **at that moment**. Honcho does not retroactively schedule reasoning for peers that join later.
This means:
- If Peer C joins a session **after** messages from Peer A and Peer B have already been sent, Peer C will **not** receive reasoning tasks for those earlier messages—even if Peer C has `observe_others=true`.
- Peer C will only begin observing new messages sent after they join the session.
- Similarly, if a peer leaves a session, they stop being included as an observer for any messages sent after their departure.
<Warning>
There is no retroactive reasoning. If your application needs an observer peer to reason about prior conversation history, add the peer to the session **before** messages are sent. Alternatively, use `peer.chat()` to include conversation history in the agent's context whether or not those messages were previously reasoned over.
</Warning>
<Note>
Conclusions are cached for fast retrieval. Use `representation()` to retrieve stored conclusions for dashboards and analytics. Use `peer.chat()` when you need query-specific reasoning with natural language.
- [Peer Card](/v3/documentation/features/advanced/peer-card) - Quick-reference profile of stable biographical facts about a peer
- [Directional Representations](/v3/documentation/features/advanced/directional-representations) - How peers build separate representations of each other
- [Scopes](/v3/documentation/features/advanced/scopes) - Named sets of sessions that act as visibility boundaries for recall
- [Dreaming](/v3/documentation/features/advanced/dreaming) - Autonomous memory consolidation and self-improvement
- [Queue Status](/v3/documentation/features/advanced/queue-status) - Monitor background processing and reasoning tasks
## Querying & Filtering
## Querying & Data
- [Search](/v3/documentation/features/advanced/search) - Search across peers, sessions, and messages
- [Filters](/v3/documentation/features/advanced/using-filters) - Filter queries with advanced parameters
- [Streaming Responses](/v3/documentation/features/advanced/streaming-response) - Stream dialectic responses in real-time
- [File Uploads](/v3/documentation/features/advanced/file-uploads) - Ingest files into peer memory
- [Deleting Data](/v3/documentation/features/advanced/deleting-data) - Delete sessions, workspaces, and conclusions
description: 'A quick-reference profile of stable biographical facts about a peer'
icon: 'id-card'
---
A **peer card** is a list of stable, biographical facts about a peer--name, occupation, preferences, standing instructions--that acts as a quick-reference profile. While the full [representation](/v3/documentation/core-concepts/representation) contains all of Honcho's reasoning (conclusions, summaries, semantic search results), the peer card captures the grounding facts that should never be forgotten.
Think of it as the front of a contact card: the information an agent needs at a glance to know who it's talking to.
## What Goes in a Peer Card
Peer cards are designed for **durable, biographical information**--things that remain true across sessions and contexts. Each fact is stored as a single string in a list.
| Traits | `"TRAIT: Detail-oriented, prefers data over anecdotes"` |
Peer cards are **not** for transient information like current mood, recent conversation topics, or reasoning traces. Those belong in conclusions and summaries.
## How Peer Cards Are Created
Peer cards are populated through two paths:
**1. Automatic (via Dreaming)**
When [dreaming](/v3/documentation/features/advanced/dreaming) runs, the deduction and induction specialists extract stable biographical facts from existing conclusions and write them to the peer card. This happens without any manual intervention--Honcho identifies facts like names, occupations, and preferences from conversation history and records them automatically.
**2. Manual (via SDK or API)**
You can set a peer card directly. This is useful for bootstrapping a peer with known information before any conversation has occurred, or for correcting facts that Honcho hasn't yet discovered.
<CodeGroup>
```python Python
from honcho import Honcho
honcho = Honcho()
peer = honcho.peer("user-123")
# Set the peer card
peer.set_card([
"Name: Alice. Also known as 'Ali'.",
"College student at MIT, studying computer science.",
"PREFERENCE: Prefers casual tone.",
"INSTRUCTION: Never mention her ex-boyfriend.",
])
# Retrieve the peer card
card = peer.get_card()
print(card)
# ["Name: Alice. Also known as 'Ali'.", "College student at MIT...", ...]
```
```typescript TypeScript
import { Honcho } from "@honcho-ai/sdk";
const honcho = new Honcho({});
const peer = await honcho.peer("user-123");
// Set the peer card
await peer.setCard([
"Name: Alice. Also known as 'Ali'.",
"College student at MIT, studying computer science.",
"PREFERENCE: Prefers casual tone.",
"INSTRUCTION: Never mention her ex-boyfriend.",
]);
// Retrieve the peer card
const card = await peer.getCard();
console.log(card);
// ["Name: Alice. Also known as 'Ali'.", "College student at MIT...", ...]
```
</CodeGroup>
## Directional Peer Cards
Peer cards follow the same observer-observed model as [representations](/v3/documentation/features/advanced/directional-representations). When `observe_others` is enabled, a peer can have a **different** card for each peer it observes.
For example, if Alice and Bob are in a session together and Alice has `observe_others: true`, Alice will build her own peer card for Bob--separate from Honcho's peer card for Bob. You can read and write these directional cards using the `target` parameter.
<CodeGroup>
```python Python
alice = honcho.peer("alice")
# Get Alice's own peer card (Honcho's view of Alice)
alice_card = alice.get_card()
# Get Alice's card for Bob (Alice's view of Bob)
alice_bob_card = alice.get_card(target="bob")
# Set Alice's card for Bob
alice.set_card(
["Bob mentioned he's allergic to peanuts."],
target="bob"
)
```
```typescript TypeScript
const alice = await honcho.peer("alice");
// Get Alice's own peer card (Honcho's view of Alice)
When you call [`peer.chat()`](/v3/documentation/features/chat), Honcho automatically injects the relevant peer cards into the system prompt. The dialectic agent sees both the observer's own card and the observed peer's card, giving it immediate grounding without needing to search memory.
### In Context Retrieval
The [`session.context()`](/v3/documentation/features/get-context) method includes the peer card when you specify a `peer_target`:
<CodeGroup>
```python Python
context = session.context(
tokens=2000,
peer_target="user-123"
)
# Access the peer card alongside the representation
print(context.peer_card) # List of peer card facts
print(context.peer_representation) # Full representation text
```
```typescript TypeScript
const context = await session.context({
tokens: 2000,
peerTarget: "user-123"
});
console.log(context.peerCard); // Array of peer card facts
console.log(context.peerRepresentation); // Full representation text
```
</CodeGroup>
### During Dreaming
The [dreaming](/v3/documentation/features/advanced/dreaming) process reads the current peer card before consolidation, then updates it with any new stable facts discovered during the deduction and induction phases.
## Limits
| Constraint | Value |
|-----------|-------|
| Maximum facts per card | **40** |
| Data type | `list[str]` (each fact is a string) |
When the dreaming process or a manual update pushes the card beyond 40 facts, it is automatically truncated to the first 40 entries. Keep facts concise and deduplicated to stay within the limit.
<Warning>
If you manually set a peer card, it **replaces** the entire card--it does not merge with existing facts. Make sure to include all facts you want to keep when calling `set_card()`.
</Warning>
## Configuration
Peer card behavior is controlled through the [configuration hierarchy](/v3/documentation/features/advanced/reasoning-configuration). You can independently toggle whether agents **use** existing peer cards and whether they **create/update** them.
<CodeGroup>
```python Python
# Disable peer card updates but still use existing cards during reasoning
session = honcho.session("my-session", config={
"peer_card": {"create": False, "use": True}
})
# Disable peer cards entirely
session = honcho.session("no-cards", config={
"peer_card": {"create": False, "use": False}
})
```
```typescript TypeScript
// Disable peer card updates but still use existing cards during reasoning
| `use` | `bool` | `true` | Whether agents read the peer card during reasoning and chat |
| `create` | `bool` | `true` | Whether agents can create or update peer cards |
Configuration can be set at the workspace, session, or message level. See [Reasoning Configuration](/v3/documentation/features/advanced/reasoning-configuration) for the full hierarchy.
## Best Practices
1. **Bootstrap with known facts.** If you already know the user's name or preferences at signup, set the peer card immediately. This gives the agent grounding from the very first interaction instead of waiting for dreaming to discover it.
2. **Use structured prefixes.** Prefixing facts with `INSTRUCTION:`, `PREFERENCE:`, or `TRAIT:` makes it easier for the agent to distinguish categories and act on them appropriately.
3. **Keep facts atomic.** Each string should contain one fact. Avoid combining multiple pieces of information into a single entry--`"Name: Alice"` and `"Location: Portland"` are better than `"Alice lives in Portland and works at Acme"`.
4. **Let dreaming handle updates.** For most applications, you don't need to manually manage the peer card after bootstrapping. The dreaming process will discover and record new facts as conversations progress.
5. **Use `set_card` for corrections.** If the automatic system has recorded something incorrect, manually set the card with the corrected facts. Remember this replaces the entire card.
ajspig
Eugene Eisenstein
Rajat Ahuja
Aakash Kattelu
steven-ji
Ulysse Pence
Vineeth Voruganti
Eri Barrett
Ken Weiner
Phil
adavyas
Serhii Zghama
Daniel Peng
Phil
ZiYun Liu
Joe-Kneeland
Vansh Sharma
Niyaz Almufti
bbasketballer75
Chris Caldwell
JUNZE
PK
Alexei Vedernikov
Leonardo Baray
papesy384
Ulysse Pence
Talha Abdur Rahman
Aru Sharma
TcDrozd
Harish Kukreja
Aakash Kattelu
Hafiz Ahmad Ashfaq
xianzuyang9-blip
Anthony Yuan
Raúl Anatol
Xiangzhe
ShellyBot
Marianne
Lily
Lily
Phil
banteg
Lily
Sanjay Santhanam
qxxaa
Alfredo Colangelo
Maestra
Luba Kaper
Vincent Koc
doria
Agney Menon
Hart
Lalit Aditya Julapalli
Viktor Szépe
LRRuan
