* feat(mock-provider): deterministic OpenAI-compatible endpoint for local and CI use
Adds src/mock_provider/, a standalone ASGI app that lets Honcho run with no
model provider, no API key, and no spend. It answers /v1/chat/completions and
/v1/embeddings with obviously-synthetic content derived from the request, so
the same request always produces the same response.
It runs as its own service from the standard Honcho image with a different
entrypoint, the way api and deriver already differ, so there is no second image
to build or keep in digest-sync. The app imports nothing from src.config or
src.db, so it boots even when the rest of the stack is misconfigured.
The chat endpoint generates from the JSON Schema it is sent rather than
answering with prose. That matters because a prose answer does not fail loudly:
repair_response_model_json swallows the parse error and returns an empty
PromptRepresentation, which reads as "the deriver found nothing" rather than
"the mock is wrong". Generation resolves $ref/$defs indirection, caps recursion
for reasoning-tree schemas, and covers json_object mode by recovering the
schema Honcho injects into the prompt. Embeddings are hash-derived, so
identical input yields an identical vector.
Tests drive the production OpenAIBackend and _EmbeddingClient against the app
over ASGI, including the strict json_schema transform that
chat.completions.parse() applies. Verified end to end against a real stack:
messages in, conclusions and 1536-dim embeddings written to pgvector, with no
calls to any real provider.
Mock embeddings carry no semantic similarity, so recall against this provider
must use lexical search. CONTRIBUTING notes that, and the load_dotenv(override=
True) behaviour that lets a stale repo .env win over exported environment
variables.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(mock-provider): validate requests with Pydantic models
Review feedback: hand-coercing the request bodies was defended on the grounds
that FastAPI answers a malformed body with a 422, and a 422 mid-deriver-run
reads as a Honcho bug. That argues against the default handler, not against the
models. Registering an exception handler fixes it — and the resulting behaviour
is more faithful, not less, because the real API answers a bad request with a
400 and an `error` envelope, which is now exactly what the mock returns.
Adds src/mock_provider/schemas.py with ChatCompletionRequest and
EmbeddingsRequest. Every model allows extra fields and every field is optional,
so validation fires on a wrong type rather than on a parameter the mock has not
heard of — a new upstream parameter must not turn a working setup into a hard
failure. dimensions is a StrictInt because bool is an int subclass and a JSON
`true` would otherwise mean a one-dimensional vector.
coerce.py stays, narrowed to serving schema_gen, which walks arbitrary
caller-supplied JSON Schema and is untyped by nature. response_format likewise
stays dict[str, Any]: only its envelope is worth typing.
Also records why schema_gen does not reuse src/utils/schema_conversion.py
despite the overlapping $ref/$defs handling — it builds a model class rather
than an instance, raises by contract where a mock must degrade, and rejects
both allOf and the recursive $ref that reasoning-tree schemas rely on.
Documents that LLM_OPENAI_API_KEY is only tested for truthiness; the previous
wording read as though the value had to be the literal string "sandbox".
Re-verified end to end after the refactor: 6 messages in, 4 conclusions and 6
1536-dim embeddings out, every real request answered 200, no calls to any real
provider.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(mock-provider): honour include_usage, generate prefixItems tuples
Three fidelity gaps where the mock answered a request differently from the
API it stands in for:
- The usage chunk was emitted on every stream. The real API sends it only
when stream_options.include_usage is set, so a caller that did not opt in
had to skip a trailing chunk with an empty choices array. stream_options
is now a typed model, which also rejects a non-boolean include_usage
instead of reading it as truthy.
- A fixed-length tuple is prefixItems with no items, which is what Pydantic
emits for tuple[str, int]. Reading only items returned [], failing the
minItems the same schema carries — the silent-empty failure schema_gen
exists to avoid.
- A zero or negative dimensions was silently replaced with 1536, answering
a bad request with a plausible-looking vector rather than a 400.
Three further deviations from JSON Schema are left in place and documented
where they occur: allOf merges properties first-wins, oneOf is treated as
anyOf, and string pattern is ignored. None is reachable from a Honcho
response model — no model emits prefixItems or oneOf, and the only pattern
constraints are on API request models — and each fix costs more than the
unreachable path is worth.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(mock-provider): strict request booleans, bounded recursion, multipleOf
Second CodeRabbit pass. All four findings reproduced first; none is reachable
from a Honcho response model, but two trace back to the previous commit.
- `include_usage` and `stream` were plain `bool`, which Pydantic coerces from
"yes"/"on"/"true"/"1". The comment added last commit claimed a string had to
fail here, and it did not — the test only passed because "definitely" is not
a recognised bool literal. Both are StrictBool now, matching why `dimensions`
is StrictInt, and the tests cover the truthy strings that actually coerced.
- `_generate_array` returned the prefix alone when `items` was absent, so
prefixItems plus a larger minItems undershot its own schema. Absent `items`
leaves those positions unconstrained rather than disallowed, so the shortfall
is filled to minItems — a bare `{"type": "array"}` still generates nothing.
- A required, non-nullable recursive $ref hit RecursionError: MAX_DEPTH only
terminates a cycle that offers a `default` or a nullable branch, and
`_generate_object` keeps descending into required properties. HARD_MAX_DEPTH
degrades to an empty container instead, since a mock must not turn its own
defect into a 500. Bounded, not plumbed into an error response — the
unreachable path does not justify touching the request path.
- `_bounded_int` ignored `multipleOf` while honouring minimum, maximum and both
exclusive bounds; 9 of 12 sampled paths produced a non-multiple. Values now
snap onto a multiple inside the bounds, and an unsatisfiable window keeps the
bounds. A fractional `multipleOf` is still ignored, as documented.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(mock-provider): correct the reason fractional multipleOf is dropped
The docstring claimed honouring it would mean returning a non-integer from an
integer schema. That is wrong: 3 is an integer and a multiple of 1.5. The real
reason is that it needs exact-decimal arithmetic to keep float drift from
deciding validity, and no Honcho response model emits multipleOf at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
afbc517cbc