2163ab1aa3
* fix(llm): forward provider_params.timeout to the OpenAI-compatible embedding client #832 and #903 added a configurable request timeout for the LLM registry and the Gemini embedding client respectively, but the OpenAI-compatible embedding client (src/embedding_client.py) was never wired up. It constructed AsyncOpenAI with no timeout at all, so a stalled socket against a slow or contended OpenAI-compatible backend (e.g. a self-hosted embedding model under load) wedges the deriver worker's event loop indefinitely — the exact failure #785/#903 describe, just via a code path #903 didn't cover. EmbeddingModelConfig now carries provider_params through from resolve_embedding_model_config, mirroring how resolve_model_config already does it for ModelConfig, and the OpenAI branch of _EmbeddingClient.__init__ extracts `timeout` via the existing request_timeout_from_extra_params helper. Unset stays unset — no existing behavior changes. Reproduced and verified against a real self-hosted deployment (local Ollama backend under load): before this fix, a single stuck embedding call blocked all deriver queue processing for 20+ minutes with no error logged, twice in one session. * fix(embedding): use first-class timeout on embedding model config provider_params is the LLM per-request escape hatch; embedding timeouts are client-construction knobs and belong next to max_batch_size. Wire the field for OpenAI and Gemini, omit the OpenAI kwarg when unset so the SDK default stays, and keep Gemini's 10-minute floor when unset. * test(embedding): live coverage for first-class embedding timeout Exercise EmbeddingModelConfig.timeout on one representative OpenAI and Gemini model: configured timeout lands on the SDK client, and a near-zero timeout aborts before the provider answers. --------- Co-authored-by: Aakash Kattelu <aakash@plasticlabs.ai> |
||
|---|---|---|
| .. | ||
| changelog | ||
| images | ||
| logo | ||
| snippets | ||
| v1 | ||
| v2 | ||
| v3 | ||
| README.md | ||
| bun.lock | ||
| docs.json | ||
| favicon.svg | ||
| package.json | ||
README.md
Honcho Docs
These docs are built using Next.js via mintlify.
Setting Up Honcho's Docs Locally
- Clone the repository:
git clone git@github.com:plastic-labs/honcho.git
- Navigate into the
docsfolder:
cd honcho/docs/
The docs folder contains the markdown files that make up the documentation. The majority of the files are in the pages directory. Some notable files in this folder include:
- Verify that you have Node.js and npm installed in your system. You can check by running:
node --version
npm --version
-
If not installed, download Node.js and npm from the respective official websites.
-
Once you have Node.js and npm running, proceed to install
pnpm- another package manager that helps to manage project dependencies:
npm install -g pnpm
- Install the project dependencies using pnpm:
pnpm i
- After the successful installation of the project dependencies, start the local server:
pnpm dev
Now, you should be able to view the docs on your local environment by visiting http://localhost:3000. You can explore the different markdown files and make changes as you see fit.