honcho/docs/v3/contributing
Joe-Kneeland 2163ab1aa3
fix(llm): forward provider_params.timeout to the OpenAI-compatible embedding client (#1024)
* fix(llm): forward provider_params.timeout to the OpenAI-compatible embedding client

#832 and #903 added a configurable request timeout for the LLM registry
and the Gemini embedding client respectively, but the OpenAI-compatible
embedding client (src/embedding_client.py) was never wired up. It
constructed AsyncOpenAI with no timeout at all, so a stalled socket
against a slow or contended OpenAI-compatible backend (e.g. a
self-hosted embedding model under load) wedges the deriver worker's
event loop indefinitely — the exact failure #785/#903 describe, just
via a code path #903 didn't cover.

EmbeddingModelConfig now carries provider_params through from
resolve_embedding_model_config, mirroring how resolve_model_config
already does it for ModelConfig, and the OpenAI branch of
_EmbeddingClient.__init__ extracts `timeout` via the existing
request_timeout_from_extra_params helper. Unset stays unset — no
existing behavior changes.

Reproduced and verified against a real self-hosted deployment (local
Ollama backend under load): before this fix, a single stuck embedding
call blocked all deriver queue processing for 20+ minutes with no
error logged, twice in one session.

* fix(embedding): use first-class timeout on embedding model config

provider_params is the LLM per-request escape hatch; embedding timeouts are
client-construction knobs and belong next to max_batch_size. Wire the field
for OpenAI and Gemini, omit the OpenAI kwarg when unset so the SDK default
stays, and keep Gemini's 10-minute floor when unset.

* test(embedding): live coverage for first-class embedding timeout

Exercise EmbeddingModelConfig.timeout on one representative OpenAI and
Gemini model: configured timeout lands on the SDK client, and a near-zero
timeout aborts before the provider answers.

---------

Co-authored-by: Aakash Kattelu <aakash@plasticlabs.ai>
2026-08-18 10:51:53 -04:00
..
changing-embeddings.mdx Make embeddings configurable (#678) 2026-05-14 15:03:35 -04:00
configuration.mdx fix(llm): forward provider_params.timeout to the OpenAI-compatible embedding client (#1024) 2026-08-18 10:51:53 -04:00
guidelines.mdx feat: retry on more httpx exceptions (#467) 2026-04-03 12:20:53 -04:00
license.mdx feat: honcho 3.0, sdks 2.0, excise stainless, update v3 docs, changelogs (#331) 2026-01-22 15:16:28 -05:00
self-hosting.mdx fix: reduce Honcho runtime image size (#1014) 2026-08-12 19:55:41 -04:00
troubleshooting.mdx docs: document pgvector preinstall for least-privilege DB roles (#984) 2026-08-06 10:23:43 -04:00