honcho/tests/llm
Joe-Kneeland 2163ab1aa3
fix(llm): forward provider_params.timeout to the OpenAI-compatible embedding client (#1024)
* fix(llm): forward provider_params.timeout to the OpenAI-compatible embedding client

#832 and #903 added a configurable request timeout for the LLM registry
and the Gemini embedding client respectively, but the OpenAI-compatible
embedding client (src/embedding_client.py) was never wired up. It
constructed AsyncOpenAI with no timeout at all, so a stalled socket
against a slow or contended OpenAI-compatible backend (e.g. a
self-hosted embedding model under load) wedges the deriver worker's
event loop indefinitely — the exact failure #785/#903 describe, just
via a code path #903 didn't cover.

EmbeddingModelConfig now carries provider_params through from
resolve_embedding_model_config, mirroring how resolve_model_config
already does it for ModelConfig, and the OpenAI branch of
_EmbeddingClient.__init__ extracts `timeout` via the existing
request_timeout_from_extra_params helper. Unset stays unset — no
existing behavior changes.

Reproduced and verified against a real self-hosted deployment (local
Ollama backend under load): before this fix, a single stuck embedding
call blocked all deriver queue processing for 20+ minutes with no
error logged, twice in one session.

* fix(embedding): use first-class timeout on embedding model config

provider_params is the LLM per-request escape hatch; embedding timeouts are
client-construction knobs and belong next to max_batch_size. Wire the field
for OpenAI and Gemini, omit the OpenAI kwarg when unset so the SDK default
stays, and keep Gemini's 10-minute floor when unset.

* test(embedding): live coverage for first-class embedding timeout

Exercise EmbeddingModelConfig.timeout on one representative OpenAI and
Gemini model: configured timeout lands on the SDK client, and a near-zero
timeout aborts before the provider answers.

---------

Co-authored-by: Aakash Kattelu <aakash@plasticlabs.ai>
2026-08-18 10:51:53 -04:00
..
test_backends fix(llm): support per-request provider timeouts (#832) 2026-08-04 12:43:00 -04:00
conftest.py Refactor clients.py to add modern features and more flexible configuration (#459) 2026-04-20 02:46:37 -04:00
test_agent_tool_schemas.py Refactor clients.py to add modern features and more flexible configuration (#459) 2026-04-20 02:46:37 -04:00
test_capture.py feat(telemetry): CloudEvents + Langfuse tracing as projections over a captured LLM stream (#845) 2026-07-02 16:49:53 -04:00
test_conversation.py Refactor clients.py to add modern features and more flexible configuration (#459) 2026-04-20 02:46:37 -04:00
test_credentials.py Refactor clients.py to add modern features and more flexible configuration (#459) 2026-04-20 02:46:37 -04:00
test_embedding_client.py fix(llm): forward provider_params.timeout to the OpenAI-compatible embedding client (#1024) 2026-08-18 10:51:53 -04:00
test_history_adapters.py Refactor clients.py to add modern features and more flexible configuration (#459) 2026-04-20 02:46:37 -04:00
test_langfuse_trace_annotation.py feat(telemetry): CloudEvents + Langfuse tracing as projections over a captured LLM stream (#845) 2026-07-02 16:49:53 -04:00
test_model_config.py feat: add model config option for json_object mode (#820) 2026-06-23 10:42:03 -04:00
test_registry.py perf: lazy-load provider SDKs to cut idle memory per process (#1011) 2026-08-12 21:39:12 -04:00
test_request_builder.py fix(llm): support per-request provider timeouts (#832) 2026-08-04 12:43:00 -04:00
test_telemetry_agent_iteration.py feat(telemetry): CloudEvents + Langfuse tracing as projections over a captured LLM stream (#845) 2026-07-02 16:49:53 -04:00
test_telemetry_agent_tool_call.py feat: add new cloudevents for api routes (#637) 2026-05-20 18:25:30 -04:00
test_telemetry_llm_call.py perf: lazy-load provider SDKs to cut idle memory per process (#1011) 2026-08-12 21:39:12 -04:00
test_tool_loop_truncation.py feat: add new cloudevents for api routes (#637) 2026-05-20 18:25:30 -04:00