honcho/tests
Niyaz Almufti a92fb1e078
Fix Gemini batch embedding for gemini-embedding-2* models (#745)
* Fix Gemini batch embedding for gemini-embedding-2* models

The google-genai SDK treats embed_content(contents=[list_of_strings]) as a
single multi-part document for gemini-embedding-2* models, silently returning
exactly 1 embedding regardless of input count. This caused a zip(...,
strict=True) ValueError in _process_batch.

Wrap each text in genai_types.Content(parts=[genai_types.Part(text=...)])
so the SDK treats each string as a separate content item. This matches the
workaround used by pydantic-ai (#4873) and graphiti (#1474).

Upstream SDK issue: googleapis/python-genai#2523
Fixes plastic-labs/honcho#744

* test(embedding): live embedding coverage for every Gemini and OpenAI model

Adds tests/live_llm/test_live_embeddings.py plus an env-driven embedding
matrix alongside the existing LLM one. Covers single embed, batched embed,
batch-vs-single alignment, chunk-to-id mapping, and the batch-split path.

Only a live call catches the gemini-embedding-2* collapse: the SDK folds a
list of bare strings into one document and returns a single embedding.
Reverting the Content wrapping fails all four batch tests for
gemini-embedding-2-preview and gemini-embedding-2 with the reported
`zip() argument 2 is shorter than argument 1`, while gemini-embedding-001
and text-embedding-3-small stay green.

Also makes the concatenation in the conclusions semantic-search validation
message explicit, so the repo-wide basedpyright pre-push hook passes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(embedding): drop the preview twin from the default embedding matrix

gemini-embedding-2 is the GA release of gemini-embedding-2-preview and
behaves identically, so running both by default doubles the Gemini cost for
no extra coverage. The preview stays reachable through
LIVE_EMBEDDING_GEMINI_MODELS.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Aakash Kattelu <aakash@plasticlabs.ai>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-11 17:50:40 -04:00
..
alembic feat: make session_name nullable for documents and update related SDKs (#347) 2026-01-26 13:33:11 -05:00
bench rename to REPRESENTATION_BATCH_TARGET_INPUT_TOKENS 2026-07-09 10:42:52 -04:00
crud Fix: vector query top k zero (#970) 2026-08-03 12:31:03 -04:00
deriver Counts documents deduped during representation, exact and semantically similar 2026-07-16 14:30:55 -02:00
dialectic feat(dialectic): optional structured outputs with limited schema for Dialectic calls (#896) 2026-07-20 18:46:49 -04:00
dreamer Session-purity invariant + card_refresh dream type (DEV-2000) (#883) 2026-07-23 14:22:07 -04:00
integration feat: defer embedding messages (#704) 2026-06-11 10:31:04 -04:00
live_llm Fix Gemini batch embedding for gemini-embedding-2* models (#745) 2026-08-11 17:50:40 -04:00
llm Fix Gemini batch embedding for gemini-embedding-2* models (#745) 2026-08-11 17:50:40 -04:00
reconciler fix: Various Codex Audits (#386) 2026-02-13 12:00:15 -05:00
routes Session-purity invariant + card_refresh dream type (DEV-2000) (#883) 2026-07-23 14:22:07 -04:00
scripts feat: add new cloudevents for api routes (#637) 2026-05-20 18:25:30 -04:00
sdk feat(dialectic): optional structured outputs with limited schema for Dialectic calls (#896) 2026-07-20 18:46:49 -04:00
sdk_typescript add read db (#773) 2026-06-10 13:28:36 -04:00
startup add read db (#773) 2026-06-10 13:28:36 -04:00
telemetry Vineeth/sentry filter consolidation (#934) 2026-07-24 15:42:41 -04:00
unified feat(dialectic): optional structured outputs with limited schema for Dialectic calls (#896) 2026-07-20 18:46:49 -04:00
utils fix: increase throughput of unit tests by changing behavior db teardown (#949) 2026-07-29 11:19:41 -04:00
vector_store fix: apply session scoping to all working-representation query paths (#881) 2026-07-24 11:41:34 -04:00
webhooks Tighten Transaction Scopes (#525) 2026-04-08 11:14:50 -04:00
__init__.py Refactor clients.py to add modern features and more flexible configuration (#459) 2026-04-20 02:46:37 -04:00
conftest.py fix: increase throughput of unit tests by changing behavior db teardown (#949) 2026-07-29 11:19:41 -04:00
test_advanced_filters.py fix: apply session scoping to all working-representation query paths (#881) 2026-07-24 11:41:34 -04:00
test_cache_redaction.py fix: increase throughput of unit tests by changing behavior db teardown (#949) 2026-07-29 11:19:41 -04:00
test_config.py fix(llm): support per-request provider timeouts (#832) 2026-08-04 12:43:00 -04:00
test_datetime_parsing.py Make embeddings configurable (#678) 2026-05-14 15:03:35 -04:00
test_db_resilience.py add read db (#773) 2026-06-10 13:28:36 -04:00
test_dependencies.py add read db (#773) 2026-06-10 13:28:36 -04:00
test_generate_jwt_script.py feat: add generate_jwt.py script for creating scoped JWTs (#757) 2026-06-09 13:49:55 -04:00
test_models_vector_dim.py feat: add new cloudevents for api routes (#637) 2026-05-20 18:25:30 -04:00
test_schema_validations.py Align API contract with DB contract for IDs (#684) 2026-05-14 16:37:39 -04:00
test_search.py Tighten Transaction Scopes (#525) 2026-04-08 11:14:50 -04:00
test_security.py Fix scoped JWTs (#679) 2026-06-22 17:30:00 -04:00
test_session_allowlist.py feat: session allowlist on dialectic and representation via filters (#882) 2026-07-28 10:55:02 -04:00