833a89e70a
* feat: init turbopuffer and lanceDB * fix: remove destructive embedding migration * fix: bug fixes * fix: LanceDB * fix: turbopuffer * fix: search and add create_observations * fix: use Async clients * fix: search; protect agaainst failed vector create/delete * fix: coderabbit comments * fix: set up compose vector store and reconciliation loop * feat: sync docs without embeddings * fix: reduce batch size; comments; types; add indexes for reconciliation * fix: add message embedding resilience * fix: clean-up and migration test * fix: cleanup 2 * fix: centralize retry logic; bump reconciliation batch; use tracked db; fix soft-delete race condition * fix: skip double query when pgvector is primary * fix: down migration * fix: remove hard-delete from critical path and make PgVectorStore deletions a no-op * fix: use soft-delete pattern for duplicate detection * fix: steps toward deprecating MessageEmbedding table * fix: remove composite and pgvector store -> make more specific * fix: migration order * fix: shorten reconciliation cycle + fix 'IN' equality check * fix: coderabbit comments * fix: add test for migration 7c0d9a4e3b1f * feat: refactor to use ReconcilerScheduler * fix: CR / opus comments * fix: work unit key and reserve system workspace * fix: make workspace_name nullable * fix: clean up sync vectors * fix: delete syntax * fix: hash namespace * External Vector Store Nits (#332) * fix: Migration naming and long held connection * chore: Comment for potential debt * chore: update typescript core package --------- Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| revisions | ||
| README.md | ||
| __init__.py | ||
| conftest.py | ||
| registry.py | ||
| scaffold.py | ||
| test_pipeline.py | ||
| verifier.py | ||
README.md
These tests validate Alembic migrations end-to-end for structure, order, and correctness. They ensure reversibility, expected schema and data, and integration with the registry and pipeline. The key components are the verifier, the test pipeline, the registry, and the revisions under test.
Verifier
- The verifier runs checks when specific revisions are applied and reverted.
- It validates the schema after upgrade, verifies data migrations such as defaults, backfills, and transforms, and confirms reversibility after downgrade.
- Assertions are grouped per-revision or feature, and helpers use the SQLAlchemy inspector to introspect the database.
Test Pipeline
- The pipeline orchestrates the database lifecycle by creating a database, applying upgrades and downgrades, running verifications, and tearing down resources.
- It typically starts from base, upgrades to the revision immediately before a target revision, seeds the DB, runs the target migration, and then validates the schema + data
- It relies on shared fixtures such as
engine,connection, andalembic_config, and it ensures isolation per test
Registry
- The registry declares revisions and test metadata used to drive scenarios.
- It defines ordering and selection, attaches verifier callbacks to specific revisions or ranges, and centralizes per revision expectations.
Revisions
- Revisions are the migration scripts under
alembic/revisions. - Each revision should provide functions decorated with
register_before_upgrade()andregister_after_upgrade(). These are used to validate schemas and data before and after a migration is run
Running the Tests
- Tests can be run all together
pytest tests/alembicor individuallypytest tests/alembic -k "revision_number". For example, to run the test against a1b2c3d4e5f6_initial_schema.py, you would run the commandpytest tests/alembic -k "a1b2c3d4e5f6"