Commit Graph

7 Commits

Author SHA1 Message Date
Rajat Ahuja 097f3b31a0
align DB state to sqlalchemy model definitions (#245)
* feat: align DB schema with sqlalchemy model definitions

* test: add test for new migration file

* fix: add pk for message embeddings table

* fix: add naming constraint

* fix: align more indexes + unique constraints --> rm redundancy

* fix: add missing indexes

* Fix Server Default Migrations and add in appropriate Server Defaults (#252)

* fix: Improve alembic migration reliability

* fix: Ensure alembic migrations use a session pooler (#247)

* fix: Ensure alembic migrations use a session pooler

* fix: bump batch size; fix document delete in 08894082221a (#249)

* chore: add alembic logging

* fix (alembic): remove order by in batches

* fix: fkey -> fk

* chore: code rabbit

---------

Co-authored-by: Rajat Ahuja <rahuja445@gmail.com>

* fix: Add server defaults to appropriate columns

* chore: (tests) Add alembic migration test

---------

Co-authored-by: Rajat Ahuja <rahuja445@gmail.com>

* fix: align models with alembic check

---------

Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
2025-11-03 14:58:12 -05:00
Vineeth Voruganti 6df41265ed
fix: Ensure alembic migrations use a session pooler (#247)
* fix: Ensure alembic migrations use a session pooler

* fix: bump batch size; fix document delete in 08894082221a (#249)

* chore: add alembic logging

* fix (alembic): remove order by in batches

* fix: fkey -> fk

* chore: code rabbit

---------

Co-authored-by: Rajat Ahuja <rahuja445@gmail.com>
2025-10-28 17:41:49 -04:00
Rajat Ahuja 5db7b4948c
feat: introduce alembic migration verification (#238)
* feat: introducer migration verification checks

* fix: move verification to tests/alembic

* feat: add verification steps for all alembic migrations

* fix: isolate test runs and implement all migration tests

* test: parametrize

* fix: CR comments

* fix: Add README

* feat: add precommit hook for validating alembic

* fix: rm pytest-alembic package

* test: create bulk resources to test migration batching

* fix: add latest test

* fix: tests to handle non-standard schema

* chore: Code Rabbit nits

* fix: CR comments 1

---------

Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
2025-10-23 16:24:35 -04:00
Rajat Ahuja a7d01d17df
feat: batch large operations in recent migrations (#242)
* feat: batch large operations in recent migrations

* fix: batch downgrade

* fix: CodeRabbit comments

* fix: delete user-created collections and documents

* fix: guard against infinite loop for internal_metadata.session_name is null

* fix: add idempotency guards
2025-10-21 11:30:56 -04:00
Benjamin McCormick 78360161d6 fix: add is_active to generated sessions 2025-10-10 14:46:41 -04:00
Benjamin McCormick 471ef778c7 fix: properly handle documents with very old session/peer setups from previous versions of internal_metadata 2025-10-10 13:32:14 -04:00
doria f988aae996
create Representation class and use it to unify all formatting (#214)
* feat: add optional JWT and webhook secrets to honcho instance creation

* chore: ignore spurious warnings

* feat: add response format if using gpt-5 model family

* feat: add response models to all apis except anthropic

* fix: raise NotImplementedError for response models in AsyncAnthropic client

* chore: address review

* [WIP] representation structure + deriver cleanup

* chore: add tests, cleanup

* feat: [WIP: semi-working] representation object

* fix: alignment

* fix: make observations hashable for dedup

* fix: datetime formatting, observation counting

* fix: switch to int for message id, clean up representation

* feat: remove need for metadata working rep

* chore: cleanup

* fix: use tenacity instead of custom fns

* feat: add representation and card to context if desired

* feat: add semantically relevant observations

* fix: pass all params to streaming, nonblocking streaming

* feat: consolidate document saving, make working representation fetching much smarter

* chore: add 100% test coverage of representation util

* feat: basic dream infra

* feat: dream queue item first pass

* chore: fixes & cleanup from coderabbit

* fix: dreams scheduled when new document count reaches a certain threshold

* feat: wip: timed dreams (not working)

* fix: test

* fix: remove useless pyright ignore

* fix: executing dreams

* feat: dreaming

* feat: [WIP] longmemeval bench

* feat: add USE_PEER_CARD setting, fix longmem test driver

* feat: get full working rep for dialectic in one swoop -- fix representation_from_documents to use the proper timestamp!

* fix: timestamps for real, handle assistant qs in longmem

* fix: remove old client, add batching to longmem

* perf: remove duplicate detection, will move to background task

* feat: track perf metrics on evals

* feat: adjust deriver prompt to use peer_id, add question date to question, clean up deriver

* fix: label metrics by task for better perf trace

* chore: code review

* feat: add efficiency score to longmem bench

* chore: tuning and cleaning up eval

* chore: bring in the big prompts

* feat: add support for vllm client

* feat: perf: bundle db calls in deriver and dialectic, increase max conns in docker db

* feat: add merge-sessions flag to longmemeval, add SUMMARY_ENABLED flag

* fix: COLLECT_METRICS default false

* chore: display start/end message ids, don't include in metrics

* fix: break large messages apart for eval

* fix: only get/create collection when needed

* feat: properly attribute documents with message id ranges and add session name column to documents

* fix: revert move of get_or_create_collection (need for fkey)

* fix: always get collection with peer name even if it's none

* chore: coderabbit

* fix: give peer card its own config, expand document schema, refactor get_context to be parallel, various cleanup chores and bugfixes

* chore: refactor: reify observer/observed system across entire codebase, including db migration

* refactor: cleanup code organization, make singletons where desired

* refactor: replace embeddings store with representation manager

* chore: coderabbit cleanup

* chore: update migration to non-null session param in documents, general review and cleanup

* chore: merge branch 'main' into ben/deriver-tidy

* chore: review fixes
2025-10-07 15:28:44 -04:00