Commit Graph

109 Commits

Author SHA1 Message Date
doria 817123de95
fixes: update unified tests, bugs in dream scheduling (#322)
* fix: swap to reasoning config in unified test files, handle new representation output format

* fix: resolve bugs in dream scheduling -- message from a peer should cancel all dreams observing them, and document counts should be checked at moment of dream, not in enqueue. clean up and test
2026-01-13 12:33:59 -05:00
Benjamin McCormick 2c66813944 fix: use real stainless releases, update to match 2026-01-12 17:27:54 -05:00
Benjamin McCormick 5b7ae0d82c feat: API renaming and cleanup
- Rename API routes for consistency
- Add backwards-compatible conclusion and queue endpoints
- SDK cleanup and representation improvements
- Add reasoning_level param validation
- Fix thinking budget validation for Anthropic

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 16:50:40 -05:00
doria 578ef2c665
feat: agentic dreamer and agentic dialectic (#309)
* feat: add better params to working representation fetch in SDKs, return messages when added

* fix: working representation routes now accepting all parameters properly, with tests

* feat: add metadata/config fields to SDK objects where viable

* fix: tests

* feat: refactor SDKs to use representation config; [TEMP STAINLESS BUILD] update API

* feat: add representation object to sdks

* fix: use stainless sdk on branch

* fix: update TypeScript SDK tsconfig to use node16 module resolution

* fix: add isolatedModules = true to tsconfig

* fix: lol

* chore: coderabbit review

* feat: make delete session real

* feat: add observations routes with delete endpoints for documents. make session deletion real.

* chore: type cleanup

* fix: tests

* chore: coderabbit review

* fix: namespace by workspace

* feat: add ability to customize messages_per_summary at both workspace and session level

* chore: tests for summary config

* chore: coderabbit cleanup

* feat: make session and workspace config totally customizeable

* feat: add search by peer knowledge (#250)

* feat: search by peer perspective

* fix: enforce workspace in filters, make messages distinct in join

* fix: batch and merge migration steps

* fix: add refresh, add config to workspace, add refresh function, make fields readonly

* fix: search distinct

* fix: merge migrations

* fix: merge migrations

* fix: batch deletions, improve comments, limit consolidate dream to 100 docs at a time, auth on observations routes

* chore: review

* chore: coderabbit

* chore: review

* chore: broken comment

* feat: add set peer card route to API

* feat: create advanced configuration parameters with message>session>workspace hierarchy

* [wip] build unified testing harness

* chore: lint

* fix: cache invalidation, naming things, etc

* feat: longmem tests

* chore: peer config refactor

* feat: consolidate dream working, refactor representation

* fix: Various CR Comment Fixes

* feat: Allow configurable Redis port for harness instances and update cleanup methods to be asynchronous.

* feat: agentic ingestion task!!!

* feat: agentic deriver

* feat: dialectic agent and dreamer agent

* chore: browbeat tests into passing

* fix: nits

* chore: remove old code, update config files

* fix: simplify deriver

* feat: dialectic agent prompt updates, re-introduce non_agent deriver, eval tweaks

* feat: fast deriver, dreamer, then dialectic

* fix: tweaks across the board

* feat: add baseline tests

* feat: truncation in tools and client, tweaks for evals

* feat: add locomo, fix longmem judge!!!

* fix: locomo f1 is trash, use llm judge

* feat: trace creation

* feat: add first draft of obex benchmark, fix embedding model, fix locomo methodology

* fix: locomo session-optimized, better logging of cache usage and better cache usage

* chore: use openrouter for baselines

* fix: add test for merge migration

* chore: opus-powered cleanup

* fix: add config for vllm, better client

* chore: clean up clients.py a bit

* chore: move magic numbers to config, add tests for agent tools

* fix: wrong mock in dialectic tests, make ToolContext a dataclass

* feat: tweak prompts, make deriver explicit-only

* feat: more prompt & tool tweaks

* chore: more tweaks

* feat: dream with subagents

* fix: make dream trigger override scheduled, play around with dream agents

* chore: cleanup deriver

* chore: cleanup dialectic

* chore: cleanup orchestrator

* chore: comment out dream stuff, WIPing

* fix: inc temp on retry, typechecking

* feat: tweak dreaming

* feat: contradiction obs

* Add dream trees

* chore: preserve reasoning_details from openrouter in client

* fix: get_observation_context correct params

* fix: use correct message id in tool

* chore: cleanup longmem runner

* chore: clean up tests, remove dream tests for now as rearchitecting around trees

* chore: update stainless deps

* Update threholding mechanism

* chore: pre-commit hooks whitespace

* chore: clean up types

* feat: add explicit bench

* fix: address additional basepyright issues

* fix: adding logging as a fixture on honcho_llm_call and supporting dialectic loging. (#305)

* fix: lock on db for tool calls

* chore: clean up experimental derivers

* chore: coderabbit review cleanup

* feat: add streaming support to agentic dialectic

* feat: prometheus token tracking for deriver and dialectic

* fix: self-loops for isolated nodes

* chore: PascalCase for prometheus parameter typing

* feat: add reasoning levels to dialectic agent

* chore: delete old file, add new fake env vars in unittest.yml

* fix: all fields needed for dialectic reasoning level configs

* feat: track dreaming usage in prometheus

* chore: Create backwards compatabile conclusion and queue endpoints

* fix: remove redundant try-catch, add trace label, move .limit to end of statement

* fix: remove vignettes (for now), review fixes, remove merge migration, config cleanup

* chore: code review / cleanup

* chore: merge fixes

* chore: clean up, remove reasoning_focus, reintroduce peer cards in dreamers

* chore: code rabbit nitpicks

* fix: add unique index for pending dreams in queue

* fix: revert removal of surprisal in dreamer config

---------

Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
Co-authored-by: 3un01a <3un01a@plasticlabs.ai>
Co-authored-by: 3un01a <3un01a.labs@gmail.com>
Co-authored-by: ajspig <46900795+ajspig@users.noreply.github.com>
2026-01-12 15:12:17 -05:00
doria 20e60cb6ba
chore: fix basedpyright warnings (#317) 2026-01-08 14:02:25 -05:00
Rajat Ahuja 201b5125b3
feat: increase iam role duration. alter s3 file name and add to discord message (#315) 2026-01-07 13:04:13 -05:00
Rajat Ahuja acfade7d4e
feat: run unified tests suite in CI (#291)
* feat: run unified tests in CI

* fix: attempt use aws secrets manager

* fix: temp add verification workflow

* fix: CodeRabbit comments

* fix: remove debugging step

* fix: only run on main

* fix: add UNIFIED_TEST_LOG_LEVEL env var; default to WARNING
2026-01-06 16:03:29 -05:00
Benjamin McCormick f0d24af827 Merge branch 'main' into vince/dev-1259 2025-12-10 10:57:40 -05:00
Vineeth Voruganti ca702cfd10
Resolve SDK Inconsistencies and Add Observation Creation Endpoints (#288)
* feat: Add Observation Creation Endpoints and SDK Cleanup

* fix: Resolve linting errors

* chore: Code Rabbit Nits

* chore: Code Rabbit Nits
2025-12-04 15:24:45 -05:00
Vineeth Voruganti 72eb0827ec
Add session cloning to Ergonomic SDKs (#285)
* feat: Add session cloning to Ergonomic SDKs

* chore: docs nit
2025-12-04 14:51:39 -05:00
doria e3d345b961
API/SDK updates: configurability, more parameters. Unified test harness (#283)
* feat: add better params to working representation fetch in SDKs, return messages when added

* fix: working representation routes now accepting all parameters properly, with tests

* feat: add metadata/config fields to SDK objects where viable

* fix: tests

* feat: refactor SDKs to use representation config; [TEMP STAINLESS BUILD] update API

* feat: add representation object to sdks

* fix: use stainless sdk on branch

* fix: update TypeScript SDK tsconfig to use node16 module resolution

* fix: add isolatedModules = true to tsconfig

* fix: lol

* chore: coderabbit review

* feat: make delete session real

* feat: add observations routes with delete endpoints for documents. make session deletion real.

* chore: type cleanup

* fix: tests

* chore: coderabbit review

* fix: namespace by workspace

* feat: add ability to customize messages_per_summary at both workspace and session level

* chore: tests for summary config

* chore: coderabbit cleanup

* feat: make session and workspace config totally customizeable

* feat: add search by peer knowledge (#250)

* feat: search by peer perspective

* fix: enforce workspace in filters, make messages distinct in join

* fix: batch and merge migration steps

* fix: add refresh, add config to workspace, add refresh function, make fields readonly

* fix: search distinct

* fix: merge migrations

* fix: merge migrations

* fix: batch deletions, improve comments, limit consolidate dream to 100 docs at a time, auth on observations routes

* chore: review

* chore: coderabbit

* chore: review

* chore: broken comment

* feat: add set peer card route to API

* feat: create advanced configuration parameters with message>session>workspace hierarchy

* [wip] build unified testing harness

* chore: lint

* fix: cache invalidation, naming things, etc

* feat: longmem tests

* chore: peer config refactor

* feat: consolidate dream working, refactor representation

* fix: Various CR Comment Fixes

* feat: Allow configurable Redis port for harness instances and update cleanup methods to be asynchronous.

* fix: version bump, api/sdk updates

* fix: observation endpoints, deletion queue, sdk observation implementation

* chore: Fix migration order

* fix: Use published stainless sdks

* chore: (docs) update api-reference

* fix: (docs) update based on api and sdk changes

* fix: Code Rabbit Comments

* fix: Code Rabbit Final Nits

* fix: dream scheduler

* fix: SDK model type consistency

---------

Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
2025-12-03 16:49:30 -05:00
doria 4ee2f8bd0c
Merge pull request #270 from plastic-labs/vince/ai-96
feat: beam benchmark initial commit
2025-11-20 17:59:41 -05:00
Benjamin McCormick 298dbbdb85 chore: coderabbit review 2025-11-20 17:56:27 -05:00
Benjamin McCormick f333e64f12 chore: tuning beam bench 2025-11-20 17:19:14 -05:00
Vineeth Voruganti a7520ce21d
Standardize DB Constraint Conventions (#272)
* fix: (db) Add standard naming conventions to SQLAlchemy Declarative Base

* chore: remove unnecessary relationships

* fix: (tests) handle old migrations being made before conventions were applied

* fix: wip migration to standardize naming for constraints

* fix (db): Migration wip)

* chore: rebase migration

* fix (db) Add migration tests

* chore: Code Review Comments

* fix: (db) fix remaining inconsistent index
2025-11-20 11:50:54 -05:00
Rajat Ahuja 0f1e1dec20
metrics for deriver / dialectic input + output tokens (#274)
* feat: separate deriver input / output tokens

* fix: track dialectic input / output tokens

* feat: track dialectic output tokens in streaming API

* fix: add component to metric ad change critical_analysis -> representation

* feat: track summary metrics in prometheus

* test: fix client streaming mocks

* fix: update tokenizer

* fix: add helper method; instrument peer card

* fix: count previous summary if not fallback
2025-11-19 21:20:34 -05:00
Benjamin McCormick 1616304b40 chore: fix up beam script 2025-11-19 17:56:12 -05:00
vintro c8b0aa9cf2
fix: timeout transparency 2025-11-19 13:43:42 -05:00
vintro a88799ca3e
fix: update code to handle 10M 2025-11-19 09:47:49 -05:00
Rajat Ahuja 2945f6bb3c
rm: level_valid constraint (#266)
* rm: level_valid constraint

* fix: create new migration

* fix: normalize data before enforcing check constraint again
2025-11-17 12:56:37 -05:00
Rajat Ahuja 7e8cf4336c
fix: add test for message seq in session (#268) 2025-11-17 12:54:23 -05:00
vintro 6fbb702ae4
feat: beam benchmark initial commit 2025-11-17 10:36:26 -05:00
Benjamin McCormick 5226b84cf3 Merge branch 'main' into ben/dev-1163 2025-11-13 12:24:39 -05:00
doria c8d75c77df
Merge pull request #256 from plastic-labs/ben/better-dedup
feat: better dedup with configurability
2025-11-13 11:46:43 -05:00
Rajat Ahuja d7bdcc3bc1
feat: codify queue columns (#254)
* feat: codify queue columns

* fix: batch with python loop control

* fix: cleanup merge

* fix: down revision

* fix: batch delete in migration

* feat: only run alembic tests for changed migration / test (#264)

* feat: only run alembic tests for changed migration / test
* fix: Run full test suite if alembic testing infra changes

* feat: codify times_derived + level on Document (#260)

* feat: codify times_derived + level on Document
* fix: CR comment

* fix: CodeRabbit comments

* fix: batch migrations; move types; remove fields from payload

* fix: rm duplicate table args

* fix: add messages.id FK
2025-11-07 13:22:24 -05:00
Rajat Ahuja 8e3f24d4f8
feat: add caching (redis) (#265)
* feat: init redis cache

* feat: initialize redis client

* fix: add fakeredis client

* fix: update docker compose

* fix: various coderabbit comments

* fix: update docker-compose.yml.example

* fix: CR comments 2

* fix: clean up client

* fix: cache key class; workspace LRU; constants -> utils

* fix: CodeRabbit comments

* feat: add redis to harness

* fix: use cashews

* fix: add locking for cache stampedes

* fix: clean up PR

* f: unify cache key templates and stop caching None

* fix: alter prefixes + prime cache on get_or_create()

* fix: re-add fakeredis

* chore: Code Rabbit nits

* CR comments; make client more resilient

* fix: BPR

* fix: set redis once but clear cache in between every test

* fix: attempt fix contextvar error

* fix: make TestClient rely on redis patch

* fix: potential working fix

* fix: namespace inheritance; address comments

* fix: namespace inheritance; address comments

---------

Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
2025-11-07 13:04:28 -05:00
Rajat Ahuja 1d0934a568
fix: message seq in session N+1 (#261)
* fix: message seq in session N+1

* test: behavior of enqueue

* fix: test

* chore: Code Rabbit Comments

---------

Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
2025-11-03 15:33:48 -05:00
Rajat Ahuja 097f3b31a0
align DB state to sqlalchemy model definitions (#245)
* feat: align DB schema with sqlalchemy model definitions

* test: add test for new migration file

* fix: add pk for message embeddings table

* fix: add naming constraint

* fix: align more indexes + unique constraints --> rm redundancy

* fix: add missing indexes

* Fix Server Default Migrations and add in appropriate Server Defaults (#252)

* fix: Improve alembic migration reliability

* fix: Ensure alembic migrations use a session pooler (#247)

* fix: Ensure alembic migrations use a session pooler

* fix: bump batch size; fix document delete in 08894082221a (#249)

* chore: add alembic logging

* fix (alembic): remove order by in batches

* fix: fkey -> fk

* chore: code rabbit

---------

Co-authored-by: Rajat Ahuja <rahuja445@gmail.com>

* fix: Add server defaults to appropriate columns

* chore: (tests) Add alembic migration test

---------

Co-authored-by: Rajat Ahuja <rahuja445@gmail.com>

* fix: align models with alembic check

---------

Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
2025-11-03 14:58:12 -05:00
doria 1df47e61c8
fix: remove list webhook db call (not needed, causes race condition) (#257)
* fix: remove get_workspace call from list_webhooks (not necessary, causes race condition)

* fix: remove old test
2025-11-03 11:26:44 -05:00
doria 342d72184a
feat: rework langfuse setup to work more cleanly; fix bug in dream scheduling (#253) 2025-11-03 11:25:29 -05:00
doria 71a6b80594
Merge pull request #258 from plastic-labs/ben/add-test-count-to-longmem
feat: add --test-count arg to longmem.py
2025-10-31 13:29:42 -04:00
Benjamin McCormick 9bdd8741a5 feat: add --test-count arg to longmem.py 2025-10-30 16:52:23 -04:00
Benjamin McCormick 0f8fc7f21c chore: remove old unuseful code 2025-10-30 14:57:25 -04:00
Benjamin McCormick 3186d2ce39 feat: add optional backup providers that kick in on retry 2025-10-29 16:53:28 -04:00
Benjamin McCormick a90156e113 feat: rework langfuse setup to work more cleanly; fix bug in dream scheduling 2025-10-29 16:10:11 -04:00
Vineeth Voruganti 6df41265ed
fix: Ensure alembic migrations use a session pooler (#247)
* fix: Ensure alembic migrations use a session pooler

* fix: bump batch size; fix document delete in 08894082221a (#249)

* chore: add alembic logging

* fix (alembic): remove order by in batches

* fix: fkey -> fk

* chore: code rabbit

---------

Co-authored-by: Rajat Ahuja <rahuja445@gmail.com>
2025-10-28 17:41:49 -04:00
Rajat Ahuja 5db7b4948c
feat: introduce alembic migration verification (#238)
* feat: introducer migration verification checks

* fix: move verification to tests/alembic

* feat: add verification steps for all alembic migrations

* fix: isolate test runs and implement all migration tests

* test: parametrize

* fix: CR comments

* fix: Add README

* feat: add precommit hook for validating alembic

* fix: rm pytest-alembic package

* test: create bulk resources to test migration batching

* fix: add latest test

* fix: tests to handle non-standard schema

* chore: Code Rabbit nits

* fix: CR comments 1

---------

Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
2025-10-23 16:24:35 -04:00
Rajat Ahuja 77a965e97f
feat: fix race condition in message sequence batching (#235)
* feat: fix race condition in message sequence batching

* fix: CodeRabbit comments; commit early to release the advisory lock before generating embeddings

* fix: use index + rm unused method

* fix: PR comments

* fix: bug in lock timeout

* fix: patch tracked_db for peers route within conftest.py
2025-10-16 11:46:55 -04:00
doria 65503955a4
fix: defensively get summary, better summary logging, gemini client token count (#236)
* fix: remove default max_distance from get_working_rep, get token count from gemini client properly, log summaries better

* fix: defensively get message public id (new field)

* fix: add defensive check to to_schema_summary

---------

Co-authored-by: Rajat Ahuja <rahuja445@gmail.com>
2025-10-10 18:18:31 -04:00
Benjamin McCormick 1eefc4af31 fix: [hotfix] return schemas, not models 2025-10-09 17:25:11 -04:00
doria 68acf38134
Misc: bug fixes, multi-db test harness, DELETE workspace (#230)
* feat: add optional JWT and webhook secrets to honcho instance creation

* chore: ignore spurious warnings

* feat: add response format if using gpt-5 model family

* feat: add response models to all apis except anthropic

* fix: raise NotImplementedError for response models in AsyncAnthropic client

* chore: address review

* [WIP] representation structure + deriver cleanup

* chore: add tests, cleanup

* feat: [WIP: semi-working] representation object

* fix: alignment

* fix: make observations hashable for dedup

* fix: datetime formatting, observation counting

* fix: switch to int for message id, clean up representation

* feat: remove need for metadata working rep

* chore: cleanup

* fix: use tenacity instead of custom fns

* feat: add representation and card to context if desired

* feat: add semantically relevant observations

* fix: pass all params to streaming, nonblocking streaming

* feat: consolidate document saving, make working representation fetching much smarter

* chore: add 100% test coverage of representation util

* feat: basic dream infra

* feat: dream queue item first pass

* chore: fixes & cleanup from coderabbit

* fix: dreams scheduled when new document count reaches a certain threshold

* feat: wip: timed dreams (not working)

* fix: test

* fix: remove useless pyright ignore

* fix: executing dreams

* feat: dreaming

* feat: [WIP] longmemeval bench

* feat: add USE_PEER_CARD setting, fix longmem test driver

* feat: get full working rep for dialectic in one swoop -- fix representation_from_documents to use the proper timestamp!

* fix: timestamps for real, handle assistant qs in longmem

* fix: remove old client, add batching to longmem

* perf: remove duplicate detection, will move to background task

* feat: track perf metrics on evals

* feat: adjust deriver prompt to use peer_id, add question date to question, clean up deriver

* fix: label metrics by task for better perf trace

* chore: code review

* feat: add efficiency score to longmem bench

* chore: tuning and cleaning up eval

* chore: bring in the big prompts

* feat: add support for vllm client

* feat: perf: bundle db calls in deriver and dialectic, increase max conns in docker db

* feat: add merge-sessions flag to longmemeval, add SUMMARY_ENABLED flag

* fix: COLLECT_METRICS default false

* chore: display start/end message ids, don't include in metrics

* fix: break large messages apart for eval

* fix: only get/create collection when needed

* feat: properly attribute documents with message id ranges and add session name column to documents

* fix: revert move of get_or_create_collection (need for fkey)

* fix: always get collection with peer name even if it's none

* chore: coderabbit

* fix: give peer card its own config, expand document schema, refactor get_context to be parallel, various cleanup chores and bugfixes

* chore: refactor: reify observer/observed system across entire codebase, including db migration

* refactor: cleanup code organization, make singletons where desired

* refactor: replace embeddings store with representation manager

* chore: coderabbit cleanup

* feat: multi-db longmem harness

* Merge branch 'main' into ben/multi-db-harness

* [WIP] feat: add delete workspace endpoint, use in bench

* chore: move excess logging to debug

feat: improve metrics block logs to include more data

fix: make longmem db deletion configurable

* fix: [CRITICAL] use async genai client

* chore: update core sdk, fix tests to use aio as well

* fix: rollback prompt changes

* chore: update version

* fix: cleanup, coderabbit, wrap delete op in try/except

---------

Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
2025-10-09 16:53:41 -04:00
doria a3d98afdfe
Add updated get_context to SDKs (#231)
* feat: add optional JWT and webhook secrets to honcho instance creation

* chore: ignore spurious warnings

* feat: add response format if using gpt-5 model family

* feat: add response models to all apis except anthropic

* fix: raise NotImplementedError for response models in AsyncAnthropic client

* chore: address review

* [WIP] representation structure + deriver cleanup

* chore: add tests, cleanup

* feat: [WIP: semi-working] representation object

* fix: alignment

* fix: make observations hashable for dedup

* fix: datetime formatting, observation counting

* fix: switch to int for message id, clean up representation

* feat: remove need for metadata working rep

* chore: cleanup

* fix: use tenacity instead of custom fns

* feat: add representation and card to context if desired

* feat: add semantically relevant observations

* fix: pass all params to streaming, nonblocking streaming

* feat: consolidate document saving, make working representation fetching much smarter

* chore: add 100% test coverage of representation util

* feat: basic dream infra

* feat: dream queue item first pass

* chore: fixes & cleanup from coderabbit

* fix: dreams scheduled when new document count reaches a certain threshold

* feat: wip: timed dreams (not working)

* fix: test

* fix: remove useless pyright ignore

* fix: executing dreams

* feat: dreaming

* feat: [WIP] longmemeval bench

* feat: add USE_PEER_CARD setting, fix longmem test driver

* feat: get full working rep for dialectic in one swoop -- fix representation_from_documents to use the proper timestamp!

* fix: timestamps for real, handle assistant qs in longmem

* fix: remove old client, add batching to longmem

* perf: remove duplicate detection, will move to background task

* feat: track perf metrics on evals

* feat: adjust deriver prompt to use peer_id, add question date to question, clean up deriver

* fix: label metrics by task for better perf trace

* chore: code review

* feat: add efficiency score to longmem bench

* chore: tuning and cleaning up eval

* chore: bring in the big prompts

* feat: add support for vllm client

* feat: perf: bundle db calls in deriver and dialectic, increase max conns in docker db

* feat: [WIP] realtime context object
note: must download custom stainless API for SDK

* feat: add merge-sessions flag to longmemeval, add SUMMARY_ENABLED flag

* fix: COLLECT_METRICS default false

* chore: display start/end message ids, don't include in metrics

* fix: break large messages apart for eval

* fix: only get/create collection when needed

* feat: properly attribute documents with message id ranges and add session name column to documents

* fix: revert move of get_or_create_collection (need for fkey)

* fix: always get collection with peer name even if it's none

* chore: coderabbit

* fix: bug in get context
feat: get context updates in ts sdk

* feat: viz

* chore: update honcho-ai/core, remove WIPs

* fix: consistent ordering, comment nits, removed excess dreamer init

* fix: test int->str

* fix: Add validation and update async python client

* fix: add validation for last_user_message as well

* fix: add deeper validation to getContext in typescript sdk

* fix: let session context take a Message object for lastUserMessage to match python sdk behavior

* fix: use PeerIdSchema

* fix: allow peer object as argument

* fix: lastUserMessage min length 1

---------

Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
2025-10-09 15:44:21 -04:00
Rajat Ahuja f38230fd92
Integrate prometheus into Honcho (#226)
* feat: add prometheus to api routes

* fix: rm multiproc setup

* feat: add prometheus to  deriver

* fix: rm deriver-specific counters (for now)

* fix: dangling issues

* feat: skip metrics endpoint; track dialectic calls; track individual message creation

* fix: CR comments and use fastapi route template. also fix dialectic call

* fix: revert to existing request_id logic

* fix: rm session_name from dialectic call

* feat: add deriver task metric

* fix: rm unused get_namespace

* rm: namespace param

* feat: add MESSAGE_INPUT_TOKENS and DERIVER_TOKENS_PROCESSED counters

* fix: rm message input tokens

* fix: PR comments; docstrings; error handle metrics exposing endpoint

* chore: Code Rabbit

---------

Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
2025-10-08 13:53:48 -04:00
Rajat Ahuja 3d0f96d339
Add QueueItem cleanup logic (#228)
* feat: start queue cleanup

* feat: add Queue cleanup logic

* feat: mark messages as errored

* fix: rename + commit

* fix: rm apscheduler

* fix: add default for created_at in migration

* fix: consolidate error handling and only throw away first message of an errored batch

* fix: move processing to within queue manager

* fix: regen migration

* fix: CR comment

* fix: rename message -> queue_item when dealing with queue_item in the queue manager
2025-10-08 12:44:33 -04:00
doria f988aae996
create Representation class and use it to unify all formatting (#214)
* feat: add optional JWT and webhook secrets to honcho instance creation

* chore: ignore spurious warnings

* feat: add response format if using gpt-5 model family

* feat: add response models to all apis except anthropic

* fix: raise NotImplementedError for response models in AsyncAnthropic client

* chore: address review

* [WIP] representation structure + deriver cleanup

* chore: add tests, cleanup

* feat: [WIP: semi-working] representation object

* fix: alignment

* fix: make observations hashable for dedup

* fix: datetime formatting, observation counting

* fix: switch to int for message id, clean up representation

* feat: remove need for metadata working rep

* chore: cleanup

* fix: use tenacity instead of custom fns

* feat: add representation and card to context if desired

* feat: add semantically relevant observations

* fix: pass all params to streaming, nonblocking streaming

* feat: consolidate document saving, make working representation fetching much smarter

* chore: add 100% test coverage of representation util

* feat: basic dream infra

* feat: dream queue item first pass

* chore: fixes & cleanup from coderabbit

* fix: dreams scheduled when new document count reaches a certain threshold

* feat: wip: timed dreams (not working)

* fix: test

* fix: remove useless pyright ignore

* fix: executing dreams

* feat: dreaming

* feat: [WIP] longmemeval bench

* feat: add USE_PEER_CARD setting, fix longmem test driver

* feat: get full working rep for dialectic in one swoop -- fix representation_from_documents to use the proper timestamp!

* fix: timestamps for real, handle assistant qs in longmem

* fix: remove old client, add batching to longmem

* perf: remove duplicate detection, will move to background task

* feat: track perf metrics on evals

* feat: adjust deriver prompt to use peer_id, add question date to question, clean up deriver

* fix: label metrics by task for better perf trace

* chore: code review

* feat: add efficiency score to longmem bench

* chore: tuning and cleaning up eval

* chore: bring in the big prompts

* feat: add support for vllm client

* feat: perf: bundle db calls in deriver and dialectic, increase max conns in docker db

* feat: add merge-sessions flag to longmemeval, add SUMMARY_ENABLED flag

* fix: COLLECT_METRICS default false

* chore: display start/end message ids, don't include in metrics

* fix: break large messages apart for eval

* fix: only get/create collection when needed

* feat: properly attribute documents with message id ranges and add session name column to documents

* fix: revert move of get_or_create_collection (need for fkey)

* fix: always get collection with peer name even if it's none

* chore: coderabbit

* fix: give peer card its own config, expand document schema, refactor get_context to be parallel, various cleanup chores and bugfixes

* chore: refactor: reify observer/observed system across entire codebase, including db migration

* refactor: cleanup code organization, make singletons where desired

* refactor: replace embeddings store with representation manager

* chore: coderabbit cleanup

* chore: update migration to non-null session param in documents, general review and cleanup

* chore: merge branch 'main' into ben/deriver-tidy

* chore: review fixes
2025-10-07 15:28:44 -04:00
Rajat Ahuja accdc79fdb
feat: Modify the Summary BaseModel to use public_id of message rather than internal ID (#224)
* feat: Modify the Summary BaseModel to use public_id of message rather than internal ID

* fix: fallback invocation

* fix: strict validation

* fix: rm ID from Message schema

* fix: make SDK changes
2025-10-07 12:37:22 -04:00
Vineeth Voruganti 73c5eb1f5a
SDK Changes (#223)
* fix: Add top level transactions for tracing in deriver

* fix: Add Delete methods to sdks for sessions

* feat: Add session deletion and session level deriver status methods

* fix: Streaming support to python sdk

* fix: Checkpoint Streaming working in SDKs with workarounds for stainless

* feat: fix pagination, add message metadata update, and get peer card methods

* feat: parity in typescript and python sdks

* fix: Address Coderabbit

* fix: Address Coderabbit

* chore: add basedpyright rules to sdk

* fix: Code Rabbit & Changelog

* fix: updated docs with api-reference
2025-10-01 16:25:30 -04:00
Rajat Ahuja 3df044e1b7
feat: use token limit from all senders for rollup (#221)
* feat: use token limit from all senders for rollup

* fix: send all messages to queue_manager

* test: fixe tests and move ownership check to separate query

* fix: if first sender message exceeds batch then we should return an empty batch

* feat: separate representation and non-representation processing

* fix: return msg object and queue items

* fix: rollup logic with interleaving messages

* fix: rm TODO

* Fix langfuse trace and add fallback for weird error in deriver

* fix: simplify query

* fix: comment out update_current_generation

* fix: BPR

---------

Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
2025-10-01 16:10:10 -04:00
Rajat Ahuja 6e01af2005
Track worker ownership (#216)
* fix: simplify; use debug logs

* fix: random type error

* fix: CR comments

* fix: move limit check before opening db connection

* fix: use single SQL call to verify ownership and fetch message batch

* fix: CR comments 2
2025-09-25 15:14:48 -04:00
Vineeth Voruganti 09a6b4f8e2
fix: add shadow .env file for compose file (#196) 2025-09-24 16:09:51 -04:00