* feat(cli): add `honcho session view` transcript command
Adds a read-only transcript view for a session, with three paging modes:
a tail window (`--last N`, the default), server pages (`--page N --size M`),
and the whole conversation (`--all`). `--reverse` selects newest-first in
every mode, `--ids` exposes message IDs, and `-p` scopes to one peer.
JSON mode emits the same shape as `message list`.
The renderer is deliberately literal: content and identifiers go through
`rich.text.Text` rather than Markdown or console markup, so newlines, tag
delimiters like `<thinking>`, and bracketed text survive intact — this is a
debugging surface, so it has to show what was actually stored. Timestamps
are converted to UTC (not just stripped of their offset) and keep
millisecond precision. Nothing is truncated with an ellipsis: a displayed
message ID is always usable with `honcho message get`.
Flags are validated before the client is built, and the session is
constructed directly instead of via the get-or-create `client.session()`,
so an invalid or mistyped invocation never reaches — or creates — anything
server-side. `--size` is bounded locally to the server's 100-item ceiling
rather than surfacing a raw 422, and the "more:" hint echoes back the size
and ordering actually in use so following it lands on the adjacent window.
Also fixes `honcho message list --last N`, which stopped at the first page
of 50: both commands now share the page-walking helper, so the same flag
returns the same window either way.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(cli): regenerate command reference for `session view`
Adds the generated `session view` accordion to the docs snippet and points
the session-debugging workflows at it. Trims the docstring to plain prose —
the RST double-backticks were rendering literally in `--help`, where every
other command uses unmarked flag names — and stops the generator emitting a
trailing blank line that tripped end-of-file-fixer on every regeneration.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(cli): carry invocation scope into the next-page hint
Addresses CodeRabbit review on #1006.
The "more:" hint echoed only `--page`, `--size`, and `--reverse`, so copying
it off a scoped invocation dropped `-w`, `-p`, and `--ids` — landing on a
different workspace or an unfiltered transcript. Hint construction moves into
`_next_page_command` in the command module, which knows the invocation; the
renderer now just prints the string it's handed and no longer needs to know
CLI flag syntax. Only flags passed explicitly are echoed, since anything from
the environment or config resolves the same way on the next run.
Also rejects non-positive `--last` on `honcho message list`, which slice
semantics turned into a silently empty result. `session view` already errored
on it; the two now agree.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(cli): read next-page hint scope from effective flag overrides
Addresses the second CodeRabbit pass on #1006.
`-w`/`-p` parse at group and top level as well as command level, all landing
in `_global_overrides`, so reading the command-level params dropped the scope
from `honcho session -w ws2 view ...`. The hint now reads the effective
overrides via a new `get_flag_overrides()`, which deliberately excludes
environment and config values since those resolve the same way on the next run.
Also shell-quotes the hint's identifiers with `shlex.join`. Note this is
hardening rather than a live injection fix: the API constrains IDs to
`^[a-zA-Z0-9_-]+$`, so an ID carrying a space or metacharacter fails the fetch
before any hint is printed. `validate_resource_id` is looser than the server
though, so quoting is the cheaper invariant to hold locally.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* docs: adding honcho-memory skill
* fix: skills to point at llm friendly content
* docs: split honcho-mcp skill out of honcho-memory; address PR review
Restructure honcho-memory into a concepts/strategy hub that routes to
per-connection path skills, and add a dedicated honcho-mcp skill holding
the MCP-tool mechanics that previously lived inline.
Addresses review feedback on #784:
- honcho-memory step 2 now leads with fast context reads, with chat as
the slower escalation
- honcho-mcp adds a "Speed: reads vs reasoning" section, describes what
each context call returns, and a reasoning-levels table
- get_representation framed as a contextualized snapshot insertable into
a system prompt
- drop schedule_dream from the tool table (manual escape hatch, not
routine guidance)
- prune queue-status references from honcho-cli; document honcho-mcp in
vibecoding skill registry
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(skills): move skills to canonical top-level skills/ with .claude symlink
Establish a single source of truth for agent skills. The real files now live in
the top-level skills/ directory (the publishing convention used by Vercel,
Supabase, and Cloudflare, and the tree Honcho's `npx skills add` distributes).
.claude/skills becomes a symlink to ../skills so Claude Code discovery keeps
working off the one tree — eliminating the parallel-copy sync burden.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: splitting context into references & verifying content is consistent.
* docs: fixing core language
* docs: fixing core language
* fix: language about observe_others
* chore: adding .agents folder for codex
* fix: add instructions.md into the mcp server & delete mcp skill in favor of including it in honcho-memory.
* chore: remove migrate docs (can be found on older versions)
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Structured outputs for dialectic
* cleanup
* rename json_schema_to_pydantic to clarify it's not a general schema converter
* clean up schema DoS guards
* simplification and cleanup of schema conversion
* chore: ruff and pyproject toml
* chore: basedpyright cleanup in test
* fix: some needed unrelated test failures
* test(schema_conversion-and-anthropic-backend): expand test coverage
include table tests
* fix(llm): support combined tool calling and structured output across backends
- OpenAI: parse() 500s on non-strict function tools; route tool-carrying
structured requests through create() with an explicit json_schema
response_format (mirrors the streaming path)
- Anthropic: skip the '{' JSON prefill when tools are present so tool_use
blocks stay reachable; make the schema instruction conditional and rely
on parse + repair
- Gemini: native response_schema + function calling is rejected before
Gemini 3; with tools present, inject a schema instruction into the final
turn instead and rely on parse + repair
- All backends: tool-call turns carry no consumable content, so skip
structured-output parsing on them
Extracted from the dialectic structured-output branch (DEV-1652) so the
transport layer can land independently.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(live_llm): exercise combined tools + structured output per provider
Two-turn live flow per backend: a forced tool-call turn (structured
parsing must be skipped) followed by a replay turn that must return a
schema-conforming answer with tools still attached. Asserts the
provider-specific request shaping: no parse() for OpenAI (500s on
non-strict tools), no '{' prefill for Anthropic, no native
response_schema for Gemini.
Verified against live OpenAI (gpt-4.1, gpt-5, gpt-5.4, gpt-5.4-mini)
and Gemini (gemini-2.5-flash).
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(unified): dialectic chat with response_format schema under tool use
Adds response_format pass-through to the unified runner's chat query and
a test case that forces the dialectic tool loop (reasoning off + global
enumeration question) while requiring a schema-conforming JSON answer —
end-to-end coverage of the combined tools + structured output transport
path on whichever provider each level is configured with.
Verified locally against a full harness run (json_match assertions pass;
the llm_judge assertion additionally runs in CI where the Anthropic key
is available).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: some needed unrelated test failures
* ci: add label-triggered live LLM test workflow
Adding the run-live-llm label to a PR (or workflow_dispatch) runs
tests/live_llm/ against real provider APIs — the only place the
--live-llm suite runs in CI. Reuses the unified-tests environment and
its Secrets Manager staging-dotenv resolution for provider keys; runs
on ubuntu-latest (no Fly runner, no Docker — the suite only touches the
LLM backends). Pins LIVE_LLM_ANTHROPIC_45_PLUS_MODELS=claude-sonnet-4-5
since the Anthropic family has no default models and would otherwise
silently collect empty.
Opt-in by design: live model behavior is variable, so this is a signal,
not a required check.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci: run live LLM tests on main pushes touching the transport
Mirrors unified-tests' push trigger, scoped to paths that can affect
the live suite (src/llm/, config, the tests, deps, and the workflow
itself) so provider API calls aren't spent on unrelated changes.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci: disable auth in live LLM test environment
The staging dotenv sets AUTH_USE_AUTH=true without a usable JWT secret,
and src/config.py validates the pair at import time — the same reason
unified-tests overrides it. This suite never runs the API server.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(live_llm): fix gpt-5.4 reasoning_effort and gemini replay-turn flake
- test_live_openai: gpt-5.4 dropped 'minimal' from the reasoning_effort
vocabulary, so the gpt5 caching test 400'd — and the OpenAI backend's
BadRequestError terminal swallowed it into an empty CompletionResult.
Pick the effort per model generation.
- test_live_tools_structured_output: use tool_choice='auto' on the
replay turn, matching the production dialectic loop (which never
forces 'none') — NONE mode is what provoked gemini-2.5-flash's empty
candidates. Drop the temperature pin so retries actually resample,
and treat a repeat tool call as a retryable attempt.
Verified live: full suite green, gemini 4/4 consecutive passes.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci: fail live LLM run when no staging secret was loaded
If the latest-tag fetch fails and no second tag exists, the fallback
step is skipped rather than failed, and the job would proceed without
provider keys — every test then skips via require_provider_key and the
run goes green. Guard on both fetch outcomes so that path fails loudly.
DEV-2035
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(live-llm-tests-GHA): remove extra comments
* feat(structured-output): enable non-recursive schema references
* docs(structured-outputs): clean up new doc
* test(structured-output): fix caching refs memory leak, add tests
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(conclusions): expose reasoning level + allow filtering by level
The `level` of a conclusion (explicit / deductive / inductive /
contradiction) was filterable server-side but stripped from the
`Conclusion` response and not surfaced in either SDK. This adds it
end-to-end so callers can list explicit-only ("not dreamed on")
conclusions without dropping to raw HTTP.
- api: add `level` to the Conclusion response schema
- python sdk: `ConclusionLevel` type, `level` on Conclusion/response,
`level=` kwarg on ConclusionScope.list() and the async variant
- ts sdk: `ConclusionLevel` type, `level` on Conclusion/response,
`level` option on list()
- tests: assert level is exposed; add level-filter list test
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(conclusions): use generic filters= on list() instead of level= kwarg
Match the documented SDK convention (peers/sessions/messages all take a
generic `filters` dict passed through to the same dynamic server-side
filter logic) instead of a one-off `level=` kwarg. `level` filtering now
works as `list(filters={"level": "explicit"})` alongside any other
supported filter/operator.
The `level` field on the Conclusion response (added in the previous
commit) is kept — it's still not otherwise returned by the API.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(conclusions): allow filtering by level on query() in py + ts SDKs
The branch's level-filter work exposed `filters=` on `list()` but left
`query()` (semantic search) hardcoding `{observer, observed}`, so callers
could filter the list endpoint by reasoning level but not semantic search —
asymmetric in both SDKs.
- Python: add keyword-only `filters` to `ConclusionScope.query` and
`ConclusionScopeAio.query`, merged over the scope's observer/observed.
- TypeScript: add optional `filters` arg to `ConclusionScope.query`,
mirroring the existing `list()` change.
The server `/conclusions/query` endpoint already honors filters in the body
(verified against production), so this is purely SDK surface parity.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(filters): document filtering conclusions by reasoning level
The using-filters page covered workspaces/peers/sessions/messages but not
conclusions. Add a "Filtering Conclusions" section showing level-based
filtering on both list() and query(), including the common "explicit only"
(exclude dream-derived) case and the in[deductive,inductive] inverse.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(conclusions): simplify filter merge to a single dict spread
Replace the merged_filters + if-block pattern in list()/query() (py sync,
aio, ts) with a single dict spread that layers the caller's filters over the
scope's observer/observed (and session). No behavior change — same merge
order (caller wins) — just less code.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(conclusions): reject scope-managed keys in SDK conclusion filters
The generic filters= argument on ConclusionScope.list()/query() spread
user-supplied filters last, so a stray observer/observed/session key
silently overrode the scope and returned data from a different peer
pair. Add a fail-loud guard in both the Python and TypeScript SDKs that
rejects scope-managed filter keys with a clear error, directing callers
to peer.conclusions / conclusions_of(target) and the session= parameter.
session_id remains a valid filter on query() (which has no dedicated
session parameter).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* Peer- and session-scoped JWTs were effectively workspace-scoped: auth() walked the route's declared scope and fell through to a workspace match, so a {w: ws-a, p: alice} token could act on any peer in ws-a.
* feat: peer keys can read sessions they belong to; require workspace on scoped keys
* fix: authorize JWTs by narrowest scope and gate member reads
Follow-up hardening on the narrowest-claim auth fix:
- Scope get_peer_config member-read to the caller's own peer; a session
member could previously read a co-member's per-session config.
- Enforce session membership on POST /peers/{id}/chat: the session_id
arrives in the body (invisible to require_auth), so a peer key could
read any session's injected message history. Check is_peer_in_session
in the handler before the dialectic runs.
- Consolidate the workspace-match check in auth() to a single hoisted
guard so no branch can silently re-open cross-workspace access.
- Normalize empty-string scope claims to None in verify_jwt so a blank
workspace can't satisfy the peer/session token-shape invariant.
- Extract scope_requires_workspace(), shared by verify_jwt and the keys
API so the creation-time guard and verification invariant can't drift.
route requires auth) and CLAUDE.md auth-scoping guidance.
- docs: describe narrow-scope key semantics in the platform reference.
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* feat: defer embedding messages
* fix: rm gauges
* feat: embed messages immediately on create with reconciler fallback (#766)
Adds embed_messages_now background task so newly created messages are
searchable within seconds instead of waiting up to the reconciler
interval. Three-phase claim/lease → embed → persist never holds a DB
session across the embedding call; the reconciler remains the fallback
for failures and stragglers.
* fix: harden immediate-embed fast path and cover its error branches
Wrap embed_messages_now in a top-level try/except so a failure in the
claim or persist phase degrades to "reconciler will retry" instead of
escaping into the background-task runner; the rows stay pending+leased
and the reconciler heals them.
Add tests for the previously-uncovered branches: external-store-unavailable
persist path, the file-upload endpoint's embed scheduling, and direct unit
tests for the shared compute_chunk_positions / build_message_vector_record
helpers.
Document the semantic-search eventual-consistency window in search.mdx
(keyword matches are immediate; vector matches lag creation by seconds).
* fix: don't hold DB session across vector-store upserts
* fix: align semantic-search function to filter null rows
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* docs(readme): repositioning pass + staleness fixes (P0-P4 audit)
Restructure README to match dual audience (AI-tool users + product
developers) per Vineeth's audit. No content deleted - long internal
sections collapsed under `<details>` for scannability.
Staleness fixes:
- Replace 404'd doc links (.../tutorial/SDK, /api-reference/introduction)
with verified replacements under /v3/documentation/reference/sdk
and /v3/api-reference/introduction
- Fix Python quickstart to pass api_key (managed default api.honcho.dev
would 401 otherwise)
- Drop hardcoded `gpt-4` model reference; read OPENAI_MODEL from env
- Replace archived Dialectic blog link with current Chat Endpoint docs
- Drop M3-Macbook-specific note; minor grammar ("deriver's" -> "derivers")
- Replace TL;DR Python-only example with side-by-side Python + TypeScript
framed around the "Honcho Loop" (store / reason / query / inject)
New sections:
- Start Here: three-path table (AI tools / building product / self-host)
- The Honcho Loop: operation model before code
- What Honcho Gives You: API-at-a-glance table
- Integrations: verified install commands for Claude Code (plugin + raw
MCP), OpenCode, OpenClaw, Hermes
- Honcho vs RAG: stubbed with TODO; copy deferred to marketing
- SDKs section with clearer Python/TypeScript landing pointers
Restructured:
- Core Concepts moved above Architecture; Collections/Documents reframed
as internal mechanism (Conclusions is the public surface)
- Storage / Reasoning / Retrieving deep-dive wrapped in <details>
- Local Development, Pre-commit hooks, Fly deployment, full config
matrix wrapped in <details>
Known follow-up (not in this branch): SDK docs at docs.honcho.dev and
PyPI PKG-INFO advertise `HONCHO_BASE_URL`, but the actual SDK code
(sdks/python/src/honcho/client.py:234, sdks/typescript/src/client.ts:154)
reads `HONCHO_URL`. README aligned with code; docs + PKG-INFO need
separate fix.
* docs(readme): restore "stateful agents" in opening sentence
Plastic Labs' canonical positioning uses "stateful agents" across
materials, and the original README opened with "for building stateful
agents." The repositioning pass in d6d60435 dropped the term entirely
(now zero occurrences) by following Vineeth's suggested opening copy
verbatim - but his audit's executive summary explicitly praised the
"stateful agents" positioning and didn't ask to remove it. Restoring
it in the bolded thesis sentence.
* docs(readme): drop self-referential "observations" in Conclusions bullet
The Conclusions definition shouldn't define itself in terms of
"observations." Per Plastic's positioning, "conclusions" is the
documentation-facing name for what the Deriver produces;
"observations" remains the internal code symbol. The README's
two remaining "observations" references (inside the <details>
Internal storage block and the Storage primitives block) are
explicit code-internal framing and stay.
* docs(readme): restore content dropped without audit instruction
Self-audit against Vineeth's audit found seven items I'd dropped that weren't in the audit's instructions to drop: outcome-marketing line, Contents TOC (audit said rename, not remove), multi-repo prose, org-onboarding detail, peer-paradigm feature bullets, Architecture "Key Features" bullets, and Learn More pointers. Also fixes two residual "Dialectic API" → "Chat Endpoint" mentions the original P0 sweep missed.
* docs(readme): add "Why Honcho" capability table + agent-skill onboarding
Closes the two gaps flagged in the freshness/repositioning audit: adds Vineeth's recommended "Why Honcho" capability table between Start Here and The Honcho Loop, and adds the `npx skills add plastic-labs/honcho` + `/honcho-integration` agent-skill path as a subsection of Integrations (verified against current docs).
* docs: split contributor-only sections out of README; trust auth for local postgres
- Move pre-commit hooks setup from README to CONTRIBUTING.md (pure
contributor content; the README still links to it).
- Move Fly.io deployment notes from README to the self-hosting docs.
- Wrap remaining <details>/<summary> blocks with markdownlint
disable/enable to clear pre-existing MD033/MD001 failures.
- Add POSTGRES_HOST_AUTH_METHOD=trust to the example compose template
with an inline warning, so host-side tests and tooling can connect
without supplying a password.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: (docs) update docs and evals urls and split pre-commit into contributing docs
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: retry on more httpx exceptions
* fix: Add retry parity to typescript and update docs
* chore: (skills) update skills to match latest state of the sdk
* chore: (docs) update stale sdk code
* chore: (docs) clean up inconsistencies in docs
* chore: Rebuild Package
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* fix: adding strict checking and updating readme
* chore: changelog and version
* fix: Add strict validation to all Python and TypeScript classes
* fix: Address Code Rabit Comments
* fix: duplicate searchQuery param in typescript session.context()
* feat: add created_at, is_active fields, and get_message method
* feat: Add pagintion params to sdk
* fix: Remove lazy initalization behavior from sdks
* fix: Address File Upload Validation, add compatibility shims, address review comments
* chore: Docs updates
* fix: Convert session config from API format in Peer.sessions()
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: Pass all args to Session constructor in Peer.sessions()
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: Preserve createdAt in Peer.refresh(), pass all data in session.peers()
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: changelog for ts
* fix: Review Comments
* fix: Add createAt and to peers call
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: replace Chatbots section with Tutorials, move Reachy Mini into it
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: restructure nav — move modeling-data to Core Concepts, file-uploads to Advanced, merge Migrations into Tutorials
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: final moving
* docs: minor changes
* docs: adding community integrations
* chore: cr updates
* fix: Rename patterns page and nitpicks on organization
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* chore: (docs) Reconcile new SDK conventions in docs
* chore: updating docs references to SDK.
* chore: final edits to docs and packages for SDK and API updates
* chore: (docs) fix typescript context() options method
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* refactor: update semantic search parameter from `last_user_message` to `search_query` across documentation and SDKs
- Changed references in documentation and code to use `search_query` instead of `last_user_message` for fetching semantically relevant observations and conclusions.
- Updated related function signatures and descriptions in Python and TypeScript SDKs to reflect this change.
- Adjusted tests to ensure compatibility with the new parameter naming.
* chore: openapi v3 formatted how we like it
* fix: reorder docs, update examples in README, update skills
* fix: message type option in sdk reference
* chore: update remaining getcontext and representation language
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
* chore: 3.0 honcho and 2.0 sdks changelog
fix: use PeerContextResponse in peer.ts
* chore: move docs to /v3/, build SDKs
* chore: code review
* feat: [WIP] migrate away from stainless in typescript sdk
* chore: move api from /v2/ to /v3/
* feat: no-stainless typescript with real tests
* feat: migrate python sdk off of stainless
* feat: clean typescript sdk
* chore: add tests for ts http client
* fix: rewrite entire python sdk in new format, update typescript sdk to use `configuration` not `config` for consistency with API
* fix: clean up SDKs, synchronize
* chore: update sdk examples
* chore: update OpenAPI documentation and SDK examples to reflect changes
* fix: better test
* fix: install deps in test runner, improve robustness of streaming in sdk, coderabbit nits
* fix: standardize around camelCase in TS SDK
* refactor: update configuration handling in SDKs to use typed models for workspace, session, and peer configurations
* docs: clarify queue status usage and remove polling methods from SDKs
add claude skills for migrations
* chore: fix links in docs
* feat: add deriver flush mode to bypass batch token threshold
- Introduced `is_deriver_flush_enabled` function to check if flush mode is active.
- Updated `QueueManager` to conditionally apply batch token thresholds based on flush mode.
- Enhanced `UnifiedTestExecutor` to enable flush mode via Redis.
- Added `flush` parameter to test cases to facilitate testing of flush mode behavior.
- Updated various test cases to utilize the new flush functionality.
* feat: implement schedule_dream functionality in SDKs, use in unified test runner
- Added `schedule_dream` method to both Python and TypeScript SDKs for scheduling dream tasks.
- Updated HTTP routes to include endpoint for scheduling dreams.
- Enhanced test runner to utilize the new `schedule_dream` method for scheduling actions.
- Updated TypeScript client to support the new scheduling functionality with appropriate parameters.
* feat: update single deriver task to support multiple observers
- Changed the `observer` parameter to `observers` as a list in multiple functions across the deriver module.
- Updated the processing logic to handle multiple observers for representation tasks.
- Adjusted related payload and queue management functions to accommodate the new observers structure.
- Modified tests to reflect changes in the representation task handling and ensure proper functionality.
* refactor: update enqueue tests to support deduplication of queue items with multiple observers
- Modified tests in `test_enqueue.py` to reflect changes in the queue item structure, where each message now results in a single queue item containing a list of observers.
- Updated assertions to validate that the `observers` field correctly includes all relevant peers, ensuring proper functionality of the deduplication logic.
- Removed redundant payload matching logic to streamline test cases and improve clarity.
* fix: add backwards compatibility for representation work unit keys and payload observers
* feat: update dialectic configuration and introduce cost calculator
- Adjusted LLM and dialectic settings in `.env.template`, `config.toml.example`, and `src/config.py` to reduce maximum tool output characters and session history tokens for cost efficiency.
- Implemented a new `dialectic_cost_calculator.py` script to estimate costs based on reasoning levels and model pricing.
- Enhanced `DialecticAgent` to utilize minimal tools and adjusted output token settings based on reasoning level to optimize performance and reduce costs.
* feat: add reasoning level to chat input in unified test runner
- Enhanced the `UnifiedTestExecutor` to include a `reasoning_level` parameter in the chat method call.
- Updated the `QueryAction` model to support the new `reasoning_level` attribute, allowing for more nuanced chat interactions.
* feat: run deriver once for multiple observers (#335)
* feat: update single deriver task to support multiple observers
- Changed the `observer` parameter to `observers` as a list in multiple functions across the deriver module.
- Updated the processing logic to handle multiple observers for representation tasks.
- Adjusted related payload and queue management functions to accommodate the new observers structure.
- Modified tests to reflect changes in the representation task handling and ensure proper functionality.
* refactor: update enqueue tests to support deduplication of queue items with multiple observers
- Modified tests in `test_enqueue.py` to reflect changes in the queue item structure, where each message now results in a single queue item containing a list of observers.
- Updated assertions to validate that the `observers` field correctly includes all relevant peers, ensuring proper functionality of the deduplication logic.
- Removed redundant payload matching logic to streamline test cases and improve clarity.
* fix: add backwards compatibility for representation work unit keys and payload observers
* feat: refactor benchmark runners to share common functionality
- Introduced a new `runner_common.py` module containing shared utilities for benchmark test runners, including common argument parsing, client creation, and queue management.
- Updated `BEAMRunner`, `LoCoMoRunner`, and `LongMemEvalRunner` to inherit from `RunnerMixin`, leveraging shared functionality for metrics collection and logging.
- Added `reasoning_level` and `redis_url` parameters to runner constructors for enhanced configuration.
- Streamlined argument parsing by utilizing `add_common_arguments` for shared command-line options across all runners.
* fix: update last_user_message handling to use message content instead of ID
* fix: standardize config vs configuration
---------
Co-authored-by: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com>
Vineeth Voruganti
Aakash Kattelu
ajspig
Eugene Eisenstein
Harish Kukreja
Rajat Ahuja
Marianne
doria
Viktor Szépe