Commit Graph

117 Commits

Author SHA1 Message Date
kshitij 9bbd956b73 fix(memory/hindsight): evict timed-out retain ops + coarser status polls
Review follow-up on the #62871 salvage (simplify pass, HIGH):

1. Ops unresolved at the wait deadline were RETAINED in the pending set.
   A permanently failing status endpoint (auth error, endless 500s, or a
   server that loses ops without 404) would grow the set forever and make
   EVERY later prefetch burn the full 10s budget re-polling it — and
   prefetch()'s bounded 3s join sits on the reply path, so that money-quote
   'adds no response latency' claim breaks. Timed-out ops are now dropped
   (identical degradation to prefetch_waits_for_retain=False: possibly
   stale recall) with a WARNING so persistent server trouble is visible.
   Guard test mutation-checked (fails with eviction disabled).

2. Status polls now spaced 0.5s (was 0.05s shared with the local drain
   poll): a wedged op cost up to ~200 get_operation_status round trips
   per prefetch; now ~20 max over the default 10s budget.
2026-08-02 22:33:43 +05:30
And 1be353bf9c fix(memory/hindsight): gate prefetch on server-side retain completion, not just queue drain
Address PR #62871 review: with the default retain_async=True, aretain_batch
returns when the write is accepted, not when it's durable/recall-visible, so
draining the local writer queue (task_done) is not a read-after-write signal.
The next-turn prefetch could still recall before the just-completed turn was
observable on the server.

- Track the async operation_id/operation_ids returned by aretain_batch
- _wait_for_retains_drained now applies two ordered, budget-bounded barriers:
  (1) local writer queue drains, then (2) tracked server-side async ops report
  completion via operations.get_operation_status (an explicit read-after-write
  condition). NotFound (completed+evicted) counts as done; transient errors
  keep waiting until the deadline
- Completed ops are removed from the pending set so later prefetches don't
  re-poll them; the whole wait stays off the reply path
- Add TestPrefetchServerRetainVisibility: op-id tracking (single/multiple),
  no-op tracking when retain_async=False, prefetch waiting for server
  completion before recall, timeout fallback on a wedged op, and NotFound /
  transient-error status handling
2026-08-02 22:33:43 +05:30
And 94b10eccf5 feat(memory/hindsight): order background prefetch after pending retains
Async retain already keeps the memory WRITE off the reply path (writes drain
on the single writer thread while the user gets their response immediately).
This closes the remaining retain/prefetch race: the next turn's warm prefetch
runs on its own thread and could recall BEFORE the just-enqueued retain write
lands, silently dropping the latest turn from recall.

- The background prefetch now waits (bounded) for pending retains to drain
  before recalling, so warmed context includes the just-completed turn.
- The wait runs only on the background prefetch thread, never the reply path,
  so it adds zero latency to the user's response and loses no writes.
- Bounded by prefetch_retain_drain_timeout (default 10s) and polls
  unfinished_tasks so a wedged write can't hang the prefetch.
- New config keys: prefetch_waits_for_retain (default true),
  prefetch_retain_drain_timeout (default 10.0).
2026-08-02 22:33:43 +05:30
kshitij a2f95e4c0e test(memory): hoisted-retriever fixture uses a real tmp db, not :memory:
Review follow-up on the #76142 salvage: MemoryStore path-resolves and
shares one process-wide connection per file, so MemoryStore(":memory:")
creates a literal ./:memory: FILE whose state leaks across test runs —
the second run of the file failed all three spy tests because the
NULL-vector test had permanently wiped hrr_vector in the leaked db.
tmp_path isolates each run; verified two consecutive runs green + full
tests/plugins/memory/ green.
2026-08-02 21:16:34 +05:30
spfcraze 89f74d58f6 perf(memory): hoist loop-invariant HRR encodes out of retrieval loops
FactRetriever.search() re-encoded the query vector once per candidate,
related() re-encoded both role atoms once per fact row, and probe()
re-encoded the role-content atom once per row. All three encoders are
deterministic (SHA-256 counter blocks), so the hoisted vectors are
bit-identical to the per-iteration values they replace.

Measured (300-fact store, dim=1024, median of 30 calls): search()
11.62 -> 1.46 ms/call (8.0x; encode_text 30 -> 1 per call), related()
63.08 -> 16.17 ms/call (3.9x; encode_atom 601 -> 3 per call), probe()
431.93 -> 389.36 ms/call (1.1x; dominated by per-fact content encoding,
which is inherent to the algorithm and unchanged).

Tests: call-count regression tests for each hoist plus a bit-exact
parity test of search() against the pre-fix per-candidate loop.
2026-08-02 21:16:34 +05:30
Teknium fae0c4f5f4 fix(hindsight): create embedded profile env file owner-only (0600)
The embedded Hindsight daemon's profile env file carries the plaintext
HINDSIGHT_API_LLM_API_KEY but was written via bare write_text(), leaving
it with umask-derived (typically world-readable) permissions.

- Create/truncate the file via os.open(..., 0o600); chmod a pre-existing
  file to 0600 BEFORE writing new secret bytes.
- Post-write validation on POSIX: verify 0600, retry chmod, and raise if
  the file still isn't owner-only.
- If validation fails, unlink the secret file so a plaintext key is never
  left behind with unverified permissions.
- Regression tests under tests/plugins/ for fresh-write mode, tightening a
  pre-existing 0644 file, and cleanup on validation failure.

Narrowed reimplementation of #74236 confined to plugins/memory/hindsight/;
the core utils.py atomic-replace opt-out from the PR was dropped.

Co-authored-by: carrion256 <carrion256@proton.me>
2026-07-31 22:39:34 -07:00
Kyzcreig 9c28600e77 test: prove concurrency with barriers/witnesses instead of wall-clock bounds
Replace OS-scheduler-dependent elapsed-time ceilings with deterministic
concurrency proofs in three tests:

- test_context_refs_concurrent: asyncio.Barrier rendezvous — all 3 URL
  fetches must be in flight simultaneously before any returns.
- test_memory_boundary_commit: positive non-blocking witness — the
  provider call list must still be empty when the async commit returns.
- test_mem0_v3 slow-prefetch: threading.Event park/release — prefetch
  must return while the backend search is still parked.

The tests/tools/test_mcp_tool.py hunk from the original PR is dropped:
main no longer carries the 'elapsed < 2.5' assertion it targeted
(superseded by a delay-relative bound).

Salvaged from #71913.
2026-07-29 21:30:53 -07:00
webtecnica 66c4c9c0b1 fix(tests): forward Windows location vars through the hermetic runner; patch Path.home() in hindsight _clean_env
Combines the Windows-hermeticity cluster (#67512 by @webtecnica, earliest;
#71112 by @Sanjays2402; #67196 by @anatolijlaptev1991-ctrl) into one fix:

- scripts/run_tests.sh: env -i forwarded only HOME, but native Windows
  CPython resolves Path.home() from USERPROFILE (or HOMEDRIVE+HOMEPATH),
  stdlib paths from LOCALAPPDATA/APPDATA, ssl/sockets need SYSTEMROOT,
  tempfile needs TEMP/TMP — the strip broke collection tree-wide on
  native Windows (issues #67385, #70813). Location vars (never
  credentials) are now forwarded, each only when actually set, so
  POSIX runs are byte-for-byte unchanged (probe-verified both ways).
  PYTHONUTF8=1 added for legacy-codepage consoles printing the
  runner's glyphs.
- tests/plugins/memory/test_hindsight_provider.py: _clean_env patched
  HOME only; on Windows Path.home() ignores HOME. Now patches
  Path.home directly into tmp_path (from #67196).

Not ported: #71112's guard test — it regex-reads run_tests.sh source,
which the test policy bans (never read source code in tests).

Fixes #67385. Fixes #70813.
2026-07-29 18:55:10 -07:00
Teknium 6b81590c55
test: prune low-value tests suite-wide (wave 1) — 46,820 → 28,106 test functions
Systematic prune per AGENTS.md test policy, one pass over every major
test tree (gateway, hermes_cli, tools, agent, run_agent, plugins, cli,
cron, tui_gateway, honcho/openviking, root-level):

- DELETE: source-reading tests (read_text/getsource on prod files),
  change-detector tests (exact catalog counts, model-name snapshots,
  config version literals), mock-echo tests (assert a mock returns what
  it was told), assertion-free/trivial tests, near-duplicate
  parametrizations (boundaries + one representative kept), async/sync
  twin duplicates, cosmetic within-file variations.
- KEEP (mandatory): security/redaction/approval guards, message-role
  alternation invariants, prompt-caching/deterministic-call-id
  invariants, issue-number regression tests (deduped), E2E tests.
- 6 test files deleted outright (script-style/no-assert or fully
  redundant); conftest.py, fakes/, fixtures/ untouched.
- tests/acp/conftest.py added: autouse fixture stubs the live
  models.dev/GitHub/Copilot/Anthropic inventory fetches that ACP server
  tests performed on every session create — test_server.py 147s → 3.4s,
  and the tests are now genuinely hermetic.
- Sleep-based slowness shrunk where safe (codex_ttfb_watchdog,
  compression_concurrent_fork, etc.); no wall-clock assertion tightened.

Verification: full hermetic suite via scripts/run_tests.sh —
2439 files, 31,130 tests passed, 0 failed, 0 flaky retries, 315s wall
(baseline: 583s wall, 13,564s subprocess CPU).
2026-07-29 13:10:23 -07:00
Teknium 0227872bf5 fix(hindsight): route setup + auto-upgrade installs through lazy_deps
Widen NS-605 to the two remaining direct-install sites in the hindsight
plugin, which still shelled out to 'uv pip install --python
sys.executable' and therefore failed (EROFS/EACCES) on immutable hosted
images with sealed venvs, and lost packages on redeploy:

- post_setup dependency install (~L835): install_specs() with ok /
  blocked-reason / stderr handling matching honcho/mem0.
- initialize()-time hindsight-client auto-upgrade (~L1240):
  install_specs(); blocked installs log the gate reason with the manual
  command instead of a raw subprocess error, and init proceeds.

Audited every other memory plugin (supermemory, byterover, holographic,
openviking, retaindb) for direct pip/uv install subprocess calls: none
remain — their deps flow through plugin.yaml pip_dependencies or
lazy_deps.ensure().

Tests: TestClientAutoUpgradeRoutesThroughLazyDeps — upgrade goes through
install_specs with the exact spec (regression guard asserts no
subprocess.run), blocked upgrade is non-fatal and surfaces the gate
reason. Updated TestPostSetupEnvEncoding stubs to the new install path.
2026-07-28 22:40:33 -07:00
solyanviktor-star f1ea4a56c2 fix(memory): cover the remaining setup-time .env reads with utf-8-sig
Follow-up to review feedback:

- mem0 _prompt_api_key read .env with the locale default, so a Notepad
  BOM hid the first key from the masked current-value lookup; read it
  with utf-8-sig + errors=replace like the canonical readers in
  hermes_cli/config.py.
- hindsight _load_simple_env used plain utf-8; it also parses the Hermes
  .env during post_setup, where a BOM stuck to the first key. Switch to
  utf-8-sig + errors=replace.
- Add hindsight regressions: BOM key matching in _load_simple_env and in
  the cloud post_setup writer, plus non-ASCII round-trip preservation,
  and a mem0 regression for the BOM'd masked-key lookup. The BOM tests
  fail without the fix on any platform.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 17:10:39 -07:00
solyanviktor-star 75afc47baa fix(memory): read/write .env as UTF-8 in mem0 and hindsight setup
The mem0 and hindsight memory-provider setup routines round-trip the
user's ~/.hermes/.env: they read existing lines, update the keys they
manage, and rewrite the whole file preserving every other line verbatim.
Both used env_path.read_text() / write_text() with no encoding.

read_text()/write_text() with no encoding fall back to the system locale
(cp1252/GBK on Windows), so on a non-UTF-8 host the preserved lines get
mangled or the call crashes on any non-ASCII value, and — because the
reader never strips a BOM — a Notepad-edited .env makes the first key
fail the in-place match and get duplicated instead of updated.

Match the canonical .env readers in hermes_cli/config.py: read with
encoding='utf-8-sig' (BOM-tolerant) and write with encoding='utf-8'.
mem0/_setup.py already pins utf-8 for mem0.json, so this just aligns the
.env path in the same file. Fixes both memory plugins in one class fix.

Adds regression tests: a BOM'd .env updates the first key in place
(locale-independent, fails without the fix) and non-ASCII existing lines
survive the round-trip.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 17:10:39 -07:00
Hao Zhe 1cfe23c6e4 fix(openviking): serialize client refresh state 2026-07-24 13:00:53 +05:30
Hao Zhe cf0bd5dd4c fix(openviking): stop pending runtime start on shutdown 2026-07-24 13:00:53 +05:30
Hao Zhe c4d0f1c1d6 fix(openviking): serialize local runtime recovery starts
Avoid spawning multiple local OpenViking server processes while a runtime autostart waiter is already active. Remote endpoints still retry on later accesses because they do not install a local waiter.
2026-07-24 13:00:53 +05:30
Hao Zhe 8fdc9c58e0 fix(openviking): use readonly config loader 2026-07-24 13:00:53 +05:30
爪爪 5ea3abc3b3 fix(openviking): match tenant-header errors structurally instead of hard-coding strings
The _needs_trusted_identity_retry method was hard-coding specific
server-side error strings to detect when a request failed due to
missing X-OpenViking-Account / X-OpenViking-User headers.  Each new
server-side error variant required another string added to the client.

Replace the string enumeration with a structural match: the error
message mentions one of the tenant headers AND the HTTP status is 400.
This covers all current error variants:

  - "Trusted mode requests must include X-OpenViking-Account and User"
  - "ROOT requests to tenant-scoped APIs must include X-OpenViking-Account"
  - "Trusted mode requests must include X-OpenViking-Account."
  - "Trusted mode requests must include X-OpenViking-User."

The 400 status guard avoids false-positives on 403 errors such as
"USER API keys cannot override X-OpenViking-User", which must not
trigger a retry.

All 176 existing tests pass.

(cherry picked from commit 5a24d6766c)
2026-07-24 13:00:53 +05:30
Hao Zhe 21a634b98a test(openviking): keep root tenant errors out of trusted retry 2026-07-24 13:00:53 +05:30
Hao Zhe 36f01d2e54 fix(openviking): sanitize splitline env separators 2026-07-24 13:00:53 +05:30
koshaji d3520944c7 fix(openviking): join runtime-autostart thread on shutdown (SIGABRT-at-exit)
`OpenVikingMemoryProvider.shutdown()` joins in-flight writers, deferred-commit
threads, and prefetch threads, but not `_runtime_start_thread` — the tracked
`daemon=True` waiter that runs `_finish_runtime_openviking_start`, which blocks
on network health probes (`_wait_for_openviking_health` polling + a
`_VikingClient.health()` request).

If the local OpenViking runtime is slow or unreachable, that waiter can still
be blocked in network I/O at interpreter exit. CPython then forcibly kills it
during `Py_FinalizeEx` (`PyThread_exit_thread` -> `__pthread_unwind` ->
`abort()`), producing SIGABRT (exit 134) with no traceback — the same daemon-
thread-at-exit failure class fixed for the Honcho provider.

Fix:
- `shutdown()` now joins `_runtime_start_thread` (timeout-bounded) alongside the
  other tracked threads.
- `_wait_for_openviking_health()` gains a `should_stop` callback; the waiter
  passes `lambda: self._shutting_down` so the poll loop bails out promptly once
  `shutdown()` flips the flag, instead of lingering up to the 60s autostart
  timeout and timing out the join (which would leave the thread alive).
- Add tests/plugins/memory/test_openviking_shutdown.py covering the short-circuit
  and the shutdown-joins-runtime-thread behaviour.

(cherry picked from commit 5471ec7021)
2026-07-24 13:00:53 +05:30
pprism13 9291b786b4 fix(openviking): sanitize embedded newlines when writing .env secrets
`_write_env_vars` in the OpenViking memory provider interpolates each
secret straight into a `KEY=VALUE` line, but the values only ever pass
through `_clean_config_value`, whose `value.strip()` trims surrounding
whitespace and leaves internal CR/LF intact. Because the file is strictly
line-oriented and is re-read via `read_text().splitlines()`, a value that
carries an embedded newline spills onto a second physical line, and the
tail is re-parsed as an independent `KEY=VALUE` entry on the next round
trip. A secret pasted with a trailing record (e.g. an `OPENVIKING_API_KEY`
copied with an extra line) therefore injects an arbitrary additional
variable into the persisted credentials file and silently corrupts it.

The fix neutralizes the line terminators at the single chokepoint where
values reach the file. A small `_env_line_safe` helper strips `\r`, `\n`,
and the NUL byte from each value, and both write sites in `_write_env_vars`
(the existing-key update branch and the appended-key branch) route through
it, so a value can only ever occupy the single line it is written on.

## What does this PR do?

Hardens the OpenViking memory provider's `.env` writer so a malformed or
pasted secret value can no longer break out of its `KEY=VALUE` line and
inject a rogue variable into the profile-scoped credentials file.

## Related Issue

N/A

## Type of Change

- [x] 🐛 Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `plugins/memory/openviking/__init__.py`: add `_env_line_safe()` which
  removes `\r`, `\n`, and `\x00` from a value, and apply it to both the
  updated-key and appended-key write branches in `_write_env_vars()`.
- `tests/plugins/memory/test_openviking_provider.py`: add two regression
  tests covering a fresh write and an in-place key update with embedded
  CR/LF, asserting no injected line survives the read-back.

## How to Test

1. Run the targeted tests:
   `pytest tests/plugins/memory/test_openviking_provider.py -k env_writer -q`
2. Reverting the `_env_line_safe` sanitization makes
   `test_openviking_env_writer_strips_embedded_newlines_in_values` and
   `test_openviking_env_writer_strips_newlines_when_updating_existing_key`
   fail with a rogue `INJECTED_KEY=`/`ROGUE=1` line appearing in the file,
   confirming the tests pin the bug.
3. `ruff check plugins/memory/openviking/__init__.py` and
   `python scripts/check-windows-footguns.py plugins/memory/openviking/__init__.py`
   both pass.

## Checklist

### Code

- [x] I've read the Contributing Guide
- [x] My commit messages follow Conventional Commits
- [x] I searched for existing PRs to make sure this isn't a duplicate
- [x] My PR contains only changes related to this fix
- [x] I've run the relevant tests and they pass
- [x] I've added tests for my changes (required for bug fixes)
- [x] I've tested on my platform: macOS 15 (Darwin 25.5)

### Documentation & Housekeeping

- [x] I've updated relevant documentation (docstrings) — or N/A
- [x] I've updated `cli-config.yaml.example` if I added/changed config keys — N/A
- [x] I've updated `CONTRIBUTING.md` or `AGENTS.md` if I changed architecture or workflows — N/A
- [x] I've considered cross-platform impact (strips CR as well as LF) — done
- [x] I've updated tool descriptions/schemas if I changed tool behavior — N/A

(cherry picked from commit f29dd2df84)
2026-07-24 13:00:53 +05:30
wernerhp cc84af9fad fix(memory): preserve genuine pre-delimiter content in merged compaction rows
teknium1 review on #57690: harvesting logic was skipping the ENTIRE merged
row when a compaction summary was appended to the tail message, discarding
real prior user content that context_compressor retains before the
_MERGED_SUMMARY_DELIMITER. Extract and harvest that pre-delimiter segment
instead of dropping it wholesale.

Revert-to-fail: reverting plugins/memory/holographic/__init__.py alone
drops test_merged_into_tail_preserves_genuine_pre_delimiter_preference
(19 passed, 1 failed); restoring the fix returns 20/20 passed.
2026-07-22 06:59:14 -07:00
wernerhp 004de13f14 fix(memory/holographic): don't harvest compaction summaries; honor auto_extract=false string
Two compounding defects in the holographic memory provider (#57682):

1. The on_session_end gate used plain truthiness on auto_extract, but the
   plugin's own config schema declares it as a string enum with default
   "false" — and not "false" is False, so extraction ran for users who
   had it configured off. Coerce with the shared utils.is_truthy_value
   (same fix class as the merged byterover no-op fix).

2. _auto_extract_facts scanned every role=user message. Context-compaction
   handoff summaries can be inserted as role=user messages and their prose
   reliably matches the decision patterns (we decided/agreed, the project
   uses), so the compactor's own output was persisted as a durable project
   fact on every rollover following a compaction — recreated even after
   manual deletion.

Adds agent.context_compressor.is_compaction_summary_message(), a public
helper that prefers the in-process COMPRESSED_SUMMARY_METADATA_KEY marker
and falls back to _is_context_summary_content() (covers merged-into-tail
and historical prefixes), since the metadata key is stripped by wire
sanitizers and doesn't survive all session-store round-trips. The plugin
skips summary messages before pattern matching.

Fixes #57682
2026-07-22 06:59:14 -07:00
Hao Zhe 8af2133009 fix(openviking): align session context with shared profile contract 2026-07-22 14:12:50 +05:30
Flownium 11c1ca01c5 fix(openviking): inject session-start memory context
(cherry picked from commit 18b474d0bd)
2026-07-22 14:12:50 +05:30
kshitij c3c80e1796 refactor: cleanup follow-up for salvaged PR #58871
- Remove dead current_sid parameter from _recover_pending_sessions
- Remove dead cleanup parameter from _release_owner_run_claim (always True)
- Set _run_lock_path after flock succeeds, not before
- Collapse redundant BlockingIOError branch (covered by OSError+errno check)
- Track _pending_marked_sids to skip re-writing marker file on every sync_turn
2026-07-22 14:05:28 +05:30
Chris Korhonen 81fc424592 fix(openviking): chunk structured session sync
Preserve ordered structured turns across OpenViking's 100-message batch limit and resume retries from the first unconfirmed message.

Based on the OpenViking batching work from commit 1a567f7067 in #58981.
2026-07-22 14:05:28 +05:30
Hao Zhe 3fd96583f4 fix(openviking): serialize orphan session recovery 2026-07-22 14:05:28 +05:30
Hao Zhe 323e9baf5d fix(openviking): recover pending session commits 2026-07-22 14:05:28 +05:30
RenoMG 95c616be20 fix(supermemory): complete self-hosted endpoint routing 2026-07-20 00:40:40 -07:00
Dhravya Shah 24ac26a3da feat(supermemory): support custom base URL for self-hosted servers
The supermemory SDK already honors SUPERMEMORY_BASE_URL, but the raw
urllib call used for session-end conversation ingest hardcoded
https://api.supermemory.ai/v4/conversations, so ingest always hit the
cloud even when pointing at a self-hosted server (e.g.
http://localhost:6767).

Resolve the base URL as config (supermemory.json base_url) >
SUPERMEMORY_BASE_URL env var > https://api.supermemory.ai, strip any
trailing slash, and use it for both the SDK client and the
/v4/conversations ingest endpoint.
2026-07-20 00:40:40 -07:00
teknium1 c84c0c5277
Merge branch 'pr-51020' into lane/c3-memory-panel 2026-07-18 15:13:04 -07:00
Teknium 07f07c7b51 fix(mem0): migrate legacy OSS base URL aliases
Normalize stale api_base keys to each mem0 provider's accepted URL field before Memory.from_config, without mutating the saved config.
2026-07-17 13:49:29 -07:00
Erosika 155a792013 Merge origin/main: keep unified declared-schema config surface
Conflict resolutions:
- web_server.py: keep the branch's unified provider-config handlers
  (declared schema served by default, profile-scoped) over main's
  surface=declared query param. Main's declared-surface helpers and the
  hermes_cli.memory_providers import are dropped — the branch deleted
  that module when provider schemas moved into their plugins, so main's
  code path could no longer import.
- hermes.ts: keep profileScoped() calls without ?surface=declared to
  match the unified backend.
- constants.ts: take main's new reasoning effort values (max, ultra),
  keep the branch's memory provider ordering.
- config-settings.tsx: take main's FallbackModelsField import, drop the
  now-unused SECTIONS import (branch replaced it with
  sectionFieldEntries).
- provider-config-panel.test.tsx: file moved to settings/memory/ on the
  branch; main's act() fixes targeted the old tests, and the rewritten
  suite produces no act() warnings, so the old path stays deleted.
- test_web_server.py: keep both sides' new tests (main's MoA endpoint
  tests plus the branch's Honcho provider tests).
2026-07-15 17:50:31 -04:00
Erosika f000fbe5c5 revert(memory): drop the provider actions extension point
No bundled provider declares actions, and the motivating request
(openviking, #56309) needs dynamic select options rather than actions.
Removing the unconsumed POST dispatch surface keeps this PR focused on
the config panel; the extension point can return as its own PR with its
first real consumer.
2026-07-10 15:41:52 -04:00
teknium1 a801046669 fix(memory): resolve() the shared-connection registry key; symlink test
Follow-ups for salvaged PR #43819: the registry key was
str(Path(db_path).expanduser()) — a symlinked or relative path to the
same DB file got its own connection, silently reintroducing the exact
multi-writer contention the registry prevents. Key on Path.resolve()
(OSError-tolerant fallback). Adds a symlink regression test and the
AUTHOR_MAP entry for adambiggs.
2026-07-09 18:17:40 -07:00
Adam Biggs b5226caff8 fix(memory): share one SQLite connection per holographic store database
Every MemoryStore instance opened its own SQLite connection guarded by
its own RLock. Several providers coexist in one process (the main agent
plus every delegate_task subagent), so instances pointing at the same
memory_store.db raced as independent WAL writers. Combined with writes
that were not rolled back on error, one connection could leave an open
write transaction that pinned the write lock and made every other
connection's writes fail with "database is locked" for the full busy
timeout.

Instances for the same database now share ONE process-wide connection
and ONE re-entrant lock, so access is fully serialized and
cross-connection contention is impossible. The shared connection is
refcounted: closing one instance never tears it out from under a live
sibling, and the last close releases it. The connection runs in
autocommit (isolation_level=None) so a write that raises mid-method can
never leave a dangling transaction holding the write lock; the existing
explicit commit() calls become harmless no-ops.

The provider's shutdown() now calls the refcount-guarded close() instead
of just dropping the reference: leaving finalization to GC kept the
connection (and its write lock) alive indefinitely on long-running
gateways, prolonging the exact contention this fix removes. The last
provider now releases the connection deterministically while siblings
stay live; regression tests fail without the wiring.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 18:17:40 -07:00
Erosika 1508ece2d9 Merge origin/main: unify declared-schema and instance-schema provider config
Main's #60569 (dashboard memory provider switching) rewrote the same
GET/PUT /api/memory/providers/{name}/config routes this PR owns — the
dashboard and desktop share one backend. Resolve by dispatching: providers
that declare a config_schema.py get the declared path (host-block storage,
locked honcho writes, profile scoping, actions); everything else keeps
main's instance get_config_schema()/save_config path unchanged, so the
dashboard's PluginsPage behavior is preserved for instance providers.
Setup manifests, the /setup endpoint, provider switching, and name
validation from #60569 apply to both paths. Instance payloads gain the
docs_url/actions keys the desktop panel expects; declared payloads gain
the setup block the dashboard expects. Main's hindsight/honcho instance-
schema tests are updated for the dispatch, and its status test gets the
HOME pin it was missing (it read the developer's real ~/.honcho).
2026-07-08 15:03:24 -04:00
teknium1 5e51b123f3 feat(mem0): add self-hosted mode to the setup wizard
The salvaged SelfHostedBackend made self-hosted servers reachable via
mem0.json / MEM0_HOST, but the setup wizard still offered only Platform
and OSS — exactly the gap users hit (Discord report: 'At memory setup
there's only 2 options'). Adds a third wizard mode:

- interactive picker: Platform / Self-hosted server / Open Source
- non-interactive: hermes memory setup mem0 --mode selfhosted
  --host http://... [--api-key ...] [--dry-run]
- host -> mem0.json (behavioral), API key -> .env as MEM0_API_KEY
  (secret), optional key for AUTH_DISABLED servers
- best-effort reachability check against the server, non-fatal
- README + memory-providers docs updated with the wizard path
2026-07-07 14:07:16 -07:00
kshitijk4poor 9a322726ae fix(mem0): prune dead get_all, wire rerank config default, warn on MEM0_HOST env override
Review follow-ups on the salvage:

- get_all() pruned from the ABC and all three backends: mem0_list (its
  only caller) was removed by the recall-tuning commit, leaving new,
  tested, unreachable code — including SelfHostedBackend's _MAX_TOP_K
  over-fetch workaround. Tests for it dropped; fake-class stubs remain
  harmlessly. (The #52921 true-total fix lives on in the PR history if
  a lister ever returns.)
- The persisted rerank config key was write-only (setup prompted for it,
  nothing read it). initialize() now parses it into _rerank_default and
  mem0_search uses it when the model doesn't pass rerank explicitly;
  per-call args still win. Guard test added.
- Platform-mode setup now warns when MEM0_HOST is set in the environment:
  the json host-clear can't help there (_load_config seeds host from the
  env var, docs tell users to put it in .env) — the user would silently
  keep routing to the self-hosted server.
- SelfHostedBackend: connect-level retries (httpx.HTTPTransport(retries=2))
  so a single transient blip doesn't count toward the provider breaker;
  transport now injectable and the test helper uses the real __init__
  instead of mirroring it via __new__.
- plugin.yaml description no longer leads with reranking (off by default,
  platform-only); docs em-dash typo fixed.
2026-07-08 01:57:11 +05:30
kshitijk4poor 53edf6f983 fix(mem0): make prompt label + platform setup honor host routing precedence
Follow-up on the salvaged #55614. The PR added host-based routing to
_create_backend (precedence: oss > host > platform) but two sibling surfaces
didn't mirror it:

- system_prompt_block() checked host before oss, so an oss+host config ran
  OSS but told the model it was self-hosted HTTP. Reordered to match routing.
- Platform setup (hermes memory setup mem0 --mode platform) left a stale host
  in mem0.json; since host beats platform, the user kept routing to the
  self-hosted server. save_config merges (no delete), so clear host to ""
  rather than pop() so the merge actually overwrites it.

Adds regression tests for both (mutation-checked).
2026-07-08 01:57:11 +05:30
kartik-mem0 2a14205ff4 feat(mem0): self-hosted dashboard backend + recall tuning (salvage #55614)
Salvage of #55614 by @kartik-mem0 (mem0 maintainer). Adds a SelfHostedBackend
that talks to a self-hosted Mem0 Docker server over httpx (X-API-Key auth,
/search + /memories routes), gated behind `host`. Also folds in the mem0
research-team recall tuning that rides with it: rerank defaults to false across
all modes, the mem0_list tool is removed (5->4 tools), search guidance is
de-shouted, and self-hosted get_all reports the true stored total (#52921).

Supersedes the self-hosted portion of #52487 (@liuhao1024, first-submitted).

Closes #52478
Fixes #52921
2026-07-08 01:57:11 +05:30
Erosika 35099685be feat(memory): provider actions extension point
Providers with behavior beyond fields (validate a server, start a local
instance, link a CLI profile) previously had no mount besides forking the
panel. Let the schema declare actions and one generic endpoint run them:
ProviderAction on ProviderConfigSchema, POST
/api/memory/providers/{name}/actions/{action} dispatching to
ACTION_HANDLERS in the plugin's config_actions.py (path-loaded and
import-light, like the schema), profile-scoped and off the event loop.
Handlers get the submitted values dict, return a JSON-able result, and
raise ValueError for user-facing 400s. The panel renders declared actions
as generic buttons beside Save. No bundled provider declares actions yet.
2026-07-07 11:00:25 -04:00
Erosika 76c063b3d9 fix(memory): profile-scope the provider config endpoints
The settings page follows the desktop's active-profile switcher, but the
provider config calls didn't: no profileScoped() on the client and no
profile param on the backend, so a multi-profile desktop edited the serving
process's config while every surrounding card showed the selected profile's.

Accept ?profile= on both endpoints and resolve inside _profile_scope (the
skills/toolsets contract), spread profileScoped() into the two client calls,
and key the schema cache on the resolved config_schema.py path instead of
the provider name — user-installed plugins are per-profile, so one profile's
lookup must never answer for another's.
2026-07-07 10:53:43 -04:00
dsad e02cef0d0d fix(memory): guard local uploads against credential reads 2026-07-05 03:14:49 -07:00
Erosika 41e59f6126 fix(memory): retry failed config schema loads instead of caching None
A syntax error in a provider's config_schema.py was cached as 'no schema'
until process restart, rendering a silent empty panel even after the file
was fixed. Cache only successful loads and genuine file-absence.
2026-07-02 18:42:31 -04:00
Erosika 2a632807e0 refactor(memory): move provider config schemas into their plugins
Each provider now declares its config surface in config_schema.py inside
its own plugin dir (plugins/memory/<name>/), loaded by file path like the
plugins themselves so plugin __init__ imports never reach the web server.
hermes_cli/memory_providers.py is gone; the shared field primitives and
loader live in plugins/memory/config_schema.py, and the schema tests move
to tests/plugins/memory/ alongside the other per-plugin suites.
2026-07-02 16:55:39 -04:00
cyb3rwr3n cb6d6d46ab fix(memory/holographic): sanitize FTS5 queries for natural-language recall
The FactRetriever's _fts_candidates passed the raw query string directly
to FTS5's MATCH operator. FTS5 defaults to AND-between-tokens, which
means any multi-word prose query like 'what happened with the deployment
rollback' required every single token to co-occur in a fact — dropping
recall to zero on the kind of queries agents actually issue via prefetch().

Fix: add _sanitize_fts_query() that:
- tokenizes the query and drops English stopwords
- strips FTS5 operator characters per token
- OR-joins the remaining content tokens as phrase literals

For pathological inputs (all stopwords, empty), falls back to the raw
query so the caller sees zero results instead of a SQL error.

This is a pure-retrieval-quality fix — the HRR + Jaccard reranking
stages still keep precision high. Ships with 10 tests covering the
sanitizer and retrieval integration.
2026-06-30 15:55:11 -07:00
Kartik c6eb7f9e72
fix(memory/mem0): recall on the current question + stronger search guidance (#55535) 2026-06-30 15:51:08 +05:30
Ben Barclay 1289f12812 fix(memory): lazy-install supermemory + mem0 SDKs like honcho/hindsight
The supermemory and mem0 memory providers shipped third-party SDKs
(supermemory / mem0ai) that are not core dependencies, but — unlike the
honcho and hindsight providers — they imported those SDKs directly with
no tools.lazy_deps.ensure() preflight and had no LAZY_DEPS allowlist
entry. On the published Docker image the agent venv is sealed
(HERMES_DISABLE_LAZY_INSTALLS=1) and lazy installs are redirected to a
writable durable target (HERMES_LAZY_INSTALL_TARGET). honcho/hindsight
route through ensure() and install fine there; supermemory/mem0 never
called it, so their SDK was never installed on a hosted instance and the
provider silently reported itself unavailable even with the API key set.

Fixes:
- Add memory.supermemory + memory.mem0 to the LAZY_DEPS allowlist
  (tools/lazy_deps.py), pinned to current PyPI releases.
- Call ensure('memory.<x>', prompt=False) at each SDK-import chokepoint
  (_SupermemoryClient.__init__; Mem0MemoryProvider._create_backend),
  mirroring honcho's wrapped try/except shape.
- Drop the SDK-import gate from supermemory's is_available() — it was a
  chicken-and-egg trap (provider never loaded on a sealed venv, so
  ensure() never ran). Now key-presence only, like honcho/mem0.
- Add matching pyproject extras [supermemory]/[mem0]; update the
  lazy-covered-extras contract test (excluded from [all] by policy).

Tests prove each path fails without the fix and the real sealed-venv
durable-target gate accepts both features.
2026-06-29 00:25:36 -07:00