fix(memory): complete discovery and registration parity for out-of-tree providers
Builds on the three salvaged commits: adds the sources and integration points they leave out, so a pip-installed memory provider is not a second-class citizen next to a directory install. Discovery - Project-local providers (./.hermes/plugins/<name>/), gated on HERMES_ENABLE_PROJECT_PLUGINS exactly as PluginManager gates its own project scan. Completes the four sources CONTRIBUTING.md and AGENTS.md already promised; memory was the only discovery system missing two of them. - find_provider_dir() now resolves a package entry point to its directory. This is load-bearing: config_schema.py (the dashboard panel) and cli.py (the `hermes <provider>` subcommands) are read from disk rather than imported, so without a directory a pip-installed provider silently lost both. - list_memory_provider_names() includes entry-point providers, so they appear in the dashboard's memory.provider dropdown. Resolution stays import-free. hermes_cli.plugins.resolve_module_origin() is extracted from _resolve_module_source() (added by the salvaged #76567) and shared, so discovery walks a module's file layout instead of importing it. find_provider_dir() is called from the dashboard and from argparse setup, long before the operator has chosen a provider — importing every installed candidate would execute third-party code on the strength of a package being present. A test asserts the resolution leaves no side effects and no sys.modules entry. Registration - PluginContext gains register_memory_provider(). Memory was the only provider category without one; context engine, image gen, video gen, web search, browser, TTS, transcription, secret source, dashboard auth and platform all have one. - _ProviderCollector delegates unknown register_* calls to a real PluginContext instead of carrying three hand-written no-ops. It silently dropped register_tool/register_hook, and had no register_auxiliary_task at all — despite PluginContext.register_auxiliary_task documenting a memory provider (hindsight's pre-retain dedup) as its worked example. It can no longer drift behind PluginContext. - A raise after register_memory_provider() no longer costs the provider. The loader caught it into a debug log, discarded the registered instance, and fell through to "instantiate any MemoryProvider subclass" — returning a different, unconfigured provider. A silent downgrade that looked like success, and the exact outcome of calling register_auxiliary_task. Activation is unchanged: still gated on memory.provider naming the plugin, and covered by a test so the real PluginContext cannot start requiring plugins.enabled — that would break every existing user-installed provider. Verified end to end against a real third-party provider (kainappsinc/elephant) installed by pip alone, with no directory copy: it appears in the dropdown, resolves its directory, loads with its tools, and renders its dashboard panel. Closes #40101.
This commit is contained in:
Greg Gibeau
Teknium
parent
a883977b12
commit
c600fd46bd
|
|
@ -828,6 +828,14 @@ Separate discovery system for pluggable memory backends. Current built-in
|
|||
providers include **honcho, mem0, supermemory, byterover, hindsight,
|
||||
holographic, openviking, retaindb**.
|
||||
|
||||
Discovery covers the same four sources as the general `PluginManager` —
|
||||
bundled, `$HERMES_HOME/plugins/`, `./.hermes/plugins/` (opt-in via
|
||||
`HERMES_ENABLE_PROJECT_PLUGINS`), and `hermes_agent.memory_providers` entry
|
||||
points — but with **bundled-first** precedence, the reverse of the general
|
||||
system's later-wins order: a memory provider is activated by name, so a
|
||||
dropped-in directory must not be able to shadow a shipped one. Discovery
|
||||
enumerates without importing; nothing runs until `memory.provider` names it.
|
||||
|
||||
Each provider implements the `MemoryProvider` ABC (see `agent/memory_provider.py`)
|
||||
and is orchestrated by `agent/memory_manager.py`. Lifecycle hooks include
|
||||
`sync_turn(turn_messages)`, `prefetch(query)`, `shutdown()`, and optional
|
||||
|
|
|
|||
|
|
@ -862,36 +862,40 @@ def _read_source_from_origin(origin: Optional[str], limit: int = 8192) -> str:
|
|||
return ""
|
||||
|
||||
|
||||
def _resolve_module_source(module_name: str, limit: int = 8192) -> str:
|
||||
"""Return the first ``limit`` chars of a module's source WITHOUT importing it.
|
||||
def resolve_module_origin(module_name: str) -> Optional[str]:
|
||||
"""Return a module's source path WITHOUT importing it, or ``None``.
|
||||
|
||||
``importlib.util.find_spec`` on a dotted name imports the parent
|
||||
package first (executing its ``__init__.py``), which would run
|
||||
arbitrary package initialization during discovery and pay the very
|
||||
import cost this classification exists to avoid — a provider whose
|
||||
heavy imports live in ``package/__init__.py`` would still pay them.
|
||||
import cost this exists to avoid — a provider whose heavy imports
|
||||
live in ``package/__init__.py`` would still pay them.
|
||||
|
||||
Only the top-level name is resolved with ``find_spec`` (import-free
|
||||
for top-level names); the remaining dotted segments are walked
|
||||
through ``submodule_search_locations`` by hand, mirroring the file
|
||||
layout conventions of the default PathFinder (``part.py`` module or
|
||||
``part/__init__.py`` package). Namespace packages, zipped modules,
|
||||
extension modules, and anything else unexpected fall back to ``""``
|
||||
(→ ``standalone``, the safe default).
|
||||
extension modules, and anything else unexpected return ``None``.
|
||||
|
||||
Shared with ``plugins/memory/__init__.py``, which needs the directory
|
||||
of a pip-installed provider to find its ``config_schema.py`` and
|
||||
``cli.py`` — both of which are loaded by path precisely so the
|
||||
provider module never has to be imported.
|
||||
"""
|
||||
parts = [p for p in module_name.split(".") if p]
|
||||
if not parts:
|
||||
return ""
|
||||
return None
|
||||
try:
|
||||
spec = importlib.util.find_spec(parts[0])
|
||||
if spec is None or not spec.origin:
|
||||
return ""
|
||||
return None
|
||||
if len(parts) == 1:
|
||||
return _read_source_from_origin(spec.origin, limit)
|
||||
return spec.origin
|
||||
|
||||
search_paths = spec.submodule_search_locations
|
||||
if not search_paths:
|
||||
return ""
|
||||
return None
|
||||
for i, part in enumerate(parts[1:], start=2):
|
||||
found_origin = None
|
||||
next_paths = None
|
||||
|
|
@ -907,13 +911,22 @@ def _resolve_module_source(module_name: str, limit: int = 8192) -> str:
|
|||
found_origin = str(mod_file)
|
||||
break
|
||||
if found_origin is None:
|
||||
return ""
|
||||
return None
|
||||
if i == len(parts) or next_paths is None:
|
||||
return _read_source_from_origin(found_origin, limit)
|
||||
return found_origin
|
||||
search_paths = next_paths
|
||||
return ""
|
||||
return None
|
||||
except Exception:
|
||||
return ""
|
||||
return None
|
||||
|
||||
|
||||
def _resolve_module_source(module_name: str, limit: int = 8192) -> str:
|
||||
"""First ``limit`` chars of a module's source, without importing it.
|
||||
|
||||
Empty string when the module cannot be resolved or read, which
|
||||
callers treat as ``standalone`` — the safe default.
|
||||
"""
|
||||
return _read_source_from_origin(resolve_module_origin(module_name), limit)
|
||||
|
||||
|
||||
@dataclass
|
||||
|
|
@ -2169,6 +2182,38 @@ class PluginContext:
|
|||
self.manifest.name, provider.prefix,
|
||||
)
|
||||
|
||||
# -- memory provider registration ---------------------------------------
|
||||
|
||||
def register_memory_provider(self, provider) -> None:
|
||||
"""Register a memory provider.
|
||||
|
||||
Memory providers are activated exclusively, by name, through
|
||||
``memory.provider`` in config.yaml, and ``plugins/memory/__init__.py``
|
||||
owns that path with its own collector. A provider reaching *this*
|
||||
implementation is therefore one the general PluginManager loaded — it
|
||||
was not classified ``exclusive`` — so the call is recorded and
|
||||
otherwise inert. Without it, such a plugin's ``register()`` dies on a
|
||||
missing attribute and the plugin fails to load at all.
|
||||
|
||||
Memory was the only provider category with no ``register_*`` here,
|
||||
which is what made that failure mode possible. The provider must be an
|
||||
instance of ``agent.memory_provider.MemoryProvider``.
|
||||
"""
|
||||
from agent.memory_provider import MemoryProvider
|
||||
|
||||
if not isinstance(provider, MemoryProvider):
|
||||
logger.warning(
|
||||
"Plugin '%s' tried to register a memory provider that does not "
|
||||
"inherit from MemoryProvider. Ignoring.",
|
||||
self.manifest.name,
|
||||
)
|
||||
return
|
||||
self._memory_provider = provider
|
||||
logger.debug(
|
||||
"Plugin '%s' registered memory provider: %s",
|
||||
self.manifest.name, getattr(provider, "name", "?"),
|
||||
)
|
||||
|
||||
# -- image gen provider registration ------------------------------------
|
||||
|
||||
@_serialized_replacement
|
||||
|
|
@ -4147,9 +4192,10 @@ class PluginManager:
|
|||
|
||||
# Auto-coerce user-installed memory providers to kind="exclusive"
|
||||
# so they're routed to plugins/memory discovery instead of being
|
||||
# loaded by the general PluginManager (which has no
|
||||
# register_memory_provider on PluginContext). Mirrors the
|
||||
# heuristic in plugins/memory/__init__.py:_is_memory_provider_dir.
|
||||
# loaded by the general PluginManager (whose PluginContext
|
||||
# register_memory_provider is a recorded no-op, not an
|
||||
# activation path). Mirrors the heuristic in
|
||||
# plugins/memory/__init__.py:_is_memory_provider_dir.
|
||||
# Bundled memory providers are already skipped via skip_names.
|
||||
if kind == "standalone" and "kind" not in data:
|
||||
init_file = plugin_dir / "__init__.py"
|
||||
|
|
|
|||
|
|
@ -1,15 +1,23 @@
|
|||
"""Memory provider plugin discovery.
|
||||
|
||||
Scans three sources for memory provider plugins:
|
||||
Scans four sources for memory provider plugins:
|
||||
|
||||
1. Bundled providers: ``plugins/memory/<name>/`` (shipped with hermes-agent)
|
||||
2. User-installed providers: ``$HERMES_HOME/plugins/<name>/``
|
||||
3. Pip-installed providers: ``hermes_agent.memory_providers`` entry points
|
||||
3. Project-local providers: ``./.hermes/plugins/<name>/``, opt-in via
|
||||
``HERMES_ENABLE_PROJECT_PLUGINS``
|
||||
4. Pip-installed providers: ``hermes_agent.memory_providers`` entry points
|
||||
|
||||
Directory providers must contain ``__init__.py`` with a class implementing
|
||||
the MemoryProvider ABC. Pip packages expose a provider or ``register(ctx)``
|
||||
callback through the entry-point group. On name collisions, bundled providers
|
||||
take precedence, followed by user-installed directories, then entry points.
|
||||
callback through the entry-point group.
|
||||
|
||||
These are the same four sources the general ``PluginManager`` scans, but the
|
||||
precedence is deliberately the reverse of its later-source-wins order: here
|
||||
**bundled wins**, then user, then project, then entry point. A memory provider
|
||||
is activated by name, so letting a directory dropped into the working tree
|
||||
shadow a shipped provider would silently redirect the agent's memory. Changing
|
||||
this order is a breaking change, not a cleanup.
|
||||
|
||||
Only ONE provider can be active at a time, selected via
|
||||
``memory.provider`` in config.yaml.
|
||||
|
|
@ -79,6 +87,24 @@ def _get_user_plugins_dir() -> Optional[Path]:
|
|||
return None
|
||||
|
||||
|
||||
def _get_project_plugins_dir() -> Optional[Path]:
|
||||
"""Return ``./.hermes/plugins/`` or None if unavailable or not opted in.
|
||||
|
||||
Gated on ``HERMES_ENABLE_PROJECT_PLUGINS`` exactly as the general
|
||||
``PluginManager`` gates its own project scan — a repository you merely
|
||||
``cd`` into must not be able to offer the agent a memory backend.
|
||||
"""
|
||||
try:
|
||||
from hermes_cli.plugins import _env_enabled
|
||||
|
||||
if not _env_enabled("HERMES_ENABLE_PROJECT_PLUGINS"):
|
||||
return None
|
||||
d = Path.cwd() / ".hermes" / "plugins"
|
||||
return d if d.is_dir() else None
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _is_memory_provider_dir(path: Path) -> bool:
|
||||
"""Heuristic: does *path* look like a memory provider plugin?
|
||||
|
||||
|
|
@ -98,8 +124,8 @@ def _is_memory_provider_dir(path: Path) -> bool:
|
|||
def _iter_provider_dirs() -> List[Tuple[str, Path]]:
|
||||
"""Yield ``(name, path)`` for all discovered provider directories.
|
||||
|
||||
Scans bundled first, then user-installed. Bundled takes precedence
|
||||
on name collisions (first-seen wins via ``seen`` set).
|
||||
Scans bundled, then user-installed, then project-local. Bundled takes
|
||||
precedence on name collisions (first-seen wins via ``seen`` set).
|
||||
"""
|
||||
seen: set = set()
|
||||
dirs: List[Tuple[str, Path]] = []
|
||||
|
|
@ -115,15 +141,18 @@ def _iter_provider_dirs() -> List[Tuple[str, Path]]:
|
|||
dirs.append((child.name, child))
|
||||
|
||||
# 2. User-installed providers ($HERMES_HOME/plugins/<name>/)
|
||||
user_dir = _get_user_plugins_dir()
|
||||
if user_dir:
|
||||
for child in sorted(user_dir.iterdir()):
|
||||
# 3. Project-local providers (./.hermes/plugins/<name>/), opt-in
|
||||
for source_dir in (_get_user_plugins_dir(), _get_project_plugins_dir()):
|
||||
if not source_dir:
|
||||
continue
|
||||
for child in sorted(source_dir.iterdir()):
|
||||
if not child.is_dir() or child.name.startswith(("_", ".")):
|
||||
continue
|
||||
if child.name in seen:
|
||||
continue # bundled takes precedence
|
||||
continue # earlier source wins
|
||||
if not _is_memory_provider_dir(child):
|
||||
continue # skip non-memory plugins
|
||||
seen.add(child.name)
|
||||
dirs.append((child.name, child))
|
||||
|
||||
return dirs
|
||||
|
|
@ -144,21 +173,63 @@ def _iter_entry_points():
|
|||
|
||||
|
||||
def find_provider_dir(name: str) -> Optional[Path]:
|
||||
"""Resolve a provider name to its directory.
|
||||
"""Resolve a provider name to the directory holding its files.
|
||||
|
||||
Checks bundled first, then user-installed.
|
||||
Checks bundled, then user-installed, then project-local, then the package
|
||||
directory of a pip entry-point provider.
|
||||
|
||||
The entry-point case matters because two of a provider's files are read
|
||||
from disk rather than imported: ``config_schema.py`` (loaded by path so the
|
||||
web server never pulls in the agent runtime — see
|
||||
``plugins/memory/config_schema.py``) and ``cli.py`` (loaded by
|
||||
``discover_plugin_cli_commands`` at argparse time). Without a directory, a
|
||||
pip-installed provider silently loses its dashboard config panel and its
|
||||
``hermes <provider>`` subcommands — working, but a second-class citizen next
|
||||
to a directory install.
|
||||
"""
|
||||
# Bundled
|
||||
bundled = _MEMORY_PLUGINS_DIR / name
|
||||
if bundled.is_dir() and (bundled / "__init__.py").exists():
|
||||
return bundled
|
||||
# User-installed
|
||||
user_dir = _get_user_plugins_dir()
|
||||
if user_dir:
|
||||
user = user_dir / name
|
||||
if user.is_dir() and _is_memory_provider_dir(user):
|
||||
return user
|
||||
return None
|
||||
# User-installed, then project-local
|
||||
for source_dir in (_get_user_plugins_dir(), _get_project_plugins_dir()):
|
||||
if not source_dir:
|
||||
continue
|
||||
candidate = source_dir / name
|
||||
if candidate.is_dir() and _is_memory_provider_dir(candidate):
|
||||
return candidate
|
||||
# Pip entry point
|
||||
return _entry_point_package_dir(find_provider_entry_point(name))
|
||||
|
||||
|
||||
def _entry_point_package_dir(entry_point) -> Optional[Path]:
|
||||
"""The directory of an entry point's module, resolved WITHOUT importing it.
|
||||
|
||||
Discovery must stay free of third-party imports: ``find_provider_dir`` is
|
||||
called from the dashboard and from argparse setup, long before the operator
|
||||
has selected a provider, so importing every installed candidate would run
|
||||
arbitrary code on the strength of a package merely being present.
|
||||
``resolve_module_origin`` walks the module's file layout instead.
|
||||
|
||||
Only package entry points (``pkg/__init__.py``) yield a directory — a
|
||||
provider pointed at a bare ``module.py`` has nowhere to put a sibling
|
||||
``config_schema.py``, so it correctly resolves to None.
|
||||
"""
|
||||
if entry_point is None:
|
||||
return None
|
||||
try:
|
||||
from hermes_cli.plugins import resolve_module_origin
|
||||
|
||||
module_name = (entry_point.value or "").split(":")[0].strip()
|
||||
origin = resolve_module_origin(module_name)
|
||||
if not origin:
|
||||
return None
|
||||
path = Path(origin)
|
||||
return path.parent if path.name == "__init__.py" else None
|
||||
except Exception as exc:
|
||||
logger.debug("Could not resolve directory for entry point '%s': %s",
|
||||
getattr(entry_point, "name", "?"), exc)
|
||||
return None
|
||||
|
||||
|
||||
def find_provider_entry_point(name: str):
|
||||
|
|
@ -177,11 +248,14 @@ def list_memory_provider_names() -> List[str]:
|
|||
"""Cheap name-only listing of discoverable memory providers.
|
||||
|
||||
Unlike :func:`discover_memory_providers`, this does NOT import provider
|
||||
modules or run availability checks — it's a directory scan only, safe to
|
||||
call at module-import time (e.g. when building the dashboard config
|
||||
schema).
|
||||
modules or run availability checks — a directory scan plus entry-point
|
||||
*enumeration*, which reads distribution metadata without executing any of
|
||||
it. Safe to call at module-import time (e.g. when building the dashboard
|
||||
config schema, where it fills the ``memory.provider`` dropdown).
|
||||
"""
|
||||
return sorted({name for name, _ in _iter_provider_dirs()})
|
||||
names = {name for name, _ in _iter_provider_dirs()}
|
||||
names.update(ep.name for ep in _iter_entry_points())
|
||||
return sorted(names)
|
||||
|
||||
|
||||
def discover_memory_providers() -> List[Tuple[str, str, bool]]:
|
||||
|
|
@ -438,10 +512,21 @@ def _load_provider_from_dir(
|
|||
collector = _ProviderCollector(name, register_skills=register_skills)
|
||||
try:
|
||||
mod.register(collector)
|
||||
if collector.provider:
|
||||
return collector.provider
|
||||
except Exception as e:
|
||||
logger.debug("register() failed for %s: %s", name, e)
|
||||
# A raise AFTER register_memory_provider() must not cost us the
|
||||
# provider. Falling through to the subclass scan below would
|
||||
# discard the instance the plugin configured and hand back a bare
|
||||
# second one — a silent downgrade that looks like success.
|
||||
if collector.provider is None:
|
||||
logger.debug("register() failed for %s: %s", name, e)
|
||||
else:
|
||||
logger.warning(
|
||||
"Memory provider '%s' raised after registering (%s) — "
|
||||
"using the registered provider; later registrations were skipped",
|
||||
name, e,
|
||||
)
|
||||
if collector.provider:
|
||||
return collector.provider
|
||||
|
||||
# Fallback: find a MemoryProvider subclass and instantiate it
|
||||
from agent.memory_provider import MemoryProvider
|
||||
|
|
@ -458,12 +543,19 @@ def _load_provider_from_dir(
|
|||
|
||||
|
||||
class _ProviderCollector:
|
||||
"""Fake plugin context that captures register_memory_provider calls."""
|
||||
"""Plugin context for memory providers.
|
||||
|
||||
Captures ``register_memory_provider`` directly — that is the one call the
|
||||
exclusive activation path owns — and delegates everything else to a real
|
||||
``PluginContext`` (see ``__getattr__``), so a memory provider has the same
|
||||
registration surface as any other plugin.
|
||||
"""
|
||||
|
||||
def __init__(self, name: str, *, register_skills: bool = True):
|
||||
self.name = name
|
||||
self.provider = None
|
||||
self._register_skills = register_skills
|
||||
self._context = None
|
||||
|
||||
def register_memory_provider(self, provider):
|
||||
self.provider = provider
|
||||
|
|
@ -471,37 +563,85 @@ class _ProviderCollector:
|
|||
def register_skill(self, *args, **kwargs):
|
||||
"""Forward plugin-provided skills to the general plugin registry.
|
||||
|
||||
Memory provider discovery uses this lightweight collector instead of
|
||||
the general PluginContext. Forwarding keeps skills registered by memory
|
||||
provider shims visible to skill_view() while preserving the exclusive
|
||||
memory-provider activation path.
|
||||
Handled explicitly rather than through ``__getattr__`` because skills
|
||||
are tracked for pruning: switching the active provider has to retract
|
||||
the skills the previous one registered, which needs the qualified name
|
||||
and resolved path recorded here.
|
||||
|
||||
Gated on ``register_skills`` so merely *inspecting* an inactive
|
||||
provider — ``hermes memory status``, the setup picker — leaves no
|
||||
registry side effects behind.
|
||||
"""
|
||||
if not self._register_skills:
|
||||
return
|
||||
try:
|
||||
from hermes_cli.plugins import PluginManifest, PluginContext, get_plugin_manager
|
||||
|
||||
manager = get_plugin_manager()
|
||||
manifest = PluginManifest(name=self.name, key=self.name)
|
||||
PluginContext(manifest, manager).register_skill(*args, **kwargs)
|
||||
manager_context = self._plugin_context()
|
||||
manager_context.register_skill(*args, **kwargs)
|
||||
skill_name = args[0] if args else kwargs.get("name")
|
||||
qualified_name = f"{self.name}:{skill_name}"
|
||||
registered_path = manager.find_plugin_skill(qualified_name)
|
||||
|
||||
from hermes_cli.plugins import get_plugin_manager
|
||||
|
||||
registered_path = get_plugin_manager().find_plugin_skill(qualified_name)
|
||||
if registered_path is not None:
|
||||
_REGISTERED_MEMORY_PROVIDER_SKILLS[qualified_name] = registered_path
|
||||
except Exception as exc:
|
||||
logger.debug("Memory provider '%s' failed to register skill: %s", self.name, exc)
|
||||
|
||||
# No-op for other registration methods
|
||||
def register_tool(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
def register_hook(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
def register_cli_command(self, *args, **kwargs):
|
||||
pass # CLI registration happens via discover_plugin_cli_commands()
|
||||
|
||||
def __getattr__(self, attr: str):
|
||||
"""Delegate any other ``register_*`` call to a real ``PluginContext``.
|
||||
|
||||
Memory providers used to get a hand-maintained stub of three no-ops
|
||||
here, which had two failure modes. Calls it *did* know about
|
||||
(``register_tool``, ``register_hook``) were silently dropped, so a
|
||||
provider's tools simply never appeared. Calls it did *not* know about
|
||||
raised ``AttributeError`` — and ``register_auxiliary_task`` is one of
|
||||
them, despite ``PluginContext.register_auxiliary_task`` documenting a
|
||||
memory provider (hindsight's pre-retain dedup) as its worked example.
|
||||
That exception surfaces as "register() failed" and costs the provider.
|
||||
|
||||
Delegating instead of enumerating means this can never drift behind
|
||||
``PluginContext`` again: a capability added there works for memory
|
||||
providers on the same commit, which is what the "widen the generic
|
||||
plugin surface" rule in AGENTS.md asks for.
|
||||
|
||||
Only ``register_*`` is forwarded. Everything else raises normally, so a
|
||||
typo still fails loudly rather than being absorbed.
|
||||
"""
|
||||
if not attr.startswith("register_"):
|
||||
raise AttributeError(attr)
|
||||
|
||||
def _forward(*args, **kwargs):
|
||||
try:
|
||||
return self._plugin_context().__getattribute__(attr)(*args, **kwargs)
|
||||
except Exception as exc:
|
||||
# A secondary registration must not cost the provider itself —
|
||||
# by the time these run, register_memory_provider has usually
|
||||
# already handed us the instance the agent needs.
|
||||
logger.warning(
|
||||
"Memory provider '%s' failed to %s: %s", self.name, attr, exc
|
||||
)
|
||||
return None
|
||||
|
||||
return _forward
|
||||
|
||||
def _plugin_context(self):
|
||||
"""A real ``PluginContext`` for this provider, built once on demand.
|
||||
|
||||
Lazy because the common case — a provider that only calls
|
||||
``register_memory_provider`` — must not pay for importing the general
|
||||
plugin manager, which discovery touches on every hermes startup.
|
||||
"""
|
||||
if self._context is None:
|
||||
from hermes_cli.plugins import PluginContext, PluginManifest, get_plugin_manager
|
||||
|
||||
manifest = PluginManifest(name=self.name, key=self.name)
|
||||
self._context = PluginContext(manifest, get_plugin_manager())
|
||||
return self._context
|
||||
|
||||
|
||||
def _get_active_memory_provider() -> Optional[str]:
|
||||
"""Read the active memory provider name from config.yaml.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,221 @@
|
|||
"""Discovery parity for out-of-tree memory providers.
|
||||
|
||||
Upstream policy closed ``plugins/memory/`` to new providers, so every new
|
||||
memory backend now lives outside this tree. These tests cover the two sources
|
||||
that reach it — project-local directories and pip entry points — and the
|
||||
integration points a directory install gets for free but a pip install
|
||||
historically did not: the dashboard config panel, the provider's CLI
|
||||
subcommands, and the ``memory.provider`` dropdown.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.metadata
|
||||
import sys
|
||||
import textwrap
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
import plugins.memory as memory_plugins
|
||||
|
||||
PROVIDER_SOURCE = """\
|
||||
from agent.memory_provider import MemoryProvider
|
||||
|
||||
|
||||
class Provider(MemoryProvider):
|
||||
@property
|
||||
def name(self):
|
||||
return "{name}"
|
||||
|
||||
def is_available(self):
|
||||
return True
|
||||
|
||||
def initialize(self, *a, **kw):
|
||||
pass
|
||||
|
||||
def get_tool_schemas(self):
|
||||
return []
|
||||
|
||||
|
||||
def register(ctx):
|
||||
ctx.register_memory_provider(Provider())
|
||||
"""
|
||||
|
||||
|
||||
class FakeEntryPoint:
|
||||
"""Mirrors the importlib.metadata EntryPoint surface discovery uses."""
|
||||
|
||||
group = "hermes_agent.memory_providers"
|
||||
|
||||
def __init__(self, name, value):
|
||||
self.name = name
|
||||
self.value = value
|
||||
|
||||
def load(self):
|
||||
import importlib
|
||||
|
||||
module_name, _, attr = self.value.partition(":")
|
||||
module = importlib.import_module(module_name)
|
||||
return getattr(module, attr) if attr else module
|
||||
|
||||
|
||||
class FakeEntryPoints(list):
|
||||
def select(self, *, group):
|
||||
return [ep for ep in self if ep.group == group]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def entry_points(monkeypatch):
|
||||
"""Install a replaceable entry-point set for the memory group."""
|
||||
registry = FakeEntryPoints()
|
||||
monkeypatch.setattr(importlib.metadata, "entry_points", lambda: registry)
|
||||
return registry
|
||||
|
||||
|
||||
def _write_provider_dir(root: Path, name: str) -> Path:
|
||||
provider = root / name
|
||||
provider.mkdir(parents=True)
|
||||
(provider / "__init__.py").write_text(PROVIDER_SOURCE.format(name=name), encoding="utf-8")
|
||||
return provider
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Project-local providers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_project_dir_is_ignored_without_opt_in(tmp_path, monkeypatch):
|
||||
"""A repo you merely cd into must not be able to offer a memory backend."""
|
||||
_write_provider_dir(tmp_path / ".hermes" / "plugins", "projectmem")
|
||||
monkeypatch.chdir(tmp_path)
|
||||
monkeypatch.delenv("HERMES_ENABLE_PROJECT_PLUGINS", raising=False)
|
||||
|
||||
assert "projectmem" not in memory_plugins.list_memory_provider_names()
|
||||
assert memory_plugins.find_provider_dir("projectmem") is None
|
||||
|
||||
|
||||
def test_project_dir_is_discovered_when_opted_in(tmp_path, monkeypatch):
|
||||
provider = _write_provider_dir(tmp_path / ".hermes" / "plugins", "projectmem")
|
||||
monkeypatch.chdir(tmp_path)
|
||||
monkeypatch.setenv("HERMES_ENABLE_PROJECT_PLUGINS", "1")
|
||||
|
||||
assert "projectmem" in memory_plugins.list_memory_provider_names()
|
||||
assert memory_plugins.find_provider_dir("projectmem") == provider
|
||||
|
||||
|
||||
def test_bundled_still_wins_over_project(tmp_path, monkeypatch):
|
||||
"""Precedence here is bundled-first, the reverse of the general
|
||||
PluginManager's later-wins order. A provider is activated by name, so a
|
||||
directory dropped into the working tree must not be able to shadow a
|
||||
shipped one and silently redirect the agent's memory."""
|
||||
_write_provider_dir(tmp_path / ".hermes" / "plugins", "honcho")
|
||||
monkeypatch.chdir(tmp_path)
|
||||
monkeypatch.setenv("HERMES_ENABLE_PROJECT_PLUGINS", "1")
|
||||
|
||||
resolved = memory_plugins.find_provider_dir("honcho")
|
||||
assert resolved == Path(memory_plugins.__file__).parent / "honcho"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pip entry-point providers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_entry_point_provider_is_listed(entry_points, tmp_path, monkeypatch):
|
||||
"""list_memory_provider_names() fills the dashboard's memory.provider
|
||||
dropdown. Enumerating entry points reads distribution metadata without
|
||||
executing any of it, so this stays safe to call at import time."""
|
||||
entry_points.append(FakeEntryPoint("pipmem", "pipmem_pkg"))
|
||||
assert "pipmem" in memory_plugins.list_memory_provider_names()
|
||||
|
||||
|
||||
def test_find_provider_dir_resolves_a_package_entry_point(entry_points, tmp_path, monkeypatch):
|
||||
"""Without a directory, a pip-installed provider silently loses its
|
||||
dashboard config panel and its `hermes <provider>` subcommands — both are
|
||||
read from disk rather than imported."""
|
||||
package = tmp_path / "pipmem_pkg"
|
||||
package.mkdir()
|
||||
(package / "__init__.py").write_text(PROVIDER_SOURCE.format(name="pipmem"), encoding="utf-8")
|
||||
(package / "config_schema.py").write_text("CONFIG_SCHEMA = None\n", encoding="utf-8")
|
||||
monkeypatch.syspath_prepend(str(tmp_path))
|
||||
entry_points.append(FakeEntryPoint("pipmem", "pipmem_pkg:register"))
|
||||
|
||||
assert memory_plugins.find_provider_dir("pipmem") == package
|
||||
|
||||
|
||||
def test_resolving_an_entry_point_does_not_import_it(entry_points, tmp_path, monkeypatch):
|
||||
"""Discovery runs before the operator has chosen a provider. Importing
|
||||
every installed candidate would execute third-party code on the strength of
|
||||
a package merely being present."""
|
||||
package = tmp_path / "sideeffect_pkg"
|
||||
package.mkdir()
|
||||
(package / "__init__.py").write_text(
|
||||
textwrap.dedent(
|
||||
"""\
|
||||
import pathlib
|
||||
pathlib.Path(__file__).with_name("IMPORTED").write_text("x")
|
||||
"""
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
monkeypatch.syspath_prepend(str(tmp_path))
|
||||
entry_points.append(FakeEntryPoint("sideeffect", "sideeffect_pkg"))
|
||||
|
||||
assert memory_plugins.find_provider_dir("sideeffect") == package
|
||||
assert not (package / "IMPORTED").exists()
|
||||
assert "sideeffect_pkg" not in sys.modules
|
||||
|
||||
|
||||
def test_bare_module_entry_point_has_no_directory(entry_points, tmp_path, monkeypatch):
|
||||
"""A single-file provider has nowhere to put a sibling config_schema.py, so
|
||||
it resolves to None rather than handing back the whole site-packages root."""
|
||||
(tmp_path / "flatmem.py").write_text(PROVIDER_SOURCE.format(name="flatmem"), encoding="utf-8")
|
||||
monkeypatch.syspath_prepend(str(tmp_path))
|
||||
entry_points.append(FakeEntryPoint("flatmem", "flatmem"))
|
||||
|
||||
assert memory_plugins.find_provider_dir("flatmem") is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Registration surface
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_a_secondary_registration_cannot_cost_the_provider(tmp_path, monkeypatch):
|
||||
"""register_auxiliary_task used to raise AttributeError on the collector —
|
||||
which the loader caught, discarded the registered provider, and replaced
|
||||
with a bare second instance built by the subclass scan. A silent downgrade
|
||||
that looked like success."""
|
||||
plugins_root = tmp_path / "plugins"
|
||||
provider = _write_provider_dir(plugins_root, "auxmem")
|
||||
(provider / "__init__.py").write_text(
|
||||
PROVIDER_SOURCE.format(name="auxmem").replace(
|
||||
" ctx.register_memory_provider(Provider())\n",
|
||||
" instance = Provider()\n"
|
||||
" instance.marked = True\n"
|
||||
" ctx.register_memory_provider(instance)\n"
|
||||
" ctx.register_auxiliary_task(\n"
|
||||
" 'auxmem_filter', display_name='Aux', description='d'\n"
|
||||
" )\n",
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
|
||||
|
||||
loaded = memory_plugins.load_memory_provider("auxmem")
|
||||
assert loaded is not None
|
||||
assert loaded.name == "auxmem"
|
||||
# The instance register() handed over, not a replacement.
|
||||
assert getattr(loaded, "marked", False)
|
||||
|
||||
|
||||
def test_activation_is_not_gated_on_plugins_enabled(tmp_path, monkeypatch):
|
||||
"""Memory providers are activated by naming them in memory.provider. Using
|
||||
a real PluginContext for secondary registrations must not start also
|
||||
requiring the plugin in plugins.enabled — that would break every existing
|
||||
user-installed provider."""
|
||||
_write_provider_dir(tmp_path / "plugins", "gatedmem")
|
||||
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
|
||||
|
||||
assert memory_plugins.load_memory_provider("gatedmem") is not None
|
||||
|
|
@ -14,15 +14,32 @@ Memory providers are one of two **provider plugin** types. The other is [Context
|
|||
|
||||
## Installation Layouts
|
||||
|
||||
Hermes discovers memory providers from bundled directories, user-installed
|
||||
directories, and installed Python package entry points. Bundled providers take
|
||||
precedence over a user directory with the same name, which takes precedence
|
||||
over a package entry point.
|
||||
Hermes discovers memory providers from four sources, in this precedence order:
|
||||
|
||||
| Source | Location | Notes |
|
||||
|---|---|---|
|
||||
| Bundled | `plugins/memory/<name>/` | Ships with Hermes. Closed to new providers — see [CONTRIBUTING](https://github.com/NousResearch/hermes-agent/blob/main/CONTRIBUTING.md). |
|
||||
| User | `$HERMES_HOME/plugins/<name>/` | Dropped in by the user, per profile. |
|
||||
| Project | `./.hermes/plugins/<name>/` | Opt-in via `HERMES_ENABLE_PROJECT_PLUGINS=1`. |
|
||||
| Package | `hermes_agent.memory_providers` entry point | `pip install`, nothing to copy. |
|
||||
|
||||
Earlier sources win on a name collision, so a directory dropped into a working
|
||||
tree can never shadow a shipped provider.
|
||||
|
||||
:::note
|
||||
This is the reverse of the general plugin system's later-wins order. A memory
|
||||
provider is activated by *name* (`memory.provider`), so shadowing would
|
||||
silently redirect the agent's memory rather than merely override a tool.
|
||||
:::
|
||||
|
||||
Discovery only *enumerates* — it never imports a provider. Nothing runs until
|
||||
`memory.provider` names it.
|
||||
|
||||
### Directory Provider
|
||||
|
||||
A directory provider lives in `plugins/memory/<name>/` when bundled with
|
||||
Hermes, or in `$HERMES_HOME/plugins/<name>/` when installed by a user:
|
||||
Hermes, in `$HERMES_HOME/plugins/<name>/` when installed by a user, or in
|
||||
`./.hermes/plugins/<name>/` for a project-local one:
|
||||
|
||||
```
|
||||
plugins/memory/my-provider/
|
||||
|
|
@ -43,9 +60,15 @@ name users select in `memory.provider`; its value points to the provider's
|
|||
my-provider = "my_provider:register"
|
||||
```
|
||||
|
||||
The package can keep its provider implementation, skills, and other resources
|
||||
inside its normal Python package layout. No copy under
|
||||
`$HERMES_HOME/plugins/` is required.
|
||||
Point the entry point at the **package**, or at a `register(ctx)` inside it, and
|
||||
keep your implementation, skills, and other resources in the normal Python
|
||||
package layout. No copy under `$HERMES_HOME/plugins/` is required.
|
||||
|
||||
A package entry point gets everything a directory install does, including the
|
||||
two files Hermes reads from disk rather than importing — `config_schema.py`
|
||||
(the dashboard config panel) and `cli.py` (your `hermes <provider>`
|
||||
subcommands). Both are found next to your package's `__init__.py`, so point the
|
||||
entry point at a package rather than a single module if you ship either.
|
||||
|
||||
## The MemoryProvider ABC
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue