From bac2955df9935629df6074040db029810b29affe Mon Sep 17 00:00:00 2001 From: Jack Maloney Date: Wed, 6 May 2026 09:17:05 -0500 Subject: [PATCH] docs: describe DCP context engine design --- website/docs/developer-guide/agent-loop.md | 24 +++++- website/docs/developer-guide/architecture.md | 12 ++- .../context-compression-and-caching.md | 82 +++++++++++++++++++ .../developer-guide/context-engine-plugin.md | 40 +++++++++ .../docs/developer-guide/prompt-assembly.md | 23 ++++++ 5 files changed, 176 insertions(+), 5 deletions(-) diff --git a/website/docs/developer-guide/agent-loop.md b/website/docs/developer-guide/agent-loop.md index 4ca66b56283c5..40f0eb36014e3 100644 --- a/website/docs/developer-guide/agent-loop.md +++ b/website/docs/developer-guide/agent-loop.md @@ -70,10 +70,13 @@ run_conversation() - chat_completions: OpenAI format as-is - codex_responses: convert to Responses API input items - anthropic_messages: convert via anthropic_adapter.py - 6. Inject ephemeral prompt layers (budget warnings, context pressure) - 7. Apply prompt caching markers if on Anthropic - 8. Make interruptible API call (_interruptible_api_call) - 9. Parse response: + 6. Let the active context engine transform the API-call copy, if supported + - DCP-style engines can add refs, compression placeholders, and nudges + - canonical conversation history must remain unchanged + 7. Inject ephemeral prompt layers (budget warnings, context pressure) + 8. Apply prompt caching markers if on Anthropic + 9. Make interruptible API call (_interruptible_api_call) + 10. Parse response: - If tool_calls: execute them, append results, loop back to step 5 - If text response: persist session, flush memory if needed, return ``` @@ -160,6 +163,19 @@ Some tools are intercepted by `run_agent.py` *before* reaching `handle_function_ These tools modify agent state directly and return synthetic tool results without going through the registry. + +### Context-engine tools + +The active context engine can expose tools via `get_tool_schemas()`. These tools +are injected into the model-visible tool list and routed back to +`handle_tool_call()` before normal registry dispatch. + +This is how DCP-style context management exposes a model-callable `compress` +tool. The tool updates context-engine state, then the next API-call transform +applies compression blocks to the outbound message copy. It should not mutate +the canonical transcript unless the engine explicitly documents a +transcript-mutating mode. + ## Callback Surfaces `AIAgent` supports platform-specific callbacks that enable real-time progress in the CLI, gateway, and ACP integrations: diff --git a/website/docs/developer-guide/architecture.md b/website/docs/developer-guide/architecture.md index c890193419946..af59e077a13f1 100644 --- a/website/docs/developer-guide/architecture.md +++ b/website/docs/developer-guide/architecture.md @@ -30,7 +30,7 @@ This page is the top-level map of Hermes Agent internals. Use it to orient yours │ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │ │ │ │ │ │ │ ┌──────┴───────┐ ┌──────┴───────┐ ┌──────┴───────┐ │ -│ │ Compression │ │ 3 API Modes │ │ Tool Registry│ │ +│ │ Context Mgmt │ │ 3 API Modes │ │ Tool Registry│ │ │ │ & Caching │ │ chat_compl. │ │ (registry.py)│ │ │ │ │ │ codex_resp. │ │ 61 tools │ │ │ │ │ │ anthropic │ │ 52 toolsets │ │ @@ -64,6 +64,7 @@ hermes-agent/ │ ├── prompt_builder.py # System prompt assembly │ ├── context_engine.py # ContextEngine ABC (pluggable) │ ├── context_compressor.py # Default engine — lossy summarization +│ ├── dcp_context_engine.py # Optional DCP-style model-guided context engine │ ├── prompt_caching.py # Anthropic prompt caching │ ├── auxiliary_client.py # Auxiliary LLM for side tasks (vision, summarization) │ ├── model_metadata.py # Model context lengths, token estimation @@ -134,6 +135,15 @@ hermes-agent/ └── tests/ # Pytest suite (~3,000+ tests) ``` + +### Context management + +Context management is handled through the `ContextEngine` interface. The default +engine is `ContextCompressor`, which performs host-triggered summarization. Other +engines can expose tools and transform the provider-bound API message copy. A +DCP-style engine uses that path to keep the stored transcript complete while +sending compressed blocks, refs, and nudges to the model. + ## Data Flow ### CLI Session diff --git a/website/docs/developer-guide/context-compression-and-caching.md b/website/docs/developer-guide/context-compression-and-caching.md index 5c6268bbce783..9ac08e109e0f0 100644 --- a/website/docs/developer-guide/context-compression-and-caching.md +++ b/website/docs/developer-guide/context-compression-and-caching.md @@ -34,6 +34,88 @@ Configure via `hermes plugins` → Provider Plugins → Context Engine, or edit For building a context engine plugin, see [Context Engine Plugins](/docs/developer-guide/context-engine-plugin). + +## DCP Context Engine + +Hermes can also run a DCP-style context engine with: + +```yaml +context: + engine: "dcp" +``` + +DCP mode is different from the built-in `ContextCompressor`. The built-in +compressor is host-driven: Hermes decides that the session is too large, calls +an auxiliary summarization model, and replaces the stored message list with a +compressed transcript. DCP mode is model-guided: Hermes exposes a `compress` +tool, adds stable message and block references to the outbound request, and +lets the active model compress completed ranges when it has enough semantic +context to know what is safe to replace. + +The DCP invariant is: + +> The canonical session transcript remains complete. DCP transforms only the +> API-call copy of the messages sent to the provider. + +A DCP engine owns separate compression state: + +- message refs such as `m0001`, `m0002` +- compression block refs such as `b1`, `b2` +- active block summaries +- duplicate-tool and old-error pruning state +- compression stats and nudge cadence + +### DCP `compress` tool + +When DCP mode is active, the context engine may expose a `compress` tool. The +tool does not rewrite the stored transcript. Instead, it records compression +blocks. The next API-call transform applies those blocks to the outbound copy. + +DCP supports two modes: + +- `range`: compress one or more contiguous spans using `{startId, endId, summary}`. +- `message`: compress individual high-volume messages using `{messageId, topic, summary}`. + +Range mode is the default because it preserves chronology and usually gives +the model enough context to summarize closed work accurately. Message mode is +more surgical and should be treated as experimental until provider-format and +cache behavior are well tested. + +### DCP nudges and automatic strategies + +DCP mode can inject ephemeral nudges when context pressure rises. Nudges tell +the model to call `compress` before continuing if a completed topic is safe to +compact. The engine may also run cheap automatic strategies over the outbound +copy: + +- deduplicate repeated tool calls with the same tool name and arguments, keeping + the latest output +- purge bulky old failed-tool inputs while preserving the error text +- protect recent user turns and configured protected tools + +These strategies are DCP-state transformations, not transcript edits. + +### Interaction with existing compression + +When `context.engine: "dcp"`, the built-in `compression:` settings do not drive +normal compaction. DCP should return `False` from `should_compress()` during +normal operation and rely on nudges plus the `compress` tool. The built-in +`ContextCompressor` may still be used as an emergency fallback for hard context +limit failures, but it should not run as a parallel primary compressor. + +Gateway session hygiene remains a safety net. Because hygiene compression +operates before the agent starts and may mutate gateway history, DCP-aware +hygiene behavior must be handled deliberately rather than implicitly reusing +the built-in compressor path. + +### Prompt caching + +DCP must run before provider cache-control markers are applied so prompt caching +sees the actual outgoing request. The transform should be deterministic and +should avoid modifying old stable content on every turn. Compression blocks and +automatic pruning should change the cached prefix only when compression state +changes, not as a side effect of moving counters or timestamps. + ## Dual Compression System Hermes has two separate compression layers that operate independently: diff --git a/website/docs/developer-guide/context-engine-plugin.md b/website/docs/developer-guide/context-engine-plugin.md index 64fea96acba18..7a60ba248a43e 100644 --- a/website/docs/developer-guide/context-engine-plugin.md +++ b/website/docs/developer-guide/context-engine-plugin.md @@ -97,6 +97,46 @@ These have sensible defaults in the ABC. Override as needed: | `handle_tool_call(name, args, **kwargs)` | Returns error JSON | You implement tool handlers | | `should_compress_preflight(messages)` | Returns `False` | You can do a cheap pre-API-call estimate | | `get_status()` | Standard token/threshold dict | You have custom metrics to expose | +| `transform_api_messages(api_messages, **kwargs)` | Returns `api_messages` unchanged | You need to alter the provider-bound copy without mutating stored history | + + +## API-call-time transforms + +Context engines that need DCP-style behavior can override +`transform_api_messages()` to transform the outbound API-call copy of the +conversation: + +```python +def transform_api_messages( + self, + api_messages: list[dict], + *, + canonical_messages: list[dict], + system_prompt: str, + tools: list[dict] | None, + api_call_count: int, + model: str, + provider: str | None, + session_id: str | None, +) -> list[dict]: + return api_messages +``` + +This hook is for ephemeral, model-facing context changes such as message refs, +compression block placeholders, and context-pressure nudges. It must not mutate +`canonical_messages`, because those messages are the authoritative session +transcript. + +Transform implementations must preserve provider-valid message order: + +- keep assistant `tool_calls` messages adjacent to their tool results +- do not orphan tool results +- do not create consecutive roles that the provider rejects +- return OpenAI-format messages that still pass Hermes sanitization + +The hook should run before provider-specific cache-control placement. Engines +should avoid churn in the stable prompt prefix because unnecessary changes lower +prompt-cache hit rates. ## Engine tools diff --git a/website/docs/developer-guide/prompt-assembly.md b/website/docs/developer-guide/prompt-assembly.md index f23705870ee9e..2496fb8ded74b 100644 --- a/website/docs/developer-guide/prompt-assembly.md +++ b/website/docs/developer-guide/prompt-assembly.md @@ -116,6 +116,29 @@ You are a CLI AI Agent. Try not to use markdown but simple text renderable inside a terminal. ``` + +## Context-engine API-call-time transforms + +After Hermes assembles the canonical conversation into provider-bound +`api_messages`, the active context engine may transform that API-call copy. This +keeps persisted session history separate from ephemeral model-facing context. + +DCP-style engines use this layer to add: + +- stable message refs such as `m0001` +- compression block refs such as `b1` +- compact placeholders for compressed ranges +- context-pressure nudges that teach the model when to call a context tool +- cheap outbound-only pruning such as duplicate tool output placeholders + +The transform must not rewrite the canonical transcript. If a session is saved, +it should still contain the full original conversation unless a user explicitly +invokes a separate transcript-mutating command. + +Because this layer changes what the provider sees, it should run before prompt +cache-control markers are applied. Engines should keep the transformed stable +prefix deterministic and avoid moving counters or timestamps in old context. + ## How SOUL.md appears in the prompt `SOUL.md` lives at `~/.hermes/SOUL.md` and serves as the agent's identity — the very first section of the system prompt. The loading logic in `prompt_builder.py` works as follows: