Replace the old '[CONTEXT SUMMARY]:' prefix on compressed summaries
with a Codex-inspired handoff framing that tells the model what happened
and how to use the summary.
What changes:
1. New SUMMARY_PREFIX constant — the text prepended to every
compressed summary:
[CONTEXT COMPACTION] An earlier part of this conversation was
summarized to preserve context space. Below is the summary — use
it to build on the work already done and avoid duplicating effort:
2. _with_summary_prefix() helper — normalizes model output by stripping
any legacy '[CONTEXT SUMMARY]:' prefix the summarization model may
have produced, then prepends the new SUMMARY_PREFIX.
3. System message annotation updated — the note appended to the system
prompt on first compression now says 'compacted into a handoff
summary' and instructs 'build on that summary rather than re-doing
work' instead of the old generic note.
Why this is better:
The old prefix ('[CONTEXT SUMMARY]: <raw text>') gave the model no
context about what the summary is or how to use it. The new prefix
explicitly frames it as a context compaction event and instructs the
model to build on prior work rather than re-doing it. This reduces
redundant tool calls and file re-reads after compression.
What does NOT change:
- The compression algorithm (positional protection, boundary alignment)
- The role alternation logic (summary role adapts to avoid consecutive
same-role messages)
- The summarization model or trigger thresholds
- LEGACY_SUMMARY_PREFIX is exported for backward compatibility
Inspired by PR #776 by @kshitijk4poor and the research in #499.
The summary message was always injected as 'user' role, which causes
consecutive user messages when the last preserved head message is also
'user'. Some APIs reject this (400 error), and it produces malformed
training data.
Fix: check the role of the last head message and pick the opposite role
for the summary — 'user' after assistant/tool, 'assistant' after user.
Based on PR #328 by johnh4098. Closes#328.
The OpenAI API returns content: null on assistant messages that only
contain tool calls. msg.get('content', '') returns None (not '') when
the key exists with value None, causing TypeError on len() and string
concatenation in _generate_summary and compress.
Fix: msg.get('content') or '' — handles both missing keys and None.
Tests from PR #216 (@Farukest). Fix also in PR #215 (@cutepawss).
Both PRs had stale branches and couldn't be merged directly.
Closes#211
Cover model_tools, toolset_distributions, context_compressor,
prompt_caching, cronjob_tools, session_search, process_registry,
and cron/scheduler with 127 new test cases.
teknium1
Bryan Young
0xbyt4