fix(codex): correct startup trust, history reads, and resume usage (#13110)
## Thinking Path
> - Paperclip runs Codex locally and in remote sandboxes.
> - The runner must preserve startup configuration and session identity.
> - Missing project trust can disable repository configuration.
> - Full-history requests use deprecated provider fields.
> - Resume usage describes old work and must not become new run usage.
> - This change corrects startup trust, state reads, and usage
classification.
## Linked Issues or Issue Description
**What happened?**
Normal Codex runs could show repository-trust and history-deprecation
warnings.
Resume could report the preceding turn's token snapshot as a late-turn
warning.
The historical last-usage value could also be attributed to the new run.
**Expected behavior**
Trust the server-selected startup root in isolated configuration. Read
lightweight
provider state and paginated evidence. Use historical cumulative usage
as a
baseline without a new charge or user-facing warning.
**Steps to reproduce**
1. Start a native Codex task in a selected repository.
2. Finish the turn and resume the provider thread.
3. Inspect provider notices, history requests, and per-run usage.
4. Repeat startup and cold resume inside a Daytona sandbox.
**Paperclip version or commit**
Codex CLI 0.153.4 is the pinned runtime and reproduced baseline.
Replayed onto master at 6abeb6733. Related authority work: Refs #13092.
This PR retains its startup cleanup and protocol-integrity checks.
**Deployment mode**
Local source checkout and disposable Daytona sandbox.
## What Changed
- Classify the exact historical resume usage event before the generic
stale-turn warning.
- Persist cumulative usage baselines across recovery of the same run.
- Use excludeTurns on resume and lightweight thread reads.
- Page turn metadata and selected turn items with cursor and identity
validation.
- Reject unsupported or incomplete history instead of guessing that
execution is idle.
- Trust the startup execution root on its host, including Git worktree
trust keys.
- Start Codex in that root and retain the selected sandbox profile on
later turns.
- Keep unrelated isolated configuration and Codex's separate hook trust
policy.
- Add Rust, TypeScript, accounting, native integration, and local
run-log documentation.
## Verification
- Codex and native-transport TypeScript: 333 passed before PR replay.
- Adjacent OpenCode/ACPX driver and accounting tests: 49 passed.
- Rust library, serialized: 226 passed. Native Codex integration: 72
passed, 1 ignored, plus two pagination regressions.
- Repository typecheck and build passed. All repository test groups have
passing coverage after fixture and resource retests; the initial
monolithic command was not clean.
- Fresh real Codex native browser tasks returned correct answers without
the three targeted notices. Answers persisted after refresh and restart.
- Real same-thread TypeScript driver tests passed locally and in
Daytona, including cold resume, configuration, skills, and an approved
harmless hook.
- Local usage summed to 64,607 tokens. Daytona usage summed to 42,737
tokens. Each sum matched its final session total exactly.
- See doc/plans/2026-09-09-codex-integration-acceptance.md for the scope
and limits of the live tests.
- After replay onto current master and review fixes: 334 Codex, backend,
and live-session tests passed, including checkpoint serialization and
real-runner process restart. TypeScript checks passed.
- The native Codex integration run passed 83 tests; the large lineage
test passed separately with the release runner (its debug build exceeded
the test deadline).
- All GitHub checks passed on the final PR head. Greptile is 5/5 with no
unresolved review threads. CI regenerates the lockfile for the added
TOML dependency, per repository policy.
- The first server shard hit a timing-dependent duplicate-key failure in
the unchanged artifact-document concurrency test. Its focused 11-test
suite passed locally. One CI retry on the same head passed all 103 files
and 1,405 tests (2 skipped): [retry
result](https://github.com/paperclipai/paperclip/actions/runs/34398832930/job/102631274667).
## Risks
- Trust applies only to the server-selected startup root and isolated
configuration. Sandbox and tool permissions remain authoritative.
- Codex still requires approval of individual hook hashes. This change
does not bypass that policy.
- Providers without the required history APIs fail explicitly.
- Daytona acceptance used the production TypeScript driver. Remote
Paperclip UI and remote Rust execution were not tested.
- No new public API, database state, recovery policy, or UI control is
included.
## Model Used
OpenAI Codex, GPT-6 (`gpt-6-astra`). Used for reasoning, code edits,
tool use,
and test execution. The exact context-window limit is not exposed in
this
session. Real-provider acceptance used Codex CLI 0.153.4 with
`gpt-5.6-sol`.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Dotta
GitHub
parent
ca96e1eb0a
commit
3b550c80fa
|
|
@ -815,6 +815,25 @@ Legacy adapters without a verified resume capability use the same automatic no-r
|
|||
|
||||
The server projection remains available for execution diagnostics. Normal working, finishing, and interaction waits add no badges or cards to task lists or feeds. A retry may briefly change the existing transcript header to Reconnecting; attempts, causes, and recovery decisions belong in the run log. There is no reconciliation dialog. Safe recovery remains automatic. If it cannot continue safely, the source-scoped recovery record resolves with a blocked no-replay disposition and the ordinary task status becomes blocked, preserving its owner. Resolving this record does not grant replay authority: dispatch continues enforcing the durable hold. Replacement history remains inspectable and the composer stays usable.
|
||||
|
||||
### Codex startup and provider state
|
||||
|
||||
Paperclip trusts the server-selected startup execution root in the isolated
|
||||
Codex configuration. Resolve that root on the execution host, including the
|
||||
main repository trust key for Git worktrees. Start the provider in that same
|
||||
root. This does not change sandbox permissions, tool authorization, secret
|
||||
access, or Codex's separate per-hook trust policy.
|
||||
|
||||
Codex retains the model conversation. Paperclip resumes with `excludeTurns: true`,
|
||||
reads lightweight thread state, and fetches paginated turn metadata or specific
|
||||
turn items only when execution reconciliation needs them. Unsupported
|
||||
or incomplete history is an explicit error, not evidence of idle execution.
|
||||
|
||||
The root-thread usage snapshot sent during resume belongs to its reported
|
||||
completed turn. Retain a bounded local diagnostic and use cumulative totals as
|
||||
a baseline; do not emit a warning or charge its historical `last` usage to the
|
||||
new run. Preserve the baseline across recovery of the same run and start a new
|
||||
delta when attaching a new run. Other stale-event and authority checks remain.
|
||||
|
||||
### Explicit Recovery Action
|
||||
|
||||
Paperclip opens an explicit recovery action when the system can identify a problem but cannot safely complete the work itself.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,93 @@
|
|||
# Codex integration acceptance — 2026-09-09
|
||||
|
||||
## Environment and scope
|
||||
|
||||
All live tests used the pinned Codex CLI 0.153.4 and gpt-5.6-sol.
|
||||
Each fixture used an isolated Codex home. A repository contained known launch
|
||||
notes, a configuration marker, a skill marker, and a harmless SessionStart hook.
|
||||
The hook only appended a line to a fixture file.
|
||||
|
||||
Streaming and feed-display changes remain separate pull requests. These tests
|
||||
used their combined implementation checkout. Each PR is also checked on the
|
||||
current master before merge.
|
||||
|
||||
## Native browser acceptance
|
||||
|
||||
A fresh test-drive instance started without tasks or prior runs. The test agent
|
||||
used paperclip_runner with Codex. Browser actions created a task in the fixture
|
||||
project, requested the notes, and sent two follow-up messages. The test server
|
||||
was restarted between the first answer and the follow-ups.
|
||||
|
||||
All three native runs succeeded. They returned the expected notes and markers,
|
||||
then answered a date-change question and recalled the original reference.
|
||||
Answers persisted after refresh. No provider notice appeared. The task composer
|
||||
remained usable.
|
||||
|
||||
The task policy opened new provider threads for completed-task follow-ups.
|
||||
This browser test does not prove same-thread resume. The next tests do.
|
||||
|
||||
## Same-thread local driver acceptance
|
||||
|
||||
The production TypeScript driver ran an initial repository read, a follow-up,
|
||||
a provider shutdown, persisted-session recovery, and a second follow-up.
|
||||
All answers used the same provider thread and retained the required context.
|
||||
|
||||
- No provider notices appeared.
|
||||
- Configuration and skill markers loaded.
|
||||
- The approved hook ran once at startup and once at cold resume.
|
||||
- Run usage was 40,445 + 13,624 + 10,538 = 64,607 tokens.
|
||||
- The sum equaled the final cumulative session usage. Historical usage was
|
||||
not charged again.
|
||||
- Resume usage produced only the bounded local diagnostic.
|
||||
|
||||
## Daytona driver acceptance
|
||||
|
||||
A disposable Daytona sandbox ran the production TypeScript driver bundle.
|
||||
The test installed Codex 0.153.4 because the image had an older version.
|
||||
Startup, warm follow-up, process shutdown, cold resume, and the second
|
||||
follow-up all succeeded on the same provider thread.
|
||||
|
||||
- Configuration and skill markers loaded.
|
||||
- The approved hook ran once at startup and once at cold resume.
|
||||
- No trust, history-deprecation, or settled-turn usage warning appeared.
|
||||
- Run usage was 23,006 + 11,611 + 8,120 = 42,737 tokens.
|
||||
- The sum equaled the final cumulative session usage.
|
||||
- A separate warning about missing system bubblewrap remained visible.
|
||||
Codex used its bundled copy. The change does not suppress that warning.
|
||||
|
||||
The sandbox was deleted after evidence collection. Remote Paperclip UI and
|
||||
remote Rust execution were not tested.
|
||||
|
||||
## Hook trust and experience limits
|
||||
|
||||
Codex reviews hook hashes separately from repository trust. The test queried
|
||||
hooks/list and approved only the harmless fixture hash through config/batchWrite.
|
||||
Product code does not bypass this policy or copy operator configuration.
|
||||
|
||||
The task feed worked correctly. Two separate issues remain outside this change:
|
||||
a dashboard preview could retain old running text, and test-drive restart could
|
||||
use a different database when its saved port did not match its actual port.
|
||||
The fixture configuration was corrected before sending further messages.
|
||||
Existing test data was preserved. Transitions were sampled rather than filmed.
|
||||
|
||||
## Automated verification before PR preparation
|
||||
|
||||
- Codex/native-transport TypeScript: 333 passed.
|
||||
- Adjacent OpenCode/ACPX driver, accounting, and recovery tests: 49 passed.
|
||||
- Same-run attach and usage baseline regression: 4 passed.
|
||||
- Rust library, serialized: 226 passed.
|
||||
- Rust Codex integration: 72 passed, 1 ignored; two new pagination tests passed.
|
||||
- Production native server integration passed after rebuilding its stale fake
|
||||
provider. The test now asks Cargo to check binary freshness.
|
||||
- Repository typecheck and build passed.
|
||||
- Repository test groups have passing coverage after environment retests.
|
||||
General-server coverage was 7,217 passed and 30 skipped. Route coverage was
|
||||
2,175 passed and 4 skipped across 144 files.
|
||||
|
||||
The initial monolithic test command did not pass cleanly. Parallel test loads
|
||||
caused timeouts; isolated reruns passed. CLI database tests reached the macOS
|
||||
shared-memory limit; they passed after unused disposable fixture servers were
|
||||
stopped. No production behavior or timeout setting changed for those failures.
|
||||
The default parallel Rust library run also had Claude fixture transport failures;
|
||||
the complete serialized rerun passed. PR checks record validation after replaying
|
||||
these changes onto current master.
|
||||
|
|
@ -0,0 +1,87 @@
|
|||
# Correct Codex startup trust, history reads, and resume notices
|
||||
|
||||
Date: 2026-09-09. Approved scope: three Codex integration fixes. This supersedes
|
||||
this document's earlier exploratory recommendations. Feed display and
|
||||
full-answer streaming are separate preceding changes.
|
||||
|
||||
## Product rules
|
||||
|
||||
1. Trust the execution root that Paperclip selects at startup. Resolve it on
|
||||
the execution host, including a Git worktree's main repository trust key.
|
||||
Write only the isolated Codex configuration. Keep sandbox, tool, and secret
|
||||
controls authoritative. Later directory changes do not grant new trust.
|
||||
2. Codex retains model conversation context. Paperclip reads provider state to
|
||||
establish execution authority or recover specific evidence. Normal resume
|
||||
must not download historical message contents.
|
||||
3. A resume usage snapshot describes completed work. It can establish a
|
||||
cumulative baseline, but must not charge that work to a new run or produce
|
||||
a warning. Other stale notifications and invalid authoritative events keep
|
||||
their existing validation.
|
||||
|
||||
## Implementation
|
||||
|
||||
### Resume usage
|
||||
|
||||
Handle the exact root-thread `thread/tokenUsage/updated` snapshot before the
|
||||
settled-turn warning branch. Keep its reported historical turn identity and a
|
||||
bounded local `codex_resume_usage_snapshot` diagnostic. The Rust normalizer
|
||||
must not emit a billable usage event for the snapshot. The TypeScript driver
|
||||
persists its cumulative baseline with the existing checkpoint and reports a
|
||||
monotonic run delta. Attachment begins a new delta at the last observed total;
|
||||
recovery of the same run preserves its baseline. Repeated snapshots are not
|
||||
additional receipts. Missing thread or turn identity does not gain authority.
|
||||
|
||||
### Supported state and history reads
|
||||
|
||||
Use `excludeTurns: true` for resume and `includeTurns: false` for thread state.
|
||||
Request turn metadata with `thread/turns/list` and `itemsView: notLoaded`.
|
||||
Only request `thread/items/list` content for a specific turn when reconciliation
|
||||
needs its final answer, tool result, or completion evidence. Follow cursors,
|
||||
keep stable order, deduplicate IDs, reject repeated cursors and incomplete
|
||||
responses. A missing API reports a compatibility/read error; there is no
|
||||
Codex full-history fallback.
|
||||
|
||||
Rust uses lightweight idle/active state, then paginated metadata when it needs
|
||||
an active turn identity. The controller's runner transport serves targeted
|
||||
recovery evidence from committed runner events; it rejects content reads
|
||||
outside the retained turn window. Existing non-Codex proxy behavior stays
|
||||
separate from Codex's protocol requirements.
|
||||
|
||||
### Startup trust
|
||||
|
||||
Before spawning Codex, canonicalize the selected root and add its trusted
|
||||
project entry to the isolated config. Preserve unrelated settings and use a
|
||||
private atomic replacement. Start the provider process in that same root,
|
||||
so startup cannot load the Paperclip server checkout by accident. Persist the
|
||||
startup directory in the existing optional session checkpoint for cold resume.
|
||||
Remote roots are resolved on the execution host.
|
||||
|
||||
Retain the server-selected permission profile on subsequent TypeScript turns,
|
||||
including Daytona's existing external sandbox profile. Do not change approval
|
||||
policy or bypass the external sandbox boundary.
|
||||
|
||||
Repository trust loads hook definitions, but Codex 0.153.4 separately reviews
|
||||
individual hook hashes. Preserve that policy. Acceptance explicitly approves
|
||||
only the harmless fixture hook through Codex's supported config API; product
|
||||
code does not bypass hook trust or invoke provider hooks itself.
|
||||
|
||||
## Verification and exclusions
|
||||
|
||||
Use Codex CLI 0.153.4, the pinned supported baseline. Test snapshot replay and
|
||||
cold recovery accounting, cross-thread isolation, full-history avoidance,
|
||||
metadata/item pagination, incomplete evidence, worktree/non-Git/canonical
|
||||
trust, malformed config, and unchanged sandbox profiles. Run focused Rust,
|
||||
TypeScript, server lifecycle/accounting tests, then repository typecheck,
|
||||
tests, and build.
|
||||
|
||||
Use fresh local test-drive data and a disposable Daytona sandbox with real
|
||||
Codex. Verify an initial repository read, two follow-ups, cold resume, config
|
||||
and skill markers, one hook execution per startup/resume, exact cumulative
|
||||
usage arithmetic, and absence of the three original warnings. Inspect the
|
||||
browser answer after refresh. Record local/native and remote/TypeScript
|
||||
proof separately, including any environment warnings or unverified behavior.
|
||||
|
||||
No new UI, task state, public API, database migration, retry policy, or session
|
||||
replacement workflow. Preserve the Gmail handoff patch and existing test data.
|
||||
|
||||
See [acceptance evidence](2026-09-09-codex-integration-acceptance.md).
|
||||
|
|
@ -119,3 +119,13 @@ section in the Observability contract.
|
|||
Provider identity diagnostics remain in the local run log. They record the notification method, expected and received thread/turn identifiers, and the classification (root, verified descendant, stale, unrelated informational, or invalid authoritative). They omit the original provider payload and credentials. Repeated informational notices are bounded.
|
||||
|
||||
Recovery lifecycle events retain the original structured failure code, retry attempt, next retry time, and predecessor/successor identifiers. Durable status delivery uses an idempotency marker; delivery grants no provider authority. Failed publication is retried without repeating provider work. These records are not first-party Telemetry.
|
||||
|
||||
## Codex resume usage snapshot
|
||||
|
||||
The native runner retains a bounded local `harness.diagnostic` event with code
|
||||
`codex_resume_usage_snapshot`. It identifies `thread/tokenUsage/updated` as
|
||||
`resume_usage_snapshot`, retains the reported thread and completed-turn IDs,
|
||||
and records cumulative usage counters. It does not include provider credentials
|
||||
or message content. The event establishes the accounting baseline; it is not a
|
||||
new billable usage receipt or a user-facing provider warning. Other provider
|
||||
identity checks remain in force.
|
||||
|
|
|
|||
|
|
@ -165,7 +165,8 @@
|
|||
"json-schema-to-ts": "^3.1.1",
|
||||
"opencode-ai": "1.18.29",
|
||||
"react-markdown": "^10.1.0",
|
||||
"remark-gfm": "^4.0.1"
|
||||
"remark-gfm": "^4.0.1",
|
||||
"smol-toml": "^1.4.2"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"react": ">=18",
|
||||
|
|
|
|||
|
|
@ -1770,6 +1770,7 @@ dependencies = [
|
|||
"sha1 0.10.7",
|
||||
"sha2 0.10.9",
|
||||
"tokio",
|
||||
"toml_edit",
|
||||
"tungstenite",
|
||||
"uuid",
|
||||
]
|
||||
|
|
@ -2677,6 +2678,30 @@ dependencies = [
|
|||
"tokio",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "toml_datetime"
|
||||
version = "0.6.11"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "22cddaf88f4fbc13c51aebbf5f8eceb5c7c5a9da2ac40a13519eb5b0a0e8f11c"
|
||||
|
||||
[[package]]
|
||||
name = "toml_edit"
|
||||
version = "0.22.27"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "41fe8c660ae4257887cf66394862d21dbca4a6ddd26f04a3560410406a2f819a"
|
||||
dependencies = [
|
||||
"indexmap",
|
||||
"toml_datetime",
|
||||
"toml_write",
|
||||
"winnow",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "toml_write"
|
||||
version = "0.1.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "5d99f8c9a7727884afe522e9bd5edbfc91a3312b36a77b5fb8926e4c31a41801"
|
||||
|
||||
[[package]]
|
||||
name = "tower"
|
||||
version = "0.5.3"
|
||||
|
|
@ -3072,6 +3097,15 @@ version = "0.52.6"
|
|||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec"
|
||||
|
||||
[[package]]
|
||||
name = "winnow"
|
||||
version = "0.7.15"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "df79d97927682d2fd8adb29682d1140b343be4ac0f08fd68b7765d9c059d3945"
|
||||
dependencies = [
|
||||
"memchr",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "wit-bindgen"
|
||||
version = "0.57.1"
|
||||
|
|
|
|||
|
|
@ -32,5 +32,6 @@ reqwest = { version = "0.12", default-features = false, features = ["blocking",
|
|||
rustix = { version = "1.1", features = ["fs"] }
|
||||
sha1 = "0.10"
|
||||
tungstenite = { version = "0.28", default-features = false, features = ["handshake", "rustls-tls-native-roots"] }
|
||||
toml_edit = "0.22"
|
||||
tokio = { version = "1.50", features = ["rt-multi-thread", "sync", "time"] }
|
||||
uuid = { version = "1.22", features = ["v4"] }
|
||||
|
|
|
|||
|
|
@ -29,6 +29,7 @@ reqwest.workspace = true
|
|||
rustix.workspace = true
|
||||
sha1.workspace = true
|
||||
tungstenite.workspace = true
|
||||
toml_edit.workspace = true
|
||||
tokio.workspace = true
|
||||
uuid.workspace = true
|
||||
|
||||
|
|
|
|||
|
|
@ -1051,6 +1051,13 @@ fn run() -> Result<(), Box<dyn std::error::Error>> {
|
|||
}}))?;
|
||||
continue;
|
||||
}
|
||||
if args
|
||||
.iter()
|
||||
.any(|arg| arg == "--require-lightweight-history")
|
||||
&& message.pointer("/params/excludeTurns") != Some(&json!(true))
|
||||
{
|
||||
return Err("thread/resume must exclude turns".into());
|
||||
}
|
||||
if require_external_sandbox
|
||||
&& (message.pointer("/params/sandbox") != Some(&json!("danger-full-access"))
|
||||
|| message.pointer("/params/permissions").is_some())
|
||||
|
|
@ -1079,6 +1086,17 @@ fn run() -> Result<(), Box<dyn std::error::Error>> {
|
|||
"id": id,
|
||||
"result": {"thread": {"id": state.thread_id, "sessionId": "codex-account-session"}}
|
||||
}))?;
|
||||
if args.iter().any(|arg| arg == "--resume-usage-snapshot")
|
||||
&& state.active_turn_id.is_none()
|
||||
&& state.next_turn > 0
|
||||
{
|
||||
for _ in 0..2 {
|
||||
send(json!({"method": "thread/tokenUsage/updated", "params": {
|
||||
"threadId": state.thread_id, "turnId": format!("provider-turn-{}", state.next_turn),
|
||||
"tokenUsage": {"total": {"inputTokens": 100, "outputTokens": 20}, "last": {"inputTokens": 50, "outputTokens": 10}}
|
||||
}}))?;
|
||||
}
|
||||
}
|
||||
if descendant_notifications {
|
||||
// Restoration must retain lineage without a replay of thread/started.
|
||||
send(json!({"method": "turn/completed", "params": {
|
||||
|
|
@ -1101,7 +1119,34 @@ fn run() -> Result<(), Box<dyn std::error::Error>> {
|
|||
}
|
||||
}
|
||||
}
|
||||
"thread/turns/list" => {
|
||||
if args
|
||||
.iter()
|
||||
.any(|arg| arg == "--require-lightweight-history")
|
||||
&& message.pointer("/params/itemsView") != Some(&json!("notLoaded"))
|
||||
{
|
||||
return Err("turn metadata must not hydrate items".into());
|
||||
}
|
||||
if args.iter().any(|arg| arg == "--repeat-history-cursor")
|
||||
|| (args.iter().any(|arg| arg == "--paginated-history")
|
||||
&& message.pointer("/params/cursor").is_none_or(Value::is_null))
|
||||
{
|
||||
send(json!({"id": id, "result": {"data": [], "nextCursor": "next-page"}}))?;
|
||||
continue;
|
||||
}
|
||||
let turns = state.active_turn_id.as_ref()
|
||||
.map(|turn_id| vec![json!({"id": turn_id, "status": "inProgress", "items": [], "itemsView": "notLoaded"})])
|
||||
.unwrap_or_default();
|
||||
send(json!({"id": id, "result": {"data": turns, "nextCursor": null}}))?;
|
||||
}
|
||||
"thread/read" => {
|
||||
if args
|
||||
.iter()
|
||||
.any(|arg| arg == "--require-lightweight-history")
|
||||
&& message.pointer("/params/includeTurns") != Some(&json!(false))
|
||||
{
|
||||
return Err("thread/read must not hydrate turns".into());
|
||||
}
|
||||
let turns = state
|
||||
.active_turn_id
|
||||
.as_ref()
|
||||
|
|
@ -1109,7 +1154,7 @@ fn run() -> Result<(), Box<dyn std::error::Error>> {
|
|||
.unwrap_or_default();
|
||||
send(json!({
|
||||
"id": id,
|
||||
"result": {"thread": {"id": state.thread_id, "turns": turns}}
|
||||
"result": {"thread": {"id": state.thread_id, "status": {"type": if state.active_turn_id.is_some() { "active" } else { "idle" }}, "turns": turns}}
|
||||
}))?;
|
||||
if fail_after_thread_read {
|
||||
return Err("configured failure after thread read".into());
|
||||
|
|
|
|||
|
|
@ -771,6 +771,14 @@ impl CodexProvider {
|
|||
observe: &mut dyn FnMut(ProviderStartupObservation) -> Result<(), LocalRunnerError>,
|
||||
) -> Result<Self, LocalRunnerError> {
|
||||
config.validate()?;
|
||||
if config.provider == "codex" {
|
||||
if let Some(home) = std::env::var_os("CODEX_HOME") {
|
||||
crate::codex_startup_trust::trust_startup_root(
|
||||
Path::new(&home),
|
||||
Path::new(&config.cwd),
|
||||
)?;
|
||||
}
|
||||
}
|
||||
if process_generation == 0 {
|
||||
return Err(LocalRunnerError::invalid(
|
||||
"Codex process generation must be positive",
|
||||
|
|
@ -834,12 +842,13 @@ impl CodexProvider {
|
|||
&environment_keys,
|
||||
)
|
||||
} else {
|
||||
SupervisedProcess::spawn_with_environment_keys(
|
||||
SupervisedProcess::spawn_in_directory_with_environment_keys(
|
||||
&config.command,
|
||||
&config.args,
|
||||
Duration::from_secs(2),
|
||||
CODEX_APP_SERVER_MAX_FRAME_BYTES,
|
||||
&environment_keys,
|
||||
Path::new(&config.cwd),
|
||||
)
|
||||
}
|
||||
})();
|
||||
|
|
@ -961,6 +970,9 @@ impl CodexProvider {
|
|||
}
|
||||
let method = if let Some(thread_id) = resume_thread_id {
|
||||
params_object.insert("threadId".to_owned(), json!(thread_id));
|
||||
if config.provider == "codex" {
|
||||
params_object.insert("excludeTurns".to_owned(), json!(true));
|
||||
}
|
||||
"thread/resume"
|
||||
} else {
|
||||
params_object.insert("experimentalRawEvents".to_owned(), json!(false));
|
||||
|
|
@ -1737,10 +1749,99 @@ impl CodexProvider {
|
|||
// It does prove the provider remained live after that terminal, so a
|
||||
// subsequent nonzero exit is a separate idle-session failure.
|
||||
self.completion_reconciliation_pending = false;
|
||||
self.request(
|
||||
if self.config.provider != "codex" {
|
||||
return self.request(
|
||||
"thread/read",
|
||||
json!({"threadId": self.thread_id, "includeTurns": true}),
|
||||
);
|
||||
}
|
||||
let mut snapshot = self.request(
|
||||
"thread/read",
|
||||
json!({"threadId": self.thread_id, "includeTurns": true}),
|
||||
)
|
||||
json!({"threadId": self.thread_id, "includeTurns": false}),
|
||||
)?;
|
||||
if snapshot.pointer("/thread/id").and_then(Value::as_str) != Some(self.thread_id.as_str()) {
|
||||
return Err(LocalRunnerError::invalid(
|
||||
"Codex thread/read returned a different thread",
|
||||
));
|
||||
}
|
||||
// Only metadata is needed to establish live authority. Never hydrate
|
||||
// message contents just to decide whether this thread has an active turn.
|
||||
match snapshot
|
||||
.pointer("/thread/status/type")
|
||||
.and_then(Value::as_str)
|
||||
{
|
||||
Some("idle") => {
|
||||
snapshot["thread"]["turns"] = json!([]);
|
||||
return Ok(snapshot);
|
||||
}
|
||||
Some("active") => {}
|
||||
_ => {
|
||||
return Err(LocalRunnerError::invalid(
|
||||
"codex_history_incomplete: unavailable thread status",
|
||||
))
|
||||
}
|
||||
};
|
||||
let mut cursor = Value::Null;
|
||||
let mut cursors = BTreeSet::new();
|
||||
let mut turns = BTreeMap::new();
|
||||
for _ in 0..10_000 {
|
||||
let page = self
|
||||
.request(
|
||||
"thread/turns/list",
|
||||
json!({
|
||||
"threadId": self.thread_id, "cursor": cursor, "limit": 100,
|
||||
"sortDirection": "desc", "itemsView": "notLoaded",
|
||||
}),
|
||||
)
|
||||
.map_err(|error| {
|
||||
LocalRunnerError::invalid(format!(
|
||||
"codex_history_read_failed: supported thread/turns/list is required: {error}"
|
||||
))
|
||||
})?;
|
||||
let data = page.get("data").and_then(Value::as_array).ok_or_else(|| {
|
||||
LocalRunnerError::invalid("codex_history_incomplete: turn page omitted data")
|
||||
})?;
|
||||
for turn in data {
|
||||
let id = bounded_provider_turn_id(turn.get("id").and_then(Value::as_str))?;
|
||||
if !matches!(
|
||||
turn.get("status").and_then(Value::as_str),
|
||||
Some("inProgress" | "completed" | "failed" | "interrupted" | "cancelled")
|
||||
) {
|
||||
return Err(LocalRunnerError::invalid(
|
||||
"codex_history_incomplete: invalid turn status",
|
||||
));
|
||||
}
|
||||
turns.insert(id, turn.clone());
|
||||
}
|
||||
let found_active = turns
|
||||
.values()
|
||||
.any(|turn| turn.get("status").and_then(Value::as_str) == Some("inProgress"));
|
||||
let next = page.get("nextCursor").cloned().unwrap_or(Value::Null);
|
||||
if next.is_null() || found_active {
|
||||
if !found_active {
|
||||
return Err(LocalRunnerError::invalid(
|
||||
"codex_history_incomplete: active thread has no active turn",
|
||||
));
|
||||
}
|
||||
snapshot["thread"]["turns"] = Value::Array(turns.into_values().collect());
|
||||
return Ok(snapshot);
|
||||
}
|
||||
let next_text = next
|
||||
.as_str()
|
||||
.filter(|value| !value.is_empty())
|
||||
.ok_or_else(|| {
|
||||
LocalRunnerError::invalid("codex_history_incomplete: invalid turn cursor")
|
||||
})?;
|
||||
if !cursors.insert(next_text.to_owned()) {
|
||||
return Err(LocalRunnerError::invalid(
|
||||
"codex_history_incomplete: repeated turn cursor",
|
||||
));
|
||||
}
|
||||
cursor = next;
|
||||
}
|
||||
Err(LocalRunnerError::invalid(
|
||||
"codex_history_incomplete: turn page limit exceeded",
|
||||
))
|
||||
}
|
||||
|
||||
pub fn resolve_runtime_request(
|
||||
|
|
@ -2338,6 +2439,21 @@ impl CodexProvider {
|
|||
if self.notification_identity_diagnostics > 32 {
|
||||
return Ok(None);
|
||||
}
|
||||
// Resume replays the cumulative usage of the last settled turn.
|
||||
// Keep its identity and baseline, but never bill its `last`
|
||||
// measurement to the newly attached run.
|
||||
if method == "thread/tokenUsage/updated"
|
||||
&& notification_thread_id(¶ms) == Some(self.thread_id.as_str())
|
||||
{
|
||||
return Ok(Some(CodexProviderEvent::Notification {
|
||||
method: "paperclip/resumeUsageSnapshot".to_owned(),
|
||||
params: json!({
|
||||
"threadId": self.thread_id,
|
||||
"turnId": notification_turn_id,
|
||||
"total": params.pointer("/tokenUsage/total"),
|
||||
}),
|
||||
}));
|
||||
}
|
||||
return Ok(Some(CodexProviderEvent::Notification {
|
||||
method: "warning".to_owned(),
|
||||
params: json!({
|
||||
|
|
|
|||
|
|
@ -0,0 +1,160 @@
|
|||
use crate::local_runner::LocalRunnerError;
|
||||
use std::fs;
|
||||
use std::io::Write;
|
||||
use std::path::{Path, PathBuf};
|
||||
use toml_edit::{value, DocumentMut};
|
||||
|
||||
// Resolve on the provider host. In particular, a Daytona path must never be
|
||||
// canonicalized against the controller's filesystem.
|
||||
fn execution_root(cwd: &Path) -> Result<PathBuf, LocalRunnerError> {
|
||||
let root = fs::canonicalize(cwd)
|
||||
.map_err(|_| LocalRunnerError::invalid("codex_startup_trust_root_unavailable"))?;
|
||||
// Resolve Git's indirection without running project hooks or Git commands.
|
||||
for ancestor in root.ancestors() {
|
||||
let git = ancestor.join(".git");
|
||||
if git.is_dir() {
|
||||
return Ok(ancestor.to_path_buf());
|
||||
}
|
||||
if git.is_file() {
|
||||
let pointer = fs::read_to_string(&git)
|
||||
.map_err(|_| LocalRunnerError::invalid("codex_startup_trust_git_unreadable"))?;
|
||||
let directory = pointer
|
||||
.trim()
|
||||
.strip_prefix("gitdir: ")
|
||||
.ok_or_else(|| LocalRunnerError::invalid("codex_startup_trust_git_invalid"))?;
|
||||
let directory = fs::canonicalize(ancestor.join(directory))
|
||||
.map_err(|_| LocalRunnerError::invalid("codex_startup_trust_git_unavailable"))?;
|
||||
let common_file = directory.join("commondir");
|
||||
if common_file.is_file() {
|
||||
let common = fs::read_to_string(common_file).map_err(|_| {
|
||||
LocalRunnerError::invalid("codex_startup_trust_common_unreadable")
|
||||
})?;
|
||||
let common = fs::canonicalize(directory.join(common.trim())).map_err(|_| {
|
||||
LocalRunnerError::invalid("codex_startup_trust_common_unavailable")
|
||||
})?;
|
||||
if common.file_name().is_some_and(|name| name == ".git") {
|
||||
return Ok(common.parent().expect(".git has a parent").to_path_buf());
|
||||
}
|
||||
}
|
||||
return Ok(ancestor.to_path_buf());
|
||||
}
|
||||
}
|
||||
Ok(root)
|
||||
}
|
||||
|
||||
pub(crate) fn trust_startup_root(home: &Path, cwd: &Path) -> Result<(), LocalRunnerError> {
|
||||
if !home.is_absolute() || !cwd.is_absolute() {
|
||||
return Err(LocalRunnerError::invalid(
|
||||
"codex_startup_trust_requires_absolute_paths",
|
||||
));
|
||||
}
|
||||
let root = execution_root(cwd)?;
|
||||
fs::create_dir_all(home)
|
||||
.map_err(|_| LocalRunnerError::invalid("codex_startup_trust_home_unavailable"))?;
|
||||
let path = home.join("config.toml");
|
||||
let existing = match fs::read_to_string(&path) {
|
||||
Ok(value) => value,
|
||||
Err(error) if error.kind() == std::io::ErrorKind::NotFound => String::new(),
|
||||
Err(_) => {
|
||||
return Err(LocalRunnerError::invalid(
|
||||
"codex_startup_trust_config_unreadable",
|
||||
))
|
||||
}
|
||||
};
|
||||
let mut document = existing
|
||||
.parse::<DocumentMut>()
|
||||
.map_err(|_| LocalRunnerError::invalid("codex_startup_trust_config_invalid"))?;
|
||||
let root_key = root
|
||||
.to_str()
|
||||
.ok_or_else(|| LocalRunnerError::invalid("codex_startup_trust_non_utf8_root"))?;
|
||||
if let Some(projects) = document.get("projects") {
|
||||
let projects = projects
|
||||
.as_table_like()
|
||||
.ok_or_else(|| LocalRunnerError::invalid("codex_startup_trust_invalid_projects"))?;
|
||||
if projects
|
||||
.get(root_key)
|
||||
.is_some_and(|project| project.as_table_like().is_none())
|
||||
{
|
||||
return Err(LocalRunnerError::invalid(
|
||||
"codex_startup_trust_invalid_project",
|
||||
));
|
||||
}
|
||||
}
|
||||
document["projects"][root_key]["trust_level"] = value("trusted");
|
||||
let temporary = home.join(format!("config.toml.{}.tmp", uuid::Uuid::new_v4()));
|
||||
let mut options = fs::OpenOptions::new();
|
||||
options.write(true).create_new(true);
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::fs::OpenOptionsExt;
|
||||
options.mode(0o600);
|
||||
}
|
||||
let result = (|| {
|
||||
let mut file = options
|
||||
.open(&temporary)
|
||||
.map_err(|_| LocalRunnerError::invalid("codex_startup_trust_write_failed"))?;
|
||||
file.write_all(document.to_string().as_bytes())
|
||||
.and_then(|_| file.sync_all())
|
||||
.map_err(|_| LocalRunnerError::invalid("codex_startup_trust_write_failed"))?;
|
||||
fs::rename(&temporary, path)
|
||||
.map_err(|_| LocalRunnerError::invalid("codex_startup_trust_commit_failed"))
|
||||
})();
|
||||
let _ = fs::remove_file(temporary);
|
||||
result
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
#[test]
|
||||
fn rejects_invalid_project_tables_without_overwriting_configuration() {
|
||||
let temp = std::env::temp_dir().join(format!("codex-trust-{}", uuid::Uuid::new_v4()));
|
||||
fs::create_dir_all(&temp).unwrap();
|
||||
let original = "projects = false\n";
|
||||
fs::write(temp.join("config.toml"), original).unwrap();
|
||||
assert!(trust_startup_root(&temp, &temp).is_err());
|
||||
assert_eq!(
|
||||
fs::read_to_string(temp.join("config.toml")).unwrap(),
|
||||
original
|
||||
);
|
||||
fs::remove_dir_all(temp).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn trusts_non_git_and_worktree_roots_without_replacing_configuration() {
|
||||
let temp = std::env::temp_dir().join(format!("codex-trust-{}", uuid::Uuid::new_v4()));
|
||||
let root = temp.join("project");
|
||||
let home = temp.join("home");
|
||||
fs::create_dir_all(&root).unwrap();
|
||||
fs::create_dir_all(&home).unwrap();
|
||||
fs::write(home.join("config.toml"), "model = 'test'\n# retain this comment\n[mcp_servers.test]\nurl = 'http://localhost/example'\n").unwrap();
|
||||
trust_startup_root(&home, &root).unwrap();
|
||||
trust_startup_root(&home, &root).unwrap();
|
||||
let config = fs::read_to_string(home.join("config.toml")).unwrap();
|
||||
assert!(config.contains("# retain this comment"));
|
||||
let parsed = config.parse::<DocumentMut>().unwrap();
|
||||
assert_eq!(parsed["model"].as_str(), Some("test"));
|
||||
assert_eq!(
|
||||
parsed["projects"][fs::canonicalize(&root).unwrap().to_str().unwrap()]["trust_level"]
|
||||
.as_str(),
|
||||
Some("trusted")
|
||||
);
|
||||
let git = root.join(".git");
|
||||
let metadata = git.join("worktrees/branch");
|
||||
fs::create_dir_all(&metadata).unwrap();
|
||||
let branch = temp.join("branch");
|
||||
fs::create_dir_all(&branch).unwrap();
|
||||
fs::write(
|
||||
branch.join(".git"),
|
||||
format!("gitdir: {}", metadata.display()),
|
||||
)
|
||||
.unwrap();
|
||||
fs::write(metadata.join("commondir"), "../..").unwrap();
|
||||
assert_eq!(
|
||||
execution_root(&branch).unwrap(),
|
||||
fs::canonicalize(&root).unwrap()
|
||||
);
|
||||
trust_startup_root(&home, &branch).unwrap();
|
||||
fs::remove_dir_all(temp).unwrap();
|
||||
}
|
||||
}
|
||||
|
|
@ -10,6 +10,7 @@ pub mod acpx_sidecar_transport;
|
|||
pub mod aws_agentcore_provider;
|
||||
pub mod claude_managed_provider;
|
||||
pub mod codex_provider;
|
||||
mod codex_startup_trust;
|
||||
pub mod durable;
|
||||
pub mod fake_harness;
|
||||
pub mod generated_acpx_sidecar_contract;
|
||||
|
|
|
|||
|
|
@ -664,6 +664,26 @@ impl SupervisedProcess {
|
|||
max_line_bytes,
|
||||
additional_environment_keys,
|
||||
None,
|
||||
None,
|
||||
)
|
||||
}
|
||||
|
||||
pub fn spawn_in_directory_with_environment_keys(
|
||||
program: &Path,
|
||||
args: &[String],
|
||||
shutdown_grace: Duration,
|
||||
max_line_bytes: usize,
|
||||
additional_environment_keys: &[&str],
|
||||
cwd: &Path,
|
||||
) -> Result<Self, LocalRunnerError> {
|
||||
Self::spawn_command(
|
||||
program,
|
||||
args,
|
||||
shutdown_grace,
|
||||
max_line_bytes,
|
||||
additional_environment_keys,
|
||||
None,
|
||||
Some(cwd),
|
||||
)
|
||||
}
|
||||
|
||||
|
|
@ -685,6 +705,7 @@ impl SupervisedProcess {
|
|||
launch
|
||||
.inherit_runtime_executable
|
||||
.then_some(inherited.program.as_path()),
|
||||
None,
|
||||
);
|
||||
#[cfg(target_os = "macos")]
|
||||
if let Ok(process) = result.as_mut() {
|
||||
|
|
@ -715,8 +736,12 @@ impl SupervisedProcess {
|
|||
max_line_bytes: usize,
|
||||
additional_environment_keys: &[&str],
|
||||
verified_runtime_executable: Option<&Path>,
|
||||
cwd: Option<&Path>,
|
||||
) -> Result<Self, LocalRunnerError> {
|
||||
let mut command = Command::new(program);
|
||||
if let Some(cwd) = cwd {
|
||||
command.current_dir(cwd);
|
||||
}
|
||||
command
|
||||
.args(args)
|
||||
.env_clear()
|
||||
|
|
|
|||
|
|
@ -652,6 +652,19 @@ pub fn normalize_codex_notification(method: &str, params: &Value) -> Vec<Normali
|
|||
}),
|
||||
);
|
||||
}
|
||||
"paperclip/resumeUsageSnapshot" => push(
|
||||
&mut events,
|
||||
"harness.diagnostic",
|
||||
EventPriority::P0,
|
||||
json!({
|
||||
"code": "codex_resume_usage_snapshot",
|
||||
"method": "thread/tokenUsage/updated",
|
||||
"classification": "resume_usage_snapshot",
|
||||
"receivedThreadId": params.get("threadId"),
|
||||
"receivedTurnId": params.get("turnId"),
|
||||
"cumulative": measurement(params.get("total").unwrap_or(&Value::Null)),
|
||||
}),
|
||||
),
|
||||
"thread/tokenUsage/updated" => {
|
||||
let cumulative = params
|
||||
.get("tokenUsage")
|
||||
|
|
|
|||
|
|
@ -5944,3 +5944,151 @@ fn durable_descendant_lineage_survives_capacity_and_provider_restoration() {
|
|||
restored.shutdown().unwrap();
|
||||
fs::remove_dir_all(directory).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resume_usage_is_a_historical_diagnostic_not_a_warning_or_charge() {
|
||||
let directory = temporary_directory("resume-usage-snapshot");
|
||||
let config = provider_config(
|
||||
&directory,
|
||||
&["--durable-turn-ids", "--resume-usage-snapshot"],
|
||||
);
|
||||
let runner_config = durable_config(&directory);
|
||||
let mut first = CodexCommandExecutor::with_runner_config(&directory, &runner_config);
|
||||
first
|
||||
.execute(&command(
|
||||
"prepare",
|
||||
1,
|
||||
"run.prepare",
|
||||
json!({
|
||||
"provider": config,
|
||||
"authorizedTools": task_context_tool_set(),
|
||||
"completionContract": {
|
||||
"revision": "sha256:settled-attach-contract",
|
||||
"criterionIds": ["criterion_settled_attach"]
|
||||
},
|
||||
}),
|
||||
))
|
||||
.expect("prepare the durable provider");
|
||||
first
|
||||
.execute(&command("open", 2, "session.open", json!({})))
|
||||
.expect("open the durable provider session");
|
||||
first
|
||||
.execute(&command(
|
||||
"turn",
|
||||
3,
|
||||
"turn.start",
|
||||
json!({"text": "Settle before rotating run authority."}),
|
||||
))
|
||||
.expect("start the provider turn");
|
||||
|
||||
let mut saw_terminal = false;
|
||||
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(5);
|
||||
while std::time::Instant::now() < deadline {
|
||||
let events = poll_and_ack(&mut first).expect("drain the settled provider turn");
|
||||
saw_terminal |= events
|
||||
.iter()
|
||||
.any(|event| event.event_type == "run.terminal");
|
||||
if saw_terminal && events.is_empty() {
|
||||
break;
|
||||
}
|
||||
if events.is_empty() {
|
||||
std::thread::sleep(std::time::Duration::from_millis(1));
|
||||
}
|
||||
}
|
||||
assert!(saw_terminal, "the first run must settle before attachment");
|
||||
first.shutdown().expect("stop the first provider process");
|
||||
drop(first);
|
||||
|
||||
let mut resumed = CodexCommandExecutor::with_runner_config(&directory, &runner_config);
|
||||
let mut events = Vec::new();
|
||||
for _ in 0..50 {
|
||||
events.extend(poll_and_ack(&mut resumed).expect("poll resume snapshots"));
|
||||
if events
|
||||
.iter()
|
||||
.filter(|event| event.payload["code"] == "codex_resume_usage_snapshot")
|
||||
.count()
|
||||
== 2
|
||||
{
|
||||
break;
|
||||
}
|
||||
std::thread::sleep(std::time::Duration::from_millis(10));
|
||||
}
|
||||
let snapshots: Vec<_> = events
|
||||
.iter()
|
||||
.filter(|event| event.payload["code"] == "codex_resume_usage_snapshot")
|
||||
.collect();
|
||||
assert_eq!(snapshots.len(), 2);
|
||||
assert_eq!(snapshots[0].payload["receivedTurnId"], "provider-turn-1");
|
||||
assert_eq!(snapshots[0].payload["cumulative"]["inputTokens"], 100);
|
||||
assert!(!events.iter().any(|event| matches!(
|
||||
event.event_type.as_str(),
|
||||
"provider.notice.recorded" | "usage.reported"
|
||||
)));
|
||||
resumed.shutdown().unwrap();
|
||||
fs::remove_dir_all(directory).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lightweight_history_pages_metadata_only_when_state_is_active() {
|
||||
let directory = temporary_directory("lightweight-history");
|
||||
let config = provider_config(
|
||||
&directory,
|
||||
&[
|
||||
"--hold-turn",
|
||||
"--require-lightweight-history",
|
||||
"--paginated-history",
|
||||
],
|
||||
);
|
||||
let mut provider = CodexProvider::start(&config, None).unwrap();
|
||||
let thread_id = provider.thread_id().to_owned();
|
||||
assert_eq!(
|
||||
provider.read_thread().unwrap()["thread"]["turns"],
|
||||
json!([])
|
||||
);
|
||||
assert!(!fs::read_to_string(directory.join("calls.log"))
|
||||
.unwrap()
|
||||
.contains("thread/turns/list"));
|
||||
provider.start_turn("Keep working", &config.cwd).unwrap();
|
||||
let active = provider.active_provider_turn_id().unwrap().to_owned();
|
||||
let state = provider.read_thread().unwrap();
|
||||
assert_eq!(state["thread"]["turns"][0]["id"], active);
|
||||
assert_eq!(state["thread"]["turns"][0]["items"], json!([]));
|
||||
assert_eq!(
|
||||
fs::read_to_string(directory.join("calls.log"))
|
||||
.unwrap()
|
||||
.matches("thread/turns/list")
|
||||
.count(),
|
||||
2
|
||||
);
|
||||
provider.shutdown().unwrap();
|
||||
let mut resumed = CodexProvider::start(&config, Some(&thread_id)).unwrap();
|
||||
assert_eq!(
|
||||
resumed.read_thread().unwrap()["thread"]["turns"][0]["id"],
|
||||
active
|
||||
);
|
||||
resumed.shutdown().unwrap();
|
||||
fs::remove_dir_all(directory).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lightweight_history_repeated_cursor_is_not_idle_evidence() {
|
||||
let directory = temporary_directory("repeated-history-cursor");
|
||||
let config = provider_config(
|
||||
&directory,
|
||||
&[
|
||||
"--hold-turn",
|
||||
"--require-lightweight-history",
|
||||
"--repeat-history-cursor",
|
||||
],
|
||||
);
|
||||
let mut provider = CodexProvider::start(&config, None).unwrap();
|
||||
provider.start_turn("Keep working", &config.cwd).unwrap();
|
||||
assert!(provider
|
||||
.read_thread()
|
||||
.unwrap_err()
|
||||
.to_string()
|
||||
.contains("repeated turn cursor"));
|
||||
assert!(provider.active_provider_turn_id().is_some());
|
||||
provider.shutdown().unwrap();
|
||||
fs::remove_dir_all(directory).unwrap();
|
||||
}
|
||||
|
|
|
|||
|
|
@ -97,6 +97,32 @@ const driver: HarnessDriver = {
|
|||
};
|
||||
|
||||
describe("HarnessDriverBackend", () => {
|
||||
it("retains Codex accounting and startup state through a serialized native checkpoint", async () => {
|
||||
const fields = { workingDirectory: "/workspace/selected", codexUsageBaseline: {
|
||||
baseline: { inputTokens: 100, outputTokens: 20 },
|
||||
latest: { inputTokens: 150, outputTokens: 35 },
|
||||
} };
|
||||
class RecoveryFieldsSession extends FakeHarnessSession {
|
||||
override async snapshot(): Promise<PersistedHarnessSession> {
|
||||
return { ...(await super.snapshot()), ...fields, semanticResult: undefined, activeTurnId: null };
|
||||
}
|
||||
}
|
||||
const original = new HarnessDriverBackend({ ...driver, openSession: async () => new RecoveryFieldsSession() });
|
||||
const session = await original.openSession({ identity: {
|
||||
runId: "run-1", sessionId: "session-1", companyId: "company-1", issueId: "issue-1", agentId: "agent-1",
|
||||
}, workingDirectory: fields.workingDirectory });
|
||||
const checkpoint = JSON.parse(JSON.stringify(await session.snapshot()));
|
||||
expect(checkpoint).toMatchObject(fields);
|
||||
const recover = vi.fn(async (_snapshot: PersistedHarnessSession) => ({ recovered: true, session: new RecoveryFieldsSession() }));
|
||||
const restarted = new HarnessDriverBackend({ ...driver, recoverSession: recover });
|
||||
const restored = await restarted.recoverSession(checkpoint, { signal: new AbortController().signal });
|
||||
expect(restored.recovered).toBe(true);
|
||||
expect(recover.mock.calls[0]![0]).toMatchObject(fields);
|
||||
expect(await restored.session!.snapshot()).toMatchObject(fields);
|
||||
checkpoint.codexUsageBaseline.latest.inputTokens = 999;
|
||||
expect(recover.mock.calls[0]![0].codexUsageBaseline!.latest.inputTokens).toBe(150);
|
||||
});
|
||||
|
||||
it("rejects and closes a provider session without a durable provider identity", async () => {
|
||||
let closed = false;
|
||||
class MissingProviderIdentitySession extends FakeHarnessSession {
|
||||
|
|
|
|||
|
|
@ -120,6 +120,12 @@ export class HarnessDriverBackend implements NativeSessionBackend {
|
|||
...(snapshot.providerIdentity === undefined
|
||||
? {}
|
||||
: { providerIdentity: structuredClone(snapshot.providerIdentity) }),
|
||||
...(snapshot.workingDirectory === undefined
|
||||
? {}
|
||||
: { workingDirectory: snapshot.workingDirectory }),
|
||||
...(snapshot.codexUsageBaseline === undefined
|
||||
? {}
|
||||
: { codexUsageBaseline: structuredClone(snapshot.codexUsageBaseline) }),
|
||||
...(snapshot.providerRecoveryPolicy === undefined
|
||||
? {}
|
||||
: { providerRecoveryPolicy: snapshot.providerRecoveryPolicy }),
|
||||
|
|
@ -774,6 +780,12 @@ class HarnessNativeSession implements NativeSession {
|
|||
...(snapshot.providerIdentity === undefined
|
||||
? {}
|
||||
: { providerIdentity: structuredClone(snapshot.providerIdentity) }),
|
||||
...(snapshot.workingDirectory === undefined
|
||||
? {}
|
||||
: { workingDirectory: snapshot.workingDirectory }),
|
||||
...(snapshot.codexUsageBaseline === undefined
|
||||
? {}
|
||||
: { codexUsageBaseline: structuredClone(snapshot.codexUsageBaseline) }),
|
||||
...(snapshot.providerRecoveryPolicy === undefined
|
||||
? {}
|
||||
: { providerRecoveryPolicy: snapshot.providerRecoveryPolicy }),
|
||||
|
|
|
|||
|
|
@ -461,6 +461,8 @@ export type PersistedHarnessProviderIdentity = AcpxSessionIdentity;
|
|||
|
||||
export interface PersistedHarnessSession {
|
||||
driverKind: string;
|
||||
/** Execution-host startup root, revalidated before a cold provider launch. */
|
||||
workingDirectory?: string;
|
||||
driverSessionId: string;
|
||||
providerSessionId?: string | null;
|
||||
runId?: string;
|
||||
|
|
@ -468,6 +470,7 @@ export interface PersistedHarnessSession {
|
|||
activeTurnId?: string | null;
|
||||
semanticResult?: PersistedHarnessSemanticResult | null;
|
||||
terminalTurns?: PersistedHarnessTurnTerminal[];
|
||||
codexUsageBaseline?: { baseline: Record<string, number>; latest: Record<string, number> };
|
||||
/** A result-less terminal task may spend this fail-closed one-shot recovery allowance. */
|
||||
dispositionOnlyRecoveryConsumed?: boolean;
|
||||
/** Exact accepted provider turn that spent the disposition-only allowance. */
|
||||
|
|
|
|||
|
|
@ -17,6 +17,7 @@ import type {
|
|||
HarnessThreadLineageEntry,
|
||||
NativeRuntimeContextCapabilities,
|
||||
PersistedHarnessProviderIdentity,
|
||||
PersistedHarnessSession,
|
||||
PersistedHarnessTurnTerminal,
|
||||
} from "./harness-driver.js";
|
||||
|
||||
|
|
@ -51,6 +52,8 @@ export interface PersistedNativeSession {
|
|||
providerSessionId?: string | null;
|
||||
/** Tagged provider-owned identity required for safe driver recovery. */
|
||||
providerIdentity?: PersistedHarnessProviderIdentity;
|
||||
workingDirectory?: string;
|
||||
codexUsageBaseline?: PersistedHarnessSession["codexUsageBaseline"];
|
||||
providerRecoveryPolicy?:
|
||||
| "same_session_only"
|
||||
| "allow_replacement_after_governed_wait"
|
||||
|
|
|
|||
|
|
@ -338,6 +338,7 @@ interface PendingRequest {
|
|||
}
|
||||
|
||||
export interface ProcessCodexTransportOptions {
|
||||
workingDirectory?: string;
|
||||
command?: string;
|
||||
args?: string[];
|
||||
environment?: NodeJS.ProcessEnv;
|
||||
|
|
@ -431,6 +432,7 @@ export class ProcessCodexAppServerTransport implements CodexAppServerTransport {
|
|||
options.command ?? "codex",
|
||||
options.args ?? ["app-server"],
|
||||
{
|
||||
cwd: options.workingDirectory,
|
||||
env: options.environment ?? createSanitizedCodexEnvironment(),
|
||||
stdio: "pipe",
|
||||
detached: this.#processGroup,
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
import { trustCodexStartupRoot } from "./codex-startup-trust.js";
|
||||
import { readCodexThreadState, readCodexTurnMetadata } from "./codex-history.js";
|
||||
import { codexExecutableReadOnlyRoots } from "./codex-security-config.js";
|
||||
import { resolve } from "node:path";
|
||||
|
||||
|
|
@ -36,8 +38,6 @@ import {
|
|||
validateCodexWorkingDirectory as validateWorkingDirectory,
|
||||
} from "./codex-boundaries.js";
|
||||
import {
|
||||
CODEX_PLANNING_PERMISSION_PROFILE as PLANNING_PERMISSION_PROFILE,
|
||||
CODEX_SKILLLESS_PERMISSION_PROFILE as SKILLLESS_PERMISSION_PROFILE,
|
||||
codexCommandEnvironment,
|
||||
createIsolatedCodexAppServerArgs,
|
||||
codexNetworkAccess,
|
||||
|
|
@ -255,7 +255,7 @@ export class CodexAppServerDriver implements HarnessDriver {
|
|||
this.#options.environment,
|
||||
this.#options.workingDirectoryAuthority,
|
||||
);
|
||||
const transport = this.#transport();
|
||||
const transport = this.#transport({ workingDirectory });
|
||||
const cancellation = bootstrapCancellation(transport, input.signal);
|
||||
try {
|
||||
await cancellation.wait(this.#persistProcessOwnership(transport));
|
||||
|
|
@ -359,6 +359,9 @@ export class CodexAppServerDriver implements HarnessDriver {
|
|||
};
|
||||
}
|
||||
const transport = this.#transport({
|
||||
workingDirectory: snapshot.workingDirectory
|
||||
? validateWorkingDirectory(snapshot.workingDirectory, this.#options.environment, this.#options.workingDirectoryAuthority)
|
||||
: undefined,
|
||||
providerRecoveryPolicy: snapshot.providerRecoveryPolicy,
|
||||
persistedSession: {
|
||||
driverSessionId: snapshot.driverSessionId,
|
||||
|
|
@ -372,13 +375,11 @@ export class CodexAppServerDriver implements HarnessDriver {
|
|||
await cancellation.wait(this.#persistProcessOwnership(transport));
|
||||
const initialize = await cancellation.wait(this.#initialize(transport));
|
||||
const existing = await cancellation.wait(
|
||||
transport.request("thread/read", {
|
||||
threadId: snapshot.driverSessionId,
|
||||
includeTurns: true,
|
||||
}),
|
||||
readCodexThreadState(transport, snapshot.driverSessionId),
|
||||
);
|
||||
await cancellation.wait(this.#persistProcessOwnership(transport));
|
||||
const existingThread = record(existing.thread);
|
||||
existingThread.turns = await cancellation.wait(readCodexTurnMetadata(transport, snapshot.driverSessionId));
|
||||
if (text(existingThread.id) !== snapshot.driverSessionId) {
|
||||
await cancellation.wait(cancellation.close());
|
||||
return {
|
||||
|
|
@ -393,6 +394,7 @@ export class CodexAppServerDriver implements HarnessDriver {
|
|||
);
|
||||
const response = await cancellation.wait(
|
||||
transport.request("thread/resume", {
|
||||
excludeTurns: true,
|
||||
threadId: snapshot.driverSessionId,
|
||||
...createSecuredCodexThreadParams(
|
||||
workingDirectory,
|
||||
|
|
@ -600,6 +602,7 @@ export class CodexAppServerDriver implements HarnessDriver {
|
|||
activeTurnId: recoveredActiveTurnId,
|
||||
semanticResult: snapshot.semanticResult ?? null,
|
||||
terminalTurns: snapshot.terminalTurns ?? [],
|
||||
codexUsageBaseline: snapshot.codexUsageBaseline,
|
||||
dispositionOnlyRecoveryConsumed,
|
||||
dispositionOnlyRecoveryTurnId,
|
||||
stalePendingRuntimeRequests: snapshot.pendingRuntimeRequests ?? [],
|
||||
|
|
@ -633,6 +636,7 @@ export class CodexAppServerDriver implements HarnessDriver {
|
|||
}
|
||||
|
||||
#transport(context?: {
|
||||
workingDirectory?: string;
|
||||
providerRecoveryPolicy?: PersistedHarnessSession["providerRecoveryPolicy"];
|
||||
persistedSession?: Pick<
|
||||
PersistedHarnessSession,
|
||||
|
|
@ -642,9 +646,14 @@ export class CodexAppServerDriver implements HarnessDriver {
|
|||
| "activeTurnId"
|
||||
>;
|
||||
}): CodexAppServerTransport {
|
||||
const workingDirectory = context?.workingDirectory ?? this.#options.environment?.PAPERCLIP_WORKSPACE_CWD;
|
||||
if (!this.#options.transportFactory && this.#options.environment?.CODEX_HOME && workingDirectory) {
|
||||
trustCodexStartupRoot(this.#options.environment.CODEX_HOME, workingDirectory);
|
||||
}
|
||||
return (
|
||||
this.#options.transportFactory?.(context) ??
|
||||
new ProcessCodexAppServerTransport({
|
||||
workingDirectory,
|
||||
args: createIsolatedCodexAppServerArgs(this.#options.environment, codexExecutableReadOnlyRoots(this.#options.environment ?? process.env)),
|
||||
environment: createSanitizedCodexEnvironment(this.#options.environment),
|
||||
onDiagnostic: this.#options.onDiagnostic,
|
||||
|
|
@ -790,9 +799,7 @@ export class CodexAppServerDriver implements HarnessDriver {
|
|||
const permissionProfileId = text(activePermissionProfile.id);
|
||||
const requestedMode = this.#options.requestedCollaborationMode ?? "default";
|
||||
const requiredPermissionProfile =
|
||||
requestedMode === "plan"
|
||||
? PLANNING_PERMISSION_PROFILE
|
||||
: SKILLLESS_PERMISSION_PROFILE;
|
||||
text(createSecuredCodexThreadParams(workingDirectory, requestedMode, true, false, this.#options.environment).permissions);
|
||||
if (
|
||||
permissionProfileId.length > 0 &&
|
||||
permissionProfileId !== requiredPermissionProfile
|
||||
|
|
@ -891,6 +898,7 @@ export class CodexAppServerDriver implements HarnessDriver {
|
|||
activeTurnId?: string | null;
|
||||
semanticResult?: PersistedHarnessSemanticResult | null;
|
||||
terminalTurns?: PersistedHarnessTurnTerminal[];
|
||||
codexUsageBaseline?: PersistedHarnessSession["codexUsageBaseline"];
|
||||
dispositionOnlyRecoveryConsumed?: boolean;
|
||||
dispositionOnlyRecoveryTurnId?: string | null;
|
||||
stalePendingRuntimeRequests?: HarnessRuntimeRequest[];
|
||||
|
|
|
|||
|
|
@ -370,6 +370,7 @@ describe("Codex app-server Codex driver", () => {
|
|||
|
||||
expect(recovered.recovered).toBe(true);
|
||||
expect(transportFactory).toHaveBeenCalledWith({
|
||||
workingDirectory: snapshot.workingDirectory,
|
||||
providerRecoveryPolicy: snapshot.providerRecoveryPolicy,
|
||||
persistedSession: {
|
||||
driverSessionId: snapshot.driverSessionId,
|
||||
|
|
|
|||
|
|
@ -91,7 +91,7 @@ describe("Codex app-server Codex driver", () => {
|
|||
second.readResponse = { thread: { id: "thread-1", sessionId: "provider-session-1", cwd: WORKSPACE, turns } };
|
||||
await original.close({ reason: "controller lost" });
|
||||
await expect(driver.recoverSession?.(snapshot)).resolves.toEqual({
|
||||
recovered: false, reason: "provider exposed ambiguous autonomous goal turn history",
|
||||
recovered: false, reason: expect.stringMatching(/ambiguous autonomous goal turn history|codex_history_incomplete/),
|
||||
});
|
||||
expect(second.calls.some((call) => call.method === "turn/start" || call.method === "thread/goal/set")).toBe(false);
|
||||
});
|
||||
|
|
@ -254,10 +254,13 @@ describe("Codex app-server Codex driver", () => {
|
|||
expect(second.calls.map((call) => call.method)).toEqual([
|
||||
"initialize",
|
||||
"thread/read",
|
||||
"thread/turns/list",
|
||||
"thread/resume",
|
||||
"thread/goal/get",
|
||||
"thread/read",
|
||||
"thread/turns/list",
|
||||
"thread/read",
|
||||
"thread/turns/list",
|
||||
]);
|
||||
expect((await recovery?.session?.snapshot())?.activeTurnId).toBe("turn-1");
|
||||
});
|
||||
|
|
@ -845,23 +848,10 @@ describe("Codex app-server Codex driver", () => {
|
|||
dispositionOnlyRecoveryTurnId:
|
||||
testCase.dispositionOnlyRecoveryTurnId,
|
||||
});
|
||||
expect(recovery).toMatchObject({ recovered: true });
|
||||
await expect(recovery!.session!.snapshot()).resolves.toMatchObject({
|
||||
activeTurnId: null,
|
||||
dispositionOnlyRecoveryConsumed: true,
|
||||
dispositionOnlyRecoveryTurnId:
|
||||
testCase.dispositionOnlyRecoveryTurnId,
|
||||
});
|
||||
await expect(recovery!.session!.startTurn({
|
||||
message: {
|
||||
role: "user",
|
||||
text: "Do not repeat the task while provider history is unknown.",
|
||||
},
|
||||
})).rejects.toThrow("session cannot start another turn");
|
||||
expect(recovery).toMatchObject({ recovered: false, reason: expect.stringContaining("codex_history_incomplete") });
|
||||
expect(
|
||||
second.calls.filter((call) => call.method === "turn/start"),
|
||||
).toHaveLength(0);
|
||||
await recovery!.session!.close({ reason: "test complete" });
|
||||
},
|
||||
);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -156,9 +156,9 @@ export class FakeCodexTransport implements CodexAppServerTransport {
|
|||
cwd: WORKSPACE,
|
||||
turns: [],
|
||||
activePermissionProfile: {
|
||||
id: planMode
|
||||
id: params.permissions ?? (planMode
|
||||
? "paperclip-runner-workspace-read-only"
|
||||
: "paperclip-runner-workspace-only",
|
||||
: "paperclip-runner-workspace-only"),
|
||||
},
|
||||
},
|
||||
model: "gpt-test",
|
||||
|
|
@ -199,8 +199,18 @@ export class FakeCodexTransport implements CodexAppServerTransport {
|
|||
this.goalState = null;
|
||||
return {};
|
||||
}
|
||||
if (method === "thread/turns/list") {
|
||||
const snapshot = this.readResponse ?? { thread: { turns: [{ id: "turn-1", status: "inProgress", items: [] }] } };
|
||||
const turns = (snapshot.thread as Record<string, unknown>).turns;
|
||||
return { data: Array.isArray(turns) ? turns.map(turn => ({ ...turn, items: [], itemsView: "notLoaded" })) : turns, nextCursor: null };
|
||||
}
|
||||
if (method === "thread/items/list") {
|
||||
const turns = ((this.readResponse?.thread as Record<string, unknown> | undefined)?.turns ?? []) as Array<Record<string, unknown>>;
|
||||
const turn = turns.find(value => value.id === params.turnId);
|
||||
return { data: ((turn?.items ?? []) as Array<Record<string, unknown>>).map(item => ({ turnId: params.turnId, item })), nextCursor: null };
|
||||
}
|
||||
if (method === "thread/read") {
|
||||
return (
|
||||
return structuredClone(
|
||||
this.readResponse ?? {
|
||||
thread: {
|
||||
id: this.threadId,
|
||||
|
|
|
|||
|
|
@ -155,9 +155,9 @@ class FakeCodexTransport implements CodexAppServerTransport {
|
|||
cwd: TEST_WORKING_DIRECTORY,
|
||||
turns: [],
|
||||
activePermissionProfile: {
|
||||
id: planMode
|
||||
id: params.permissions ?? (planMode
|
||||
? "paperclip-runner-workspace-read-only"
|
||||
: "paperclip-runner-workspace-only",
|
||||
: "paperclip-runner-workspace-only"),
|
||||
},
|
||||
},
|
||||
model: "gpt-test",
|
||||
|
|
@ -198,8 +198,18 @@ class FakeCodexTransport implements CodexAppServerTransport {
|
|||
this.goalState = null;
|
||||
return {};
|
||||
}
|
||||
if (method === "thread/turns/list") {
|
||||
const snapshot = this.readResponse ?? { thread: { turns: [{ id: "turn-1", status: "inProgress", items: [] }] } };
|
||||
const turns = (snapshot.thread as Record<string, unknown>).turns;
|
||||
return { data: Array.isArray(turns) ? turns.map(turn => ({ ...turn, items: [], itemsView: "notLoaded" })) : turns, nextCursor: null };
|
||||
}
|
||||
if (method === "thread/items/list") {
|
||||
const turns = ((this.readResponse?.thread as Record<string, unknown> | undefined)?.turns ?? []) as Array<Record<string, unknown>>;
|
||||
const turn = turns.find(value => value.id === params.turnId);
|
||||
return { data: ((turn?.items ?? []) as Array<Record<string, unknown>>).map(item => ({ turnId: params.turnId, item })), nextCursor: null };
|
||||
}
|
||||
if (method === "thread/read") {
|
||||
return (
|
||||
return structuredClone(
|
||||
this.readResponse ?? {
|
||||
thread: {
|
||||
id: this.threadId,
|
||||
|
|
@ -3370,10 +3380,13 @@ describe("Codex app-server Codex driver", () => {
|
|||
expect(second.calls.map((call) => call.method)).toEqual([
|
||||
"initialize",
|
||||
"thread/read",
|
||||
"thread/turns/list",
|
||||
"thread/resume",
|
||||
"thread/goal/get",
|
||||
"thread/read",
|
||||
"thread/turns/list",
|
||||
"thread/read",
|
||||
"thread/turns/list",
|
||||
]);
|
||||
expect((await recovery?.session?.snapshot())?.activeTurnId).toBe("turn-1");
|
||||
});
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
import { readCodexThreadState, readCodexTurnMetadata, readCodexTurnItems } from "./codex-history.js";
|
||||
import { codexRunUsage } from "./codex-usage-baseline.js";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { NativeProviderTerminalFailure } from "../../contracts/native-session-backend.js";
|
||||
|
||||
|
|
@ -105,6 +107,10 @@ export class CodexHarnessSession
|
|||
this.pendingRuntimeRequestMap.clear();
|
||||
this.eventQueue.clear();
|
||||
}
|
||||
if (this.codexUsageBaseline && input.runId !== this.runId) {
|
||||
this.codexUsageBaseline = { baseline: { ...this.codexUsageBaseline.latest }, latest: { ...this.codexUsageBaseline.latest } };
|
||||
this.usageSnapshot = codexRunUsage(this.codexUsageBaseline);
|
||||
}
|
||||
this.runId = input.runId;
|
||||
this.result = null;
|
||||
this.resultFingerprint = null;
|
||||
|
|
@ -202,9 +208,9 @@ export class CodexHarnessSession
|
|||
threadId: this.opened.threadId,
|
||||
cwd: this.opened.context.workingDirectory,
|
||||
permissions:
|
||||
requestedMode === "plan"
|
||||
text(record(record(this.opened.context.sandbox).permissionProfile).id) || (requestedMode === "plan"
|
||||
? PLANNING_PERMISSION_PROFILE
|
||||
: SKILLLESS_PERMISSION_PROFILE,
|
||||
: SKILLLESS_PERMISSION_PROFILE),
|
||||
runtimeWorkspaceRoots: [this.opened.context.workingDirectory],
|
||||
...(this.opened.collaborationMode === null
|
||||
? {}
|
||||
|
|
@ -615,10 +621,7 @@ export class CodexHarnessSession
|
|||
this.assertProtocolIntegrity();
|
||||
this.requireCapability("read");
|
||||
try {
|
||||
const snapshot = await this.transport.request("thread/read", {
|
||||
threadId: this.opened.threadId,
|
||||
includeTurns: true,
|
||||
});
|
||||
const snapshot = await readCodexThreadState(this.transport, this.opened.threadId);
|
||||
this.assertProtocolIntegrity();
|
||||
return snapshot;
|
||||
} catch (error) {
|
||||
|
|
@ -646,7 +649,15 @@ export class CodexHarnessSession
|
|||
"thread/read returned a different provider session",
|
||||
);
|
||||
}
|
||||
const turns = Array.isArray(thread.turns) ? thread.turns.map(record) : [];
|
||||
const turns = await readCodexTurnMetadata(this.transport, this.opened.threadId);
|
||||
thread.turns = turns;
|
||||
for (const turn of turns) {
|
||||
if ((text(turn.id) === this.activeTurnId || this.terminalTurns.has(text(turn.id)))
|
||||
&& text(turn.status) !== "inProgress") {
|
||||
turn.items = await readCodexTurnItems(this.transport, this.opened.threadId, text(turn.id));
|
||||
turn.itemsView = "full";
|
||||
}
|
||||
}
|
||||
const reconciledUsage = boundedPayload(
|
||||
record(thread.tokenUsage ?? snapshot.tokenUsage),
|
||||
);
|
||||
|
|
@ -730,6 +741,7 @@ export class CodexHarnessSession
|
|||
this.assertProtocolIntegrity();
|
||||
return {
|
||||
driverKind: this.driverKind,
|
||||
workingDirectory: this.opened.context.workingDirectory,
|
||||
driverSessionId: this.opened.threadId,
|
||||
providerSessionId: this.opened.providerSessionId,
|
||||
...(this.opened.providerIdentity === undefined
|
||||
|
|
@ -749,6 +761,7 @@ export class CodexHarnessSession
|
|||
callId: this.resultCallId,
|
||||
turnId: this.resultTurnId,
|
||||
},
|
||||
...(this.codexUsageBaseline ? { codexUsageBaseline: structuredClone(this.codexUsageBaseline) } : {}),
|
||||
terminalTurns: [...this.terminalTurns].map(([turnId, fingerprint]) => ({
|
||||
turnId,
|
||||
fingerprint,
|
||||
|
|
|
|||
|
|
@ -0,0 +1,112 @@
|
|||
import { describe, it, expect, vi } from "vitest";
|
||||
import {
|
||||
readCodexThreadState,
|
||||
readCodexTurnMetadata,
|
||||
readCodexTurnItems,
|
||||
} from "./codex-history.js";
|
||||
|
||||
describe("Codex paginated history", () => {
|
||||
it("reads state without hydrating history", async () => {
|
||||
const request = vi.fn().mockResolvedValue({ thread: { id: "thread" } });
|
||||
await readCodexThreadState({ request }, "thread");
|
||||
expect(request).toHaveBeenCalledExactlyOnceWith("thread/read", {
|
||||
threadId: "thread",
|
||||
includeTurns: false,
|
||||
});
|
||||
});
|
||||
it("follows empty pages and preserves ordering while updating duplicate turn anchors", async () => {
|
||||
const request = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({
|
||||
data: [{ id: "one", status: "inProgress" }],
|
||||
nextCursor: "a",
|
||||
})
|
||||
.mockResolvedValueOnce({ data: [], nextCursor: "b" })
|
||||
.mockResolvedValueOnce({
|
||||
data: [
|
||||
{ id: "one", status: "completed" },
|
||||
{ id: "two", status: "inProgress" },
|
||||
],
|
||||
nextCursor: null,
|
||||
});
|
||||
expect(await readCodexTurnMetadata({ request }, "thread")).toEqual([
|
||||
{ id: "one", status: "completed" },
|
||||
{ id: "two", status: "inProgress" },
|
||||
]);
|
||||
expect(request.mock.calls.map((call) => call[1].cursor)).toEqual([
|
||||
undefined,
|
||||
"a",
|
||||
"b",
|
||||
]);
|
||||
expect(
|
||||
request.mock.calls.every((call) => call[1].itemsView === "notLoaded"),
|
||||
).toBe(true);
|
||||
});
|
||||
it("loads full items only for the requested turn and deduplicates stable IDs", async () => {
|
||||
const command = {
|
||||
turnId: "one",
|
||||
item: { id: "cmd", type: "commandExecution", exitCode: 0 },
|
||||
};
|
||||
const answer = {
|
||||
turnId: "one",
|
||||
item: { id: "answer", type: "agentMessage", text: "saved answer" },
|
||||
};
|
||||
const request = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({ data: [command], nextCursor: "a" })
|
||||
.mockResolvedValueOnce({ data: [command, answer] });
|
||||
expect(await readCodexTurnItems({ request }, "thread", "one")).toEqual([
|
||||
command.item,
|
||||
answer.item,
|
||||
]);
|
||||
expect(
|
||||
request.mock.calls.every(
|
||||
(call) => call[0] === "thread/items/list" && call[1].turnId === "one",
|
||||
),
|
||||
).toBe(true);
|
||||
});
|
||||
it.each([
|
||||
[
|
||||
{ data: [], nextCursor: "a" },
|
||||
{ data: [], nextCursor: "a" },
|
||||
],
|
||||
[{ data: [], nextCursor: "a" }, {}],
|
||||
[{ data: [{ id: "one" }] }],
|
||||
])("rejects incomplete history", async (...pages) => {
|
||||
const request = vi.fn();
|
||||
for (const page of pages) request.mockResolvedValueOnce(page);
|
||||
await expect(readCodexTurnMetadata({ request }, "thread")).rejects.toThrow(
|
||||
"codex_history_incomplete",
|
||||
);
|
||||
});
|
||||
it("does not turn an interrupted page or unsupported method into idle history", async () => {
|
||||
const request = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({ data: [], nextCursor: "next" })
|
||||
.mockRejectedValueOnce(new Error("method not found"));
|
||||
await expect(readCodexTurnMetadata({ request }, "thread")).rejects.toThrow(
|
||||
"supported paginated history",
|
||||
);
|
||||
});
|
||||
it("rejects another thread or turn", async () => {
|
||||
await expect(
|
||||
readCodexThreadState(
|
||||
{ request: vi.fn().mockResolvedValue({ thread: { id: "other" } }) },
|
||||
"thread",
|
||||
),
|
||||
).rejects.toThrow("different driver session");
|
||||
await expect(
|
||||
readCodexTurnItems(
|
||||
{
|
||||
request: vi
|
||||
.fn()
|
||||
.mockResolvedValue({
|
||||
data: [{ turnId: "other", item: { id: "a" } }],
|
||||
}),
|
||||
},
|
||||
"thread",
|
||||
"one",
|
||||
),
|
||||
).rejects.toThrow("different turn");
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,119 @@
|
|||
import type { CodexAppServerTransport } from "./app-server-transport.js";
|
||||
import { HarnessReconciliationError } from "../../contracts/harness-driver.js";
|
||||
import { record, text } from "./codex-driver-values.js";
|
||||
|
||||
type Requester = Pick<CodexAppServerTransport, "request">;
|
||||
|
||||
/** A partial or unsupported provider read is never evidence that work is idle. */
|
||||
async function pages(
|
||||
transport: Requester,
|
||||
method: "thread/turns/list" | "thread/items/list",
|
||||
params: Record<string, unknown>,
|
||||
identity: (value: Record<string, unknown>) => string,
|
||||
): Promise<Record<string, unknown>[]> {
|
||||
const values = new Map<string, Record<string, unknown>>();
|
||||
const cursors = new Set<string>();
|
||||
let cursor: string | undefined;
|
||||
for (let page = 0; page < 10_000; page += 1) {
|
||||
let response: Record<string, unknown>;
|
||||
try {
|
||||
response = await transport.request(method, {
|
||||
...params,
|
||||
limit: 100,
|
||||
...(cursor ? { cursor } : {}),
|
||||
});
|
||||
} catch (error) {
|
||||
throw new HarnessReconciliationError(
|
||||
`codex_history_read_failed: ${method} requires supported paginated history; ${String(error)}`,
|
||||
);
|
||||
}
|
||||
if (!Array.isArray(response.data))
|
||||
throw new HarnessReconciliationError(
|
||||
`codex_history_incomplete: ${method} omitted data`,
|
||||
);
|
||||
for (const raw of response.data) {
|
||||
const value = record(raw);
|
||||
const id = identity(value);
|
||||
if (!id)
|
||||
throw new HarnessReconciliationError(
|
||||
`codex_history_incomplete: ${method} omitted an identity`,
|
||||
);
|
||||
// Later pages can repeat the cursor anchor with its newly completed state.
|
||||
values.set(id, value);
|
||||
}
|
||||
const next = response.nextCursor;
|
||||
if (next == null) return [...values.values()];
|
||||
if (typeof next !== "string" || !next || cursors.has(next)) {
|
||||
throw new HarnessReconciliationError(
|
||||
`codex_history_incomplete: ${method} repeated or invalid cursor`,
|
||||
);
|
||||
}
|
||||
cursors.add(next);
|
||||
cursor = next;
|
||||
}
|
||||
throw new HarnessReconciliationError(
|
||||
`codex_history_incomplete: ${method} page limit exceeded`,
|
||||
);
|
||||
}
|
||||
|
||||
export async function readCodexThreadState(
|
||||
transport: Requester,
|
||||
threadId: string,
|
||||
): Promise<Record<string, unknown>> {
|
||||
const snapshot = await transport.request("thread/read", {
|
||||
threadId,
|
||||
includeTurns: false,
|
||||
});
|
||||
if (text(record(snapshot.thread).id) !== threadId)
|
||||
throw new HarnessReconciliationError(
|
||||
"thread/read returned a different driver session",
|
||||
);
|
||||
return snapshot;
|
||||
}
|
||||
|
||||
export function readCodexTurnMetadata(
|
||||
transport: Requester,
|
||||
threadId: string,
|
||||
): Promise<Record<string, unknown>[]> {
|
||||
return pages(
|
||||
transport,
|
||||
"thread/turns/list",
|
||||
{ threadId, sortDirection: "asc", itemsView: "notLoaded" },
|
||||
(value) => {
|
||||
if (
|
||||
![
|
||||
"inProgress",
|
||||
"completed",
|
||||
"failed",
|
||||
"interrupted",
|
||||
"cancelled",
|
||||
].includes(text(value.status))
|
||||
) {
|
||||
throw new HarnessReconciliationError(
|
||||
"codex_history_incomplete: invalid turn status",
|
||||
);
|
||||
}
|
||||
return text(value.id);
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
export async function readCodexTurnItems(
|
||||
transport: Requester,
|
||||
threadId: string,
|
||||
turnId: string,
|
||||
): Promise<Record<string, unknown>[]> {
|
||||
const entries = await pages(
|
||||
transport,
|
||||
"thread/items/list",
|
||||
{ threadId, turnId, sortDirection: "asc" },
|
||||
(value) => {
|
||||
if (text(value.turnId) !== turnId)
|
||||
throw new HarnessReconciliationError(
|
||||
"thread/items/list returned a different turn",
|
||||
);
|
||||
return text(record(value.item).id);
|
||||
},
|
||||
);
|
||||
return entries.map((entry) => record(entry.item));
|
||||
}
|
||||
|
|
@ -0,0 +1,145 @@
|
|||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
FakeCodexTransport,
|
||||
makeDriver,
|
||||
WORKSPACE,
|
||||
collectUntilTerminal,
|
||||
} from "./codex-app-server-driver.test-support.js";
|
||||
|
||||
describe("Codex resume accounting through the production driver", () => {
|
||||
it("retains a run delta across repeated historical snapshots and a cold resume", async () => {
|
||||
const first = new FakeCodexTransport();
|
||||
const second = new FakeCodexTransport();
|
||||
const driver = makeDriver([first, second], { conversationMode: "direct" });
|
||||
let session = await driver.openSession({
|
||||
runId: "first",
|
||||
normalizedSessionId: "session",
|
||||
workingDirectory: WORKSPACE,
|
||||
});
|
||||
await session.startTurn({ message: { role: "user", text: "First" } });
|
||||
first.push("thread/tokenUsage/updated", {
|
||||
threadId: "thread-1",
|
||||
turnId: "turn-1",
|
||||
tokenUsage: {
|
||||
total: { inputTokens: 100, outputTokens: 10 },
|
||||
last: { inputTokens: 100, outputTokens: 10 },
|
||||
},
|
||||
});
|
||||
first.push("turn/completed", {
|
||||
threadId: "thread-1",
|
||||
turn: { id: "turn-1", status: "completed", items: [] },
|
||||
});
|
||||
await collectUntilTerminal(session.events());
|
||||
const persisted = JSON.parse(JSON.stringify(await session.snapshot!()));
|
||||
await session.close();
|
||||
second.readResponse = {
|
||||
thread: {
|
||||
id: "thread-1",
|
||||
sessionId: "provider-session-1",
|
||||
cwd: WORKSPACE,
|
||||
turns: [{ id: "turn-1", status: "completed", items: [] }],
|
||||
},
|
||||
};
|
||||
const recovered = await driver.recoverSession(persisted);
|
||||
expect(recovered.recovered).toBe(true);
|
||||
session = recovered.session!;
|
||||
await session.attachRun!({ runId: "second" });
|
||||
for (let i = 0; i < 3; i++)
|
||||
second.push("thread/tokenUsage/updated", {
|
||||
threadId: "thread-1",
|
||||
turnId: "turn-1",
|
||||
tokenUsage: {
|
||||
total: { inputTokens: 100, outputTokens: 10 },
|
||||
last: { inputTokens: 100, outputTokens: 10 },
|
||||
},
|
||||
});
|
||||
// Consume the historical diagnostics before admitting the next turn.
|
||||
const iterator = session.events()[Symbol.asyncIterator]();
|
||||
let historical = 0;
|
||||
while (historical < 3) {
|
||||
const { value } = await iterator.next();
|
||||
expect(value.eventType).not.toBe("provider.notice.recorded");
|
||||
if (value.payload.code === "codex_resume_usage_snapshot") historical++;
|
||||
expect(value.payload.kind).not.toBe("usage");
|
||||
}
|
||||
expect((await session.snapshot!()).codexUsageBaseline).toEqual({
|
||||
baseline: { inputTokens: 100, outputTokens: 10 },
|
||||
latest: { inputTokens: 100, outputTokens: 10 },
|
||||
});
|
||||
second.turnStartResponse = Promise.resolve({
|
||||
turn: { id: "turn-2", status: "inProgress", items: [] },
|
||||
});
|
||||
await session.startTurn({ message: { role: "user", text: "Second" } });
|
||||
for (let i = 0; i < 2; i++)
|
||||
second.push("thread/tokenUsage/updated", {
|
||||
threadId: "thread-1",
|
||||
turnId: "turn-2",
|
||||
tokenUsage: {
|
||||
total: { inputTokens: 140, outputTokens: 16 },
|
||||
last: { inputTokens: 20, outputTokens: 3 },
|
||||
},
|
||||
});
|
||||
second.push("thread/tokenUsage/updated", {
|
||||
threadId: "foreign-thread",
|
||||
turnId: "turn-2",
|
||||
tokenUsage: { total: { inputTokens: 9999 } },
|
||||
});
|
||||
second.push("turn/completed", {
|
||||
threadId: "thread-1",
|
||||
turn: { id: "turn-2", status: "completed", items: [] },
|
||||
});
|
||||
const events = await collectUntilTerminal(session.events());
|
||||
const usages = events.filter((event) => event.payload.kind === "usage");
|
||||
expect(usages).toHaveLength(2);
|
||||
for (const event of usages)
|
||||
expect(event.payload.usage).toMatchObject({
|
||||
runDelta: { inputTokens: 40, outputTokens: 6 },
|
||||
});
|
||||
expect(
|
||||
(await session.snapshot!()).codexUsageBaseline?.latest.inputTokens,
|
||||
).toBe(140);
|
||||
await session.attachRun!({ runId: "second" });
|
||||
expect(
|
||||
(await session.snapshot!()).codexUsageBaseline?.baseline.inputTokens,
|
||||
).toBe(100);
|
||||
expect(await session.usage!()).toMatchObject({
|
||||
runDelta: { inputTokens: 40, outputTokens: 6 },
|
||||
});
|
||||
expect(
|
||||
second.calls.filter((call) => call.method === "thread/resume")[0]?.params
|
||||
.excludeTurns,
|
||||
).toBe(true);
|
||||
expect(
|
||||
second.calls
|
||||
.filter((call) => call.method === "thread/read")
|
||||
.every((call) => call.params.includeTurns === false),
|
||||
).toBe(true);
|
||||
await session.close();
|
||||
});
|
||||
|
||||
it("keeps the startup external sandbox profile on subsequent turns", async () => {
|
||||
const transport = new FakeCodexTransport();
|
||||
const session = await makeDriver([transport], {
|
||||
conversationMode: "direct",
|
||||
environment: {
|
||||
PATH: "/bin",
|
||||
HOME: "/isolated/home",
|
||||
CODEX_HOME: "/isolated/codex",
|
||||
PAPERCLIP_RUNNER_EXTERNAL_SANDBOX: "1",
|
||||
},
|
||||
}).openSession({
|
||||
runId: "external",
|
||||
normalizedSessionId: "external-session",
|
||||
workingDirectory: WORKSPACE,
|
||||
});
|
||||
await session.startTurn({ message: { role: "user", text: "Read notes" } });
|
||||
const start = transport.calls.find(
|
||||
(call) => call.method === "thread/start",
|
||||
)!;
|
||||
expect(
|
||||
transport.calls.find((call) => call.method === "turn/start")?.params
|
||||
.permissions,
|
||||
).toBe(start.params.permissions);
|
||||
await session.close();
|
||||
});
|
||||
});
|
||||
|
|
@ -1,3 +1,4 @@
|
|||
import { observeCodexUsage, codexRunUsage } from "./codex-usage-baseline.js";
|
||||
import { classifyCodexNotification } from "./codex-notification-identity.js";
|
||||
import { paperclipWorkspaceFileReferencesFromText } from "../../live/workspace-file-reference.js";
|
||||
import { canonicalProviderEventsFromCodex, isCanonicalProviderEventType } from "../../provider-events.js";
|
||||
|
|
@ -112,6 +113,20 @@ async function mapNotificationBody(state: CodexSessionState, notification: Codex
|
|||
rootThreadId: state.opened.threadId, activeTurnId: state.activeTurnId,
|
||||
knownThreads: new Set(state.lineageByThread.keys()), settledTurns: new Set(state.terminalTurns.keys()),
|
||||
});
|
||||
if ((identity.classification === "root" || identity.classification === "stale_turn")
|
||||
&& identity.threadId === state.opened.threadId
|
||||
&& notification.method === "thread/tokenUsage/updated"
|
||||
&& identity.turnId !== null
|
||||
&& identity.turnId !== state.activeTurnId
|
||||
&& (state.activeTurnId === null || (identity.turnId !== null && state.terminalTurns.has(identity.turnId)))) {
|
||||
state.codexUsageBaseline = observeCodexUsage(state.codexUsageBaseline, record(notification.params.tokenUsage).total, true);
|
||||
state.usageSnapshot = codexRunUsage(state.codexUsageBaseline);
|
||||
if (state.notificationIdentityDiagnostics++ < 32) state.emit("harness.diagnostic", {
|
||||
code: "codex_resume_usage_snapshot", classification: "resume_usage_snapshot",
|
||||
receivedThreadId: identity.threadId, receivedTurnId: identity.turnId,
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (identity.classification !== "root") {
|
||||
if (state.notificationIdentityDiagnostics < 32) {
|
||||
state.notificationIdentityDiagnostics += 1;
|
||||
|
|
@ -140,7 +155,12 @@ async function mapNotificationBody(state: CodexSessionState, notification: Codex
|
|||
state.failProtocol("provider_event_type_invalid", "Unknown canonical provider event type.");
|
||||
return;
|
||||
}
|
||||
state.emit(params.eventType, record(params.payload), { turnId: turnId || undefined, itemId: itemId || undefined });
|
||||
const canonicalPayload = record(params.payload);
|
||||
if (params.eventType === "harness.diagnostic" && canonicalPayload.code === "codex_resume_usage_snapshot") {
|
||||
state.codexUsageBaseline = observeCodexUsage(state.codexUsageBaseline, canonicalPayload.cumulative, true);
|
||||
state.usageSnapshot = codexRunUsage(state.codexUsageBaseline);
|
||||
}
|
||||
state.emit(params.eventType, canonicalPayload, { turnId: turnId || undefined, itemId: itemId || undefined });
|
||||
return;
|
||||
}
|
||||
if (notification.method === "paperclip/workspaceChange/updated") {
|
||||
|
|
@ -513,6 +533,10 @@ async function mapNotificationBody(state: CodexSessionState, notification: Codex
|
|||
}
|
||||
if (notification.method === "thread/tokenUsage/updated") {
|
||||
state.usageSnapshot = boundedPayload(record(params.tokenUsage));
|
||||
if (state.driverKind === "codex_app_server" && Object.keys(record(record(params.tokenUsage).total)).length > 0) {
|
||||
state.codexUsageBaseline = observeCodexUsage(state.codexUsageBaseline, record(params.tokenUsage).total, false);
|
||||
state.usageSnapshot = { ...state.usageSnapshot, ...codexRunUsage(state.codexUsageBaseline) };
|
||||
}
|
||||
// Codex can replay a thread-scoped usage snapshot while a resumed thread
|
||||
// is being attached, before the next turn has started. Keep the snapshot,
|
||||
// but do not turn that benign replay into a fatal turn-binding violation.
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import { type CodexUsageBaseline, codexRunUsage } from "./codex-usage-baseline.js";
|
||||
import type {
|
||||
HarnessRuntimeRequest,
|
||||
HarnessThreadGoal,
|
||||
|
|
@ -105,6 +106,7 @@ export class CodexSessionState {
|
|||
sourceSequence: number;
|
||||
activeTurnId: string | null;
|
||||
usageSnapshot: Record<string, unknown> | null = null;
|
||||
codexUsageBaseline: CodexUsageBaseline | null = null;
|
||||
result: PrpStructuredRunResult | null = null;
|
||||
resultFingerprint: string | null = null;
|
||||
resultCallId: string | null = null;
|
||||
|
|
@ -154,6 +156,7 @@ export class CodexSessionState {
|
|||
activeTurnId?: string | null;
|
||||
semanticResult?: PersistedHarnessSemanticResult | null;
|
||||
terminalTurns?: PersistedHarnessTurnTerminal[];
|
||||
codexUsageBaseline?: CodexUsageBaseline;
|
||||
dispositionOnlyRecoveryConsumed?: boolean;
|
||||
dispositionOnlyRecoveryTurnId?: string | null;
|
||||
stalePendingRuntimeRequests?: HarnessRuntimeRequest[];
|
||||
|
|
@ -171,6 +174,8 @@ export class CodexSessionState {
|
|||
dynamicTools: readonly Readonly<Record<string, unknown>>[];
|
||||
dynamicToolHandler?: CodexAppServerDriverOptions["dynamicToolHandler"];
|
||||
}) {
|
||||
this.codexUsageBaseline = input.codexUsageBaseline ?? null;
|
||||
if (this.codexUsageBaseline) this.usageSnapshot = codexRunUsage(this.codexUsageBaseline);
|
||||
this.transport = input.transport;
|
||||
this.runId = input.runId;
|
||||
this.sourceSequence = 0;
|
||||
|
|
|
|||
|
|
@ -0,0 +1,129 @@
|
|||
import {
|
||||
mkdtempSync,
|
||||
mkdirSync,
|
||||
readFileSync,
|
||||
rmSync,
|
||||
writeFileSync,
|
||||
realpathSync,
|
||||
symlinkSync,
|
||||
} from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { parse } from "smol-toml";
|
||||
import { trustCodexStartupRoot } from "./codex-startup-trust.js";
|
||||
|
||||
describe("isolated Codex startup trust", () => {
|
||||
it.each(["missing", "table", "empty-table", "inline", "empty-inline", "inline-projects"])(
|
||||
"preserves managed comments and unrelated content with %s project configuration", (kind) => {
|
||||
const temp = realpathSync(mkdtempSync(join(tmpdir(), "codex-trust-comments-")));
|
||||
try {
|
||||
const cwd = join(temp, "project"); const home = join(temp, "home");
|
||||
mkdirSync(cwd); mkdirSync(home);
|
||||
const root = JSON.stringify(cwd);
|
||||
const project = {
|
||||
missing: "",
|
||||
table: `[projects.${root}]\ntrust_level = "untrusted" # retain trust comment\n`,
|
||||
"empty-table": `[projects.${root}] # retain table comment\n`,
|
||||
inline: `[projects]\n${root} = {trust_level = 'untrusted', label = "unchanged"}\n`,
|
||||
"empty-inline": `[projects]\n${root} = {} # retain inline comment\n`,
|
||||
"inline-projects": `projects = {${root} = {label = "unchanged"}}\n`,
|
||||
}[kind]!;
|
||||
const managed = '# BEGIN PAPERCLIP MANAGED MCP\n[mcp_servers.fixture]\nurl = "https://example.invalid/old"\n# END PAPERCLIP MANAGED MCP\n';
|
||||
const misleading = 'text = """\n[projects.fake]\ntrust_level = "untrusted"\n"""\n';
|
||||
const original = (kind === "inline-projects" ? project + misleading : misleading + project) + managed;
|
||||
writeFileSync(join(home, "config.toml"), original);
|
||||
const expected = parse(original);
|
||||
expected.projects ??= {};
|
||||
const projects = expected.projects as Record<string, Record<string, unknown>>;
|
||||
projects[cwd] = { ...projects[cwd], trust_level: "trusted" };
|
||||
trustCodexStartupRoot(home, cwd);
|
||||
const updated = readFileSync(join(home, "config.toml"), "utf8");
|
||||
expect(parse(updated)).toEqual(expected);
|
||||
expect(updated).toContain(managed);
|
||||
expect(updated).toContain(misleading);
|
||||
trustCodexStartupRoot(home, cwd);
|
||||
expect(readFileSync(join(home, "config.toml"), "utf8")).toBe(updated);
|
||||
const refreshed = updated.replace(managed, managed.replace('/old', '/new'));
|
||||
expect((parse(refreshed).mcp_servers as Record<string, {url: string}>).fixture.url).toBe('https://example.invalid/new');
|
||||
} finally { rmSync(temp, { recursive: true, force: true }); }
|
||||
},
|
||||
);
|
||||
|
||||
it("trusts the exact non-Git root and preserves unrelated config", () => {
|
||||
const temp = realpathSync(mkdtempSync(join(tmpdir(), "codex-trust-")));
|
||||
try {
|
||||
const cwd = join(temp, "project");
|
||||
const home = join(temp, "home");
|
||||
mkdirSync(cwd);
|
||||
mkdirSync(home);
|
||||
writeFileSync(
|
||||
join(home, "config.toml"),
|
||||
'model = "test"\n[mcp_servers.fixture]\nurl = "http://localhost/example"\n',
|
||||
);
|
||||
trustCodexStartupRoot(home, cwd);
|
||||
trustCodexStartupRoot(home, cwd);
|
||||
expect(parse(readFileSync(join(home, "config.toml"), "utf8"))).toEqual({
|
||||
model: "test",
|
||||
mcp_servers: { fixture: { url: "http://localhost/example" } },
|
||||
projects: { [cwd]: { trust_level: "trusted" } },
|
||||
});
|
||||
expect(readFileSync(join(home, "config.toml"), "utf8")).not.toContain(
|
||||
`[projects."${temp}"]`,
|
||||
);
|
||||
} finally {
|
||||
rmSync(temp, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
it("uses the canonical main repository key for worktrees and symlinks", () => {
|
||||
const temp = realpathSync(mkdtempSync(join(tmpdir(), "codex-trust-")));
|
||||
try {
|
||||
const main = join(temp, "main");
|
||||
const worktree = join(temp, "branch");
|
||||
const home = join(temp, "home");
|
||||
mkdirSync(main);
|
||||
execFileSync("git", ["init", main], { stdio: "ignore" });
|
||||
execFileSync(
|
||||
"git",
|
||||
[
|
||||
"-C",
|
||||
main,
|
||||
"-c",
|
||||
"user.name=Test",
|
||||
"-c",
|
||||
"user.email=test@example.com",
|
||||
"commit",
|
||||
"--allow-empty",
|
||||
"-m",
|
||||
"init",
|
||||
],
|
||||
{ stdio: "ignore" },
|
||||
);
|
||||
execFileSync(
|
||||
"git",
|
||||
["-C", main, "worktree", "add", "-b", "test", worktree],
|
||||
{ stdio: "ignore" },
|
||||
);
|
||||
symlinkSync(worktree, join(temp, "alias"));
|
||||
trustCodexStartupRoot(home, join(temp, "alias"));
|
||||
expect(
|
||||
parse(readFileSync(join(home, "config.toml"), "utf8")).projects,
|
||||
).toEqual({ [main]: { trust_level: "trusted" } });
|
||||
} finally {
|
||||
rmSync(temp, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
it("fails without replacing malformed configuration", () => {
|
||||
const temp = realpathSync(mkdtempSync(join(tmpdir(), "codex-trust-")));
|
||||
try {
|
||||
writeFileSync(join(temp, "config.toml"), "invalid = [");
|
||||
expect(() => trustCodexStartupRoot(temp, temp)).toThrow();
|
||||
expect(readFileSync(join(temp, "config.toml"), "utf8")).toBe(
|
||||
"invalid = [",
|
||||
);
|
||||
} finally {
|
||||
rmSync(temp, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,122 @@
|
|||
import { execFileSync } from "node:child_process";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import {
|
||||
existsSync,
|
||||
realpathSync,
|
||||
readFileSync,
|
||||
writeFileSync,
|
||||
mkdirSync,
|
||||
renameSync,
|
||||
rmSync,
|
||||
} from "node:fs";
|
||||
import { dirname, isAbsolute, join, resolve } from "node:path";
|
||||
import { parse, type TomlTable } from "smol-toml";
|
||||
import { isDeepStrictEqual } from "node:util";
|
||||
|
||||
/** Keep managed-region comments and all unrelated bytes. The parser validates
|
||||
* every candidate against the exact intended semantic change; text inside a
|
||||
* comment or multiline string can never qualify as the target assignment. */
|
||||
function editTrust(source: string, root: string, expected: TomlTable): string {
|
||||
const matches = (candidate: string) => {
|
||||
try { return isDeepStrictEqual(parse(candidate), expected); }
|
||||
catch { return false; }
|
||||
};
|
||||
if (matches(source)) return source;
|
||||
const appended = `${source}\n[projects.${JSON.stringify(root)}]\ntrust_level = "trusted"\n`;
|
||||
if (matches(appended)) return appended;
|
||||
const values = /(?:"trust_level"|'trust_level'|\btrust_level)\s*=\s*("""[\s\S]*?"""|'''[\s\S]*?'''|"(?:\\.|[^"\\])*"|'[^']*')/g;
|
||||
for (const match of source.matchAll(values)) {
|
||||
const end = match.index! + match[0].length;
|
||||
const start = end - match[1]!.length;
|
||||
const candidate = source.slice(0, start) + '"trusted"' + source.slice(end);
|
||||
if (matches(candidate)) return candidate;
|
||||
}
|
||||
// An existing project table may not yet have a trust field.
|
||||
for (const match of source.matchAll(/^[ \t]*\[(?!\[)[^\r\n]*\][^\r\n]*(?:\r?\n|$)/gm)) {
|
||||
const end = match.index! + match[0].length;
|
||||
const candidate = source.slice(0, end) + '\ntrust_level = "trusted"\n' + source.slice(end);
|
||||
if (matches(candidate)) return candidate;
|
||||
}
|
||||
// Inline project tables are closed to appended table headers. Insert only
|
||||
// when reparsing proves this is the intended object, not a brace in text.
|
||||
const fields = ['trust_level = "trusted"', `${JSON.stringify(root)} = { trust_level = "trusted" }`];
|
||||
for (const match of source.matchAll(/\{/g)) {
|
||||
const end = match.index! + 1;
|
||||
for (const field of fields) {
|
||||
for (const separator of [', ', '']) {
|
||||
const candidate = source.slice(0, end) + field + separator + source.slice(end);
|
||||
if (matches(candidate)) return candidate;
|
||||
}
|
||||
}
|
||||
}
|
||||
throw new Error("codex_startup_trust_cannot_preserve_configuration");
|
||||
}
|
||||
|
||||
/** Run on the execution host, before the provider process loads project config. */
|
||||
export function trustCodexStartupRoot(codexHome: string, cwd: string): void {
|
||||
if (!isAbsolute(codexHome) || !isAbsolute(cwd))
|
||||
throw new Error("codex_startup_trust_requires_absolute_paths");
|
||||
const startup = realpathSync(cwd);
|
||||
let root = startup;
|
||||
try {
|
||||
const top = execFileSync(
|
||||
"git",
|
||||
["-C", startup, "rev-parse", "--show-toplevel"],
|
||||
{ encoding: "utf8", timeout: 5000, stdio: ["ignore", "pipe", "ignore"] },
|
||||
).trim();
|
||||
const common = execFileSync(
|
||||
"git",
|
||||
[
|
||||
"-C",
|
||||
startup,
|
||||
"rev-parse",
|
||||
"--path-format=absolute",
|
||||
"--git-common-dir",
|
||||
],
|
||||
{ encoding: "utf8", timeout: 5000, stdio: ["ignore", "pipe", "ignore"] },
|
||||
).trim();
|
||||
// Linked worktrees share Codex's trust key with the main checkout.
|
||||
root = realpathSync(common.endsWith("/.git") ? dirname(common) : top);
|
||||
} catch (error) {
|
||||
// Non-Git folders have their own exact startup trust boundary. Failures
|
||||
// inside a repository must not guess a different trust key.
|
||||
for (let ancestor = startup; ; ancestor = dirname(ancestor)) {
|
||||
if (existsSync(join(ancestor, ".git")))
|
||||
throw new Error("codex_startup_trust_git_resolution_failed", {
|
||||
cause: error,
|
||||
});
|
||||
if (dirname(ancestor) === ancestor) break;
|
||||
}
|
||||
}
|
||||
mkdirSync(codexHome, { recursive: true, mode: 0o700 });
|
||||
const path = join(codexHome, "config.toml");
|
||||
const source = existsSync(path) ? readFileSync(path, "utf8") : "";
|
||||
const config = parse(source);
|
||||
const projects = config.projects ?? {};
|
||||
if (
|
||||
typeof projects !== "object" ||
|
||||
Array.isArray(projects) ||
|
||||
projects instanceof Date
|
||||
)
|
||||
throw new Error("codex_startup_trust_invalid_projects");
|
||||
const project = projects[root] ?? {};
|
||||
if (
|
||||
typeof project !== "object" ||
|
||||
Array.isArray(project) ||
|
||||
project instanceof Date
|
||||
)
|
||||
throw new Error("codex_startup_trust_invalid_project");
|
||||
config.projects = {
|
||||
...projects,
|
||||
[root]: { ...project, trust_level: "trusted" },
|
||||
};
|
||||
const updated = editTrust(source, root, config);
|
||||
if (updated === source) return;
|
||||
const temporary = resolve(codexHome, `config.toml.${randomUUID()}.tmp`);
|
||||
try {
|
||||
writeFileSync(temporary, updated, { mode: 0o600, flag: "wx" });
|
||||
renameSync(temporary, path);
|
||||
} finally {
|
||||
rmSync(temporary, { force: true });
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
import { describe, it, expect } from "vitest";
|
||||
import { observeCodexUsage, codexRunUsage } from "./codex-usage-baseline.js";
|
||||
describe("Codex resume usage accounting", () => {
|
||||
it("never charges historical snapshots, including repetitions after restart", () => {
|
||||
let state = observeCodexUsage(
|
||||
null,
|
||||
{ inputTokens: 100, outputTokens: 20 },
|
||||
true,
|
||||
);
|
||||
state = observeCodexUsage(
|
||||
JSON.parse(JSON.stringify(state)),
|
||||
{ inputTokens: 100, outputTokens: 20 },
|
||||
true,
|
||||
);
|
||||
expect(codexRunUsage(state).runDelta).toEqual({
|
||||
inputTokens: 0,
|
||||
outputTokens: 0,
|
||||
});
|
||||
state = observeCodexUsage(
|
||||
state,
|
||||
{ inputTokens: 140, outputTokens: 26 },
|
||||
false,
|
||||
);
|
||||
state = observeCodexUsage(
|
||||
state,
|
||||
{ inputTokens: 140, outputTokens: 26 },
|
||||
false,
|
||||
);
|
||||
state = observeCodexUsage(
|
||||
state,
|
||||
{ inputTokens: 100, outputTokens: 20 },
|
||||
true,
|
||||
);
|
||||
expect(codexRunUsage(state).runDelta).toEqual({
|
||||
inputTokens: 40,
|
||||
outputTokens: 6,
|
||||
});
|
||||
});
|
||||
it("retains all current usage for a fresh session", () => {
|
||||
expect(
|
||||
codexRunUsage(
|
||||
observeCodexUsage(null, { inputTokens: 40, outputTokens: 6 }, false),
|
||||
).runDelta,
|
||||
).toEqual({ inputTokens: 40, outputTokens: 6 });
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
/** Session totals are monotonic observations, not additional billable receipts. */
|
||||
export interface CodexUsageBaseline {
|
||||
baseline: Record<string, number>;
|
||||
latest: Record<string, number>;
|
||||
}
|
||||
|
||||
export function codexUsageMeasurement(value: unknown): Record<string, number> {
|
||||
if (value === null || typeof value !== "object") return {};
|
||||
return Object.fromEntries(
|
||||
Object.entries(value).filter(
|
||||
(entry): entry is [string, number] =>
|
||||
typeof entry[1] === "number" &&
|
||||
Number.isFinite(entry[1]) &&
|
||||
entry[1] >= 0,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
export function observeCodexUsage(
|
||||
state: CodexUsageBaseline | null,
|
||||
total: unknown,
|
||||
historical: boolean,
|
||||
): CodexUsageBaseline {
|
||||
const measurement = codexUsageMeasurement(total);
|
||||
const baseline = state?.baseline ?? (historical ? measurement : {});
|
||||
const latest = { ...state?.latest };
|
||||
for (const [key, value] of Object.entries(measurement)) {
|
||||
latest[key] = Math.max(latest[key] ?? 0, value);
|
||||
}
|
||||
return { baseline, latest };
|
||||
}
|
||||
|
||||
export function codexRunUsage(
|
||||
state: CodexUsageBaseline,
|
||||
): Record<string, unknown> {
|
||||
return {
|
||||
total: state.latest,
|
||||
runDelta: Object.fromEntries(
|
||||
Object.entries(state.latest).map(([key, value]) => [
|
||||
key,
|
||||
Math.max(0, value - (state.baseline[key] ?? 0)),
|
||||
]),
|
||||
),
|
||||
};
|
||||
}
|
||||
|
|
@ -3345,6 +3345,35 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
|
|||
await this.#command("turn.interrupt", params);
|
||||
return {};
|
||||
}
|
||||
if (method === "thread/turns/list" || method === "thread/items/list") {
|
||||
if (params.threadId !== this.#threadId) throw new Error("codex_history_identity_mismatch");
|
||||
const snapshot = await this.request("thread/read", { threadId: this.#threadId, includeTurns: false });
|
||||
const turns = record(snapshot.thread).turns as Array<Record<string, unknown>>;
|
||||
let data: Array<Record<string, unknown>>;
|
||||
if (method === "thread/turns/list") {
|
||||
data = turns.map(turn => ({ ...turn, items: [], itemsView: "notLoaded" }));
|
||||
} else {
|
||||
if (params.turnId !== this.#turnId) throw new Error("codex_history_unavailable: requested turn is outside the retained runner event window");
|
||||
const items = new Map<string, Record<string, unknown>>();
|
||||
let observedTurn = "";
|
||||
let observedStart = false;
|
||||
for (const event of this.#core?.store.state.committedEvents ?? []) {
|
||||
const payload = record(record(event.envelope.payload).payload);
|
||||
if (event.eventType === "turn.started") observedTurn = String(payload.providerTurnId ?? payload.turnId ?? record(payload.turn).id ?? "");
|
||||
if (event.eventType === "turn.started" && observedTurn === params.turnId) observedStart = true;
|
||||
if (event.eventType !== "item.completed" || observedTurn !== params.turnId) continue;
|
||||
const item = record(rehydrateRunnerdItemNotification(payload, this.#threadId, observedTurn).item);
|
||||
if (typeof item.id === "string") items.set(item.id, { turnId: observedTurn, item });
|
||||
}
|
||||
if (!observedStart) throw new Error("codex_history_incomplete: requested turn start is outside the retained runner event window");
|
||||
data = [...items.values()];
|
||||
}
|
||||
if (params.sortDirection === "desc") data.reverse();
|
||||
const offset = params.cursor == null ? 0 : Number(params.cursor);
|
||||
if (!Number.isSafeInteger(offset) || offset < 0 || offset > data.length) throw new Error("codex_history_invalid_cursor");
|
||||
const limit = typeof params.limit === "number" ? Math.max(1, Math.min(100, params.limit)) : 100;
|
||||
return { data: data.slice(offset, offset + limit), nextCursor: offset + limit < data.length ? String(offset + limit) : null };
|
||||
}
|
||||
if (method === "thread/read") {
|
||||
if (this.#core === null) {
|
||||
this.#recoveryTurnBindingPending = true;
|
||||
|
|
|
|||
|
|
@ -81,6 +81,19 @@ function handleRequest(message) {
|
|||
});
|
||||
return;
|
||||
}
|
||||
if (method === "thread/turns/list") {
|
||||
const turns = Object.entries(state.turns).map(([turnId, status]) => ({
|
||||
id: turnId, status, items: [], itemsView: "notLoaded",
|
||||
}));
|
||||
if (params.sortDirection === "desc") turns.reverse();
|
||||
const offset = Number(params.cursor ?? 0);
|
||||
const limit = params.limit ?? 100;
|
||||
send({ id, result: {
|
||||
data: turns.slice(offset, offset + limit),
|
||||
nextCursor: offset + limit < turns.length ? String(offset + limit) : null,
|
||||
} });
|
||||
return;
|
||||
}
|
||||
if (method === "thread/read") {
|
||||
send({
|
||||
id,
|
||||
|
|
@ -88,7 +101,10 @@ function handleRequest(message) {
|
|||
thread: {
|
||||
id: state.threadId,
|
||||
sessionId: state.sessionId,
|
||||
turns: Object.entries(state.turns).map(([turnId, status]) => ({ id: turnId, status })),
|
||||
status: { type: Object.values(state.turns).includes("inProgress") ? "active" : "idle" },
|
||||
...(params.includeTurns ? {
|
||||
turns: Object.entries(state.turns).map(([turnId, status]) => ({ id: turnId, status })),
|
||||
} : {}),
|
||||
tokenUsage: {
|
||||
total: {
|
||||
inputTokens: state.nextTurn * 10,
|
||||
|
|
|
|||
|
|
@ -1,6 +1,5 @@
|
|||
import { execFileSync } from "node:child_process";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { existsSync } from "node:fs";
|
||||
import { mkdtemp, readFile, rm } from "node:fs/promises";
|
||||
import { createServer, type Server } from "node:http";
|
||||
import { tmpdir } from "node:os";
|
||||
|
|
@ -59,7 +58,8 @@ const fakeCodexBinary = resolve(
|
|||
);
|
||||
|
||||
function ensureRunnerTestBinaries(): void {
|
||||
if (existsSync(runnerBinary) && existsSync(fakeCodexBinary)) return;
|
||||
// Cargo's freshness check is necessary even when the files exist: an older
|
||||
// fake provider can otherwise exercise a different protocol than the source.
|
||||
execFileSync("cargo", [
|
||||
"build",
|
||||
"--release",
|
||||
|
|
|
|||
Loading…
Reference in New Issue