## Thinking Path
> - Paperclip manages AI agents that perform work.
> - Paperclip Runner connects durable task runs to local provider
processes.
> - The full-stack paid matrix exposed failures after the runner
integrity repair.
> - Verified JavaScript entrypoints lost their relative module graph
when Linux executed them through descriptor paths.
> - Returned provider startup errors also remained pending and became
indeterminate after recovery.
> - Sparse Codex tool lifecycle events lost the `write_document`
identity before task transcript projection.
> - This pull request repairs those three boundaries and makes the
structured-question fixture deterministic.
> - The benefit is repeatable provider startup, exact failure replay,
and correct inline Plan placement.
## Linked Issues or Issue Description
Refs #12721 and #12700.
**What happened?**
The paid runner matrix failed ACPX and OpenCode startup before provider
session creation. The runner journal then replaced the original startup
error with an indeterminate recovery result. Native Codex saved a Plan
but rendered it only as a fallback card. A legacy Claude waiting reply
could also echo the reserved terminal marker before the answer arrived.
**Expected behavior**
Verified JavaScript providers must start from immutable
descriptor-backed artifacts. Returned startup failures must persist as
terminal failed command results. Native tool lifecycle updates must
preserve the `write_document` boundary. Pre-answer fixture output must
not contain the reserved terminal marker.
**Steps to reproduce**
1. Run the local provider cells in the Runner Full-Stack E2E workflow.
2. Observe ACPX and OpenCode fail during `session.open` before provider
execution.
3. Observe recovery report `execution_indeterminate` instead of the
original startup error.
4. Run the native Codex Plan cell and observe the fallback Plan card
after the tool activity row.
5. Run the legacy Claude structured-question resume cell and observe an
early marker echo in waiting prose.
**Paperclip version or commit**
`0f9452101740835ce0b1488a204bf48acd5bafc3`
**Deployment mode**
Local development with the paid GitHub Actions acceptance workflow.
## What Changed
- Bundle the ACPX sidecar and OpenCode proxy as self-contained Node ESM
entrypoints before hashing and verified descriptor launch.
- Anchor ACPX dynamic provider package resolution at a
controller-derived provider-pack root and keep that root out of the
provider child environment.
- Persist executor-returned startup errors as redacted durable failed
command results while retaining indeterminate recovery for true process
death.
- Coalesce sparse native tool items by stable ID so a late
`write_document` name, input, and result reach the transcript boundary
once.
- Forbid the structured-question fixture from spelling or announcing its
reserved terminal marker before the user answers.
## Verification
- Rust and TypeScript regression tests cover durable failed replay, true
crash ambiguity, bundle closure, package-root derivation, environment
filtering, exact Codex tool lifecycle coalescing, and prompt
determinism.
- Local execution is intentionally limited to formatters and static diff
checks. GitHub Actions will run tests, type checks, builds, and security
checks.
- After ordinary CI is green, scoped paid cells will validate one ACPX
launch, one OpenCode launch, native Codex Plan projection, and legacy
Claude structured resume before a complete matrix rerun.
- Prior failing matrix:
https://github.com/paperclipai/paperclip/actions/runs/33682434315
## Risks
- Bundling changes the bytes covered by provider launch hashes.
Provider-pack generation already hashes the final built files.
- ACPX still loads qualified provider packages dynamically. The
controller supplies a normalized package root, while existing version,
digest, path, and descriptor checks remain active.
- Durable `failed` is terminal. Replays return the same redacted result
and do not execute the provider effect twice.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
OpenAI Codex based on GPT-5 with agentic reasoning, repository
inspection, code editing, Git, parallel subagents, and GitHub Actions
coordination. The exact deployed snapshot and context-window size are
not exposed to this task.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked related public work or described the bug in
this PR
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
ticket id
- [ ] I have run tests locally and they pass (intentionally deferred to
GitHub Actions)
- [x] I have added or updated tests where applicable
- [x] No documentation change is required for this runtime repair
- [x] I have considered and documented the risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Local CLI adapters are responsible for starting agent runtimes and
validating that their configured models are usable before a run starts.
> - The OpenCode local adapter checks `opencode models` during model
discovery and preflight validation.
> - On hosts with a shared Ollama daemon, that lightweight metadata call
can transiently queue behind an active generation and time out or return
a short failure.
> - Treating that transient contention as a hard adapter failure
prevents otherwise valid local OpenCode runs from starting.
> - This pull request adds a small bounded retry/backoff around OpenCode
model discovery while keeping the existing per-attempt timeout and
surfacing a final failure when retries are exhausted.
> - The benefit is fewer false adapter failures during local Ollama
contention without changing shared Ollama configuration or hiding
genuinely stuck model discovery.
## Linked Issues or Issue Description
No public GitHub issue exists for this adapter reliability bug.
Bug description:
- What happened: `opencode models` can transiently time out or fail
while a shared local Ollama daemon is busy serving another OpenCode
generation, causing the adapter preflight to fail before the actual run
starts.
- Expected behavior: transient model-list contention should be retried
briefly before declaring the adapter unavailable.
- Steps to reproduce: run an OpenCode local adapter using an
Ollama-backed model while another `opencode run` is actively generating
against the same daemon, then trigger model discovery/preflight during
that contention window.
- Paperclip version/commit: observed on the current Paperclip
master-line OpenCode local adapter before this change.
- Deployment mode: local trusted / local CLI adapter execution with a
shared local Ollama daemon.
Related search:
- Searched public GitHub issues for `opencode models preflight retry`;
no matching issue found.
- Searched public GitHub PRs for `opencode models preflight retry`; no
matching PR found. The only search hit was unrelated OpenClaw gateway
authentication work (#6121).
## What Changed
- Added bounded retry/backoff to OpenCode model discovery: three total
attempts with 2s and 4s waits between failures.
- Preserved the existing 20s per-attempt `opencode models` timeout.
- Retry covers timeout and non-zero process exits, while spawn-level
failures still surface immediately.
- Added unit coverage for transient fail -> timeout -> success behavior
and exhausted retry behavior.
- Updated existing OpenCode environment diagnostic tests with explicit
timeouts for the intentional retry/backoff path.
## Verification
- `pnpm --filter @paperclipai/adapter-opencode-local exec vitest run
src/server/models.test.ts src/server/execute.test.ts` -> 2 files passed,
13 tests passed.
- `pnpm --filter @paperclipai/adapter-opencode-local typecheck` ->
passed.
- `pnpm vitest run
server/src/__tests__/opencode-local-adapter-environment.test.ts` -> 1
file passed, 3 tests passed.
- Branch diff against current `upstream/master` is limited to
`packages/adapters/opencode-local/src/server/models.ts`,
`packages/adapters/opencode-local/src/server/models.test.ts`, and
`server/src/__tests__/opencode-local-adapter-environment.test.ts`.
## Risks
Low risk. This only changes OpenCode model discovery behavior and keeps
the preflight bounded. A genuinely unavailable `opencode models` call
still fails after three attempts, and command spawn failures are not
masked.
## Model Used
OpenAI Codex, GPT-5.5 coding agent, tool-enabled repository editing and
shell verification in a local Paperclip workspace.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Co-authored-by: Test <test@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents run through adapters; the `opencode-local` adapter shells out
to the OpenCode CLI and, before each run, does a pre-flight `opencode
models` **availability probe** to fail fast on a misconfigured
`provider/model`.
> - That probe was written to **throw on any probe failure** — a
timeout, a non-zero exit, or a transient `Unexpected error` from the CLI
— which aborts the whole heartbeat run.
> - In practice the CLI probe fails transiently (provider hiccup, cold
cache, momentary CLI error). When that happens *after* the agent has
already done its work, the run dies before its terminal disposition is
written, so the platform reopens the issue and re-runs it — a spurious
crash/re-run loop that affects every agent on the OpenCode adapter.
> - This PR makes the probe **non-fatal when it cannot run**: it warns
and proceeds with the configured model, letting the real invocation be
authoritative.
> - It deliberately **keeps** the genuine guard: when the probe
*succeeds* and the configured model is absent from a non-empty list, it
still throws (this is what catches misconfigured slugs).
> - The benefit is that a best-effort pre-flight check can no longer
take down an otherwise-healthy run, while the useful misconfiguration
guard is retained.
## Linked Issues or Issue Description
No public GitHub issue exists; describing inline (bug).
**What happened:** an OpenCode-adapter agent run terminated at the
adapter level with `` `opencode models` failed: Unexpected error ``. The
failure landed after the agent had produced its work, so the
terminal-status update never applied and the run was reopened and
re-executed.
**Expected:** a transient failure of the `opencode models` availability
*probe* should not abort the run — the probe is a best-effort pre-flight
guard, not a gate.
**Actual:** the probe threw on timeout / non-zero exit / empty output,
aborting the run and discarding the completed work + disposition.
**Scope:** both the local (`models.ts`) and remote/SSH (`execute.ts`)
probe paths; affects any agent on the `opencode_local` adapter.
Related PRs (context / prior art):
- Refs #5119 — added the remote execution-target model-probe validation
this PR softens.
- Refs #3291 — closed prior attempt to make the `opencode_local` model
probe non-blocking (at agent-create time; different entry point).
- Refs #8014 — related open work raising the probe timeout (20s → 60s);
complementary, not overlapping.
## What Changed
- `models.ts` (`ensureOpenCodeModelConfiguredAndAvailable`): if
discovery throws (probe can't run) or returns an empty list, **warn and
proceed** with the configured model instead of throwing. The "model
present in a non-empty list" check is unchanged and still throws when
the configured model is genuinely absent.
- `execute.ts` (`ensureRemoteOpenCodeModelConfiguredAndAvailable`):
remote probe **timeout / non-zero exit / empty output** now warn and
return (proceed) instead of throwing. The remote model-absent guard
still throws.
- `models.test.ts`: the local "discovery cannot run" case now asserts
the probe **proceeds** with the configured model (was: asserts it
rejects).
- `execute.test.ts`: added remote regression tests — non-zero exit,
timeout, and empty output all proceed; a successful probe missing the
configured model still rejects.
## Verification
```bash
pnpm --filter @paperclipai/adapter-opencode-local typecheck # clean
# opencode-local server suite (default 5s per-test timeout is too tight for the
# heavy SSH tests on some machines; use a realistic timeout):
node node_modules/.pnpm/vitest@*/node_modules/vitest/vitest.mjs run \
packages/adapters/opencode-local/src/server/models.test.ts \
packages/adapters/opencode-local/src/server/execute.test.ts \
packages/adapters/opencode-local/src/server/execute.remote.test.ts \
--testTimeout=45000
```
Result: typecheck clean; all opencode-local server tests pass, including
the new remote fail-open tests and the retained "model unavailable on
the remote target" guard test.
## Risks
- **Fail-open behavior (intentional).** When the probe can't run, a
genuinely misconfigured model is no longer caught at pre-flight — it
surfaces at the real invocation instead. This is the accepted tradeoff:
the probe is best-effort, and the real invocation is authoritative. The
high-value guard (probe succeeds + model absent from a non-empty list)
is retained, so the common misconfiguration — a bad `provider/model`
slug — is still caught.
- No API, schema, or migration changes. Behavior change is confined to
the two probe helpers. Low risk overall.
## Model Used
Anthropic **Claude Opus 4.8** (`claude-opus-4-8`), used via Claude Code
with agentic tool use (repo search, file editing, shell/code execution)
and extended reasoning. Used to diagnose the crash, implement the fix,
and write the tests; the change was reviewed before submission.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change
(`fix/opencode-model-probe-non-fatal`) and contains no internal ticket
id
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes — N/A
(internal adapter behavior; no user-facing docs affected)
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green (functional gates:
tests/build/e2e/typecheck/security). Review/Greptile gate re-running
after this update.
- [ ] Greptile is 5/5 with no open P2s — re-triggered after addressing
both P2s (remote test coverage + this template-complete description)
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The `opencode-local` adapter runs the OpenCode harness; its
model/provider routing assumes built-in providers (anthropic/openai/...)
and their default models
> - Deployments increasingly put an OpenAI/Anthropic-compatible LLM
gateway between the harness and the model for cost, governance, or
data-residency reasons: LiteLLM, OpenRouter, Portkey, Kong, a corporate
proxy, self-hosted models (vLLM/Ollama), or region-pinned/sovereign
endpoints. But OpenCode only resolves `--model provider/model` when the
model is registered in a provider's `models` map, and
`OPENCODE_ALLOW_ALL_MODELS` does NOT bypass its internal `getModel()`
> - Several lanes also fall back to built-in default models the gateway
may not serve: the auxiliary/title model (e.g. `claude-haiku-*`) and the
budget/recovery "cheap" lane (`openai/gpt-5.1-codex-mini`); these abort
runs with "no keys found that support model"
> - This pull request makes the adapter's provider/model wiring
declarative via env, so any such deployment can register gateway models
+ pin the auxiliary/budget lanes without code changes; nothing here is
specific to one hosting setup
> - The benefit is OpenCode works behind any compatible gateway with
config only; with no env set, behavior is unchanged
## Linked Issues or Issue Description
No existing issue; describing in-PR (feature / adapter enhancement).
- **Gap:** there is no supported way to register custom/gateway
providers + models for `opencode-local`, nor to pin the auxiliary
(title-gen) and budget (recovery) model lanes, so routing OpenCode
through a gateway fails at `getModel()` or on the default helper models.
- Related: #5737 (exe.dev sandbox installs for gemini/opencode local),
#5823 (unblock claude_local on remote sandbox providers).
> Note on ROADMAP: this is adapter-level, opt-in config (defaults
unchanged) that *enables* gateway routing for one harness; it is not the
core "Cloud / Sandbox agents" platform work itself. Happy to
redirect/discuss in #dev if preferred.
## What Changed
- `PAPERCLIP_OPENCODE_PROVIDERS`: merge custom/extended providers
(OpenCode `provider` shape) into the runtime `opencode.json`, so gateway
models are registered and `--model provider/model` resolves. `{env:VAR}`
placeholders are expanded server-side (so a key need not depend on the
sandbox run env).
- A malformed `PAPERCLIP_OPENCODE_PROVIDERS` is no longer silently
ignored: invalid JSON, a non-object value, and individual provider
entries with non-object values (which are skipped by name) each append a
visible note to the run notes so the misconfiguration is diagnosable
(addresses both review P1s).
- `PAPERCLIP_OPENCODE_SMALL_MODEL` / `PAPERCLIP_OPENCODE_CHEAP_MODEL`:
pin the auxiliary (title-generation) and budget (recovery-retry) lanes
to gateway-served models; defaults unchanged.
- Honour `OPENCODE_ALLOW_ALL_MODELS` on the **remote** execution path
too (was local-only, a parity gap).
- `PAPERCLIP_OPENCODE_PRINT_LOGS`: optional toggle adding `--print-logs`
so OpenCode logs surface on stderr for diagnosing remote/sandbox runs.
- `buildOpenCodeModelProfiles()` guards its `process.env` default with
`typeof process` so the shared client/server module stays browser-safe
(a bare `process.env` at module load threw ReferenceError in the browser
under Vite dev middleware and broke UI rendering in the e2e lane).
## Verification
- `pnpm --filter @paperclipai/adapter-opencode-local build` and
`typecheck` (tsc clean)
- `pnpm exec vitest run packages/adapters/opencode-local/src` shows 33
passing (incl. new tests for the provider merge, `{env:}` expansion, the
malformed/non-object/skipped-entry provider notes, small/cheap-model
resolution, and the remote allow-all bypass)
- Manually verified end-to-end against a real
OpenAI-/Anthropic-compatible gateway: with the providers + small/cheap
model set, both the title-gen and main task route to the configured
gateway model and the agent completes (a real completion is returned and
billed). That deployment supplies the verification evidence; the
mechanism is gateway-agnostic.
## Risks
Low. Everything is env-driven and opt-in; with no env set the generated
config output is unchanged, and the cheap model profile keeps its model
(the only difference is its updated human-readable description).
Defaults preserved: built-in providers, Codex-mini cheap lane with
`variant: low`, no `--print-logs`. No migration/UI impact.
## Model Used
Claude Opus 4.8 (`claude-opus-4-8`, 1M context), extended thinking +
tool use, via Claude Code.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md (adapter-level opt-in config enabling
gateway routing; not the core sandbox-platform work, noted above)
- [x] I have searched GitHub for duplicate or related PRs and linked
them above (#5737, #5823)
- [x] I have either (a) linked existing issues OR (b) described the
issue in-PR following the relevant issue template
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots (n/a, no UI)
- [ ] I have updated relevant documentation to reflect my changes (env
vars documented inline via comments; no central doc references the
adapter env yet)
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(the P1 about silently dropped malformed providers JSON is addressed in
6eeb803, the follow-up P1 about silently skipped non-object entries in
2f4045a; the latest review has no further findings, and a re-review is
requested for the final note-copy/test-fixture polish at head)
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
> **Stacked PR (part 4 of 7).** Depends on:
- PR #5114
- PR #5115
- PR #5116
> Diff against `master` includes commits from earlier PRs in the stack —
the new commit in this PR is the topmost one.
## Thinking Path
> - Paperclip orchestrates AI agents for zero-human companies
> - When creating an OpenCode-local agent, Paperclip currently validates
> `adapterConfig.model` against the *Paperclip host's* `opencode models`
output
> - SSH testing surfaced that this blocks creating an OpenCode agent for
an SSH
> environment: the model that exists on the SSH target isn't visible to
the
> host, so creation fails with "OpenCode requires `adapterConfig.model`
in
> provider/model format" even when the operator picked a real remote
model
> - The initial direction was environment-aware model discovery; the
final
> decision was to keep OpenCode on the same explicit-model pattern as
other
> adapters (default + curated list + manual override) and stop blocking
> creation on host-side discovery
> - This PR does both: the adapter-models endpoint now accepts
`environmentId` and
> probes against the target environment, and the create-time hard gate
is
> replaced by `requireOpenCodeModelId` which validates `provider/model`
*format*
> without requiring host-local discovery. Test/run-time still surfaces
real
> auth/availability problems
> - The benefit is that operators can create OpenCode agents for remote
> environments without out-of-band setup, and the model picker in the UI
> reflects the actually-targeted environment
## What Changed
- Added `requireOpenCodeModelId(input)` in
`opencode-local/src/server/models.ts`,
exported it from the adapter index
- `ensureOpenCodeModelConfiguredAndAvailable` now delegates the format
check to
`requireOpenCodeModelId`
- `agentsApi.adapterModels(companyId, adapterType, { environmentId })`
now accepts
an environment ID and passes it as a query parameter
- `queryKeys.agents.adapterModels` now keys on `(companyId, adapterType,
environmentId)`
- `server/src/routes/agents.ts` reads and validates the new query
parameter,
forwarding it to the adapter's model probe
- `AgentConfigForm.tsx` and `OnboardingWizard.tsx` build the model query
key from
the currently selected default environment ID and disable autodetect for
`opencode_local` (model selection is explicit)
- `NewAgent.tsx` simplified — no longer special-cases OpenCode
autodetect
- `company-portability.ts` no longer needs OpenCode-specific autodetect
handling
- Tests added/updated:
`adapter-model-refresh-routes.test.ts`, `adapter-models.test.ts`,
`agent-permissions-routes.test.ts`,
`opencode-local/src/server/models.test.ts`
## Verification
- `pnpm --filter @paperclipai/server test -- adapter-models
adapter-model-refresh agent-permissions`
- `pnpm --filter @paperclipai/adapter-opencode-local test`
- `pnpm --filter @paperclipai/ui test -- AgentConfigForm
OnboardingWizard NewAgent`
- Manual QA in browser:
1. Boot Paperclip on Tailscale-bound port (so it's reachable from
another
machine), create an OpenCode-local agent, switch the default environment
between two installed sandboxes, and confirm the model list refreshes
per-environment
2. Submit with a malformed `provider/model` string and verify the new
`requireOpenCodeModelId` error surfaces
- Before/after screenshots attached for `AgentConfigForm` model picker
## Risks
- Behavioural shift: switching default environment now triggers a model
refetch.
Should be cheap but introduces a new UI loading state for OpenCode
users.
- Removing dynamic autodetect for OpenCode: if any user configured an
agent
without specifying `model` and relied on autodetect populating it, that
agent
will now fail at submit time. Mitigation: validation error is explicit
and
actionable.
- New query string parameter on `/api/companies/:id/adapter-models` —
older
clients that omit it still work (parameter is optional and defaults to
null).
## Model Used
- OpenAI GPT-5.4 (reasoning effort: high) via Codex CLI
- Provider: OpenAI
- Used to author the code changes in this PR
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [ ] I have updated relevant documentation to reflect my changes — N/A
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
Dotta
dmndbrp-oss
Philip D'Souza
Jannes Stubbemann
Devin Foley
Konan69