Commit Graph

108 Commits

Author SHA1 Message Date
Dotta 16b2b84d84
[codex] Improve agent runtime recovery and governance (#4086)
## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies.
> - The heartbeat runtime, agent import path, and agent configuration
defaults determine whether work is dispatched safely and predictably.
> - Several accumulated fixes all touched agent execution recovery, wake
routing, import behavior, and runtime concurrency defaults.
> - Those changes need to land together so the heartbeat service and
agent creation defaults stay internally consistent.
> - This pull request groups the runtime/governance changes from the
split branch into one standalone branch.
> - The benefit is safer recovery for stranded runs, bounded high-volume
reads, imported-agent approval correctness, skill-template support, and
a clearer default concurrency policy.

## What Changed

- Fixed stranded continuation recovery so successful automatic retries
are requeued instead of incorrectly blocking the issue.
- Bounded high-volume issue/log reads across issue, heartbeat, agent,
project, and workspace paths.
- Fixed imported-agent approval and instruction-path permission
handling.
- Quarantined seeded worktree execution state during worktree
provisioning.
- Queued approval follow-up wakes and hardened SQL_ASCII heartbeat
output handling.
- Added reusable agent instruction templates for hiring flows.
- Set the default max concurrent agent runs to five and updated related
UI/tests/docs.

## Verification

- `pnpm install --frozen-lockfile`
- `pnpm exec vitest run server/src/__tests__/company-portability.test.ts
server/src/__tests__/heartbeat-process-recovery.test.ts
server/src/__tests__/heartbeat-comment-wake-batching.test.ts
server/src/__tests__/heartbeat-list.test.ts
server/src/__tests__/issues-service.test.ts
server/src/__tests__/agent-permissions-routes.test.ts
packages/adapter-utils/src/server-utils.test.ts
ui/src/lib/new-agent-runtime-config.test.ts`
- Split integration check: merged this branch first, followed by the
other [PAP-1614](/PAP/issues/PAP-1614) branches, with no merge
conflicts.
- Confirmed this branch does not include `pnpm-lock.yaml`.

## Risks

- Medium risk: touches heartbeat recovery, queueing, and issue list
bounds in central runtime paths.
- Imported-agent and concurrency default behavior changes may affect
existing automation that assumes one-at-a-time default runs.
- No database migrations are included.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, GPT-5.4 tool-enabled coding model, agentic
code-editing/runtime with local shell and GitHub CLI access; exact
context window and reasoning mode are not exposed by the Paperclip
harness.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-04-20 06:19:48 -05:00
Dotta e89d3f7e11
[codex] Add backup endpoint and dev runtime hardening (#4087)
## Thinking Path

> - Paperclip is a local-first control plane for AI-agent companies.
> - Operators need predictable local dev behavior, recoverable instance
data, and scripts that do not churn the running app.
> - Several accumulated changes improve backup streaming, dev-server
health, static UI caching/logging, diagnostic-file ignores, and instance
isolation.
> - These are operational improvements that can land independently from
product UI work.
> - This pull request groups the dev-infra and backup changes from the
split branch into one standalone branch.
> - The benefit is safer local operation, easier manual backups, less
noisy dev output, and less cross-instance auth leakage.

## What Changed

- Added a manual instance database backup endpoint and route tests.
- Streamed backup/restore handling to avoid materializing large payloads
at once.
- Reduced dev static UI log/cache churn and ignored Node diagnostic
report captures.
- Added guarded dev auto-restart health polling coverage.
- Preserved worktree config during provisioning and scoped auth cookies
by instance.
- Added a Discord daily digest helper script and environment
documentation.
- Hardened adapter-route and startup feedback export tests around the
changed infrastructure.

## Verification

- `pnpm install --frozen-lockfile`
- `pnpm exec vitest run packages/db/src/backup-lib.test.ts
server/src/__tests__/instance-database-backups-routes.test.ts
server/src/__tests__/server-startup-feedback-export.test.ts
server/src/__tests__/adapter-routes.test.ts
server/src/__tests__/dev-runner-paths.test.ts
server/src/__tests__/health-dev-server-token.test.ts
server/src/__tests__/http-log-policy.test.ts
server/src/__tests__/vite-html-renderer.test.ts
server/src/__tests__/workspace-runtime.test.ts
server/src/__tests__/better-auth.test.ts`
- Split integration check: merged after the runtime/governance branch
and before UI branches with no merge conflicts.
- Confirmed this branch does not include `pnpm-lock.yaml`.

## Risks

- Medium risk: touches server startup, backup streaming, auth cookie
naming, dev health checks, and worktree provisioning.
- Backup endpoint behavior depends on existing board/admin access
controls and database backup helpers.
- No database migrations are included.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, GPT-5.4 tool-enabled coding model, agentic
code-editing/runtime with local shell and GitHub CLI access; exact
context window and reasoning mode are not exposed by the Paperclip
harness.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-04-20 06:08:55 -05:00
Dotta 7f893ac4ec
[codex] Harden execution reliability and heartbeat tooling (#3679)
## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies
> - Reliable execution depends on heartbeat routing, issue lifecycle
semantics, telemetry, and a fast enough local verification loop to keep
regressions visible
> - The remaining commits on this branch were mostly server/runtime
correctness fixes plus test and documentation follow-ups in that area
> - Those changes are logically separate from the UI-focused
issue-detail and workspace/navigation branches even when they touch
overlapping issue APIs
> - This pull request groups the execution reliability, heartbeat,
telemetry, and tooling changes into one standalone branch
> - The benefit is a focused review of the control-plane correctness
work, including the follow-up fix that restored the implicit
comment-reopen helpers after branch splitting

## What Changed

- Hardened issue/heartbeat execution behavior, including self-review
stage skipping, deferred mention wakes during active execution, stranded
execution recovery, active-run scoping, assignee resolution, and
blocked-to-todo wake resumption
- Reduced noisy polling/logging overhead by trimming issue run payloads,
compacting persisted run logs, silencing high-volume request logs, and
capping heartbeat-run queries in dashboard/inbox surfaces
- Expanded telemetry and status semantics with adapter/model fields on
task completion plus clearer status guidance in docs/onboarding material
- Updated test infrastructure and verification defaults with faster
route-test module isolation, cheaper default `pnpm test`, e2e isolation
from local state, and repo verification follow-ups
- Included docs/release housekeeping from the branch and added a small
follow-up commit restoring the implicit comment-reopen helpers that were
dropped during branch reconstruction

## Verification

- `pnpm vitest run
server/src/__tests__/issue-comment-reopen-routes.test.ts
server/src/__tests__/issue-telemetry-routes.test.ts`
- `pnpm vitest run server/src/__tests__/http-log-policy.test.ts
server/src/__tests__/heartbeat-run-log.test.ts
server/src/__tests__/health.test.ts`
- `server/src/__tests__/activity-service.test.ts`,
`server/src/__tests__/heartbeat-comment-wake-batching.test.ts`, and
`server/src/__tests__/heartbeat-process-recovery.test.ts` were attempted
on this host but the embedded Postgres harness reported
init-script/data-dir problems and skipped or failed to start, so they
are noted as environment-limited

## Risks

- Medium: this branch changes core issue/heartbeat routing and
reopen/wakeup behavior, so regressions would affect agent execution flow
rather than isolated UI polish
- Because it also updates verification infrastructure, reviewers should
pay attention to whether the new tests are asserting the right failure
modes and not just reshaping harness behavior

## Model Used

- OpenAI Codex coding agent (GPT-5-class runtime in Codex CLI; exact
deployed model ID is not exposed in this environment), reasoning
enabled, tool use and local code execution enabled

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-04-14 13:34:52 -05:00
Dotta e89076148a
[codex] Improve workspace runtime and navigation ergonomics (#3680)
## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies
> - That operator experience depends not just on issue chat, but also on
how workspaces, inbox groups, and navigation state behave over
long-running sessions
> - The current branch included a separate cluster of workspace-runtime
controls, inbox grouping, sidebar ordering, and worktree lifecycle fixes
> - Those changes cross server, shared contracts, database state, and UI
navigation, but they still form one coherent operator workflow area
> - This pull request isolates the workspace/runtime and navigation
ergonomics work into one standalone branch
> - The benefit is better workspace recovery and navigation persistence
without forcing reviewers through the unrelated issue-detail/chat work

## What Changed

- Improved execution workspace and project workspace controls, request
wiring, layout, and JSON editor ergonomics
- Hardened linked worktree reuse/startup behavior and documented the
`worktree repair` flow for recovering linked worktrees safely
- Added inbox workspace grouping, mobile collapse, archive undo,
keyboard navigation, shared group-header styling, and persisted
collapsed-group behavior
- Added persistent sidebar order preferences with the supporting DB
migration, shared/server contracts, routes, services, hooks, and UI
integration
- Scoped issue-list preferences by context and added targeted UI/server
tests for workspace controls, inbox behavior, sidebar preferences, and
worktree validation

## Verification

- `pnpm vitest run
server/src/__tests__/sidebar-preferences-routes.test.ts
ui/src/pages/Inbox.test.tsx
ui/src/components/ProjectWorkspaceSummaryCard.test.tsx
ui/src/components/WorkspaceRuntimeControls.test.tsx
ui/src/api/workspace-runtime-control.test.ts`
- `server/src/__tests__/workspace-runtime.test.ts` was attempted, but
the embedded Postgres suite self-skipped/hung on this host after
reporting an init-script issue, so it is not counted as a local pass
here

## Risks

- Medium: this branch includes migration-backed preference storage plus
worktree/runtime behavior, so merge review should pay attention to state
persistence and worktree recovery semantics
- The sidebar preference migration is standalone, but it should still be
watched for conflicts if another migration lands first

## Model Used

- OpenAI Codex coding agent (GPT-5-class runtime in Codex CLI; exact
deployed model ID is not exposed in this environment), reasoning
enabled, tool use and local code execution enabled

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-04-14 12:57:11 -05:00
Dotta b6115424b1 fix: isolate dev runner worktree env 2026-04-11 08:27:25 -05:00
dotta 2a84e53c1b Introduce bind presets for deployment setup
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-11 07:09:07 -05:00
Dotta 8bdf4081ee chore: improve worktree tooling and security docs 2026-04-10 22:26:30 -05:00
dotta 46892ded18 Add worktree reseed command
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-09 06:14:12 -05:00
dotta 34589ad457 Add worktree reseed command 2026-04-08 06:02:34 -05:00
dotta bd6d07d0b4 fix(ui): polish issue detail timelines and attachments
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-02 11:51:40 -05:00
Cody (Radius Red) 420cd4fd8d chore(docker): improve base image and organize docker files
- Add wget, ripgrep, python3, and GitHub CLI (gh) to base image
- Add OPENCODE_ALLOW_ALL_MODELS=true to production ENV
- Move compose files, onboard-smoke Dockerfile to docker/
- Move entrypoint script to scripts/docker-entrypoint.sh
- Add Podman Quadlet unit files (pod, app, db containers)
- Add docker/README.md with build, compose, and quadlet docs
- Add scripts/docker-build-test.sh for local build validation
- Update all doc references for new file locations
- Keep main Dockerfile at project root (no .dockerignore changes needed)

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-01 11:36:27 +00:00
Dotta 5575399af1
Merge pull request #2048 from remdev/fix/codex-rpc-client-spawn-error
fix(codex) rpc client spawn error
2026-03-30 14:24:33 -05:00
dotta 868cfa8c50 Auto-apply dev:once migrations
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-29 10:55:26 -05:00
dotta 6793dde597 Add idempotent local dev service management
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-29 10:55:26 -05:00
Mikhail Batukhtin c98af52590 test(codex-local): regression for CodexRpcClient spawn ENOENT
Add a Vitest case that mocks `node:child_process.spawn` so the child
emits `error` (ENOENT) after the constructor attaches listeners.
`getQuotaWindows()` must resolve with `ok: false` instead of leaving an
unhandled `error` event on the process.

Register `packages/adapters/codex-local` in the root Vitest workspace.

Document in DEVELOPING.md that a missing `codex` binary should not take
down the API server during quota polling.
2026-03-29 14:43:51 +03:00
dotta a8894799e4 Align worktree provision with worktree init
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-26 12:35:19 -05:00
dotta d53714a145 fix: manage codex home per company by default 2026-03-20 14:44:27 -05:00
dotta 8fc399f511 Add guarded dev restart handling
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-20 08:50:00 -05:00
Dotta 6f931b8405 Add Docker setup for untrusted PR review in isolated containers
Adds a dedicated Docker environment for reviewing untrusted pull requests
with codex/claude, keeping CLI auth state in volumes and using a separate
scratch workspace for PR checkouts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 14:30:53 -05:00
Dotta aa799bba4c Fix worktree seed source selection
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-13 15:07:42 -05:00
Dotta cce9941464 Add worktree UI branding 2026-03-13 11:12:43 -05:00
Dotta b888f92718 Revert "Merge pull request #707 from paperclipai/nm/premerge-lockfile-refresh"
This reverts commit 56df8d3cf0, reversing
changes made to ac82cae39a.
2026-03-12 12:13:39 -05:00
Dotta 8808a33fe1 ci: refresh pnpm lockfile before merge 2026-03-12 10:52:17 -05:00
Dotta 397e6d0915 Document worktree CLI commands with full option reference
Add a "Worktree CLI Reference" subsection to doc/DEVELOPING.md with
complete option tables and examples for worktree init, worktree:make,
and worktree env subcommands.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-12 07:31:14 -05:00
Dotta 3a003e11cc
Merge pull request #545 from paperclipai/feat/worktree-and-routing-polish
Add worktree:make and routing polish
2026-03-10 17:38:17 -05:00
Dotta 80d87d3b4e Add paperclipai worktree:make command 2026-03-10 16:52:26 -05:00
Dotta c799fca313 Add worktree-specific favicon branding 2026-03-10 16:15:11 -05:00
Dotta 93a8b55ff8 Copy git hooks during worktree init 2026-03-10 14:55:35 -05:00
Dotta dfbb4f1ccb Add command-based worktree provisioning 2026-03-10 12:42:36 -05:00
Dotta 83738b45cd Fix worktree minimal clone startup 2026-03-10 10:13:05 -05:00
Dotta 4a67db6a4d Add minimal worktree seed mode 2026-03-10 10:13:05 -05:00
Dotta 0704854926 Add worktree init CLI for isolated development instances 2026-03-10 10:13:05 -05:00
zvictor bee9fdd207 ci: split workflows and move pnpm lockfile ownership to GitHub Actions 2026-03-06 21:21:28 -03:00
Dotta 36013c35d9 dev: make pnpm dev watch workspace package changes 2026-03-06 15:48:35 -06:00
Dotta 7a2ecff4f0 Add invite onboarding network host suggestions 2026-03-05 12:28:27 -06:00
Dotta bee24e880f Add Paperclip host networking guidance to OpenClaw smoke script 2026-03-05 12:22:14 -06:00
Dotta 1b98c2b279 Isolate OpenClaw smoke state to avoid stale auth drift 2026-03-05 11:42:50 -06:00
Dotta 529d53acc0 Pin OpenClaw Docker UI smoke defaults to OpenAI models 2026-03-05 11:29:29 -06:00
Dotta 6f98c5f25c Clarify zero-flag OpenClaw Docker UI smoke defaults 2026-03-05 11:14:30 -06:00
Dotta 9dbd72cffd Improve OpenClaw Docker UI smoke pairing ergonomics 2026-03-05 11:04:14 -06:00
Dotta 34d9122b45 Add one-command OpenClaw Docker UI smoke script 2026-03-05 09:15:46 -06:00
Dotta ec0b7daca2 Add paperclipai db:backup CLI command 2026-03-05 06:02:12 -06:00
Dotta c145074daf Add configurable automatic database backup scheduling 2026-03-04 18:03:23 -06:00
Dotta d4a2fc6464 Improve OpenClaw smoke auth error handling 2026-03-04 16:37:55 -06:00
Dotta be50daba42 Add OpenClaw onboarding text endpoint and join smoke harness 2026-03-04 16:29:14 -06:00
Dotta f60c1001ec refactor: rename packages to @paperclipai and CLI binary to paperclipai
Rename all workspace packages from @paperclip/* to @paperclipai/* and
the CLI binary from `paperclip` to `paperclipai` in preparation for
npm publishing. Bump CLI version to 0.1.0 and add package metadata
(description, keywords, license, repository, files). Update all
imports, documentation, user-facing messages, and tests accordingly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 08:45:26 -06:00
Dotta cabd16bc70 docs: sync docs and skills updates from backup branch 2026-03-02 16:44:10 -06:00
Forgotten e2c5b6698c feat: join request claim secrets, onboarding API, and company branding
Add secure claim secret flow for agent join requests with timing-safe
comparison, expiry, and one-time use. Expose machine-readable onboarding
manifests and skill index API endpoints. Add company brand color with
hex validation, pattern icon generation, and settings page integration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 16:33:20 -06:00
Forgotten 1e11806fa3 feat: Docker quickstart with Compose, docs, and improved Dockerfile
Rewrites Dockerfile to use bookworm-slim base, installs Claude and Codex
CLIs, adds docker-compose.quickstart.yml for one-command setup, and adds
DOCKER.md with usage instructions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 10:32:33 -06:00
Forgotten 9f049aa4f3 feat: resolve agent workspace from session/project/fallback
Heartbeat service resolves cwd from task session, project primary
workspace, or agent home directory (~/.paperclip/instances/.../workspaces/).
Adapters receive workspace context and forward it as env vars and
session params. cwd is now optional in adapter config.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 08:38:58 -06:00
Forgotten 85c0b9a3dc feat: private hostname guard for authenticated/private mode
Reject requests from unrecognised Host headers when running
authenticated/private. Adds server middleware, CLI `allowed-hostname`
command, config-schema field, and prompt support for configuring
allowed hostnames during onboard/configure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 19:43:52 -06:00
Forgotten 7f58e783dd feat: add dev-runner script with tailscale-auth mode
Replace inline env vars in package.json dev scripts with a dedicated
node script that supports --tailscale-auth for private-network dev.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 16:08:17 -06:00
Forgotten 21c506dcae docs: add deployment modes documentation and update plans
Add DEPLOYMENT-MODES.md with canonical mode taxonomy. Update CLI.md,
DEVELOPING.md, PRODUCT.md, and SPEC-implementation.md with local_trusted/
authenticated nomenclature. Revise humans-and-permissions plan with Better
Auth choice, bootstrap flow, unified invite semantics, and expanded criteria.
Add implementation guide and additional plan documents for cursor cloud
adapter and deployment auth mode consolidation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 14:41:35 -06:00
Forgotten fe1ede32b6 docs: add storage section to developer guide
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 10:33:47 -06:00
Forgotten 8f3fc077fa feat(cli): add client commands and home-based local runtime defaults 2026-02-20 07:10:58 -06:00
Forgotten 80a8ec26f1 Add secrets documentation and inline env migration script
Document secret storage in DATABASE.md and DEVELOPING.md. Update
SPEC-implementation with company_secrets schema and indexes. Add
migrate-inline-env-secrets script for converting existing plain
env values to managed secrets (dry-run by default, --apply to commit).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 15:44:11 -06:00
Forgotten cc24722090 Replace PGlite with embedded-postgres and add startup banner
Switch from PGlite (WebAssembly) to embedded-postgres for zero-config
local development — provides a real PostgreSQL server with full
compatibility. Add startup banner with config summary on server boot.
Improve server bootstrap with auto port detection, database creation,
and migration on startup. Update DATABASE.md, DEVELOPING.md, and
SPEC-implementation.md to reflect the change. Update CLI database
check and prompts. Simplify OnboardingWizard database options.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 11:45:43 -06:00
Forgotten fade29fc3e Reorganize docs and add implementation spec
Move GOAL.md, PRODUCT.md, SPEC.md from repo root into doc/. Add
AGENTS.md (contributor guidance), doc/DEVELOPING.md (dev setup),
doc/SPEC-implementation.md (V1 implementation contract), and
doc/specs/ui.md (UI design spec). Update ClipHub doc.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 09:07:15 -06:00