paperclip/packages/plugins/plugin-llm-wiki
Dotta ad961227f5
feat(secrets): add user-specific runtime secrets (#8825)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent runs often need provider credentials, API tokens, and other
environment-bound secrets.
> - Company-level secrets work for shared credentials, but they do not
model values that should differ by human operator.
> - Without a user-scoped model, a run can dispatch without knowing
whether the responsible human has supplied the needed value.
> - Paperclip also needs run attribution to make those user-scoped
runtime checks deterministic and auditable.
> - This pull request adds user-specific secret definitions, per-user
values, environment bindings, responsible-user attribution, and runtime
resolution gates.
> - The benefit is that teams can define the secret once, let each user
provide their own value, and block runs before dispatch when required
user secrets or active definitions are unavailable.

## Linked Issues or Issue Description

Refs #224
Refs #6057

This PR implements user-specific secret support as a core
secret-management capability rather than a one-off adapter setting. It
is related to existing public work on company secrets UI and runtime
secret refs, but is distinct because the value is owned by the
responsible user and resolved at run dispatch time.

Related PR search before opening found existing secrets work such as
#1550, #8256, #8614, #8634, and #8647; none of those add the full
user-secret definition/value/runtime gate covered here.

## What Changed

- Added user-secret definitions and per-user "My secrets" values,
keeping stored values out of access metadata.
- Added `user_secret_ref` environment bindings and UI affordances to
pick them alongside existing secret refs.
- Added responsible-user runtime resolution so user-secret refs resolve
against the human responsible for the run.
- Added pre-dispatch missing-secret gates so runs fail before adapter
dispatch when required user values are absent or definitions are
inactive.
- Added low-trust allowlist hardening for user-secret runtime access.
- Added issue, routine, run, and agent API key responsible-user
attribution and fail-closed dispatch behavior when attribution cannot be
resolved.
- Added denial-copy mapping so responsible-user authorization failures
surface as actionable run outcomes instead of opaque setup failures.
- Added OpenAPI documentation for the user-secret routes.
- Rebases cleanly on current `master`; migrations were renumbered
incrementally as `0128_user_specific_secrets`,
`0129_agent_api_key_responsible_user`, and
`0130_run_responsible_user_invariant` after upstream `0126`/`0127`
migrations.
- Removed previously committed local design screenshots so the PR
contains code/docs/tests only.

## Verification

- PASS: PR head `2527febd106bcf3ca264ca0da7fca491084192d6` is based on
`paperclipai/paperclip:master`.
- PASS: `git diff --check`
- PASS: `git diff --name-only public/master...HEAD | rg
'^(pnpm-lock\\.yaml|\\.github/workflows/|screenshots/)' || true`
produced no files.
- PASS: migration journal audit confirmed unique indexes through `130`
with tail entries `0126_issue_comment_derived_attribution`,
`0127_environment_custom_images_instance_scoped`,
`0128_user_specific_secrets`, `0129_agent_api_key_responsible_user`, and
`0130_run_responsible_user_invariant`.
- PASS: `pnpm --filter @paperclipai/ui typecheck`
- PASS: `pnpm --filter @paperclipai/server typecheck`
- PASS: `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/heartbeat-responsible-user-invariant.test.ts`
- PASS: `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/heartbeat-active-run-output-watchdog.test.ts
src/__tests__/heartbeat-stale-queue-invalidation.test.ts
src/__tests__/heartbeat-workspace-finalize-branch.test.ts
src/__tests__/issue-monitor-scheduler.test.ts`
- PASS: `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/heartbeat-comment-wake-batching.test.ts
src/__tests__/heartbeat-retry-scheduling.test.ts
src/__tests__/heartbeat-accepted-plan-workspace-refresh.test.ts
src/__tests__/heartbeat-plugin-environment.test.ts`
- PASS: `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/low-trust-red-team-routes.test.ts`
- PASS: `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/secrets-service.test.ts` (55 tests)
- PASS: `pnpm vitest run server/src/__tests__/secrets-routes.test.ts
server/src/__tests__/secrets-service.test.ts` (89 tests after final
Greptile cleanup fixes)
- PASS: `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/heartbeat-issue-liveness-escalation.test.ts` (17 tests
after the final rebase CI fix)
- PASS: focused server Vitest batches covering heartbeat recovery,
project env, plugin env, routines, low-trust, pipelines, monitors,
watchdog, and stale queue paths.
- PASS: GitHub checks are green on
`2527febd106bcf3ca264ca0da7fca491084192d6`, including Typecheck +
Release Registry, Build, General tests, serialized server suites, e2e,
Canary Dry Run, verify, security checks, and Greptile Review.
- PASS: Greptile Review completed successfully on
`2527febd106bcf3ca264ca0da7fca491084192d6` with Confidence Score 5/5,
and GraphQL review-thread audit returned zero unresolved non-outdated
threads.

## Risks

- Runtime behavior now depends on a run having a correct responsible
user; missing or incorrect responsibility assignment can block runs
before adapter dispatch.
- `user_secret_ref` bindings intentionally expose metadata without
values, but UI/API callers may need to handle the new binding kind
explicitly.
- External secret providers and IAM policies are not automatically
provisioned by this PR; operators still need to configure provider-side
access for non-local vaults.
- The PR is broad across db/shared/server/UI/runtime paths, so release
validation should include both API and UI secret workflows before merge.
- The migration renumbering is intentionally incremental after upstream
migrations; the branch migrations use guarded
column/table/index/constraint creation so users who tested the older
draft numbering should not hit duplicate DDL for the existing objects.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, GPT-5-based coding agent (`gpt-5`), Codex local adapter
with shell/tool use and code execution. Context window and internal
reasoning mode are not exposed by the runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 05:58:20 -05:00
..
agents/wiki-maintainer
fixtures/basic-root
migrations
skills
src [codex] Roll up May 17 branch changes (#6210) 2026-05-17 17:15:06 -05:00
templates
tests feat(secrets): add user-specific runtime secrets (#8825) 2026-07-05 05:58:20 -05:00
.gitignore
README.md
esbuild.config.mjs
package.json build(deps): align react-dom with react@19.2.7 (replaces dependabot #8471) (#8557) 2026-06-23 11:56:23 -07:00
rollup.config.mjs
tsconfig.json
vitest.config.ts

README.md

LLM Wiki

Local-file LLM Wiki plugin for source ingestion, wiki browsing, query, lint, and maintenance workflows.

Scope

This package is the standalone home for LLM Wiki behavior. Wiki-specific routes, UI, prompts, tools, local-folder templates, migrations, fixtures, and tests live here rather than in Paperclip core.

The alpha surface includes:

  • manifest-declared Wiki page, sidebar entry, and settings page
  • trusted local folder declaration for raw/, wiki/, AGENTS.md, IDEA.md, wiki/index.md, and wiki/log.md
  • plugin database namespace migration for wiki instances, sources, pages, operations, query sessions, and resource bindings
  • managed Wiki Maintainer agent, managed LLM Wiki project, and paused managed routines for wiki update processing, lint, and index refresh
  • plugin-operation issue creation using surfaceVisibility: "plugin_operation"
  • local source capture into raw/ with metadata rows in the plugin DB namespace
  • opt-in company-scoped Paperclip event ingestion controls for issues, comments, and documents; event ingestion is disabled by default and routes captured raw provenance into the default space only
  • manual Paperclip project/root issue distillation and bounded backfill actions with explicit work items, operation issues, source caps, and estimated cost recording
  • Paperclip-derived distillation (cursor windows, manual distill-now, backfill) always writes into the default wiki space in Phase 1; non-default spaces remain on manual / raw-file ingest until per-space Paperclip ingestion profiles ship
  • Paperclip-derived distillation maintains wiki/projects/<slug>/standup.md as the executive current-state view for each represented project, alongside durable wiki/projects/<slug>/index.md knowledge pages
  • wiki page writes with plugin path validation, atomic local-folder writes, metadata/revision rows, backlink extraction, and optional stale-hash protection
  • wiki tools for search/read/write/propose patch/source/log/index/backlinks workflows

Phase 5 Security Gate

Paperclip-derived text ingestion stays limited to issue titles/descriptions, issue comments, and issue documents.

  • Issue attachments/assets are metadata-only in Phase 5.
  • Issue work products are metadata-only in Phase 5.
  • The wiki must not fetch /api/assets/:id/content, dereference work-product url fields, or store those capability-bearing links in source bundles/snapshots.

The accepted policy lives in doc/plans/2026-05-06-llm-wiki-paperclip-asset-security-gate.md.

Development

pnpm install
pnpm dev            # watch builds
pnpm dev:ui         # local dev server with hot-reload events
pnpm test

From the Paperclip repo root:

pnpm --filter @paperclipai/plugin-llm-wiki typecheck
pnpm --filter @paperclipai/plugin-llm-wiki test
pnpm --filter @paperclipai/plugin-llm-wiki build

Alpha Verification

Run these commands from the Paperclip repo root before handing off alpha plugin changes:

pnpm --filter @paperclipai/plugin-llm-wiki typecheck
pnpm --filter @paperclipai/plugin-llm-wiki test
pnpm --filter @paperclipai/plugin-llm-wiki build

The focused Vitest suite covers:

  • standalone package boundaries and package-local harness dependencies
  • required local folder bootstrap writes
  • raw source capture plus ingest metadata persistence
  • hidden plugin-operation issue creation for ingest/query/file-as-page workflows
  • disabled and enabled Paperclip event ingestion paths
  • managed routine declarations, manual distill/backfill work items, source cap handling, and backfill project/date scoping
  • atomic page writes, metadata/revision rows, backlinks, and stale-hash refusal
  • query session creation, run-id recording, stream event forwarding, and completion updates
  • filing a streamed query answer back into the wiki through a hidden operation

Remaining alpha gaps:

  • Browser screenshot capture is maintained separately under tests/screenshots; generated screenshots/ outputs are local artifacts and are ignored by git.
  • Host-level plugin install and live agent invocation still need Paperclip server/runtime smoke coverage when preparing a release candidate.

Install Into Paperclip

curl -X POST http://127.0.0.1:3100/api/plugins/install \
  -H "Content-Type: application/json" \
  -d '{"packageName":"/Users/dotta/paperclip/.paperclip/worktrees/PAP-3179-design-a-llm-wiki-plugin/packages/plugins/plugin-llm-wiki","isLocalPath":true}'

Build Options

  • pnpm build uses esbuild presets from @paperclipai/plugin-sdk/bundlers.
  • pnpm build:rollup uses rollup presets from the same SDK.

After changing manifest-loaded assets such as skills, agent instructions, or templates, recompile the local plugin before re-enabling it:

pnpm --filter @paperclipai/plugin-llm-wiki build

The package-local dist/ directory is ignored by git, but local Paperclip installs load the compiled dist/manifest.js and dist/worker.js files at runtime. If activation failed before the rebuild, re-enable the plugin or restart the Paperclip dev server so the host imports the fresh bundle.

Local File Layout

<configured-wiki-root>/
  AGENTS.md
  IDEA.md
  .gitignore
  raw/
    .gitkeep
  wiki/
    index.md
    log.md
    sources/
      .gitkeep
    projects/
      .gitkeep
      <project-slug>/
        index.md
        standup.md
        decisions.md
        history.md
    entities/
      .gitkeep
    concepts/
      .gitkeep
    synthesis/
      .gitkeep

Use the settings page or bootstrap-root action to configure the folder and write the starter files. The plugin uses Paperclip's local folder API for path containment, symlink checks, read/write validation, and atomic writes.

Bootstrap preserves existing files rather than overwriting operator edits. The default first-install skeleton is copied from the vanilla LLM Wiki layout, with CLAUDE.md renamed to AGENTS.md and Paperclip project overviews, standups, decisions, and history kept together under wiki/projects/<slug>/.

Managed Agent Instructions

Plugin-managed agent instruction bundles live under:

agents/<agent-key>/AGENTS.md

For this plugin the Wiki Maintainer source bundle is agents/wiki-maintainer/AGENTS.md. Any additional files in that folder are installed as sibling instruction files for the managed agent. The settings health check reports drift from these defaults, and resetting the managed agent asks for confirmation before replacing customized instructions.