Add Paperclip page publishing skill (#8664)

Reviewed by CTO for PAP-12039. Additive operational skill with validated S3/CloudFront publishing helper, defensive overwrite/symlink/hidden-file checks, focused Node tests, and full green CI/security/Greptile checks. Merged via maintainer path after CODEOWNERS review request was accepted as good enough for now.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Dotta 2026-06-26 13:49:13 -05:00 committed by GitHub
parent 574543d7d3
commit de3b143fef
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 1278 additions and 0 deletions

View File

@ -0,0 +1,621 @@
# Paperclip Page Skill
`paperclip-page` publishes static page directories to a Paperclip-controlled S3
bucket served through CloudFront. It is the durable Paperclip-owned replacement
for quick `here.now`-style page sharing.
The v1 security posture is:
- CloudFront + ACM + Origin Access Control in front of a private S3 REST origin.
- Public content only.
- Dedicated uploader IAM identity, separate from Paperclip attachment storage.
- No `s3:DeleteObject`, no `aws s3 sync --delete`, and no bucket/IAM/DNS changes
from the publish helper.
- Symlinks, hidden files, unsafe slugs, and accidental overwrites are rejected.
## Agent Quick Start
Build or prepare a static directory with `index.html` at its root:
```bash
site/
index.html
assets/app.css
assets/app.js
```
Validate without AWS writes:
```bash
skills/paperclip-page/scripts/publish.sh ./site --slug demo --dry-run
```
Publish:
```bash
skills/paperclip-page/scripts/publish.sh ./site --slug demo
```
Update an existing page from the same source directory:
```bash
skills/paperclip-page/scripts/publish.sh ./site --slug demo --update
```
The helper prints:
- public URL
- S3 key prefix
- local ownership state path
## Source Directory Rules
- `index.html` must exist at the directory root.
- Source directory itself must not be a symlink.
- No symlinks anywhere in the tree.
- No hidden files or dot paths in published content.
- `.paperclip-page/state.json` is allowed and excluded from uploads.
- Do not publish secrets, credentials, internal logs, private company material,
customer data, or regulated data.
Add this to the publishing repo or generated site `.gitignore` when the source
directory lives in a git checkout:
```gitignore
.paperclip-page/
```
## Environment Variables
Required for live publishes:
```bash
export AWS_REGION=us-east-1
export PAPERCLIP_PAGE_BUCKET=paperclip-pages-prod
export PAPERCLIP_PAGE_BASE_URL=https://pages.paperclip.ing
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
```
Optional:
```bash
export PAPERCLIP_PAGE_DEFAULT_PREFIX=""
export PAPERCLIP_PAGE_AWS_PROFILE=paperclip-page-uploader
```
Recommended Paperclip secret names:
- `paperclip-page-aws-access-key-id`
- `paperclip-page-aws-secret-access-key`
Bind those secrets into publisher agents as `AWS_ACCESS_KEY_ID` and
`AWS_SECRET_ACCESS_KEY`. Do not reuse Paperclip's internal S3 attachment/object
storage credentials.
## AWS Setup
Run setup with an operator/admin AWS profile. Agents using this skill should not
create buckets, mutate IAM, change DNS, or manage CloudFront.
```bash
export AWS_PROFILE=paperclip-admin
export AWS_REGION=us-east-1
export BUCKET=paperclip-pages-prod
export DOMAIN=pages.paperclip.ing
export UPLOADER_USER=paperclip-page-uploader
export CLOUDFRONT_COMMENT="Paperclip pages"
aws sts get-caller-identity --profile "$AWS_PROFILE"
```
Create the bucket:
```bash
aws s3api create-bucket \
--profile "$AWS_PROFILE" \
--region "$AWS_REGION" \
--bucket "$BUCKET"
```
For regions other than `us-east-1`, add:
```bash
--create-bucket-configuration LocationConstraint="$AWS_REGION"
```
Disable ACLs and keep ownership bucket-enforced:
```bash
aws s3api put-bucket-ownership-controls \
--profile "$AWS_PROFILE" \
--bucket "$BUCKET" \
--ownership-controls '{"Rules":[{"ObjectOwnership":"BucketOwnerEnforced"}]}'
```
Block public access. CloudFront reads through OAC, so the bucket does not need a
public website policy:
```bash
aws s3api put-public-access-block \
--profile "$AWS_PROFILE" \
--bucket "$BUCKET" \
--public-access-block-configuration \
'BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true'
```
Enable versioning:
```bash
aws s3api put-bucket-versioning \
--profile "$AWS_PROFILE" \
--bucket "$BUCKET" \
--versioning-configuration Status=Enabled
```
Enable default encryption:
```bash
aws s3api put-bucket-encryption \
--profile "$AWS_PROFILE" \
--bucket "$BUCKET" \
--server-side-encryption-configuration '{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"AES256"}}]}'
```
Upload an operator-managed root `404.html` before creating the distribution:
```bash
mkdir -p /tmp/paperclip-pages-bootstrap
printf '<!doctype html><title>Not found</title><h1>Not found</h1>\n' \
> /tmp/paperclip-pages-bootstrap/404.html
aws s3 cp /tmp/paperclip-pages-bootstrap/404.html "s3://$BUCKET/404.html" \
--profile "$AWS_PROFILE" \
--content-type text/html \
--cache-control 'public,max-age=60'
```
Create an ACM certificate in `us-east-1` for CloudFront:
```bash
export ACM_REGION=us-east-1
aws acm request-certificate \
--profile "$AWS_PROFILE" \
--region "$ACM_REGION" \
--domain-name "$DOMAIN" \
--validation-method DNS \
--idempotency-token paperclippages \
> /tmp/paperclip-pages-acm.json
export CERT_ARN="$(jq -r '.CertificateArn' /tmp/paperclip-pages-acm.json)"
aws acm describe-certificate \
--profile "$AWS_PROFILE" \
--region "$ACM_REGION" \
--certificate-arn "$CERT_ARN" \
--query 'Certificate.DomainValidationOptions[].ResourceRecord'
```
Add the returned DNS validation record in Cloudflare, then wait:
```bash
aws acm wait certificate-validated \
--profile "$AWS_PROFILE" \
--region "$ACM_REGION" \
--certificate-arn "$CERT_ARN"
```
Create a CloudFront Origin Access Control:
```bash
aws cloudfront create-origin-access-control \
--profile "$AWS_PROFILE" \
--origin-access-control-config "{
\"Name\":\"paperclip-pages-oac\",
\"Description\":\"OAC for $BUCKET\",
\"SigningProtocol\":\"sigv4\",
\"SigningBehavior\":\"always\",
\"OriginAccessControlOriginType\":\"s3\"
}" \
> /tmp/paperclip-pages-oac.json
export OAC_ID="$(jq -r '.OriginAccessControl.Id' /tmp/paperclip-pages-oac.json)"
```
Create and publish a CloudFront Function so clean page URLs such as `/demo/`
load `/demo/index.html` from the S3 REST origin:
```bash
cat > paperclip-pages-index-router.js <<'EOF'
function handler(event) {
var request = event.request;
var uri = request.uri;
if (uri.endsWith('/')) {
request.uri = uri + 'index.html';
return request;
}
var lastSegment = uri.substring(uri.lastIndexOf('/') + 1);
if (lastSegment.indexOf('.') === -1) {
request.uri = uri + '/index.html';
}
return request;
}
EOF
aws cloudfront create-function \
--profile "$AWS_PROFILE" \
--name paperclip-pages-index-router \
--function-config 'Comment=Rewrite clean page URLs to index.html,Runtime=cloudfront-js-2.0' \
--function-code fileb://paperclip-pages-index-router.js \
> /tmp/paperclip-pages-function.json
export FUNCTION_ETAG="$(jq -r '.ETag' /tmp/paperclip-pages-function.json)"
aws cloudfront publish-function \
--profile "$AWS_PROFILE" \
--name paperclip-pages-index-router \
--if-match "$FUNCTION_ETAG" \
> /tmp/paperclip-pages-function-live.json
export FUNCTION_ARN="$(jq -r '.FunctionSummary.FunctionMetadata.FunctionARN' /tmp/paperclip-pages-function-live.json)"
```
Create `cloudfront-config.json`:
```bash
export CALLER_REFERENCE="paperclip-pages-$(date +%s)"
jq -n \
--arg caller "$CALLER_REFERENCE" \
--arg comment "$CLOUDFRONT_COMMENT" \
--arg domain "$DOMAIN" \
--arg bucket "$BUCKET" \
--arg oac "$OAC_ID" \
--arg functionArn "$FUNCTION_ARN" \
--arg cert "$CERT_ARN" \
'{
CallerReference: $caller,
Comment: $comment,
Enabled: true,
IsIPV6Enabled: true,
Aliases: {Quantity: 1, Items: [$domain]},
Origins: {
Quantity: 1,
Items: [{
Id: "s3-origin",
DomainName: ($bucket + ".s3.amazonaws.com"),
OriginAccessControlId: $oac,
S3OriginConfig: {OriginAccessIdentity: ""}
}]
},
DefaultRootObject: "index.html",
DefaultCacheBehavior: {
TargetOriginId: "s3-origin",
ViewerProtocolPolicy: "redirect-to-https",
AllowedMethods: {Quantity: 2, Items: ["GET", "HEAD"], CachedMethods: {Quantity: 2, Items: ["GET", "HEAD"]}},
Compress: true,
CachePolicyId: "658327ea-f89d-4fab-a63d-7e88639e58f6",
OriginRequestPolicyId: "88a5eaf4-2fd4-4709-b370-b4c650ea3fcf",
FunctionAssociations: {
Quantity: 1,
Items: [{EventType: "viewer-request", FunctionARN: $functionArn}]
}
},
CustomErrorResponses: {
Quantity: 1,
Items: [{ErrorCode: 403, ResponsePagePath: "/404.html", ResponseCode: "404", ErrorCachingMinTTL: 60}]
},
ViewerCertificate: {
ACMCertificateArn: $cert,
SSLSupportMethod: "sni-only",
MinimumProtocolVersion: "TLSv1.2_2021"
},
Restrictions: {GeoRestriction: {RestrictionType: "none", Quantity: 0}}
}' > cloudfront-config.json
```
Create the distribution:
```bash
aws cloudfront create-distribution \
--profile "$AWS_PROFILE" \
--distribution-config file://cloudfront-config.json \
> /tmp/paperclip-pages-cloudfront.json
export DISTRIBUTION_ID="$(jq -r '.Distribution.Id' /tmp/paperclip-pages-cloudfront.json)"
export DISTRIBUTION_DOMAIN="$(jq -r '.Distribution.DomainName' /tmp/paperclip-pages-cloudfront.json)"
```
Grant CloudFront read access to the private bucket:
```bash
export ACCOUNT_ID="$(aws sts get-caller-identity --profile "$AWS_PROFILE" --query Account --output text)"
jq -n \
--arg bucket "$BUCKET" \
--arg account "$ACCOUNT_ID" \
--arg distribution "$DISTRIBUTION_ID" \
'{
Version: "2012-10-17",
Statement: [{
Sid: "AllowCloudFrontServicePrincipalReadOnly",
Effect: "Allow",
Principal: {Service: "cloudfront.amazonaws.com"},
Action: "s3:GetObject",
Resource: ("arn:aws:s3:::" + $bucket + "/*"),
Condition: {
StringEquals: {
"AWS:SourceArn": ("arn:aws:cloudfront::" + $account + ":distribution/" + $distribution)
}
}
}]
}' > bucket-policy.json
aws s3api put-bucket-policy \
--profile "$AWS_PROFILE" \
--bucket "$BUCKET" \
--policy file://bucket-policy.json
```
Create the uploader IAM user:
```bash
aws iam create-user \
--profile "$AWS_PROFILE" \
--user-name "$UPLOADER_USER"
```
Create `paperclip-page-uploader-policy.json`. This policy supports collision
checks and additive uploads under slug prefixes while protecting root bootstrap
objects such as `404.html`.
```bash
jq -n \
--arg bucket "$BUCKET" \
'{
Version: "2012-10-17",
Statement: [
{
Sid: "ListPublishedPagePrefixes",
Effect: "Allow",
Action: ["s3:ListBucket"],
Resource: ("arn:aws:s3:::" + $bucket)
},
{
Sid: "ReadPublishedPages",
Effect: "Allow",
Action: ["s3:GetObject"],
Resource: ("arn:aws:s3:::" + $bucket + "/*")
},
{
Sid: "WritePublishedPageObjects",
Effect: "Allow",
Action: ["s3:PutObject"],
Resource: ("arn:aws:s3:::" + $bucket + "/*/*")
},
{
Sid: "DenyReservedRootWrites",
Effect: "Deny",
Action: ["s3:PutObject", "s3:DeleteObject", "s3:PutObjectTagging"],
Resource: [
("arn:aws:s3:::" + $bucket + "/404.html"),
("arn:aws:s3:::" + $bucket + "/index.html")
]
}
]
}' > paperclip-page-uploader-policy.json
```
Attach it:
```bash
aws iam put-user-policy \
--profile "$AWS_PROFILE" \
--user-name "$UPLOADER_USER" \
--policy-name PaperclipPagePublisher \
--policy-document file://paperclip-page-uploader-policy.json
```
Create access keys and treat the output as secret material:
```bash
aws iam create-access-key \
--profile "$AWS_PROFILE" \
--user-name "$UPLOADER_USER" \
> /tmp/paperclip-page-uploader-key.json
chmod 600 /tmp/paperclip-page-uploader-key.json
```
## Cloudflare DNS
Use DNS-only or proxied CNAME to CloudFront. Do not point v1 at the S3 website
endpoint.
Cloudflare UI:
- Open the `paperclip.ing` zone.
- Add `CNAME`:
- Name: `pages`
- Target: the CloudFront domain, for example `d111111abcdef8.cloudfront.net`
- Proxy status: DNS only or Proxied
- TTL: Auto
API equivalent:
```bash
export CF_ZONE_ID=<paperclip.ing-zone-id>
export CF_API_TOKEN=<token-with-zone-dns-edit>
curl -sS -X POST "https://api.cloudflare.com/client/v4/zones/$CF_ZONE_ID/dns_records" \
-H "Authorization: Bearer $CF_API_TOKEN" \
-H "Content-Type: application/json" \
--data "$(jq -n \
--arg name pages \
--arg content "$DISTRIBUTION_DOMAIN" \
'{type:"CNAME", name:$name, content:$content, ttl:1, proxied:false}')"
```
Smoke check:
```bash
curl -I "https://$DOMAIN/404.html"
```
## Paperclip Secrets
Create secrets from environment variables so values do not land in shell history:
```bash
export PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID="$(jq -r '.AccessKey.AccessKeyId' /tmp/paperclip-page-uploader-key.json)"
export PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY="$(jq -r '.AccessKey.SecretAccessKey' /tmp/paperclip-page-uploader-key.json)"
pnpm paperclipai secrets create \
--company-id <company-id> \
--name paperclip-page-aws-access-key-id \
--value-env PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID
pnpm paperclipai secrets create \
--company-id <company-id> \
--name paperclip-page-aws-secret-access-key \
--value-env PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY
```
Bind runtime env to publishing agents:
```json
{
"AWS_ACCESS_KEY_ID": {
"type": "secret_ref",
"secretId": "<access-key-secret-id>",
"version": "latest"
},
"AWS_SECRET_ACCESS_KEY": {
"type": "secret_ref",
"secretId": "<secret-key-secret-id>",
"version": "latest"
},
"AWS_REGION": { "type": "plain", "value": "us-east-1" },
"PAPERCLIP_PAGE_BUCKET": { "type": "plain", "value": "paperclip-pages-prod" },
"PAPERCLIP_PAGE_BASE_URL": { "type": "plain", "value": "https://pages.paperclip.ing" },
"PAPERCLIP_PAGE_DEFAULT_PREFIX": { "type": "plain", "value": "" }
}
```
## Install And Attach
Create or update the company skill from this package:
```bash
pnpm paperclipai skills create \
--company-id <company-id> \
--name "Paperclip Page" \
--slug paperclip-page \
--description "Publish static pages to the Paperclip pages host" \
--body-file skills/paperclip-page/SKILL.md
```
Attach it to an agent:
```bash
pnpm paperclipai skills agent sync <agent-id-or-shortname> \
--company-id <company-id> \
--skill paperclip-page
```
Ensure the agent can read this directory or copy the package into the installed
company skill location with `scripts/publish.sh` preserved as executable.
## Credential Rotation
1. Create a second access key:
```bash
aws iam create-access-key \
--profile "$AWS_PROFILE" \
--user-name "$UPLOADER_USER" \
> /tmp/paperclip-page-uploader-key-rotation.json
chmod 600 /tmp/paperclip-page-uploader-key-rotation.json
```
2. Store new secret versions in Paperclip Secrets.
3. Update agent env bindings to the new versions or `latest`.
4. Run a dry-run and a small publish smoke.
5. Disable the old key:
```bash
aws iam update-access-key \
--profile "$AWS_PROFILE" \
--user-name "$UPLOADER_USER" \
--access-key-id <old-access-key-id> \
--status Inactive
```
6. Delete the old key after the next successful publish:
```bash
aws iam delete-access-key \
--profile "$AWS_PROFILE" \
--user-name "$UPLOADER_USER" \
--access-key-id <old-access-key-id>
```
## Troubleshooting
`AccessDenied` on upload:
- Confirm agent env contains the uploader key, not the admin key.
- Confirm the uploader policy allows `s3:ListBucket`, `s3:GetObject`, and
`s3:PutObject`.
- Confirm uploads target `<slug>/...` so the `arn:aws:s3:::<bucket>/*/*`
object ARN matches.
`Slug already exists`:
- Use a different slug.
- Or run `--update` from the original source directory that has
`.paperclip-page/state.json`.
Generated slug collides:
- The helper appends a short suffix for generated slugs when AWS reports a
collision. Explicit slugs fail instead of silently changing the URL.
URL 404s after upload:
- Check `curl -I https://<domain>/<slug>/`.
- Check CloudFront distribution deployment status.
- Check DNS CNAME target.
- Check the object exists at `s3://<bucket>/<slug>/index.html`.
Stale browser cache:
- The helper uses `Cache-Control: public,max-age=60`.
- Wait a minute or issue a CloudFront invalidation if the operator wants an
immediate refresh.
CloudFront returns 403:
- Confirm the bucket policy references the correct distribution ARN.
- Confirm OAC is attached to the S3 origin.
- Confirm the bucket is private and public access block is enabled.
## Public Content Security Notes
Anything published with this skill is public. The tool cannot reliably classify
generated files, so the publishing agent must inspect content before uploading.
Do not publish:
- API keys, OAuth tokens, cookies, or `.env` files
- internal customer data
- private company docs
- unpublished security reports
- raw transcripts that may contain secrets
Recovery after accidental overwrite uses S3 versioning. Because v1 uploader
credentials cannot delete objects, rollback should be performed by an operator
with admin credentials.

View File

@ -0,0 +1,90 @@
---
name: paperclip-page
description: >
Publish static HTML pages and asset folders to the Paperclip-approved S3 and
CloudFront website host. Use when asked to deploy, publish, host, or share a
persistent Paperclip page, wireframe viewer, prototype, report, or other static
site without using here.now.
---
# Paperclip Page
Use this skill to publish a static directory to the configured Paperclip pages
host, for example `https://pages.paperclip.ing/<slug>/`.
## Requirements
- Source directory contains `index.html` at its root.
- `aws` CLI v2, `curl`, and `jq` are available on PATH for live publishes.
- Environment variables are configured:
- `PAPERCLIP_PAGE_BUCKET`
- `PAPERCLIP_PAGE_BASE_URL`
- `AWS_REGION`
- AWS credentials via Paperclip Secrets or an approved AWS vault
- Optional environment variables:
- `PAPERCLIP_PAGE_DEFAULT_PREFIX`
- `PAPERCLIP_PAGE_AWS_PROFILE`
## Workflow
1. Inspect the source directory and confirm it is public static content only.
2. Run `scripts/publish.sh <dir> --dry-run` to validate local structure and see
the resolved URL/prefix.
3. Choose a slug:
- Use `--slug <slug>` when the user gave a stable URL path.
- Omit `--slug` to derive one from the source directory name.
4. Publish:
```bash
skills/paperclip-page/scripts/publish.sh ./site --slug my-page
```
5. Return the printed public URL and S3 prefix to the issue/user.
## Update Workflow
Updates are additive overwrites only. The helper never deletes remote objects.
```bash
skills/paperclip-page/scripts/publish.sh ./site --slug my-page --update
```
When the target prefix already exists, `--update` requires local ownership proof
from `./site/.paperclip-page/state.json` generated by an earlier publish from
that same source directory. Without that state, create a new slug instead of
overwriting another page.
## Safety Rules
- Publish public content only. Do not publish secrets, customer data, private
company material, credentials, or internal logs.
- Never print AWS secret values.
- Never change bucket policy, IAM, DNS, CloudFront, or ACM settings from this
skill. Setup belongs to an operator runbook, not the publish helper.
- Never upload outside the configured bucket and prefix.
- Never use `aws s3 sync --delete` or require `s3:DeleteObject` in v1.
- The helper forces `--no-follow-symlinks` and fails if any source symlink is
present.
- The helper rejects hidden files and dot-segment paths except its own
`.paperclip-page/state.json`.
- Slugs and prefix segments must use lowercase ASCII letters, digits, and
hyphens only.
- Keep site-wide root objects such as `404.html` operator-managed; publishes
always target `<slug>/...` or `<default-prefix>/<slug>/...`.
## Troubleshooting
- `Slug already exists`: choose a different slug or use `--update` from the
original source directory containing `.paperclip-page/state.json`.
- `Missing index.html`: build the static site first or point the helper at the
directory that contains the root HTML file.
- `Found symlink`: replace symlinks with real files before publishing.
- `AccessDenied`: confirm the uploader IAM policy allows `ListBucket`,
`GetObject`, and `PutObject` for the configured bucket/prefix, and that the
agent received the Paperclip Secrets.
- Public URL verification failed: check CloudFront deployment/DNS, object
existence, and that the distribution uses HTTPS with the private S3 REST
origin.
See `README.md` next to this skill for operator setup, AWS policy examples,
credential rotation, and install/attach commands.

View File

@ -0,0 +1,388 @@
#!/usr/bin/env bash
set -euo pipefail
usage() {
cat <<'EOF'
Usage:
publish.sh <dir> [--slug slug] [--update] [--dry-run]
Publishes a static directory with a root index.html to the configured Paperclip
pages bucket and prints the public URL and S3 prefix.
Required environment for live publish:
PAPERCLIP_PAGE_BUCKET, PAPERCLIP_PAGE_BASE_URL, AWS_REGION, AWS credentials
Optional environment:
PAPERCLIP_PAGE_DEFAULT_PREFIX, PAPERCLIP_PAGE_AWS_PROFILE
Options:
--slug SLUG Lowercase URL slug. Allowed: a-z, 0-9, hyphen.
--update Additively overwrite an owned existing prefix. Never deletes.
--dry-run Validate and print the planned target without AWS writes.
--help, -h Show this help.
EOF
}
die() {
printf 'paperclip-page: %s\n' "$*" >&2
exit 1
}
require_command() {
if ! command -v "$1" >/dev/null 2>&1; then
die "missing required command: $1"
fi
}
normalize_base_url() {
local value="$1"
value="${value%/}"
[[ "$value" == https://* ]] || die "PAPERCLIP_PAGE_BASE_URL must be an https URL"
[[ ! "$value" =~ [[:space:]] ]] || die "PAPERCLIP_PAGE_BASE_URL cannot contain whitespace"
printf '%s\n' "$value"
}
validate_segment() {
local value="$1"
local label="$2"
[[ -n "$value" ]] || die "$label cannot be empty"
[[ "${#value}" -le 64 ]] || die "$label is too long; max length is 64 characters"
[[ "$value" =~ ^[a-z0-9]([a-z0-9-]*[a-z0-9])?$ ]] || die "$label must use lowercase letters, digits, and hyphens only"
[[ "$value" != "." && "$value" != ".." ]] || die "$label cannot be a dot segment"
}
normalize_slug() {
local value="$1"
value="${value#/}"
value="${value%/}"
[[ "$value" != *"/"* ]] || die "slug must be one path segment, not a nested path"
validate_segment "$value" "slug"
case "$value" in
404|404-html|index|index-html|root|assets)
die "slug '$value' is reserved"
;;
esac
printf '%s\n' "$value"
}
derive_slug() {
local source_dir="$1"
local base
base="$(basename "$source_dir")"
base="$(printf '%s' "$base" | tr '[:upper:]' '[:lower:]' | sed -E 's/[^a-z0-9]+/-/g; s/^-+//; s/-+$//; s/-{2,}/-/g')"
if [[ -z "$base" ]]; then
base="paperclip-page"
fi
printf '%.48s\n' "$base" | sed -E 's/-+$//'
}
random_suffix() {
if command -v openssl >/dev/null 2>&1; then
openssl rand -hex 3
else
od -An -N3 -tx1 /dev/urandom | tr -d ' \n'
fi
}
normalize_default_prefix() {
local raw="${1:-}"
local segment
local normalized=""
raw="${raw#/}"
raw="${raw%/}"
[[ "$raw" != *"//"* ]] || die "PAPERCLIP_PAGE_DEFAULT_PREFIX cannot contain empty path segments"
if [[ -z "$raw" ]]; then
printf '\n'
return
fi
IFS='/' read -r -a segments <<<"$raw"
for segment in "${segments[@]}"; do
validate_segment "$segment" "prefix segment"
if [[ -z "$normalized" ]]; then
normalized="$segment"
else
normalized="$normalized/$segment"
fi
done
printf '%s\n' "$normalized"
}
join_prefix() {
local default_prefix="$1"
local slug="$2"
if [[ -n "$default_prefix" ]]; then
printf '%s/%s/\n' "$default_prefix" "$slug"
else
printf '%s/\n' "$slug"
fi
}
aws_base_args=()
aws_cli() {
aws "${aws_base_args[@]}" "$@"
}
object_exists() {
local bucket="$1"
local prefix="$2"
local key
key="$(aws_cli s3api list-objects-v2 \
--bucket "$bucket" \
--prefix "$prefix" \
--max-keys 1 \
--query 'Contents[0].Key' \
--output text)"
[[ "$key" != "None" && -n "$key" ]]
}
assert_safe_source_tree() {
local source_dir="$1"
local found
[[ ! -L "$source_dir" ]] || die "source directory must not be a symlink"
[[ -f "$source_dir/index.html" ]] || die "source directory must contain root index.html"
found="$(find "$source_dir" -type l -print -quit)"
[[ -z "$found" ]] || die "found symlink in source tree: $found"
found="$(
cd "$source_dir"
find . -mindepth 1 \
\( -path './.paperclip-page' -o -path './.paperclip-page/*' \) -prune -o \
\( -name '.*' -o -path '*/.*' \) -print -quit
)"
[[ -z "$found" ]] || die "hidden files and dot paths are not allowed in published content: $found"
}
read_state_value() {
local state_file="$1"
local expression="$2"
jq -r "$expression // empty" "$state_file"
}
assert_update_ownership() {
local source_dir="$1"
local bucket="$2"
local prefix="$3"
local state_file="$source_dir/.paperclip-page/state.json"
local state_bucket
local state_prefix
[[ -f "$state_file" ]] || die "update of an existing prefix requires ownership state at $state_file"
state_bucket="$(read_state_value "$state_file" '.bucket')"
state_prefix="$(read_state_value "$state_file" '.prefix')"
[[ "$state_bucket" == "$bucket" ]] || die "state bucket does not match target bucket"
[[ "$state_prefix" == "$prefix" ]] || die "state prefix does not match target prefix"
}
compute_source_hash() {
local source_dir="$1"
if ! command -v sha256sum >/dev/null 2>&1; then
printf 'unavailable\n'
return
fi
(
cd "$source_dir"
find . -type f ! -path './.paperclip-page/*' -print0 |
LC_ALL=C sort -z |
while IFS= read -r -d '' path; do
sha256sum "$path"
done
) | sha256sum | awk '{print $1}'
}
write_state() {
local source_dir="$1"
local bucket="$2"
local prefix="$3"
local slug="$4"
local url="$5"
local base_url="$6"
local source_hash="$7"
local state_dir="$source_dir/.paperclip-page"
local state_file="$state_dir/state.json"
local temp_file
mkdir -p "$state_dir"
temp_file="$(mktemp "$state_dir/state.json.tmp.XXXXXX")"
jq -n \
--arg bucket "$bucket" \
--arg prefix "$prefix" \
--arg slug "$slug" \
--arg url "$url" \
--arg baseUrl "$base_url" \
--arg publishedAt "$(date -u +'%Y-%m-%dT%H:%M:%SZ')" \
--arg sourceHash "$source_hash" \
'{
bucket: $bucket,
prefix: $prefix,
slug: $slug,
url: $url,
baseUrl: $baseUrl,
publishedAt: $publishedAt,
sourceHash: $sourceHash,
version: 1
}' >"$temp_file"
mv "$temp_file" "$state_file"
}
source_arg=""
slug_arg=""
update=0
dry_run=0
while [[ $# -gt 0 ]]; do
case "$1" in
--slug)
slug_arg="${2:-}"
shift 2
;;
--update)
update=1
shift
;;
--dry-run)
dry_run=1
shift
;;
--help|-h)
usage
exit 0
;;
--*)
die "unknown argument: $1"
;;
*)
if [[ -n "$source_arg" ]]; then
die "unexpected positional argument: $1"
fi
source_arg="$1"
shift
;;
esac
done
[[ -n "$source_arg" ]] || {
usage >&2
exit 1
}
require_command jq
require_command find
require_command sed
[[ -d "$source_arg" ]] || die "source path is not a directory: $source_arg"
source_dir="$(cd "$source_arg" && pwd -P)"
assert_safe_source_tree "$source_dir"
bucket="${PAPERCLIP_PAGE_BUCKET:-}"
base_url="${PAPERCLIP_PAGE_BASE_URL:-}"
region="${AWS_REGION:-}"
default_prefix="$(normalize_default_prefix "${PAPERCLIP_PAGE_DEFAULT_PREFIX:-}")"
[[ -n "$bucket" ]] || die "PAPERCLIP_PAGE_BUCKET is required"
[[ "$bucket" =~ ^[a-z0-9][a-z0-9.-]{1,61}[a-z0-9]$ ]] || die "PAPERCLIP_PAGE_BUCKET does not look like a valid S3 bucket name"
[[ -n "$base_url" ]] || die "PAPERCLIP_PAGE_BASE_URL is required"
base_url="$(normalize_base_url "$base_url")"
explicit_slug=0
if [[ -n "$slug_arg" ]]; then
explicit_slug=1
slug="$(normalize_slug "$slug_arg")"
else
slug="$(normalize_slug "$(derive_slug "$source_dir")")"
fi
if [[ "$dry_run" == "0" ]]; then
require_command aws
require_command curl
[[ -n "$region" ]] || die "AWS_REGION is required for live publish"
aws_base_args=(--region "$region")
if [[ -n "${PAPERCLIP_PAGE_AWS_PROFILE:-}" ]]; then
aws_base_args+=(--profile "$PAPERCLIP_PAGE_AWS_PROFILE")
fi
fi
prefix="$(join_prefix "$default_prefix" "$slug")"
target_exists=0
if [[ "$update" == "1" ]]; then
assert_update_ownership "$source_dir" "$bucket" "$prefix"
fi
if [[ "$dry_run" == "0" ]]; then
if object_exists "$bucket" "$prefix"; then
target_exists=1
fi
if [[ "$target_exists" == "1" && "$update" == "0" ]]; then
if [[ "$explicit_slug" == "1" ]]; then
die "slug already exists: $slug. Use --update from the owning source directory or choose a new slug."
fi
for _ in 1 2 3 4 5; do
candidate="${slug}-$(random_suffix)"
candidate="$(printf '%.64s' "$candidate" | sed -E 's/-+$//')"
validate_segment "$candidate" "generated slug"
candidate_prefix="$(join_prefix "$default_prefix" "$candidate")"
if ! object_exists "$bucket" "$candidate_prefix"; then
slug="$candidate"
prefix="$candidate_prefix"
target_exists=0
break
fi
done
[[ "$target_exists" == "0" ]] || die "could not find an unused generated slug after 5 attempts"
fi
fi
url="${base_url}/${prefix}"
mode="publish"
if [[ "$update" == "1" ]]; then
mode="update"
fi
if [[ "$dry_run" == "1" ]]; then
cat <<EOF
paperclip-page dry run
mode: $mode
source: $source_dir
bucket: $bucket
prefix: $prefix
url: $url
EOF
exit 0
fi
aws_cli s3 sync "$source_dir/" "s3://$bucket/$prefix" \
--no-follow-symlinks \
--exclude '.paperclip-page/*' \
--cache-control 'public,max-age=60' \
--only-show-errors
source_hash="$(compute_source_hash "$source_dir")"
write_state "$source_dir" "$bucket" "$prefix" "$slug" "$url" "$base_url" "$source_hash"
curl -fsSIL --max-time 20 "$url" >/dev/null
cat <<EOF
paperclip-page published
mode: $mode
url: $url
bucket: $bucket
prefix: $prefix
state: $source_dir/.paperclip-page/state.json
EOF

View File

@ -0,0 +1,179 @@
import assert from "node:assert/strict";
import { execFileSync } from "node:child_process";
import {
chmodSync,
existsSync,
mkdirSync,
mkdtempSync,
readFileSync,
rmSync,
statSync,
writeFileSync,
} from "node:fs";
import { tmpdir } from "node:os";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import { after, test } from "node:test";
const scriptPath = join(dirname(fileURLToPath(import.meta.url)), "publish.sh");
const tempDirs = new Set();
after(() => {
for (const dir of tempDirs) {
rmSync(dir, { force: true, recursive: true });
}
});
function createSite(name = "paperclip-page-test") {
const siteDir = mkdtempSync(join(tmpdir(), `${name}-`));
tempDirs.add(siteDir);
writeFileSync(join(siteDir, "index.html"), "<!doctype html><title>Paperclip</title>\n");
return siteDir;
}
function writeExecutable(path, body) {
writeFileSync(path, body, { mode: 0o755 });
chmodSync(path, 0o755);
}
function writeState(siteDir, state) {
mkdirSync(join(siteDir, ".paperclip-page"), { recursive: true });
writeFileSync(join(siteDir, ".paperclip-page", "state.json"), `${JSON.stringify(state)}\n`);
}
function runPublish(args, env = {}) {
try {
return {
output: execFileSync("bash", [scriptPath, ...args], {
encoding: "utf8",
env: {
...process.env,
PAPERCLIP_PAGE_BUCKET: "paperclip-pages-test",
PAPERCLIP_PAGE_BASE_URL: "https://pages.example.test/",
...env,
},
stdio: ["ignore", "pipe", "pipe"],
}),
status: 0,
};
} catch (error) {
return {
output: `${error.stdout ?? ""}${error.stderr ?? ""}`,
status: error.status ?? 1,
};
}
}
test("publish helper stays executable", () => {
assert.equal(statSync(scriptPath).mode & 0o111, 0o111);
});
test("dry run validates and prints the planned target without requiring AWS", () => {
const result = runPublish([
createSite(),
"--slug",
"demo-page",
"--dry-run",
]);
assert.equal(result.status, 0);
assert.match(result.output, /^paperclip-page dry run$/m);
assert.match(result.output, /^mode: publish$/m);
assert.match(result.output, /^bucket: paperclip-pages-test$/m);
assert.match(result.output, /^prefix: demo-page\/$/m);
assert.match(result.output, /^url: https:\/\/pages\.example\.test\/demo-page\/$/m);
});
test("dry run normalizes a safe default prefix", () => {
const result = runPublish(
[createSite(), "--slug", "demo-page", "--dry-run"],
{ PAPERCLIP_PAGE_DEFAULT_PREFIX: "/reports/launches/" },
);
assert.equal(result.status, 0);
assert.match(result.output, /^prefix: reports\/launches\/demo-page\/$/m);
assert.match(result.output, /^url: https:\/\/pages\.example\.test\/reports\/launches\/demo-page\/$/m);
});
test("dry run update requires matching local ownership state", () => {
const siteDir = createSite();
const missingState = runPublish([siteDir, "--slug", "demo-page", "--update", "--dry-run"]);
assert.notEqual(missingState.status, 0);
assert.match(missingState.output, /requires ownership state/);
writeState(siteDir, {
bucket: "paperclip-pages-test",
prefix: "demo-page/",
});
const result = runPublish([siteDir, "--slug", "demo-page", "--update", "--dry-run"]);
assert.equal(result.status, 0);
assert.match(result.output, /^mode: update$/m);
});
test("rejects nested slugs", () => {
const result = runPublish([createSite(), "--slug", "nested/path", "--dry-run"]);
assert.notEqual(result.status, 0);
assert.match(result.output, /slug must be one path segment/);
});
test("rejects hidden files in the source tree", () => {
const siteDir = createSite();
mkdirSync(join(siteDir, "assets"));
writeFileSync(join(siteDir, "assets", ".secret"), "do not publish\n");
const result = runPublish([siteDir, "--slug", "demo-page", "--dry-run"]);
assert.notEqual(result.status, 0);
assert.match(result.output, /hidden files and dot paths are not allowed/);
});
test("live publish writes state before URL verification", () => {
const siteDir = createSite();
const binDir = mkdtempSync(join(tmpdir(), "paperclip-page-bin-"));
tempDirs.add(binDir);
writeExecutable(
join(binDir, "aws"),
`#!/usr/bin/env bash
set -euo pipefail
while [[ "$1" == "--region" || "$1" == "--profile" ]]; do
shift 2
done
if [[ "$1" == "s3api" ]]; then
echo "None"
exit 0
fi
if [[ "$1" == "s3" && "$2" == "sync" ]]; then
exit 0
fi
echo "unexpected aws call: $*" >&2
exit 1
`,
);
writeExecutable(
join(binDir, "curl"),
`#!/usr/bin/env bash
set -euo pipefail
echo "simulated CloudFront propagation miss" >&2
exit 22
`,
);
const result = runPublish([siteDir, "--slug", "demo-page"], {
AWS_REGION: "us-east-1",
PATH: `${binDir}:${process.env.PATH}`,
});
assert.notEqual(result.status, 0);
assert.match(result.output, /simulated CloudFront propagation miss/);
assert.equal(existsSync(join(siteDir, ".paperclip-page", "state.json")), true);
const state = JSON.parse(readFileSync(join(siteDir, ".paperclip-page", "state.json"), "utf8"));
assert.equal(state.bucket, "paperclip-pages-test");
assert.equal(state.prefix, "demo-page/");
assert.equal(state.url, "https://pages.example.test/demo-page/");
});