feat(agents): persist personas and render cached avatar URLs
Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Dotta
parent
4042eb1c48
commit
f5cd8de1b3
|
|
@ -1308,3 +1308,9 @@ Networking behavior for this smoke script:
|
|||
### GitHub identity for shared agents
|
||||
|
||||
See [execution GitHub identity](execution-github-identity.md) for the operation-time credential contract, continuation rules, runtime rollout, and acceptance-test requirements.
|
||||
|
||||
### Agent persona Storybook
|
||||
|
||||
See [agent-personas.md](agent-personas.md) for the dynamic avatar endpoint, cache,
|
||||
and character stories. Set `PAPERCLIP_STORYBOOK_API_URL` to your isolated
|
||||
Paperclip API URL when running Storybook. Avatar PNGs are generated on demand.
|
||||
|
|
|
|||
|
|
@ -1561,3 +1561,10 @@ Export/import behavior in V1:
|
|||
- import supports preview (dry-run) before apply
|
||||
- import preview reports skill-policy and legacy-grant mappings before apply and rejects unknown policy schema versions
|
||||
- GitHub imports warn on unpinned refs instead of blocking
|
||||
|
||||
## Agent visual identity
|
||||
|
||||
Agent appearances are stable, versioned ClipLab end-cap personas, separate from
|
||||
behavioral instructions. Compact surfaces use on-demand cached PNG URLs; larger
|
||||
placements may use a lazy live character. See [agent-personas.md](agent-personas.md)
|
||||
for persistence, migration, rendering, and integration contracts.
|
||||
|
|
|
|||
|
|
@ -543,3 +543,10 @@ Things Paperclip explicitly does **not** do:
|
|||
7. **Atomic ownership.** Single assignee per task. Atomic checkout prevents conflicts.
|
||||
8. **Progressive deployment.** Trivial to start local, straightforward to scale to hosted.
|
||||
9. **Extensible core.** Clean boundaries so plugins can add capabilities (Adapters, knowledge base, revenue tracking) without modifying core.
|
||||
|
||||
## Agent visual identity
|
||||
|
||||
Agent appearances are stable, versioned ClipLab end-cap personas, separate from
|
||||
behavioral instructions. Compact surfaces use on-demand cached PNG URLs; larger
|
||||
placements may use a lazy live character. See [agent-personas.md](agent-personas.md)
|
||||
for persistence, migration, rendering, and integration contracts.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,166 @@
|
|||
# Agent personas
|
||||
|
||||
Agents have a persisted visual identity independent of prompts and runtime
|
||||
configuration: `{ schemaVersion: 1, characterVersion: "cap-v1", paletteId }`.
|
||||
The cap-v1 library contains 17 permanent palettes and a presentation-only gray
|
||||
Muted dream palette. New agents get one random assignment; existing rows are
|
||||
backfilled with the same ID-based mapping used by legacy clients. Duplicating
|
||||
an agent generates another assignment. Export/import preserves it.
|
||||
|
||||
## Rendering and URLs
|
||||
|
||||
`GET /api/agent-avatars/cap-v1/bubblegum-sky/rest.png?size=24&scale=2`
|
||||
|
||||
The endpoint is public preset artwork, contains no agent/company identifiers,
|
||||
and works without the UI. `Agent.avatarUrl` is the 512px resting portrait URL.
|
||||
Resolve relative URLs against the instance base URL for integrations.
|
||||
|
||||
Supported logical sizes: 16, 20, 24, 32, 40, 48, 64, 96, 128, 256, 512.
|
||||
Density is 1 or 2. Poses: rest, idle, listening, thinking, working, success,
|
||||
confused, sleepy, loading. Other inputs receive 400. Display size controls
|
||||
face detail separately from raster dimensions: 24px at 2× remains eyes-only.
|
||||
|
||||
A cold request samples ClipLab's deterministic scene, exports SVG using the
|
||||
same geometry and face code as the live renderer, and rasterizes it with sharp
|
||||
in a worker thread. No Chromium, GPU, pre-rendering command, or per-agent asset
|
||||
row is required. Two workers serve a bounded queue, coalesce identical requests
|
||||
within the process, time out failed rendering, and stop after an idle interval.
|
||||
|
||||
The configured local-disk/S3 provider stores results in
|
||||
`generated-agent-avatars/<version>/<palette>/<pose>-<size>-<scale>.png`.
|
||||
This preset-only cache intentionally uses the storage provider directly;
|
||||
company asset APIs retain their existing authorization boundary. Cache data
|
||||
can be removed and regenerates on demand. Independent replicas can render the
|
||||
same key safely; successful writes are complete objects. ETags hash PNG bytes.
|
||||
A small `.png.json` sidecar stores the content digest and length; it is published
|
||||
after the complete PNG. Warm requests stream stored PNG bytes without re-rendering
|
||||
or buffering the image in the API process. Responses are immutable for one year. Render/storage failures return 503 with
|
||||
Retry-After and no-store rather than caching a broken image.
|
||||
|
||||
cap-v1 is frozen: change the version when changing palette values, poses,
|
||||
rendering, or dependencies in a way that changes pixels. Keep old versions
|
||||
available. The SVG exporter approximates 3D gradients with a planar gradient;
|
||||
front-facing identity portraits minimize the difference from WebGL.
|
||||
|
||||
## Components
|
||||
|
||||
- `AgentAvatar`: image only; pass the agent record or appearance and a semantic
|
||||
size. No per-agent queries, live-renderer imports, or circle cropping.
|
||||
- `AgentIdentity`: agent avatar and name. Human identities keep `Identity`.
|
||||
- `AgentCharacter`: lazy live hero with state and optional tracking-region
|
||||
props, plus an explicit `trackingScope="page"` for onboarding and agent headers. One live renderer per view; other instances keep their still image.
|
||||
Reduced motion, offscreen content, renderer failure, and static states do
|
||||
not run the animation loop. Pointer tracking defaults to its region and is off for touch.
|
||||
|
||||
Onboarding stores the eventual palette in its existing draft and presents gray
|
||||
until verified connection/hiring succeeds. Reconnect never randomizes identity.
|
||||
Names, explicit status badges, and status text remain authoritative.
|
||||
|
||||
## Development and verification
|
||||
|
||||
ClipLab provenance/licenses are under `packages/shared/src/cliplab/`.
|
||||
Palette tokens originate in `ui/src/index.css`. After deliberately introducing
|
||||
new versioned artwork, `node scripts/sync-agent-palette-tokens.mjs` synchronizes
|
||||
the TS palette data; `--check` detects drift. This does not generate images.
|
||||
|
||||
Storybook: **Agents / Personas**. Run with
|
||||
`PAPERCLIP_STORYBOOK_API_URL=http://localhost:<isolated-port> pnpm storybook`.
|
||||
Static Storybook hosting must proxy `/api/agent-avatars` to an instance.
|
||||
|
||||
Focused checks:
|
||||
|
||||
```sh
|
||||
pnpm exec vitest run packages/shared/src/agent-appearance.test.ts server/src/__tests__/agent-avatars.test.ts
|
||||
node scripts/sync-agent-palette-tokens.mjs --check
|
||||
pnpm check:token-gates
|
||||
pnpm build-storybook
|
||||
```
|
||||
|
||||
The 500-avatar story must create zero WebGL contexts and load no live runtime.
|
||||
Use fixed poses/times for screenshots and Linux for authoritative visual
|
||||
baselines. Verify cold and warm URLs, reduced motion, reconnect, and saved
|
||||
appearance after refresh alongside normal typecheck/test/build checks.
|
||||
|
||||
Linux visual/performance checks (against the built Storybook with the API proxy):
|
||||
|
||||
```sh
|
||||
PAPERCLIP_PERSONA_STORYBOOK_URL=http://localhost:<storybook-port> \
|
||||
pnpm exec playwright test --config tests/storybook-visual/agent-personas.config.ts
|
||||
```
|
||||
|
||||
Use the Playwright 1.62.1 Noble image for authoritative Linux baselines. The suite
|
||||
covers both themes, palette and size grids, every expression, fixed-pose SVG/WebGL
|
||||
pixel comparisons, repeated mounts, context loss, delayed/failed images, and the
|
||||
500-avatar no-WebGL/no-live-download/no-frame-loop contract. Baselines follow the
|
||||
existing Storybook visual artifact workflow; they are not application assets.
|
||||
|
||||
## Acceptance record — 2026-09-10
|
||||
|
||||
Implemented in `codex/agent-personas`, based on `5cb4f061d`, with the original
|
||||
checkout left unchanged. Verification used disposable embedded-Postgres data,
|
||||
a local MinIO container, and Playwright's Linux Noble image.
|
||||
|
||||
| Check | Result |
|
||||
| --- | --- |
|
||||
| Repository typecheck and build | Passed |
|
||||
| Token gates and palette-token synchronization | Passed |
|
||||
| Storybook production build | Passed |
|
||||
| Linux visual/performance suite | 30 passed; exact snapshot comparison passed |
|
||||
| Static/live agreement | Rest at 16/24/48/256 logical pixels, both densities; all eight animated expression snapshots at 128px/2× |
|
||||
| Avatar endpoint | Cold/warm requests, concurrency, deletion, retry, ETags and invalid parameters passed |
|
||||
| Real local-disk and S3-compatible storage | Passed; persisted warm cache reused by a fresh service instance |
|
||||
| Compiled API-only smoke | Passed with TypeScript stripping disabled, no UI and a native worker |
|
||||
| Persistence and contracts | Creation, saved draft, SQL backfill, approvals, duplication, portability and revision restoration passed |
|
||||
| UI/runtime lifecycle | Reduced motion, one live owner, scoped pointers, hidden/offscreen suspension, failures and disposal passed |
|
||||
| Hands-on app | Stable rename/reload, fresh duplicate assignment, paused static portrait, matching task/list/configuration identities passed |
|
||||
|
||||
The broad `pnpm test:run` verification was completed in its groups/shards after
|
||||
resource-contention retries. General server (8,347 tests), UI (5,619), CLI (484),
|
||||
DB (133), and adapter/plugin source suites (2,235) passed. Remaining route suites
|
||||
and the added persona/contract tests passed after the OpenAPI coverage update.
|
||||
The queued-comment suite exposed a pre-existing fixture-cleanup failure: run
|
||||
claims created dependent runtime rows, so swallowed foreign-key errors left the
|
||||
`QUE` company behind. Its isolated-database cleanup now truncates the company
|
||||
fixture graph with cascade; all 17 tests pass together. No production queued-comment
|
||||
behavior changed. Verification completed across the repository runner's groups
|
||||
and shards, with targeted reruns after fixes, rather than another monolithic run.
|
||||
|
||||
Actual provider sign-in was not completed: the disposable instance had no managed
|
||||
sandbox available for sign-in. Onboarding gray/loading/success transitions were
|
||||
covered in Storybook and the onboarding tests. The app is usable in local-trusted
|
||||
mode; this standalone worktree has no managed issue identity or login handoff.
|
||||
|
||||
The reviewed Linux candidate archive is generated at
|
||||
`tests/storybook-visual/baseline-review/snapshots.tgz`. It remains a local review
|
||||
artifact; the existing baseline manifest was not repointed to an unpublished URL.
|
||||
|
||||
## Placement and sharpness refinement — 2026-09-10
|
||||
|
||||
The overview/configuration header owns the live character beside the agent name
|
||||
and follows the pointer across the whole page; there is no second hero in the
|
||||
overview body. The new-agent dialog and setup page
|
||||
use a larger, padded character frame with page-wide mouse tracking. Other
|
||||
placements retain region-scoped tracking. Touch, reduced motion, hidden views,
|
||||
and unmount cleanup still disable tracking and frame scheduling.
|
||||
|
||||
Live canvases render at twice the display density (capped at 4×), with a 1024px
|
||||
face texture and padded framing for rotations and expression props. This affects
|
||||
only the live renderer: existing versioned PNG URLs retain their original pixels.
|
||||
The Snapshot Agreement story provides explicit 1×/2× controls and labels the
|
||||
actual PNG and WebGL pixel dimensions for a fair comparison.
|
||||
|
||||
`Agents / Personas / Full pages` includes the actual application shell and route
|
||||
components for all agents, agent overview, task detail, dashboard, the new-agent
|
||||
dialog, and the connection page. Fixtures stay in Storybook; these examples do
|
||||
not read or mutate the running company's data. Dashboard activity rows now use
|
||||
the same agent avatars as its active-agent and task placements.
|
||||
|
||||
Refinement verification: 38 Linux Playwright checks passed, including exact
|
||||
comparison with the reviewed snapshots for all six full-page stories and
|
||||
corner-pointer clipping checks at 1×/2× display density. The 500-avatar view
|
||||
still loads no live renderer or WebGL contexts. All 18 targeted appearance,
|
||||
component/runtime, and avatar endpoint tests passed, as did shared/UI typechecks,
|
||||
token gates, UI build, and Storybook build. Hands-on inspection confirmed the
|
||||
single animated header at `/PER/agents/persona-tester-renamed/overview` and the
|
||||
larger onboarding character in the real route components. The repository-wide
|
||||
checks above were not repeated for this UI refinement.
|
||||
|
|
@ -41,7 +41,7 @@
|
|||
"release:rollback": "./scripts/rollback-latest.sh",
|
||||
"release:bootstrap-package": "node scripts/bootstrap-npm-package.mjs",
|
||||
"check:tokens": "node scripts/check-forbidden-tokens.mjs",
|
||||
"check:token-gates": "node scripts/check-token-gates.mjs",
|
||||
"check:token-gates": "node scripts/check-token-gates.mjs && node scripts/sync-agent-palette-tokens.mjs --check",
|
||||
"check:node-version": "node scripts/check-node-version-policy.mjs",
|
||||
"check:no-git-push": "node scripts/check-no-git-push.mjs",
|
||||
"check:module-boundaries": "node scripts/check-module-boundaries.mjs",
|
||||
|
|
|
|||
|
|
@ -0,0 +1,11 @@
|
|||
ALTER TABLE "agents" ADD COLUMN IF NOT EXISTS "appearance" jsonb;
|
||||
--> statement-breakpoint
|
||||
WITH RECURSIVE appearance_hash AS (
|
||||
SELECT id, id::text AS identity_text, 0 AS position, 0 AS hash FROM agents WHERE appearance IS NULL
|
||||
UNION ALL
|
||||
SELECT id, identity_text, position + 1, (hash * 31 + ascii(substr(identity_text, position + 1, 1))) % 17
|
||||
FROM appearance_hash WHERE position < length(identity_text)
|
||||
)
|
||||
UPDATE agents SET appearance = jsonb_build_object('schemaVersion', 1, 'characterVersion', 'cap-v1',
|
||||
'paletteId', (ARRAY['bubblegum-sky','pink-lemonade','orchid-peach','coral-mint','lime-lagoon','arctic-blue','solar-flare','violet-ember','deep-tide','coral-current','golden-hour','tangerine-cobalt','electric-grove','flamingo-jade','cherry-pop','turquoise-cherry','ultraviolet-tide'])[appearance_hash.hash + 1])
|
||||
FROM appearance_hash WHERE agents.id = appearance_hash.id AND appearance_hash.position = length(appearance_hash.identity_text) AND agents.appearance IS NULL;
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -1891,6 +1891,13 @@
|
|||
"when": 1788999440971,
|
||||
"tag": "0271_woozy_silver_surfer",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 272,
|
||||
"version": "7",
|
||||
"when": 1789075418387,
|
||||
"tag": "0272_marvelous_madame_web",
|
||||
"breakpoints": true
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -1,3 +1,4 @@
|
|||
import type { AgentAppearance } from "@paperclipai/shared";
|
||||
import {
|
||||
type AnyPgColumn,
|
||||
pgTable,
|
||||
|
|
@ -21,6 +22,7 @@ export const agents = pgTable(
|
|||
role: text("role").notNull().default("general"),
|
||||
title: text("title"),
|
||||
icon: text("icon"),
|
||||
appearance: jsonb("appearance").$type<AgentAppearance>(),
|
||||
status: text("status").notNull().default("idle"),
|
||||
reportsTo: uuid("reports_to").references((): AnyPgColumn => agents.id),
|
||||
capabilities: text("capabilities"),
|
||||
|
|
|
|||
|
|
@ -40,16 +40,18 @@
|
|||
"dist"
|
||||
],
|
||||
"scripts": {
|
||||
"build": "tsc",
|
||||
"build": "tsc && mkdir -p dist/cliplab && cp src/cliplab/LICENSE src/cliplab/PROVENANCE.md dist/cliplab/",
|
||||
"clean": "rm -rf dist",
|
||||
"typecheck": "tsc --noEmit"
|
||||
},
|
||||
"dependencies": {
|
||||
"zod": "^4.4.3"
|
||||
"zod": "^4.4.3",
|
||||
"three": "0.185.1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^24.0.0",
|
||||
"typescript": "^7.0.2"
|
||||
"typescript": "^7.0.2",
|
||||
"@types/three": "0.185.4"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=24.11.0"
|
||||
|
|
|
|||
|
|
@ -0,0 +1,29 @@
|
|||
import { describe, expect, it } from "vitest";
|
||||
import { AGENT_PALETTE_IDS, agentAppearanceSchema, appearanceForPalette, legacyAgentAppearance, randomAgentAppearance, resolveAgentAppearance, agentAvatarUrl } from "./agent-appearance.js";
|
||||
import { renderAgentSvg } from "./cliplab/static.js";
|
||||
|
||||
describe("agent appearance", () => {
|
||||
it("assigns only the 17 permanent cap palettes", () => {
|
||||
for (let i = 0; i < 100; i++) expect(AGENT_PALETTE_IDS).toContain(randomAgentAppearance().paletteId);
|
||||
expect(agentAppearanceSchema.safeParse({ schemaVersion: 1, characterVersion: "cap-v1", paletteId: "muted-dream" }).success).toBe(false);
|
||||
});
|
||||
it("preserves a saved appearance and resolves legacy IDs deterministically", () => {
|
||||
const appearance = appearanceForPalette("deep-tide");
|
||||
expect(resolveAgentAppearance(appearance, "different-id")).toEqual(appearance);
|
||||
expect(resolveAgentAppearance(null, "agent-1")).toEqual(legacyAgentAppearance("agent-1"));
|
||||
expect(legacyAgentAppearance("agent-1")).toEqual(legacyAgentAppearance("agent-1"));
|
||||
expect(agentAvatarUrl(appearance, 24, 2)).toBe("/api/agent-avatars/cap-v1/deep-tide/rest.png?size=24&scale=2");
|
||||
});
|
||||
it("renders without a browser and preserves logical-size detail at high density", () => {
|
||||
const appearance = appearanceForPalette("bubblegum-sky");
|
||||
const small = renderAgentSvg(appearance, 16, 2);
|
||||
expect(small).toContain('width="32"');
|
||||
expect(small).not.toContain('id="agent-face-visible"');
|
||||
const eyes = renderAgentSvg(appearance, 24, 2);
|
||||
expect(eyes).toContain('width="48"');
|
||||
expect(eyes).toContain('id="agent-face-visible"');
|
||||
expect(eyes).not.toContain('id="agent-candle-light"');
|
||||
expect(renderAgentSvg(appearance, 48, 1)).toContain('id="agent-candle-light"');
|
||||
expect(renderAgentSvg(appearance, 24, 2)).toBe(eyes);
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,52 @@
|
|||
import { z } from "zod";
|
||||
|
||||
export const AGENT_PALETTE_IDS = ["bubblegum-sky", "pink-lemonade", "orchid-peach", "coral-mint", "lime-lagoon", "arctic-blue", "solar-flare", "violet-ember", "deep-tide", "coral-current", "golden-hour", "tangerine-cobalt", "electric-grove", "flamingo-jade", "cherry-pop", "turquoise-cherry", "ultraviolet-tide"] as const;
|
||||
export type AgentPaletteId = typeof AGENT_PALETTE_IDS[number];
|
||||
export type CharacterPaletteId = AgentPaletteId | "muted-dream";
|
||||
export const AGENT_AVATAR_SIZES = [16, 20, 24, 32, 40, 48, 64, 96, 128, 256, 512] as const;
|
||||
export type AgentAvatarSize = typeof AGENT_AVATAR_SIZES[number];
|
||||
export const CHARACTER_STATES = ["rest", "idle", "listening", "thinking", "working", "success", "confused", "sleepy", "loading"] as const;
|
||||
export type CharacterState = typeof CHARACTER_STATES[number];
|
||||
export const agentAppearanceSchema = z.object({
|
||||
schemaVersion: z.literal(1),
|
||||
characterVersion: z.literal("cap-v1"),
|
||||
paletteId: z.enum(AGENT_PALETTE_IDS),
|
||||
}).strict();
|
||||
export type AgentAppearance = z.infer<typeof agentAppearanceSchema>;
|
||||
|
||||
export function appearanceForPalette(paletteId: AgentPaletteId): AgentAppearance {
|
||||
return { schemaVersion: 1, characterVersion: "cap-v1", paletteId };
|
||||
}
|
||||
/** A persisted choice, never randomize while rendering. */
|
||||
export function randomAgentAppearance(): AgentAppearance {
|
||||
const bytes = new Uint32Array(1);
|
||||
// Rejection sampling avoids modulo bias.
|
||||
const limit = Math.floor(0x100000000 / AGENT_PALETTE_IDS.length) * AGENT_PALETTE_IDS.length;
|
||||
do { globalThis.crypto.getRandomValues(bytes); } while (bytes[0] >= limit);
|
||||
return appearanceForPalette(AGENT_PALETTE_IDS[bytes[0] % AGENT_PALETTE_IDS.length]);
|
||||
}
|
||||
/** Must match the migration: rolling base-31 hash, modulo 17 at every step. */
|
||||
export function legacyAgentAppearance(id: string): AgentAppearance {
|
||||
let hash = 0;
|
||||
for (const char of id) hash = (hash * 31 + char.charCodeAt(0)) % AGENT_PALETTE_IDS.length;
|
||||
return appearanceForPalette(AGENT_PALETTE_IDS[hash]);
|
||||
}
|
||||
export function resolveAgentAppearance(appearance: unknown, id = "agent"): AgentAppearance {
|
||||
const parsed = agentAppearanceSchema.safeParse(appearance);
|
||||
return parsed.success ? parsed.data : legacyAgentAppearance(id);
|
||||
}
|
||||
export function agentAvatarUrl(appearance: AgentAppearance, size: AgentAvatarSize = 512, scale: 1 | 2 = 1, pose: CharacterState = "rest", muted = false): string {
|
||||
return `/api/agent-avatars/${appearance.characterVersion}/${muted ? "muted-dream" : appearance.paletteId}/${pose}.png?size=${size}&scale=${scale}`;
|
||||
}
|
||||
export function characterStateForAgent(status: string): CharacterState {
|
||||
if (status === "running") return "working";
|
||||
if (status === "error") return "confused";
|
||||
if (status === "paused" || status === "terminated" || status === "pending_approval") return "rest";
|
||||
return "idle";
|
||||
}
|
||||
|
||||
/** Hydrate compact agent projections without an additional per-agent request. */
|
||||
export function withAgentAppearance<T extends { id: string; appearance?: AgentAppearance | null }>(agent: T) {
|
||||
const appearance = resolveAgentAppearance(agent.appearance, agent.id);
|
||||
return { ...agent, appearance, avatarUrl: agentAvatarUrl(appearance) };
|
||||
}
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
MIT License
|
||||
|
||||
Copyright (c) 2026 Jérémy Perret (bloub, the project this is forked from)
|
||||
Copyright (c) 2026 Tonio (Cliplab, modifications)
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
# ClipLab cap-v1
|
||||
|
||||
Source: https://github.com/tonio-alucema/cliplab
|
||||
Commit: a050f7246738db1d0763e286862747633293e2af
|
||||
License: MIT (see LICENSE). Three.js is MIT licensed.
|
||||
|
||||
Paperclip adaptations: ESM extensions, optional graphics backend for Node SVG
|
||||
snapshots, region-scoped runtime input (with explicit page scope for onboarding),
|
||||
suspended frame scheduling, and supersampled live textures/canvases. Live-only
|
||||
framing leaves the versioned static snapshot geometry unchanged. The Vue
|
||||
studio and media encoders are not included. Keep this version immutable after
|
||||
release; new artwork or rasterization changes require a new character version.
|
||||
|
||||
Palette source: https://cliptoon-color-library.vercel.app/
|
||||
The 17 assignable colors and Muted dream are frozen as cap-v1 tokens in
|
||||
ui/src/index.css. Generated palette data is checked by
|
||||
scripts/sync-agent-palette-tokens.mjs --check.
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
import type { AgentAppearance, CharacterPaletteId, CharacterState } from "../agent-appearance.js";
|
||||
import { defaultProject, definitionOf, sampleDefinition, BASE_POSE, type Definition, type Sample } from "./model.js";
|
||||
import { CAP_V1_COLORS } from "./palette-tokens.js";
|
||||
|
||||
export function characterDefinition(appearance: AgentAppearance, muted = false): Definition {
|
||||
const project = defaultProject();
|
||||
const colors = CAP_V1_COLORS[(muted ? "muted-dream" : appearance.paletteId) as CharacterPaletteId];
|
||||
const character = { ...project.characters[1], id: "paperclip-cap-v1", name: "Agent", color: colors.a, color2: colors.b,
|
||||
gradientAngle: 0, iris: false, toon: true, shadow: false, followCursor: true, followRotation: true };
|
||||
const definition = definitionOf(project, character);
|
||||
for (const animation of definition.animations) if (animation.id === "happy") animation.loop = false;
|
||||
return definition;
|
||||
}
|
||||
export function animationId(state: CharacterState) { return state === "success" ? "happy" : state === "rest" ? "idle" : state; }
|
||||
export function characterStill(definition: Definition, state: CharacterState): Sample {
|
||||
if (state === "rest") return { pose: { ...BASE_POSE }, blink: 0, bob: 0, breathe: 0, expressionId: "idle", beatIndex: 0, stepIndex: 0 };
|
||||
return sampleDefinition(definition, animationId(state), 0.6);
|
||||
}
|
||||
|
|
@ -0,0 +1,205 @@
|
|||
// Vendored from ClipLab a050f724; see PROVENANCE.md and LICENSE.
|
||||
import type { Detail, FaceLayer, Pose } from './model.js'
|
||||
import { irisOffset, type Gaze } from './gaze.js'
|
||||
|
||||
export type Point = { x: number; y: number }
|
||||
const COUNT = 128, TAU = Math.PI * 2
|
||||
const pt = (x: number, y: number): Point => ({ x, y })
|
||||
const distance = (a: Point, b: Point) => Math.hypot(b.x - a.x, b.y - a.y)
|
||||
const unit = (a: Point) => { const n = Math.hypot(a.x, a.y) || 1; return pt(a.x / n, a.y / n) }
|
||||
const arc = (x: number, y: number, rx: number, ry: number, from = 0, to = TAU, count = 64) => Array.from({ length: count + 1 }, (_, i) => { const a = from + (to - from) * i / count; return pt(x + Math.cos(a) * rx, y + Math.sin(a) * ry) })
|
||||
const cubic = (a: Point, b: Point, c: Point, d: Point, n = 32) => Array.from({ length: n + 1 }, (_, i) => {
|
||||
const t = i / n, s = 1 - t
|
||||
return pt(s ** 3 * a.x + 3 * s ** 2 * t * b.x + 3 * s * t ** 2 * c.x + t ** 3 * d.x, s ** 3 * a.y + 3 * s ** 2 * t * b.y + 3 * s * t ** 2 * c.y + t ** 3 * d.y)
|
||||
})
|
||||
const quad = (a: Point, b: Point, c: Point) => cubic(a, pt(a.x + (b.x - a.x) * 2 / 3, a.y + (b.y - a.y) * 2 / 3), pt(c.x + (b.x - c.x) * 2 / 3, c.y + (b.y - c.y) * 2 / 3), c)
|
||||
const area = (p: Point[]) => p.reduce((sum, a, i) => { const b = p[(i + 1) % p.length]!; return sum + a.x * b.y - b.x * a.y }, 0)
|
||||
function clean(points: Point[]) {
|
||||
const p = points.filter((point, i) => i === 0 || distance(point, points[i - 1]!) > .00001)
|
||||
if (p.length > 1 && distance(p[0]!, p.at(-1)!) < .00001) p.pop()
|
||||
return p
|
||||
}
|
||||
/** Stable winding, anchor and perimeter correspondence; never align against a live blend. */
|
||||
export function contour(points: Point[]): Point[] {
|
||||
let p = clean(points)
|
||||
if (p.length < 2) return Array.from({ length: COUNT }, () => ({ ...(p[0] ?? pt(0, 0)) }))
|
||||
if (area(p) < 0) p.reverse()
|
||||
let anchor = 0
|
||||
for (let i = 1; i < p.length; i++) if (p[i]!.x < p[anchor]!.x - .00001 || (Math.abs(p[i]!.x - p[anchor]!.x) < .00001 && p[i]!.y < p[anchor]!.y)) anchor = i
|
||||
p = [...p.slice(anchor), ...p.slice(0, anchor)]
|
||||
const lengths = p.map((a, i) => distance(a, p[(i + 1) % p.length]!)), total = lengths.reduce((a, b) => a + b, 0)
|
||||
let segment = 0, traversed = 0
|
||||
return Array.from({ length: COUNT }, (_, i) => {
|
||||
const target = total * i / COUNT
|
||||
while (segment < p.length - 1 && traversed + lengths[segment]! < target) traversed += lengths[segment++]!
|
||||
const a = p[segment]!, b = p[(segment + 1) % p.length]!, t = (target - traversed) / (lengths[segment] || 1)
|
||||
return pt(a.x + (b.x - a.x) * t, a.y + (b.y - a.y) * t)
|
||||
})
|
||||
}
|
||||
function ribbon(points: Point[], width: number): Point[] {
|
||||
const p = clean(points), h = width / 2
|
||||
const normals = p.slice(1).map((b, i) => { const a = p[i]!; return unit(pt(a.y - b.y, b.x - a.x)) })
|
||||
if (!normals.length) return arc(p[0]!.x, p[0]!.y, h, h)
|
||||
const side = (sign: number) => p.flatMap((a, i) => {
|
||||
const before = normals[Math.max(0, i - 1)]!, after = normals[Math.min(normals.length - 1, i)]!
|
||||
const cross = before.x * after.y - before.y * after.x
|
||||
if (i > 0 && i < p.length - 1 && cross * sign < -.0001) {
|
||||
const from = Math.atan2(before.y * sign, before.x * sign)
|
||||
const delta = Math.atan2(cross, before.x * after.x + before.y * after.y)
|
||||
return arc(a.x, a.y, h, h, from, from + delta, Math.max(2, Math.ceil(Math.abs(delta) * 10)))
|
||||
}
|
||||
const n = unit(pt(before.x + after.x, before.y + after.y)), reach = h / Math.max(.35, n.x * after.x + n.y * after.y)
|
||||
return [pt(a.x + n.x * reach * sign, a.y + n.y * reach * sign)]
|
||||
})
|
||||
const end = p.at(-1)!, first = p[0]!, endAngle = Math.atan2(normals.at(-1)!.y, normals.at(-1)!.x), startAngle = Math.atan2(normals[0]!.y, normals[0]!.x)
|
||||
return [...side(1), ...arc(end.x, end.y, h, h, endAngle, endAngle - Math.PI, 16), ...side(-1).reverse(), ...arc(first.x, first.y, h, h, startAngle - Math.PI, startAngle - TAU, 16)]
|
||||
}
|
||||
function expand(points: Point[], amount: number) {
|
||||
const p = clean(points); if (area(p) < 0) p.reverse()
|
||||
return p.flatMap((a, i) => {
|
||||
const before = p[(i - 1 + p.length) % p.length]!, after = p[(i + 1) % p.length]!
|
||||
const u = unit(pt(a.y - before.y, before.x - a.x)), v = unit(pt(after.y - a.y, a.x - after.x)), n = unit(pt(u.x + v.x, u.y + v.y))
|
||||
const cross = u.x * v.y - u.y * v.x
|
||||
if (cross > .0001) {
|
||||
const from = Math.atan2(u.y, u.x), delta = Math.atan2(cross, u.x * v.x + u.y * v.y)
|
||||
return arc(a.x, a.y, amount, amount, from, from + delta, Math.max(2, Math.ceil(delta * 10)))
|
||||
}
|
||||
const reach = amount / Math.max(.5, n.x * v.x + n.y * v.y)
|
||||
return [pt(a.x + n.x * reach, a.y + n.y * reach)]
|
||||
})
|
||||
}
|
||||
export const traitWeight = (layers: FaceLayer[], predicate: (pose: FaceLayer['traits']) => boolean) => layers.reduce((n, layer) => n + (predicate(layer.traits) ? layer.weight : 0), 0)
|
||||
export function blendContours(contours: Point[][], layers: FaceLayer[]): Point[] {
|
||||
return Array.from({ length: COUNT }, (_, i) => contours.reduce((point, shape, j) => pt(point.x + shape[i]!.x * layers[j]!.weight, point.y + shape[i]!.y * layers[j]!.weight), pt(0, 0)))
|
||||
}
|
||||
function trace(ctx: CanvasRenderingContext2D, shape: Point[]) {
|
||||
ctx.beginPath(); ctx.moveTo(shape[0]!.x, shape[0]!.y)
|
||||
for (const point of shape.slice(1)) ctx.lineTo(point.x, point.y)
|
||||
ctx.closePath()
|
||||
}
|
||||
export function isLineEye(pose: Pick<Pose, 'eye' | 'mouth'>, side: number, detail: Detail) {
|
||||
return ['closed', 'arc-up', 'arc-down', 'squint'].includes(pose.eye) || (pose.eye === 'wink' && side > 0) || (detail === 'eyes' && ['open', 'grin'].includes(pose.mouth) && pose.eye === 'dot')
|
||||
}
|
||||
export function eyeContour(pose: Pose, r: number, side: number, blink: number, detail: Detail): Point[] {
|
||||
const stroke = Math.max(13, r * .43)
|
||||
if ((pose.faceSet === 'set-2' && pose.eye === 'wink' && side > 0) || pose.eye === 'squint') {
|
||||
const sign = pose.eye === 'wink' ? -1 : side
|
||||
return contour(ribbon([pt(-r * sign * .7, -r * .8), pt(r * sign * .6, 0), pt(-r * sign * .7, r * .8)], stroke))
|
||||
}
|
||||
if (isLineEye(pose, side, detail)) {
|
||||
const happy = ['open', 'grin', 'smile', 'u-smile'].includes(pose.mouth)
|
||||
const up = pose.eye === 'arc-up' || (pose.eye !== 'arc-down' && happy)
|
||||
return contour(ribbon(arc(0, 0, r, r, up ? Math.PI : 0, up ? TAU : Math.PI), stroke))
|
||||
}
|
||||
const h = Math.max(.12, 1 - blink) * pose.eyeHeight * (pose.eye === 'soft' ? .65 : 1)
|
||||
let shape: Point[]
|
||||
if (pose.eye === 'star') shape = Array.from({ length: 10 }, (_, i) => { const a = i / 10 * TAU - Math.PI / 2, size = r * 1.2 * (i % 2 ? .46 : 1); return pt(Math.cos(a) * size, Math.sin(a) * size) })
|
||||
else if (pose.eye === 'heart') shape = [...cubic(pt(0, r), pt(-r * 2, -r * .1), pt(-r, -r * 1.5), pt(0, -r * .55)), ...cubic(pt(0, -r * .55), pt(r, -r * 1.5), pt(r * 2, -r * .1), pt(0, r))]
|
||||
else if (pose.eye === 'half-lidded') shape = arc(0, 0, r, r, 0, Math.PI)
|
||||
else { const size = r * (pose.eye === 'pupil' && detail === 'full' ? 1.5 : 1); shape = arc(0, 0, size, size) }
|
||||
const open = contour(shape.map(p => pt(p.x, p.y * h)))
|
||||
const closing = Math.max(0, Math.min(1, (blink - .5) / .5))
|
||||
if (!closing) return open
|
||||
const closed = contour(ribbon([pt(-r, 0), pt(r, 0)], stroke))
|
||||
return open.map((p, i) => pt(p.x + (closed[i]!.x - p.x) * closing, p.y + (closed[i]!.y - p.y) * closing))
|
||||
}
|
||||
export function mouthGeometry(pose: Pose): { outline: Point[]; opening: Point[]; width: number; height: number } {
|
||||
const broad = ['open', 'grin', 'cry'].includes(pose.mouth), w = (broad ? 124 : pose.mouth === 'oh' ? 51 : 61) * pose.mouthWidth
|
||||
const h = (broad ? 48 : 22) + 72 * pose.mouthOpen, stroke = 17 * pose.mouthStroke
|
||||
let path: Point[], filled = false
|
||||
if (pose.mouth === 'smile') path = arc(0, -10, w, w, Math.PI * .18, Math.PI * .82)
|
||||
else if (pose.mouth === 'u-smile') path = arc(0, -8, w * .56, w * .56, 0, Math.PI)
|
||||
else if (pose.mouth === 'frown') path = arc(0, 39, w, w, Math.PI * 1.2, Math.PI * 1.8)
|
||||
else if (pose.mouth === 'line' || pose.mouth === 'sleep') path = [pt(-w * .58, 0), pt(w * .58, 0)]
|
||||
else if (pose.mouth === 'kiss') path = [...cubic(pt(-w * .25, -21), pt(w * .48, -34), pt(w * .53, -2), pt(0, 0)), ...cubic(pt(0, 0), pt(w * .53, 2), pt(w * .48, 34), pt(-w * .25, 21))]
|
||||
else if (pose.mouth === 'wave') path = cubic(pt(-w, 4), pt(-w * .35, -25), pt(w * .35, 25), pt(w, -4))
|
||||
else if (pose.mouth === 'tongue-out') path = quad(pt(-w, -9), pt(0, -2), pt(w, -9))
|
||||
else {
|
||||
filled = true
|
||||
if (pose.mouth === 'oh') { const r = w * (.55 + .3 * pose.mouthOpen); path = arc(0, 14, r, r) }
|
||||
else if (pose.mouth === 'cry') path = [...cubic(pt(-w, 30), pt(-w * 1.1, -h), pt(w * 1.1, -h), pt(w, 30)), ...quad(pt(w, 30), pt(w, 44), pt(w * .76, 38)), ...quad(pt(w * .76, 38), pt(0, 24), pt(-w * .76, 38)), ...quad(pt(-w * .76, 38), pt(-w, 44), pt(-w, 30))].map(p => pt(p.x, p.y + 36 / (4 / 3)))
|
||||
else { const tilt = pose.mouth === 'grin' ? 24 : 0; path = [...quad(pt(-w, -11), pt(0, 5), pt(w, -11 - tilt)), ...cubic(pt(w, -11 - tilt), pt(w * 1.02, h), pt(-w * 1.02, h), pt(-w, -11))] }
|
||||
}
|
||||
return { outline: contour(filled ? expand(path, stroke / 2) : ribbon(path, stroke)), opening: filled ? contour(path) : Array.from({ length: COUNT }, () => pt(0, 0)), width: w, height: h }
|
||||
}
|
||||
function colorMix(colors: { color: string; weight: number }[]) {
|
||||
const value = [0, 0, 0]
|
||||
for (const { color, weight } of colors) for (let i = 0; i < 3; i++) value[i]! += parseInt(color.slice(1 + i * 2, 3 + i * 2), 16) * weight
|
||||
return `rgb(${value.map(v => Math.round(v)).join(',')})`
|
||||
}
|
||||
export function drawMorphEye(ctx: CanvasRenderingContext2D, pose: Pose, layers: FaceLayer[], r: number, side: number, blink: number, detail: Detail, ink: string, iris: boolean, gaze: Gaze) {
|
||||
const shape = blendContours(layers.map(layer => eyeContour({ ...pose, ...layer.traits }, r, side, blink, detail)), layers)
|
||||
const filled = (traits: FaceLayer['traits']) => !isLineEye(traits, side, detail)
|
||||
const cheeks = traitWeight(layers, t => t.cheeks && filled(t))
|
||||
ctx.save()
|
||||
if (cheeks > 0) {
|
||||
const h = Math.max(.12, 1 - blink) * pose.eyeHeight * (1 - .35 * traitWeight(layers, t => t.eye === 'soft')), cut = r * .82 * Math.sqrt(cheeks)
|
||||
ctx.beginPath(); ctx.rect(-r * 3, -r * 3, r * 6, r * 6); ctx.moveTo(cut, r * 1.13 * h); ctx.ellipse(0, r * 1.13 * h, cut, cut * h, 0, 0, TAU); ctx.clip('evenodd')
|
||||
}
|
||||
ctx.fillStyle = colorMix(layers.map(({ traits, weight }) => ({ weight, color: detail === 'full' && traits.eye === 'pupil' ? '#fffef9' : detail === 'full' && traits.eye === 'heart' && traits.faceSet === 'set-2' ? '#ff3f58' : ink })))
|
||||
trace(ctx, shape); ctx.fill(); ctx.clip()
|
||||
ctx.scale(1, Math.max(.12, 1 - blink) * pose.eyeHeight * (1 - .35 * traitWeight(layers, t => t.eye === 'soft')))
|
||||
const pupil = detail === 'full' ? traitWeight(layers, t => t.eye === 'pupil') : 0
|
||||
if (pupil > 0) {
|
||||
ctx.globalAlpha = pupil * (1 - blink); ctx.fillStyle = ink
|
||||
ctx.beginPath(); ctx.arc(gaze.x * r * 1.5 * .52, -gaze.y * r * 1.5 * .52, r * .6, 0, TAU); ctx.fill()
|
||||
}
|
||||
const visible = iris && detail === 'full' ? traitWeight(layers, t => filled(t) && t.eye !== 'pupil' && t.eye !== 'half-lidded') : 0
|
||||
if (visible > 0) {
|
||||
const dot = irisOffset(r, gaze, 0, cheeks)
|
||||
ctx.globalAlpha = visible * (1 - blink); ctx.fillStyle = '#ffffff'
|
||||
ctx.beginPath(); ctx.arc(dot.x, dot.y, r * .28, 0, TAU); ctx.fill()
|
||||
}
|
||||
ctx.restore()
|
||||
}
|
||||
/** Find the lower contour at the drool's right-side attachment point. Using
|
||||
* the rendered outline keeps it attached throughout mouth-shape morphs. */
|
||||
export function droolAnchor(outline: Point[], width: number): Point {
|
||||
const x = Math.min(width * .35, Math.max(...outline.map(p => p.x)) * .75)
|
||||
let bottom = -Infinity
|
||||
for (let i = 0; i < outline.length; i++) {
|
||||
const a = outline[i]!, b = outline[(i + 1) % outline.length]!
|
||||
if ((a.x <= x && b.x >= x) || (b.x <= x && a.x >= x)) {
|
||||
const y = Math.abs(b.x - a.x) < 1e-8 ? Math.max(a.y, b.y) : a.y + (b.y - a.y) * (x - a.x) / (b.x - a.x)
|
||||
bottom = Math.max(bottom, y)
|
||||
}
|
||||
}
|
||||
// The round cap overlaps the lip by two units so there is no visible gap.
|
||||
return { x, y: (Number.isFinite(bottom) ? bottom : 0) + 6 }
|
||||
}
|
||||
export function drawDrool(ctx: CanvasRenderingContext2D, anchor: Point, amount = 1) {
|
||||
ctx.save(); ctx.globalAlpha = amount; ctx.strokeStyle = '#fffef5'; ctx.lineWidth = 16
|
||||
ctx.beginPath(); ctx.moveTo(anchor.x, anchor.y); ctx.lineTo(anchor.x + 3, anchor.y + 34 * amount); ctx.stroke()
|
||||
ctx.fillStyle = '#fffef5'; ctx.beginPath(); ctx.arc(anchor.x + 3, anchor.y + 38 * amount, 8 * amount, 0, TAU); ctx.fill(); ctx.restore()
|
||||
}
|
||||
export function drawMorphMouth(ctx: CanvasRenderingContext2D, pose: Pose, layers: FaceLayer[], ink: string) {
|
||||
const geometry = layers.map(layer => mouthGeometry({ ...pose, ...layer.traits }))
|
||||
const outline = blendContours(geometry.map(g => g.outline), layers), opening = blendContours(geometry.map(g => g.opening), layers)
|
||||
const w = geometry.reduce((sum, g, i) => sum + g.width * layers[i]!.weight, 0), h = geometry.reduce((sum, g, i) => sum + g.height * layers[i]!.weight, 0)
|
||||
ctx.fillStyle = ink; trace(ctx, outline); ctx.fill()
|
||||
ctx.save(); trace(ctx, outline); ctx.clip(); trace(ctx, opening); ctx.clip()
|
||||
const tongue = traitWeight(layers, t => t.tongue), teeth = traitWeight(layers, t => t.teeth)
|
||||
ctx.translate(0, 27 * traitWeight(layers, t => t.mouth === 'cry'))
|
||||
if (tongue > 0) { ctx.globalAlpha = tongue; ctx.fillStyle = '#f37b83'; ctx.beginPath(); ctx.ellipse(10, h * .65, w * .65, h * .34, -.1, 0, TAU); ctx.fill() }
|
||||
if (teeth > 0) { ctx.globalAlpha = teeth; ctx.fillStyle = '#fffef8'; ctx.beginPath(); ctx.roundRect(-w * .76, -23, w * 1.52, 30, 12); ctx.fill() }
|
||||
ctx.restore()
|
||||
const out = traitWeight(layers, t => t.mouth === 'tongue-out')
|
||||
if (out > 0) {
|
||||
const w = 61 * pose.mouthWidth
|
||||
ctx.save(); ctx.globalAlpha = out; ctx.fillStyle = '#ff526c'; ctx.beginPath(); ctx.moveTo(-w * .6 * out, 5); ctx.lineTo(w * .6 * out, 5); ctx.bezierCurveTo(w * .88 * out, (85 * pose.mouthOpen + 28) * out, -w * .88 * out, (85 * pose.mouthOpen + 28) * out, -w * .6 * out, 5); ctx.fill(); ctx.restore()
|
||||
}
|
||||
const drool = traitWeight(layers, t => t.drool)
|
||||
if (drool > 0) drawDrool(ctx, droolAnchor(outline, w), drool)
|
||||
}
|
||||
|
||||
export function drawMorphBrow(ctx: CanvasRenderingContext2D, layers: FaceLayer[], r: number, side: number, ink: string) {
|
||||
if (!layers.some(layer => layer.traits.brows !== 'none')) return
|
||||
const shapes = layers.map(({ traits }) => {
|
||||
if (traits.brows === 'none') return contour([pt(0, 0)])
|
||||
const path = traits.brows === 'raised' ? arc(0, 10, r * .85, r * .85, Math.PI * 1.15, Math.PI * 1.85)
|
||||
: traits.brows === 'worried' ? quad(pt(side * r, 2), pt(-side * r * .1, 10), pt(-side * r * .75, -16))
|
||||
: [pt(side * r, -10), pt(-side * r * .75, 8)]
|
||||
return contour(ribbon(path, 12))
|
||||
})
|
||||
ctx.fillStyle = ink; trace(ctx, blendContours(shapes, layers)); ctx.fill()
|
||||
}
|
||||
|
|
@ -0,0 +1,65 @@
|
|||
// Vendored from ClipLab a050f724; see PROVENANCE.md and LICENSE.
|
||||
export interface Gaze { x: number; y: number }
|
||||
export interface PointerLook extends Gaze { weight: number }
|
||||
export interface EyeGazes { left: Gaze; right: Gaze }
|
||||
export const centeredGaze = (): Gaze => ({ x: 0, y: 0 })
|
||||
export const inactivePointer = (): PointerLook => ({ x: 0, y: 0, weight: 0 })
|
||||
|
||||
/** Geometric canvas coordinates retain the target's position, even outside the canvas. */
|
||||
export function pointerLook(pointer: { clientX: number; clientY: number }, bounds: { left: number; top: number; width: number; height: number }): PointerLook {
|
||||
return { x: (pointer.clientX - bounds.left) / Math.max(1, bounds.width) * 2 - 1, y: 1 - (pointer.clientY - bounds.top) / Math.max(1, bounds.height) * 2, weight: 1 }
|
||||
}
|
||||
|
||||
export function easePointer(current: PointerLook, target: PointerLook, seconds: number, immediate = false): PointerLook {
|
||||
return { ...easeGaze(current, target, seconds, immediate), weight: easeGaze({ x: current.weight, y: 0 }, { x: target.weight, y: 0 }, seconds, immediate).x }
|
||||
}
|
||||
|
||||
/** Solve in the projected eye's own axes, then gently saturate travel with distance. */
|
||||
export function gazeAtPoint(pointer: Gaze, center: Gaze, right: Gaze, up: Gaze): Gaze {
|
||||
const rx = right.x - center.x, ry = right.y - center.y, ux = up.x - center.x, uy = up.y - center.y
|
||||
const determinant = rx * uy - ry * ux
|
||||
if (Math.abs(determinant) < 1e-10) return centeredGaze()
|
||||
const dx = pointer.x - center.x, dy = pointer.y - center.y
|
||||
const x = (dx * uy - dy * ux) / determinant, y = (rx * dy - ry * dx) / determinant
|
||||
const distance = Math.hypot(x, y, 1.15)
|
||||
return { x: x / distance, y: y / distance }
|
||||
}
|
||||
|
||||
export function localGaze(gaze: Gaze, rotation: number): Gaze {
|
||||
const angle = rotation * Math.PI / 180
|
||||
return clampGaze({ x: gaze.x * Math.cos(angle) - gaze.y * Math.sin(angle), y: gaze.x * Math.sin(angle) + gaze.y * Math.cos(angle) })
|
||||
}
|
||||
|
||||
export function clampGaze({ x, y }: Gaze): Gaze {
|
||||
x = Number.isFinite(x) ? x : 0; y = Number.isFinite(y) ? y : 0
|
||||
const length = Math.max(1, Math.hypot(x, y))
|
||||
return { x: x / length, y: y / length }
|
||||
}
|
||||
|
||||
export function pointerGaze(pointer: { clientX: number; clientY: number }, bounds: { left: number; top: number; width: number; height: number }): Gaze {
|
||||
return clampGaze({
|
||||
x: (pointer.clientX - bounds.left - bounds.width / 2) / Math.max(80, bounds.width / 2),
|
||||
y: -(pointer.clientY - bounds.top - bounds.height / 2) / Math.max(80, bounds.height / 2)
|
||||
})
|
||||
}
|
||||
|
||||
/** Time-based easing, shared by the studio and paused or playing app characters. */
|
||||
export function easeGaze(current: Gaze, target: Gaze, seconds: number, immediate = false): Gaze {
|
||||
const amount = immediate ? 1 : 1 - Math.exp(-Math.max(0, seconds) * 16)
|
||||
const next = { x: current.x + (target.x - current.x) * amount, y: current.y + (target.y - current.y) * amount }
|
||||
return Math.hypot(next.x - target.x, next.y - target.y) < .0005 ? { ...target } : next
|
||||
}
|
||||
|
||||
/** Keep the complete white dot inside the eye, with a small margin at every angle. */
|
||||
export function irisOffset(radius: number, gaze: Gaze, eyeRotation = 0, cheeks: boolean | number = false): Gaze {
|
||||
const look = clampGaze(gaze), angle = eyeRotation * Math.PI / 180
|
||||
const x = look.x * radius * .62, y = -look.y * radius * .62
|
||||
const dot = { x: x * Math.cos(angle) + y * Math.sin(angle), y: -x * Math.sin(angle) + y * Math.cos(angle) }
|
||||
// Stay above the cheek's circular cutout, including the dot radius and a small gap.
|
||||
if (cheeks) {
|
||||
const amount = typeof cheeks === 'number' ? Math.max(0, Math.min(1, cheeks)) : 1
|
||||
const clearance = radius * (.82 * Math.sqrt(amount) + .305)
|
||||
if (Math.abs(dot.x) < clearance) dot.y = Math.min(dot.y, radius * 1.13 - Math.sqrt(clearance ** 2 - dot.x ** 2))
|
||||
}
|
||||
return dot
|
||||
}
|
||||
|
|
@ -0,0 +1,323 @@
|
|||
// Vendored from ClipLab a050f724; see PROVENANCE.md and LICENSE.
|
||||
export type Shape = 'capsule' | 'cap' | 'sphere'
|
||||
export type Eye = 'dot' | 'soft' | 'closed' | 'wink' | 'star' | 'heart' | 'squint' | 'wide' | 'arc-up' | 'arc-down' | 'half-lidded' | 'pupil'
|
||||
export type Mouth = 'smile' | 'open' | 'line' | 'frown' | 'oh' | 'wave' | 'sleep' | 'grin' | 'cry' | 'u-smile' | 'kiss' | 'tongue-out'
|
||||
export type Prop = 'none' | 'zzz' | 'sparkle' | 'heart' | 'question' | 'sweat' | 'crown'
|
||||
export type Detail = 'body' | 'eyes' | 'full'
|
||||
|
||||
export interface Pose {
|
||||
eye: Eye; mouth: Mouth; prop: Prop
|
||||
faceSet: 'set-1' | 'set-2'; brows: 'none' | 'raised' | 'worried' | 'angry'; blush: number
|
||||
eyeSize: number; eyeHeight: number; spacing: number; eyeTilt: number
|
||||
leftScale: number; rightScale: number; gazeX: number; gazeY: number
|
||||
leftX: number; rightX: number; leftY: number; rightY: number; leftRotation: number; rightRotation: number
|
||||
mouthWidth: number; mouthOpen: number; faceScale: number; faceY: number
|
||||
rotationX: number; rotationY: number; rotationZ: number; squash: number
|
||||
tongue: boolean; teeth: boolean; drool: boolean; cheeks: boolean; tears: boolean; mouthStroke: number
|
||||
}
|
||||
export interface Beat { id: string; name: string; duration: number; pose: Pose; gradientAction?: 'rotate' | 'hold' }
|
||||
export interface Expression { id: string; name: string; description: string; beats: Beat[]; poseExpressionId?: string }
|
||||
export interface Step { id: string; expressionId: string; duration: number }
|
||||
export interface Animation { id: string; name: string; steps: Step[]; loop: boolean }
|
||||
export interface Character {
|
||||
id: string; name: string; shape: Shape; color: string; color2: string; gradient: boolean
|
||||
gradientAngle: number; toon: boolean; trueFront: boolean; lockPosition: boolean; eyeColor: string; iris: boolean
|
||||
elevated: boolean; elevation: number; shadow: boolean
|
||||
motion: number; speed: number; blink: boolean; blinkInterval: number; followCursor: boolean; followRotation: boolean
|
||||
}
|
||||
export interface Definition { version: 1; character: Character; expressions: Expression[]; animations: Animation[] }
|
||||
export interface Project { version: 1; name: string; characters: Character[]; expressions: Expression[]; animations: Animation[]; defaultsRevision?: number }
|
||||
export type FaceTraits = Pick<Pose, 'eye' | 'mouth' | 'faceSet' | 'brows' | 'cheeks' | 'tongue' | 'teeth' | 'drool'>
|
||||
export interface FaceLayer { traits: FaceTraits; weight: number }
|
||||
export interface Sample { pose: Pose; blink: number; bob: number; breathe: number; expressionId: string; beatIndex: number; stepIndex: number; effectPhase?: number; propAmount?: number; tearAmount?: number; gradientRotation?: number; gradientMix?: number; faceLayers?: FaceLayer[] }
|
||||
|
||||
export const EYES: Eye[] = ['dot', 'soft', 'closed', 'wink', 'star', 'heart', 'squint', 'wide', 'arc-up', 'arc-down', 'half-lidded', 'pupil']
|
||||
export const MOUTHS: Mouth[] = ['smile', 'open', 'line', 'frown', 'oh', 'wave', 'sleep', 'grin', 'cry', 'u-smile', 'kiss', 'tongue-out']
|
||||
export const PROPS: Prop[] = ['none', 'zzz', 'sparkle', 'heart', 'question', 'sweat', 'crown']
|
||||
export const SHAPES: { id: Shape; name: string; ratio: string }[] = [
|
||||
{ id: 'capsule', name: 'Capsule', ratio: '1:2' }, { id: 'cap', name: 'End cap', ratio: '1:1' }, { id: 'sphere', name: 'Circle', ratio: '1:1' }
|
||||
]
|
||||
export const PALETTES = [
|
||||
['#ff986d', '#ffd092'], ['#81d4c1', '#c6efd2'], ['#a79ce8', '#d8cafa'],
|
||||
['#f0ce58', '#fff1a5'], ['#ee98b2', '#ffd2da'], ['#72b6de', '#b9e4f5']
|
||||
]
|
||||
|
||||
export function detailAt(size: number): Detail { return size <= 16 ? 'body' : size <= 24 ? 'eyes' : 'full' }
|
||||
export function uid(prefix = 'item') { return `${prefix}-${crypto.randomUUID().slice(0, 8)}` }
|
||||
export function clone<T>(value: T): T { return JSON.parse(JSON.stringify(value)) as T }
|
||||
export const BASE_POSE: Pose = {
|
||||
eye: 'dot', mouth: 'smile', prop: 'none', faceSet: 'set-1', brows: 'none', blush: 0, eyeSize: 1, eyeHeight: 1, spacing: 1, eyeTilt: 0,
|
||||
leftScale: 1, rightScale: 1, gazeX: 0, gazeY: 0, mouthWidth: 1, mouthOpen: .5,
|
||||
leftX: 0, rightX: 0, leftY: 0, rightY: 0, leftRotation: 0, rightRotation: 0,
|
||||
faceScale: .75, faceY: 0, rotationX: 0, rotationY: 0, rotationZ: 0, squash: 1,
|
||||
tongue: false, teeth: false, drool: false, cheeks: false, tears: false, mouthStroke: 1
|
||||
}
|
||||
const p = (v: Partial<Pose> = {}): Pose => ({ ...BASE_POSE, ...v })
|
||||
const b = (id: string, name: string, duration: number, pose: Partial<Pose>): Beat => ({ id, name, duration, pose: p(pose) })
|
||||
|
||||
export function defaultExpressions(): Expression[] {
|
||||
return [
|
||||
{ id: 'idle', name: 'Idle', description: 'A little breath. A quiet smile.', beats: [
|
||||
b('idle-1', 'Settle', 1.8, {}), b('idle-2', 'Look around', 1.4, { gazeX: .22, rotationZ: -3, rotationY: 5 }), b('idle-3', 'Return', 1.8, {})
|
||||
] },
|
||||
{ id: 'listening', name: 'Listening', description: 'Present, curious, and all ears.', beats: [
|
||||
b('listen-1', 'Notice', 1.2, { eye: 'wide', mouth: 'line', eyeSize: 1.06, rotationX: -4 }), b('listen-2', 'Lean in', 1.5, { mouth: 'smile', eyeHeight: 1.13, rotationZ: 6, rotationX: -7 }), b('listen-3', 'Nod', 1.1, { mouth: 'smile', rotationX: 5, rotationZ: 3 })
|
||||
] },
|
||||
{ id: 'thinking', name: 'Thinking', description: 'A glance up. Something is taking shape.', beats: [
|
||||
b('think-1', 'Wonder', 1.5, { mouth: 'oh', mouthOpen: .23, mouthWidth: .6, gazeX: .6, gazeY: .55, rotationZ: -7, prop: 'question' }), b('think-2', 'Consider', 1.6, { eye: 'soft', mouth: 'wave', gazeX: -.35, gazeY: .2, rotationY: -8, rotationZ: 4 }), b('think-3', 'Almost', 1.2, { mouth: 'smile', gazeY: .25, rotationZ: -3 })
|
||||
] },
|
||||
{ id: 'working', name: 'Working', description: 'Small, focused, steady movements.', beats: [
|
||||
b('work-1', 'Focus left', 1.0, { eye: 'soft', mouth: 'line', mouthWidth: .65, gazeX: -.5, gazeY: -.3, rotationY: -7, rotationX: 4 }), b('work-2', 'Focus right', 1.0, { eye: 'soft', mouth: 'line', mouthWidth: .65, gazeX: .5, gazeY: -.3, rotationY: 7, rotationX: 4 }), b('work-3', 'Check', 1.4, { mouth: 'smile', rotationX: -3 })
|
||||
] },
|
||||
{ id: 'happy', name: 'Happy', description: 'A small open smile says it all.', beats: [
|
||||
b('happy-1', 'Light up', .9, { mouth: 'open', mouthOpen: .62, rotationZ: -5, squash: 1.025 }), b('happy-2', 'Delight', 1.3, { eye: 'arc-up', mouth: 'grin', mouthOpen: .8, prop: 'sparkle', rotationZ: 5, squash: .97 }), b('happy-3', 'Glow', 1.5, { mouth: 'open', mouthOpen: .5, rotationZ: -2 })
|
||||
] },
|
||||
{ id: 'confused', name: 'Confused', description: 'A little uneven. A second look.', beats: [
|
||||
b('confused-1', 'Wait', 1.2, { eye: 'soft', leftScale: .7, rightScale: 1.1, mouth: 'wave', rotationZ: 10, prop: 'question' }), b('confused-2', 'Really?', 1.4, { leftScale: 1.1, rightScale: .7, mouth: 'oh', mouthOpen: .25, rotationZ: -9, gazeX: -.2 }), b('confused-3', 'Hmm', 1.3, { eye: 'soft', mouth: 'frown', mouthWidth: .75, rotationZ: 4 })
|
||||
] },
|
||||
{ id: 'sleepy', name: 'Sleepy', description: 'Slow breathing, drifting sleep marks.', beats: [
|
||||
b('sleep-1', 'Doze', 2, { eye: 'arc-down', mouth: 'sleep', prop: 'zzz', drool: true, rotationZ: -6, rotationX: 4, squash: .97 }), b('sleep-2', 'Breathe in', 2, { eye: 'arc-down', mouth: 'oh', mouthOpen: .25, mouthWidth: .65, prop: 'zzz', drool: true, rotationZ: -3, squash: 1.025 }), b('sleep-3', 'Breathe out', 2, { eye: 'arc-down', mouth: 'sleep', prop: 'zzz', drool: true, rotationZ: -7, squash: .98 })
|
||||
] },
|
||||
{ id: 'sad', name: 'Sad', description: 'A round frown, a little lower.', beats: [
|
||||
b('sad-1', 'Low', 1.6, { mouth: 'frown', rotationZ: -3 }), b('sad-2', 'Sigh', 1.8, { eye: 'arc-down', mouth: 'frown', rotationX: 4 }), b('sad-3', 'Look up', 1.6, { mouth: 'frown', gazeY: .2 })
|
||||
] },
|
||||
{ id: 'tearful', name: 'Tearful', description: 'A trembling mouth and falling tears.', beats: [
|
||||
b('tear-1', 'Well up', 1.5, { mouth: 'cry', tears: true, mouthOpen: .6, rotationZ: -3 }), b('tear-2', 'Let go', 1.8, { mouth: 'cry', tears: true, mouthOpen: .85, eye: 'arc-down', rotationZ: 3 }), b('tear-3', 'Breathe', 1.8, { mouth: 'cry', tears: true, mouthOpen: .5 })
|
||||
] },
|
||||
{ id: 'playful', name: 'Playful', description: 'A wink, a grin, a little mischief.', beats: [
|
||||
b('play-1', 'Peek', 1.1, { mouth: 'open', tongue: true, rotationY: -10, rotationZ: -8 }), b('play-2', 'Wink', 1.2, { eye: 'squint', mouth: 'u-smile', tongue: true, mouthOpen: .7, rotationZ: 9, prop: 'sparkle' }), b('play-3', 'Grin', 1.3, { mouth: 'grin', cheeks: true, teeth: true, mouthOpen: .45, rotationZ: -3 })
|
||||
] },
|
||||
{ id: 'loading', name: 'Loading', description: 'Focused Working gestures with a gently eased gradient turn.', poseExpressionId: 'working', beats: [
|
||||
{ ...b('loading-rotate', 'Rotate', 2, {}), gradientAction: 'rotate' },
|
||||
{ ...b('loading-hold', 'Hold', .5, {}), gradientAction: 'hold' }
|
||||
] }
|
||||
]
|
||||
}
|
||||
export function defaultProject(): Project {
|
||||
const base: Character = {
|
||||
id: 'milo', name: 'Milo', shape: 'capsule', color: '#ff986d', color2: '#ffd092', gradient: true,
|
||||
gradientAngle: 20, toon: true, trueFront: false, lockPosition: false, eyeColor: '#080909', iris: false, elevated: false, elevation: .065,
|
||||
shadow: true, motion: .45, speed: 1, blink: true, blinkInterval: 4.2, followCursor: false, followRotation: false
|
||||
}
|
||||
const expressions = defaultExpressions()
|
||||
const sequences: Record<string, [string, number][]> = {
|
||||
listening: [['listening', 3.8], ['idle', 1.6]],
|
||||
thinking: [['thinking', 4.3], ['working', 1.8]],
|
||||
working: [['working', 6.8], ['thinking', 2.2]],
|
||||
happy: [['happy', 3.7], ['playful', 2.4], ['idle', 1.8]],
|
||||
confused: [['confused', 3.9], ['thinking', 2.1]],
|
||||
playful: [['playful', 3.6], ['happy', 2.2]]
|
||||
}
|
||||
return {
|
||||
version: 1, name: 'My character studio', defaultsRevision: 2,
|
||||
characters: [base, { ...base, id: 'pip', name: 'Pip', shape: 'cap', color: '#81d4c1', color2: '#c6efd2' }, { ...base, id: 'lumi', name: 'Lumi', shape: 'sphere', color: '#a79ce8', color2: '#d8cafa' }],
|
||||
expressions,
|
||||
animations: expressions.map((e) => ({
|
||||
id: e.id, name: e.id === 'happy' ? 'Success' : e.name, loop: true,
|
||||
steps: (sequences[e.id] ?? [[e.id, expressionDuration(e)] as [string, number]]).map(([expressionId, duration], i) => ({ id: `${e.id}-step-${i + 1}`, expressionId, duration }))
|
||||
}))
|
||||
}
|
||||
}
|
||||
const safeDuration = (value: number) => Number.isFinite(value) ? Math.max(.2, value) : 1
|
||||
export function expressionDuration(expression: Expression) { return expression.beats.reduce((n, beat) => n + safeDuration(beat.duration), 0) }
|
||||
export function animationDuration(animation: Animation) { return animation.steps.reduce((n, step) => n + safeDuration(step.duration), 0) }
|
||||
export function isGradientExpression(expression: Expression) { return expression.beats.some(beat => beat.gradientAction !== undefined) }
|
||||
export function addLoadingAnimation(project: Project): Animation {
|
||||
let expression = project.expressions.find(e => e.id === 'loading' && isGradientExpression(e)) ?? project.expressions.find(isGradientExpression)
|
||||
const existing = expression && project.animations.find(a => a.steps.length === 1 && a.steps[0]!.expressionId === expression!.id)
|
||||
if (existing) return existing
|
||||
const needsWorking = (!expression || expression.poseExpressionId === 'working') && !project.expressions.some(e => e.id === 'working')
|
||||
if (project.animations.length >= 100 || project.expressions.length + (expression ? 0 : 1) + (needsWorking ? 1 : 0) > 100) throw new Error('Make room in the library before adding Loading. Projects support up to 100 expressions and animations.')
|
||||
if (!expression) {
|
||||
expression = defaultExpressions().find(e => e.id === 'loading')!
|
||||
if (project.expressions.some(e => e.id === expression!.id)) expression.id = uid('loading')
|
||||
project.expressions.push(expression)
|
||||
}
|
||||
if (expression.poseExpressionId === 'working' && !project.expressions.some(e => e.id === 'working')) {
|
||||
project.expressions.push(defaultExpressions().find(e => e.id === 'working')!)
|
||||
}
|
||||
const animation: Animation = { id: project.animations.some(a => a.id === 'loading') ? uid('loading') : 'loading', name: 'Loading', loop: true, steps: [{ id: uid('step'), expressionId: expression.id, duration: expressionDuration(expression) }] }
|
||||
project.animations.push(animation)
|
||||
return animation
|
||||
}
|
||||
export function definitionOf(project: Project, character: Character, animationIds?: string[]): Definition {
|
||||
const animations = project.animations.filter(a => !animationIds || animationIds.includes(a.id))
|
||||
const used = new Set(animations.flatMap(a => a.steps.map(s => s.expressionId)))
|
||||
// A selected Loading export also carries the expression playing alongside it.
|
||||
for (const id of used) {
|
||||
const source = project.expressions.find(e => e.id === id)?.poseExpressionId
|
||||
if (source) used.add(source)
|
||||
}
|
||||
return clone({ version: 1, character, expressions: animationIds ? project.expressions.filter(e => used.has(e.id)) : project.expressions, animations })
|
||||
}
|
||||
const numericKeys = Object.keys(BASE_POSE).filter(k => typeof BASE_POSE[k as keyof Pose] === 'number') as (keyof Pose)[]
|
||||
const faceKeys = ['eye', 'mouth', 'faceSet', 'brows', 'cheeks', 'tongue', 'teeth', 'drool'] as const
|
||||
export function faceLayers(pose: Pose): FaceLayer[] {
|
||||
return [{ traits: Object.fromEntries(faceKeys.map(key => [key, pose[key]])) as FaceTraits, weight: 1 }]
|
||||
}
|
||||
export function mixFaceLayers(a: FaceLayer[], b: FaceLayer[], t: number): FaceLayer[] {
|
||||
const combined = new Map<string, FaceLayer>()
|
||||
for (const [layers, amount] of [[a, 1 - t], [b, t]] as const) for (const layer of layers) {
|
||||
const weight = layer.weight * amount
|
||||
if (weight <= 0) continue
|
||||
const key = JSON.stringify(layer.traits), prior = combined.get(key)
|
||||
if (prior) prior.weight += weight
|
||||
else combined.set(key, { traits: layer.traits, weight })
|
||||
}
|
||||
return [...combined.values()]
|
||||
}
|
||||
export function mixPose(a: Pose, b: Pose, t: number): Pose {
|
||||
const result = { ...(t < .5 ? a : b) }
|
||||
for (const key of numericKeys) {
|
||||
let delta = (b[key] as number) - (a[key] as number)
|
||||
if (key.startsWith('rotation')) delta = ((delta + 180) % 360 + 360) % 360 - 180
|
||||
;(result as unknown as Record<string, unknown>)[key] = (a[key] as number) + delta * t
|
||||
}
|
||||
return result
|
||||
}
|
||||
const smooth = (t: number) => { const v = Math.max(0, Math.min(1, t)); return v * v * (3 - 2 * v) }
|
||||
function featureAmount(a: unknown, b: unknown, blend: number) { return a === b ? 1 : blend < .5 ? 1 - smooth(blend * 2) : smooth((blend - .5) * 2) }
|
||||
const mod = (t: number, d: number) => ((t % d) + d) % d
|
||||
export function sampleExpression(expression: Expression, time: number, expressions: Expression[] = [], visited = new Set<string>()): { pose: Pose; beatIndex: number; propAmount: number; tearAmount: number; gradientRotation?: number; gradientMix?: number; faceLayers: FaceLayer[] } {
|
||||
const duration = expressionDuration(expression)
|
||||
let t = mod(time, duration)
|
||||
const gradient = isGradientExpression(expression)
|
||||
let gradientStart = 0
|
||||
for (let i = 0; i < expression.beats.length; i++) {
|
||||
const beat = expression.beats[i]!
|
||||
const beatDuration = safeDuration(beat.duration)
|
||||
if (t < beatDuration || i === expression.beats.length - 1) {
|
||||
const previous = expression.beats[(i - 1 + expression.beats.length) % expression.beats.length]!
|
||||
const transition = Math.min(.45, beatDuration * .4)
|
||||
const progress = Math.min(t / transition, 1)
|
||||
const eased = progress * progress * (3 - 2 * progress)
|
||||
const turn = Math.min(1, t / beatDuration)
|
||||
const turnEased = (1 - Math.cos(Math.PI * turn)) / 2
|
||||
const source = expressions.find(e => e.id === expression.poseExpressionId && !visited.has(e.id) && e.id !== expression.id)
|
||||
const face = source ? sampleExpression(source, mod(time, duration) / duration * expressionDuration(source), expressions, new Set([...visited, expression.id])) : undefined
|
||||
return { pose: face?.pose ?? mixPose(previous.pose, beat.pose, eased), faceLayers: face?.faceLayers ?? mixFaceLayers(faceLayers(previous.pose), faceLayers(beat.pose), eased), beatIndex: i, propAmount: face?.propAmount ?? featureAmount(previous.pose.prop, beat.pose.prop, eased), tearAmount: face?.tearAmount ?? featureAmount(previous.pose.tears, beat.pose.tears, eased), ...(gradient ? { gradientRotation: gradientStart + (beat.gradientAction === 'rotate' ? 360 * turnEased : 0), gradientMix: 1 } : {}) }
|
||||
}
|
||||
if (beat.gradientAction === 'rotate') gradientStart += 360
|
||||
t -= beatDuration
|
||||
}
|
||||
return { pose: p(), faceLayers: faceLayers(p()), beatIndex: 0, propAmount: 1, tearAmount: 1 }
|
||||
}
|
||||
export function sampleDefinition(def: Definition, animationId: string, time: number, expressionId?: string): Sample {
|
||||
const speedTime = Math.max(0, time) * def.character.speed
|
||||
const animation = def.animations.find(a => a.id === animationId) ?? def.animations[0]
|
||||
let expression = def.expressions.find(e => e.id === expressionId) ?? def.expressions[0]!
|
||||
let local = speedTime, stepIndex = 0
|
||||
let previousExpression: Expression | undefined
|
||||
let blend = 1
|
||||
if (!expressionId && animation?.steps.length) {
|
||||
const total = animationDuration(animation)
|
||||
local = animation.loop ? mod(speedTime, total) : Math.min(speedTime, total - .00001)
|
||||
for (let i = 0; i < animation.steps.length; i++) {
|
||||
const step = animation.steps[i]!
|
||||
const stepDuration = safeDuration(step.duration)
|
||||
if (local < stepDuration || i === animation.steps.length - 1) {
|
||||
expression = def.expressions.find(e => e.id === step.expressionId) ?? expression
|
||||
stepIndex = i
|
||||
if (animation.steps.length > 1 && (i > 0 || animation.loop)) {
|
||||
const prev = animation.steps[(i - 1 + animation.steps.length) % animation.steps.length]!
|
||||
previousExpression = def.expressions.find(e => e.id === prev.expressionId)
|
||||
blend = Math.min(1, local / Math.min(.35, stepDuration * .2))
|
||||
}
|
||||
local = local / stepDuration * expressionDuration(expression)
|
||||
break
|
||||
}
|
||||
local -= stepDuration
|
||||
}
|
||||
}
|
||||
const sample = sampleExpression(expression, local, def.expressions)
|
||||
if (previousExpression && blend < 1) {
|
||||
const previous = sampleExpression(previousExpression, expressionDuration(previousExpression) - .00001, def.expressions)
|
||||
const eased = smooth(blend)
|
||||
const oldProp = previous.pose.prop, oldTears = previous.pose.tears
|
||||
sample.propAmount *= featureAmount(oldProp, sample.pose.prop, eased)
|
||||
sample.tearAmount *= featureAmount(oldTears, sample.pose.tears, eased)
|
||||
sample.pose = mixPose(previous.pose, sample.pose, eased)
|
||||
sample.faceLayers = mixFaceLayers(previous.faceLayers, sample.faceLayers, eased)
|
||||
if (previous.gradientRotation !== undefined || sample.gradientRotation !== undefined) {
|
||||
const from = previous.gradientRotation ?? 0, to = sample.gradientRotation ?? 0
|
||||
// Align the previous completed turn to the next expression's starting angle.
|
||||
// Keep the incoming turn unwrapped so crossing 180 degrees cannot flip it.
|
||||
const alignedFrom = from - Math.round(from / 360) * 360
|
||||
sample.gradientRotation = alignedFrom + (to - alignedFrom) * eased
|
||||
const fromMix = previous.gradientMix ?? 0
|
||||
sample.gradientMix = fromMix + ((sample.gradientMix ?? 0) - fromMix) * eased
|
||||
}
|
||||
}
|
||||
// All secondary motion uses the cycle period, so exported loops close exactly.
|
||||
const period = expressionId ? expressionDuration(expression) : animation ? animationDuration(animation) : expressionDuration(expression)
|
||||
const phase = 2 * Math.PI * mod(speedTime, period) / period
|
||||
const blinkCount = Math.max(1, Math.round(period / def.character.blinkInterval))
|
||||
const blinkPhase = mod(speedTime + period * .17, period / blinkCount)
|
||||
const gradientOnly = isGradientExpression(expression) && !expression.poseExpressionId
|
||||
const blink = !gradientOnly && def.character.blink && blinkPhase < .19 ? Math.sin(blinkPhase / .19 * Math.PI) ** 2 : 0
|
||||
return { ...sample, expressionId: expression.id, stepIndex, blink, effectPhase: mod(speedTime, period) / period * Math.max(1, Math.round(period / 2.4)), bob: gradientOnly ? 0 : Math.sin(phase) * def.character.motion, breathe: gradientOnly ? 0 : Math.cos(phase) * def.character.motion }
|
||||
}
|
||||
|
||||
function obj(v: unknown): Record<string, unknown> { if (!v || typeof v !== 'object' || Array.isArray(v)) throw new Error('This file is not a ClipLab project.'); return v as Record<string, unknown> }
|
||||
function num(v: unknown, fallback: number, low: number, high: number) { return typeof v === 'number' && Number.isFinite(v) ? Math.min(high, Math.max(low, v)) : fallback }
|
||||
function str(v: unknown, fallback: string, max = 80) { return typeof v === 'string' && v.trim() ? v.trim().slice(0, max) : fallback }
|
||||
function color(v: unknown, fallback: string) { return typeof v === 'string' && /^#[0-9a-f]{6}$/i.test(v) ? v : fallback }
|
||||
function choice<T extends string>(v: unknown, options: T[], fallback: T) { return options.includes(v as T) ? v as T : fallback }
|
||||
function boolean(v: unknown, fallback: boolean) { return typeof v === 'boolean' ? v : fallback }
|
||||
function parsePose(value: unknown): Pose {
|
||||
const v = obj(value), pose = { ...BASE_POSE }
|
||||
const ranges: Record<string, [number, number]> = { blush: [0, 1], mouthStroke: [.5, 2.2], leftX: [-60, 60], rightX: [-60, 60], leftY: [-60, 60], rightY: [-60, 60], leftRotation: [-90, 90], rightRotation: [-90, 90], eyeSize: [.35, 2], eyeHeight: [.15, 2], spacing: [.5, 1.6], eyeTilt: [-45, 45], leftScale: [.3, 1.8], rightScale: [.3, 1.8], gazeX: [-1, 1], gazeY: [-1, 1], mouthWidth: [.3, 1.7], mouthOpen: [.1, 1], faceScale: [.6, 1.4], faceY: [-.3, .3], rotationX: [-180, 180], rotationY: [-180, 180], rotationZ: [-180, 180], squash: [.8, 1.2] }
|
||||
for (const key of numericKeys) { const [lo, hi] = ranges[key]!; (pose as unknown as Record<string, unknown>)[key] = num(v[key], BASE_POSE[key] as number, lo, hi) }
|
||||
return { ...pose, faceSet: choice(v.faceSet, ['set-1', 'set-2'], 'set-1'), brows: choice(v.brows, ['none', 'raised', 'worried', 'angry'], 'none'), eye: choice(v.eye, EYES, 'dot'), mouth: choice(v.mouth, MOUTHS, 'smile'), prop: choice(v.prop, PROPS, 'none'), tongue: boolean(v.tongue, false), teeth: boolean(v.teeth, false), drool: boolean(v.drool, false), cheeks: boolean(v.cheeks, false), tears: boolean(v.tears, false) }
|
||||
}
|
||||
export function parseCharacter(value: unknown): Character {
|
||||
const v = obj(value), d = defaultProject().characters[0]!
|
||||
return { ...d, id: str(v.id, uid('character')), name: str(v.name, 'Character'), shape: choice(v.shape, ['capsule', 'cap', 'sphere'], 'capsule'), color: color(v.color, d.color), color2: color(v.color2, d.color2), gradient: boolean(v.gradient, true), gradientAngle: num(v.gradientAngle, 20, -180, 180), toon: boolean(v.toon, d.toon), trueFront: boolean(v.trueFront, false), lockPosition: boolean(v.lockPosition, false), eyeColor: color(v.eyeColor, d.eyeColor), iris: boolean(v.iris, false), elevated: boolean(v.elevated, false), elevation: num(v.elevation, .065, .005, .2), shadow: boolean(v.shadow, true), motion: num(v.motion, .45, 0, 1), speed: num(v.speed, 1, .25, 3), blink: boolean(v.blink, true), blinkInterval: num(v.blinkInterval, 4.2, 1, 12), followCursor: boolean(v.followCursor, false), followRotation: boolean(v.followRotation, false) }
|
||||
}
|
||||
export function parseProject(value: unknown): Project {
|
||||
const v = obj(value)
|
||||
if (v.version !== 1 || !Array.isArray(v.expressions) || !Array.isArray(v.animations)) throw new Error('Choose a ClipLab version 1 project or character definition.')
|
||||
const charactersRaw = Array.isArray(v.characters) ? v.characters : v.character ? [v.character] : []
|
||||
if (!charactersRaw.length || charactersRaw.length > 30 || !v.expressions.length || v.expressions.length > 100 || !v.animations.length || v.animations.length > 100) throw new Error('The project has an unsupported number of characters or sequences.')
|
||||
const expressions = v.expressions.map((item, index): Expression => {
|
||||
const e = obj(item)
|
||||
if (!Array.isArray(e.beats) || !e.beats.length || e.beats.length > 32) throw new Error('Each expression needs 1–32 beats.')
|
||||
return { id: str(e.id, `expression-${index}`), name: str(e.name, 'Expression'), description: str(e.description, '', 160), ...(typeof e.poseExpressionId === 'string' && e.poseExpressionId ? { poseExpressionId: e.poseExpressionId } : {}), beats: e.beats.map((item, i) => { const b = obj(item); return { id: str(b.id, `beat-${i}`), name: str(b.name, `Beat ${i + 1}`), duration: num(b.duration, 1.5, .2, 15), pose: parsePose(b.pose), ...(b.gradientAction === 'rotate' || b.gradientAction === 'hold' ? { gradientAction: b.gradientAction } : {}) } }) }
|
||||
})
|
||||
if (new Set(expressions.map(e => e.id)).size !== expressions.length) throw new Error('Expression names in the file must have unique IDs.')
|
||||
for (const expression of expressions) {
|
||||
const seen = new Set([expression.id])
|
||||
let sourceId = expression.poseExpressionId
|
||||
while (sourceId) {
|
||||
const source = expressions.find(e => e.id === sourceId)
|
||||
if (!source) throw new Error('An expression refers to missing face motion.')
|
||||
if (seen.has(sourceId)) throw new Error('Face motion expressions cannot refer back to themselves.')
|
||||
seen.add(sourceId); sourceId = source.poseExpressionId
|
||||
}
|
||||
}
|
||||
const animations = v.animations.map((item, index): Animation => {
|
||||
const a = obj(item)
|
||||
if (!Array.isArray(a.steps) || !a.steps.length || a.steps.length > 64) throw new Error('Each animation needs 1–64 steps.')
|
||||
return { id: str(a.id, `animation-${index}`), name: str(a.name, 'Animation'), loop: boolean(a.loop, true), steps: a.steps.map((item, i) => { const s = obj(item); if (!expressions.some(e => e.id === s.expressionId)) throw new Error('An animation refers to a missing expression.'); return { id: str(s.id, `step-${i}`), expressionId: s.expressionId as string, duration: num(s.duration, 3, .2, 30) } }) }
|
||||
})
|
||||
const characters = charactersRaw.map(parseCharacter)
|
||||
if (new Set(characters.map(c => c.id)).size !== characters.length || new Set(animations.map(a => a.id)).size !== animations.length) throw new Error('Characters and animations need unique IDs.')
|
||||
return { version: 1, name: str(v.name, 'My character studio'), characters, expressions, animations, ...(typeof v.defaultsRevision === 'number' ? { defaultsRevision: num(v.defaultsRevision, 2, 0, 1000) } : {}) }
|
||||
}
|
||||
|
||||
/** Upgrade only untouched studio defaults, once; portable definitions stay literal. */
|
||||
export function upgradeStudioDefaults(project: Project): Project {
|
||||
if ((project.defaultsRevision ?? 0) >= 2) return project
|
||||
const defaults = defaultExpressions()
|
||||
for (const expression of project.expressions) {
|
||||
const preset = defaults.find(e => e.id === expression.id)
|
||||
if (!preset) continue
|
||||
for (const beat of expression.beats) if (beat.pose.faceScale === 1 && preset.beats.some(b => b.id === beat.id)) beat.pose.faceScale = .75
|
||||
if (expression.id === 'loading' && !expression.poseExpressionId && expression.beats.length === 2 && expression.beats.every(beat => {
|
||||
const original = preset.beats.find(b => b.id === beat.id)
|
||||
return original && original.gradientAction === beat.gradientAction && Object.keys(BASE_POSE).every(key => beat.pose[key as keyof Pose] === BASE_POSE[key as keyof Pose])
|
||||
}) && project.expressions.some(e => e.id === 'working')) expression.poseExpressionId = 'working'
|
||||
}
|
||||
project.defaultsRevision = 2
|
||||
return project
|
||||
}
|
||||
|
|
@ -0,0 +1,75 @@
|
|||
// Generated from ui/src/index.css by scripts/sync-agent-palette-tokens.mjs.
|
||||
export const CAP_V1_COLORS = {
|
||||
"bubblegum-sky": {
|
||||
"a": "#8bd4ff",
|
||||
"b": "#ff51bf"
|
||||
},
|
||||
"pink-lemonade": {
|
||||
"a": "#ff26a8",
|
||||
"b": "#fff78a"
|
||||
},
|
||||
"orchid-peach": {
|
||||
"a": "#e771ff",
|
||||
"b": "#ffd87c"
|
||||
},
|
||||
"coral-mint": {
|
||||
"a": "#beffe8",
|
||||
"b": "#ff797b"
|
||||
},
|
||||
"lime-lagoon": {
|
||||
"a": "#b4ffa4",
|
||||
"b": "#26dfff"
|
||||
},
|
||||
"arctic-blue": {
|
||||
"a": "#97fff3",
|
||||
"b": "#0084ff"
|
||||
},
|
||||
"solar-flare": {
|
||||
"a": "#ecca5c",
|
||||
"b": "#fe3c3f"
|
||||
},
|
||||
"violet-ember": {
|
||||
"a": "#ff5c43",
|
||||
"b": "#6262ff"
|
||||
},
|
||||
"deep-tide": {
|
||||
"a": "#003d60",
|
||||
"b": "#32fffc"
|
||||
},
|
||||
"coral-current": {
|
||||
"a": "#26e2ff",
|
||||
"b": "#ff6666"
|
||||
},
|
||||
"golden-hour": {
|
||||
"a": "#ffcd4f",
|
||||
"b": "#fde5ba"
|
||||
},
|
||||
"tangerine-cobalt": {
|
||||
"a": "#ffad32",
|
||||
"b": "#4169ff"
|
||||
},
|
||||
"electric-grove": {
|
||||
"a": "#b4ff32",
|
||||
"b": "#008d58"
|
||||
},
|
||||
"flamingo-jade": {
|
||||
"a": "#ff4f9a",
|
||||
"b": "#36edaa"
|
||||
},
|
||||
"cherry-pop": {
|
||||
"a": "#ff69db",
|
||||
"b": "#d51c46"
|
||||
},
|
||||
"turquoise-cherry": {
|
||||
"a": "#37f0db",
|
||||
"b": "#ff4f64"
|
||||
},
|
||||
"ultraviolet-tide": {
|
||||
"a": "#7516cf",
|
||||
"b": "#00e8d1"
|
||||
},
|
||||
"muted-dream": {
|
||||
"a": "#a6aaad",
|
||||
"b": "#44464a"
|
||||
}
|
||||
} as const;
|
||||
|
|
@ -0,0 +1,463 @@
|
|||
// Vendored from ClipLab a050f724; see PROVENANCE.md and LICENSE.
|
||||
import * as THREE from 'three'
|
||||
import { BASE_POSE, detailAt, type Character, type Detail, type FaceLayer, type Pose, type Sample, type Shape } from './model.js'
|
||||
import { clampGaze, gazeAtPoint, irisOffset, localGaze, type EyeGazes, type Gaze, type PointerLook } from './gaze.js'
|
||||
import { drawMorphBrow, drawMorphEye, drawMorphMouth, drawDrool, droolAnchor, mouthGeometry, traitWeight } from './face-morph.js'
|
||||
|
||||
export interface RenderOptions {
|
||||
width: number; height: number; displaySize?: number; background?: string | null
|
||||
cursor?: { x: number; y: number }; rotation?: { x: number; y: number; z: number }; zoom?: number; framing?: 'portrait' | 'character'; pixelRatio?: number
|
||||
pointerLook?: PointerLook; eyeGazes?: EyeGazes; reducedMotion?: boolean
|
||||
}
|
||||
/** Cursor pitch owns the vertical look direction, independent of the resting tilt. */
|
||||
export function characterRotation(character: Pick<Character, 'trueFront' | 'followRotation'>, pose: Pick<Pose, 'rotationX' | 'rotationY' | 'rotationZ'>, rotation = { x: -5, y: -12, z: -7 }, cursor: Gaze = { x: 0, y: 0 }) {
|
||||
if (character.trueFront) return { x: 0, y: 0, z: 0 }
|
||||
return {
|
||||
x: character.followRotation ? -cursor.y * 16 : rotation.x + pose.rotationX,
|
||||
y: rotation.y + pose.rotationY + (character.followRotation ? cursor.x * 28 : 0),
|
||||
z: rotation.z + pose.rotationZ
|
||||
}
|
||||
}
|
||||
/** One rounded light region; the body-following toggle activates
|
||||
* a subtle light-source shift. The shared eased cursor keeps motion smooth. */
|
||||
export function toonLightOffset(character: Pick<Character, 'followRotation'>, cursor: Gaze = { x: 0, y: 0 }, reducedMotion = false): Gaze {
|
||||
const look = !reducedMotion && character.followRotation ? clampGaze(cursor) : { x: 0, y: 0 }
|
||||
return { x: -.015 + look.x * .045, y: .015 + look.y * .045 }
|
||||
}
|
||||
/** Size adaptation is render-only; larger sizes restore the authored appearance. */
|
||||
export function faceForSize(character: Character, sample: Sample, size: number) {
|
||||
const simpleEyes = size > 16 && size <= 48
|
||||
const frontOnly = size <= 24
|
||||
const flatFill = size < 48
|
||||
return {
|
||||
character: simpleEyes || flatFill ? { ...character, ...(simpleEyes ? { iris: false } : {}), ...(flatFill ? { toon: false, shadow: false } : {}), ...(frontOnly ? { trueFront: true, lockPosition: true, followRotation: false } : {}) } : character,
|
||||
sample: simpleEyes || frontOnly ? { ...sample, pose: { ...sample.pose, faceScale: sample.pose.faceScale * (simpleEyes ? 1.3 : 1), eyeSize: sample.pose.eyeSize * (size === 24 ? 1.2 : 1), ...(frontOnly ? { squash: 1 } : {}), ...(size === 24 ? { gazeX: -4, gazeY: 0, leftX: 0, rightX: 0, leftY: 0, rightY: 0, spacing: BASE_POSE.spacing * .7, faceY: BASE_POSE.faceY } : {}) } } : sample,
|
||||
simpleEyes
|
||||
}
|
||||
}
|
||||
export const bodyHeight = (shape: Shape) => shape === 'capsule' ? 2 : 1
|
||||
export function radiusAt(shape: Shape, y: number): number {
|
||||
if (shape === 'sphere') return Math.sqrt(Math.max(0, .25 - y * y))
|
||||
if (shape === 'capsule') { const dy = Math.max(Math.abs(y) - .5, 0); return Math.sqrt(Math.max(0, .25 - dy * dy)) }
|
||||
if (y >= 0) return Math.sqrt(Math.max(0, .25 - y * y))
|
||||
if (y >= -.43) return .5
|
||||
return .43 + Math.sqrt(Math.max(0, .07 ** 2 - (y + .43) ** 2))
|
||||
}
|
||||
function geometryFor(shape: Shape, squareBottom = false): THREE.BufferGeometry {
|
||||
if (shape === 'sphere') return new THREE.SphereGeometry(.5, 80, 64)
|
||||
if (shape === 'capsule') return new THREE.CapsuleGeometry(.5, 1, 24, 80)
|
||||
const points = [new THREE.Vector2(0, -.5), new THREE.Vector2(squareBottom ? .5 : .43, -.5)]
|
||||
for (let i = 1; !squareBottom && i <= 12; i++) { const a = -Math.PI / 2 + i / 12 * Math.PI / 2; points.push(new THREE.Vector2(.43 + .07 * Math.cos(a), -.43 + .07 * Math.sin(a))) }
|
||||
points.push(new THREE.Vector2(.5, 0))
|
||||
for (let i = 1; i <= 32; i++) { const a = i / 32 * Math.PI / 2; points.push(new THREE.Vector2(.5 * Math.cos(a), .5 * Math.sin(a))) }
|
||||
return new THREE.LatheGeometry(points, 80)
|
||||
}
|
||||
const vertexShader = `
|
||||
uniform vec3 fillScale;
|
||||
uniform vec3 fillOffset;
|
||||
varying vec3 vPosition;
|
||||
void main() {
|
||||
vPosition = position * fillScale + fillOffset;
|
||||
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
|
||||
}
|
||||
`
|
||||
const fragmentShader = `
|
||||
uniform vec3 colorA;
|
||||
uniform vec3 colorB;
|
||||
uniform float gradientOn;
|
||||
uniform float toonOn;
|
||||
uniform float insetFill;
|
||||
uniform float angle;
|
||||
uniform float bodyHeight;
|
||||
varying vec3 vPosition;
|
||||
void main() {
|
||||
float t = clamp(0.5 + vPosition.y / bodyHeight * cos(angle) + vPosition.x * sin(angle), 0.0, 1.0);
|
||||
vec3 color = mix(colorA, mix(colorB, colorA, t), gradientOn);
|
||||
// The light fill uses an inset copy of the silhouette, with one crisp shade step.
|
||||
float shade = mix(0.8, 1.0, insetFill);
|
||||
gl_FragColor = vec4(color * mix(1.0, shade, toonOn), 1.0);
|
||||
#include <colorspace_fragment>
|
||||
}
|
||||
`
|
||||
function heart(ctx: CanvasRenderingContext2D, x: number, y: number, r: number) {
|
||||
ctx.beginPath(); ctx.moveTo(x, y + r)
|
||||
ctx.bezierCurveTo(x - r * 2, y - r * .1, x - r, y - r * 1.5, x, y - r * .55)
|
||||
ctx.bezierCurveTo(x + r, y - r * 1.5, x + r * 2, y - r * .1, x, y + r); ctx.closePath(); ctx.fill()
|
||||
}
|
||||
function star(ctx: CanvasRenderingContext2D, x: number, y: number, r: number, points = 5) {
|
||||
ctx.beginPath()
|
||||
for (let i = 0; i < points * 2; i++) { const a = i / (points * 2) * Math.PI * 2 - Math.PI / 2; const d = i % 2 ? r * .46 : r; const px = x + Math.cos(a) * d, py = y + Math.sin(a) * d; i ? ctx.lineTo(px, py) : ctx.moveTo(px, py) }
|
||||
ctx.closePath(); ctx.fill()
|
||||
}
|
||||
|
||||
const faceAspect = .76 / .57
|
||||
const smoothstep = (a: number, b: number, x: number) => { const t = Math.max(0, Math.min(1, (x - a) / (b - a))); return t * t * (3 - 2 * t) }
|
||||
const fract = (v: number) => ((v % 1) + 1) % 1
|
||||
export function effectEnvelope(phase: number) { const t = fract(phase); return smoothstep(0, .18, t) * (1 - smoothstep(.72, 1, t)) }
|
||||
export function eyeRadius(pose: Pose, character: Character, detail: Detail, side: number) { return (detail === 'eyes' ? 29 : 27) * pose.eyeSize * (side < 0 ? pose.leftScale : pose.rightScale) * (character.iris ? 1.2 : 1) }
|
||||
|
||||
export function projectedEye(character: Character, pose: Pose, blink: number, side: number, matrix: THREE.Matrix4, camera: THREE.Camera, layers?: FaceLayer[]) {
|
||||
const radius = eyeRadius(pose, character, 'full', side)
|
||||
const cx = 256 + side * 103 * pose.spacing + (side < 0 ? pose.leftX : pose.rightX)
|
||||
const cy = 197 - (side < 0 ? pose.leftY : pose.rightY)
|
||||
const angle = (pose.eyeTilt * side + (side < 0 ? pose.leftRotation : pose.rightRotation)) * Math.PI / 180
|
||||
const h = Math.max(.12, 1 - blink) * pose.eyeHeight * (1 - .35 * (layers ? traitWeight(layers, t => t.eye === 'soft') : pose.eye === 'soft' ? 1 : 0))
|
||||
const shell = character.elevated ? 1 + character.elevation : 1.003
|
||||
const project = (dx: number, dy: number) => {
|
||||
const tx = cx + Math.cos(angle) * dx - Math.sin(angle) * dy * h
|
||||
const ty = cy + faceAspect * (Math.sin(angle) * dx + Math.cos(angle) * dy * h)
|
||||
const x = (tx / 512 - .5) * .76 * pose.faceScale
|
||||
const y = (.5 - ty / 512) * .57 * pose.faceScale + (character.shape === 'capsule' ? -.14 : -.025) + pose.faceY
|
||||
const r = shell * radiusAt(character.shape, y / shell)
|
||||
return new THREE.Vector3(x, y, Math.sqrt(Math.max(.001, r * r - x * x))).applyMatrix4(matrix).project(camera)
|
||||
}
|
||||
const center = project(0, 0)
|
||||
// Central differences follow the tangent of the curved face at this eye.
|
||||
const right = project(radius * .1, 0).sub(project(-radius * .1, 0)).multiplyScalar(5).add(center)
|
||||
const up = project(0, -radius * .1).sub(project(0, radius * .1)).multiplyScalar(5).add(center)
|
||||
return { center, right, up }
|
||||
}
|
||||
|
||||
/** Resting highlights are local to the face, so they remain up-left when tilted. */
|
||||
function irisRestGaze(character: Pick<Character, 'iris' | 'followCursor'>, gaze: Gaze, reducedMotion = false): Gaze {
|
||||
if (!character.iris) return gaze
|
||||
if (reducedMotion) return { x: -.22, y: .22 }
|
||||
return character.followCursor ? gaze : { x: gaze.x - .22, y: gaze.y + .22 }
|
||||
}
|
||||
|
||||
export function resolveEyeGazes(character: Character, pose: Pose, blink: number, matrix: THREE.Matrix4, camera: THREE.Camera, gaze: Gaze, pointer?: PointerLook, layers?: FaceLayer[]): EyeGazes {
|
||||
const eye = (side: number) => {
|
||||
const base = localGaze({ x: pose.gazeX + gaze.x, y: pose.gazeY + gaze.y }, pose.eyeTilt * side + (side < 0 ? pose.leftRotation : pose.rightRotation))
|
||||
if (!pointer?.weight) return base
|
||||
const { center, right, up } = projectedEye(character, pose, blink, side, matrix, camera, layers)
|
||||
const look = gazeAtPoint(pointer, center, right, up), weight = Math.max(0, Math.min(1, pointer.weight))
|
||||
return { x: base.x + (look.x - base.x) * weight, y: base.y + (look.y - base.y) * weight }
|
||||
}
|
||||
return { left: eye(-1), right: eye(1) }
|
||||
}
|
||||
function drop(ctx: CanvasRenderingContext2D, x: number, y: number, r: number) {
|
||||
ctx.beginPath(); ctx.moveTo(x, y - r * 1.5); ctx.bezierCurveTo(x - r * .25, y - r * .8, x - r, y - r * .2, x - r, y + r * .35); ctx.bezierCurveTo(x - r, y + r * 1.65, x + r, y + r * 1.65, x + r, y + r * .35); ctx.bezierCurveTo(x + r, y - r * .2, x + r * .25, y - r * .8, x, y - r * 1.5); ctx.fill()
|
||||
}
|
||||
export function drawFace(ctx: CanvasRenderingContext2D, pose: Pose, character: Character, blink: number, detail: Detail, gaze: { x: number; y: number }, effects: { phase?: number; tearAmount?: number; eyeGazes?: EyeGazes; faceLayers?: FaceLayer[]; simpleEyes?: boolean } = {}) {
|
||||
ctx.clearRect(0, 0, 512, 512)
|
||||
if (detail === 'body') return
|
||||
const ink = character.eyeColor, eyeY = detail === 'eyes' ? 254 : 197, spacing = 103 * pose.spacing
|
||||
const internalGaze = character.iris && detail === 'full'
|
||||
const gx = internalGaze ? 0 : (pose.gazeX + gaze.x) * 20, gy = internalGaze ? 0 : -(pose.gazeY + gaze.y) * 15
|
||||
const layers = effects.faceLayers
|
||||
const pupilAmount = !effects.simpleEyes && detail === 'full' ? layers ? traitWeight(layers, t => t.eye === 'pupil') : pose.eye === 'pupil' ? 1 : 0 : 0
|
||||
ctx.lineCap = 'round'; ctx.lineJoin = 'round'
|
||||
for (const side of [-1, 1]) {
|
||||
const r = eyeRadius(pose, character, detail, side)
|
||||
const eyeGaze = side < 0 ? effects.eyeGazes?.left : effects.eyeGazes?.right
|
||||
const pupilEyes = !effects.simpleEyes && pose.eye === 'pupil' && detail === 'full'
|
||||
const x = 256 + side * spacing + gx * (1 - pupilAmount) + (side < 0 ? pose.leftX : pose.rightX), y = eyeY + gy * (1 - pupilAmount) - (side < 0 ? pose.leftY : pose.rightY)
|
||||
if (detail === 'full' && pose.blush > 0) {
|
||||
ctx.save(); ctx.globalAlpha = pose.blush * .8; ctx.fillStyle = '#ff647a'; ctx.beginPath(); ctx.ellipse(x + side * 20, y + 56, 31, 24 * faceAspect, 0, 0, Math.PI * 2); ctx.fill(); ctx.restore()
|
||||
}
|
||||
const localRotation = side < 0 ? pose.leftRotation : pose.rightRotation
|
||||
// Equal world units in X/Y: the face mesh is wider than it is tall.
|
||||
ctx.save(); ctx.translate(x, y); ctx.scale(1, faceAspect); ctx.rotate((pose.eyeTilt * side + localRotation) * Math.PI / 180)
|
||||
ctx.fillStyle = ink; ctx.strokeStyle = ink; ctx.lineWidth = Math.max(13, r * .43)
|
||||
const happyMouth = ['open', 'grin', 'smile', 'u-smile'].includes(pose.mouth)
|
||||
const closed = ['closed', 'arc-up', 'arc-down'].includes(pose.eye) || (pose.eye === 'wink' && side > 0) || (detail === 'eyes' && ['open', 'grin'].includes(pose.mouth) && pose.eye === 'dot')
|
||||
if (effects.simpleEyes) {
|
||||
// Keep literal round dots even at eyes-only sizes or between styled beats.
|
||||
ctx.fillStyle = '#000000'; ctx.scale(1, Math.max(.12, 1 - blink))
|
||||
ctx.beginPath(); ctx.arc(0, 0, r, 0, Math.PI * 2); ctx.fill()
|
||||
} else if (layers) {
|
||||
const look = eyeGaze ?? localGaze({ x: pose.gazeX + gaze.x, y: pose.gazeY + gaze.y }, pose.eyeTilt * side + localRotation)
|
||||
drawMorphEye(ctx, pose, layers, r, side, blink, detail, ink, character.iris, look)
|
||||
} else if (pose.faceSet === 'set-2' && pose.eye === 'wink' && side > 0) {
|
||||
ctx.beginPath(); ctx.moveTo(r * .7, -r * .65); ctx.lineTo(-r * .55, 0); ctx.lineTo(r * .7, r * .65); ctx.stroke()
|
||||
} else if (closed) {
|
||||
const up = pose.eye === 'arc-up' || (pose.eye !== 'arc-down' && happyMouth)
|
||||
ctx.beginPath(); ctx.arc(0, 0, r, up ? Math.PI : 0, up ? Math.PI * 2 : Math.PI); ctx.stroke()
|
||||
} else if (pose.eye === 'squint') {
|
||||
ctx.beginPath(); ctx.moveTo(-r * side * .7, -r * .8); ctx.lineTo(r * side * .6, 0); ctx.lineTo(-r * side * .7, r * .8); ctx.stroke()
|
||||
} else if (blink > .7) {
|
||||
ctx.beginPath(); ctx.moveTo(-r, 0); ctx.lineTo(r, 0); ctx.stroke()
|
||||
} else {
|
||||
const h = Math.max(.12, 1 - blink) * pose.eyeHeight * (pose.eye === 'soft' ? .65 : 1)
|
||||
ctx.scale(1, h)
|
||||
if (pose.cheeks) {
|
||||
// The cheek is transparent, revealing the live shaded body beneath the eye.
|
||||
ctx.beginPath(); ctx.rect(-r * 2, -r * 2, r * 4, r * 4); ctx.moveTo(r * .82, r * 1.13); ctx.arc(0, r * 1.13, r * .82, 0, Math.PI * 2); ctx.clip('evenodd')
|
||||
}
|
||||
if (pose.eye === 'star') star(ctx, 0, 0, r * 1.2)
|
||||
else if (pose.eye === 'heart') { if (pose.faceSet === 'set-2' && detail === 'full') ctx.fillStyle = '#ff3f58'; heart(ctx, 0, 0, r) }
|
||||
else if (pose.eye === 'half-lidded') { ctx.beginPath(); ctx.arc(0, 0, r, 0, Math.PI); ctx.closePath(); ctx.fill() }
|
||||
else if (pupilEyes) {
|
||||
const outer = r * 1.5
|
||||
ctx.fillStyle = '#fffef9'; ctx.beginPath(); ctx.arc(0, 0, outer, 0, Math.PI * 2); ctx.fill(); ctx.clip()
|
||||
const look = eyeGaze ?? { x: pose.gazeX + gaze.x, y: pose.gazeY + gaze.y }
|
||||
ctx.fillStyle = ink; ctx.beginPath(); ctx.arc(look.x * outer * .52, -look.y * outer * .52, outer * .4, 0, Math.PI * 2); ctx.fill()
|
||||
}
|
||||
else { ctx.beginPath(); ctx.arc(0, 0, r, 0, Math.PI * 2); ctx.fill() }
|
||||
if (internalGaze && !pupilEyes && pose.eye !== 'half-lidded') {
|
||||
// The eye path also clips star/heart shapes and intersects cheek cutouts.
|
||||
ctx.clip()
|
||||
const dot = irisOffset(r, eyeGaze ?? { x: pose.gazeX + gaze.x, y: pose.gazeY + gaze.y }, eyeGaze ? 0 : pose.eyeTilt * side + localRotation, pose.cheeks)
|
||||
ctx.fillStyle = '#ffffff'; ctx.beginPath(); ctx.arc(dot.x, dot.y, r * .28, 0, Math.PI * 2); ctx.fill()
|
||||
}
|
||||
}
|
||||
ctx.restore()
|
||||
if (detail === 'full' && layers) {
|
||||
ctx.save(); ctx.translate(x, y - r * (1.45 + .35 * pupilAmount)); ctx.scale(1, faceAspect)
|
||||
drawMorphBrow(ctx, layers, r, side, ink); ctx.restore()
|
||||
} else if (detail === 'full' && pose.brows !== 'none') {
|
||||
ctx.save(); ctx.translate(x, y - (pupilEyes ? r * 1.8 : r * 1.45)); ctx.scale(1, faceAspect); ctx.strokeStyle = ink; ctx.lineWidth = 12; ctx.beginPath()
|
||||
if (pose.brows === 'raised') { ctx.arc(0, 10, r * .85, Math.PI * 1.15, Math.PI * 1.85) }
|
||||
else if (pose.brows === 'worried') { ctx.moveTo(side * r, 2); ctx.quadraticCurveTo(-side * r * .1, 10, -side * r * .75, -16) }
|
||||
else { ctx.moveTo(side * r, -10); ctx.lineTo(-side * r * .75, 8) }
|
||||
ctx.stroke(); ctx.restore()
|
||||
}
|
||||
if (pose.tears && detail === 'full') {
|
||||
const phase = fract((effects.phase ?? .35) + (side < 0 ? .46 : 0)), alpha = effectEnvelope(phase) * (effects.tearAmount ?? 1)
|
||||
ctx.save(); ctx.translate(x + side * (r + 18), y + 23 + phase * phase * 123); ctx.scale(1, faceAspect); ctx.rotate(-side * .2); ctx.globalAlpha = alpha; ctx.fillStyle = '#f6fdff'; drop(ctx, 0, 0, 12 + phase * 3); ctx.restore()
|
||||
}
|
||||
}
|
||||
if (detail === 'eyes') return
|
||||
if (layers) {
|
||||
ctx.save(); ctx.translate(256 + gx * .25, 293 + gy * .25); ctx.scale(1, faceAspect)
|
||||
drawMorphMouth(ctx, pose, layers, ink); ctx.restore(); return
|
||||
}
|
||||
const x = 256 + gx * .25, y = (pose.mouth === 'cry' ? 329 : 293) + gy * .25
|
||||
const broad = ['open', 'grin', 'cry'].includes(pose.mouth), w = (broad ? 124 : pose.mouth === 'oh' ? 51 : 61) * pose.mouthWidth
|
||||
const line = 17 * pose.mouthStroke
|
||||
ctx.save(); ctx.translate(x, y); ctx.scale(1, faceAspect); ctx.fillStyle = ink; ctx.strokeStyle = ink; ctx.lineWidth = line; ctx.beginPath()
|
||||
if (pose.mouth === 'smile') { ctx.arc(0, -10, w, Math.PI * .18, Math.PI * .82); ctx.stroke() }
|
||||
else if (pose.mouth === 'u-smile') { ctx.arc(0, -8, w * .56, 0, Math.PI); ctx.stroke() }
|
||||
else if (pose.mouth === 'frown') { ctx.arc(0, 39, w, Math.PI * 1.2, Math.PI * 1.8); ctx.stroke() }
|
||||
else if (pose.mouth === 'line' || pose.mouth === 'sleep') { ctx.moveTo(-w * .58, 0); ctx.lineTo(w * .58, 0); ctx.stroke() }
|
||||
else if (pose.mouth === 'kiss') { ctx.moveTo(-w * .25, -21); ctx.bezierCurveTo(w * .48, -34, w * .53, -2, 0, 0); ctx.bezierCurveTo(w * .53, 2, w * .48, 34, -w * .25, 21); ctx.stroke() }
|
||||
else if (pose.mouth === 'tongue-out') {
|
||||
ctx.beginPath(); ctx.moveTo(-w, -9); ctx.quadraticCurveTo(0, -2, w, -9); ctx.stroke()
|
||||
ctx.fillStyle = '#ff526c'; ctx.beginPath(); ctx.moveTo(-w * .6, 5); ctx.lineTo(w * .6, 5); ctx.bezierCurveTo(w * .88, 85 * pose.mouthOpen + 28, -w * .88, 85 * pose.mouthOpen + 28, -w * .6, 5); ctx.fill()
|
||||
}
|
||||
else if (pose.mouth === 'wave') { ctx.moveTo(-w, 4); ctx.bezierCurveTo(-w * .35, -25, w * .35, 25, w, -4); ctx.stroke() }
|
||||
else {
|
||||
const h = (broad ? 48 : 22) + 72 * pose.mouthOpen
|
||||
if (pose.mouth === 'oh') ctx.arc(0, 14, w * (.55 + .3 * pose.mouthOpen), 0, Math.PI * 2)
|
||||
else if (pose.mouth === 'cry') { ctx.moveTo(-w, 30); ctx.bezierCurveTo(-w * 1.1, -h, w * 1.1, -h, w, 30); ctx.quadraticCurveTo(w, 44, w * .76, 38); ctx.quadraticCurveTo(0, 24, -w * .76, 38); ctx.quadraticCurveTo(-w, 44, -w, 30) }
|
||||
else { const tilt = pose.mouth === 'grin' ? 24 : 0; ctx.moveTo(-w, -11); ctx.quadraticCurveTo(0, 5, w, -11 - tilt); ctx.bezierCurveTo(w * 1.02, h, -w * 1.02, h, -w, -11) }
|
||||
ctx.closePath(); ctx.fill(); ctx.save(); ctx.clip()
|
||||
if (pose.tongue) { ctx.fillStyle = '#f37b83'; ctx.beginPath(); ctx.ellipse(10, h * .65, w * .65, h * .34, -.1, 0, Math.PI * 2); ctx.fill() }
|
||||
if (pose.teeth) { ctx.fillStyle = '#fffef8'; ctx.beginPath(); ctx.roundRect(-w * .76, -23, w * 1.52, 30, 12); ctx.fill() }
|
||||
ctx.restore()
|
||||
// Rebuild the outer path after the clipped interior details changed the canvas path.
|
||||
ctx.beginPath()
|
||||
if (pose.mouth === 'oh') ctx.arc(0, 14, w * (.55 + .3 * pose.mouthOpen), 0, Math.PI * 2)
|
||||
else if (pose.mouth === 'cry') { ctx.moveTo(-w, 30); ctx.bezierCurveTo(-w * 1.1, -h, w * 1.1, -h, w, 30); ctx.quadraticCurveTo(w, 44, w * .76, 38); ctx.quadraticCurveTo(0, 24, -w * .76, 38); ctx.quadraticCurveTo(-w, 44, -w, 30) }
|
||||
else { const tilt = pose.mouth === 'grin' ? 24 : 0; ctx.moveTo(-w, -11); ctx.quadraticCurveTo(0, 5, w, -11 - tilt); ctx.bezierCurveTo(w * 1.02, h, -w * 1.02, h, -w, -11) }
|
||||
ctx.closePath(); ctx.strokeStyle = ink; ctx.stroke()
|
||||
}
|
||||
if (pose.drool) {
|
||||
const anchor = droolAnchor(mouthGeometry(pose).outline, w)
|
||||
if (pose.mouth === 'cry') anchor.y -= 27
|
||||
drawDrool(ctx, anchor)
|
||||
}
|
||||
ctx.restore()
|
||||
}
|
||||
|
||||
export function drawProp(ctx: CanvasRenderingContext2D, prop: Pose['prop'], color: string, phase?: number) {
|
||||
ctx.clearRect(0, 0, 256, 256)
|
||||
const count = ['zzz', 'sparkle', 'heart'].includes(prop) ? 3 : 1
|
||||
for (let i = 0; i < count; i++) {
|
||||
const p = fract((phase ?? .34) + i * .29), ease = p * p
|
||||
ctx.save(); ctx.globalAlpha = phase === undefined ? 1 : effectEnvelope(p)
|
||||
ctx.translate(prop === 'zzz' ? 45 + i * 64 + ease * 36 : count === 1 ? 128 : 51 + i * 69, prop === 'zzz' ? 204 - i * 58 - ease * 54 : count === 1 ? 132 - ease * 26 : 198 - i * 60 - ease * 38)
|
||||
const scale = phase === undefined ? 1 : .62 + .38 * smoothstep(0, .38, p)
|
||||
ctx.scale(scale, scale); ctx.fillStyle = color; ctx.strokeStyle = color; ctx.lineWidth = 11; ctx.lineCap = 'round'; ctx.lineJoin = 'round'
|
||||
if (prop === 'zzz') { const r = 14 + i * 5; ctx.beginPath(); ctx.moveTo(-r, -r); ctx.lineTo(r, -r); ctx.lineTo(-r, r); ctx.lineTo(r, r); ctx.stroke() }
|
||||
else if (prop === 'sparkle') star(ctx, 0, 0, 23 + i * 4, 4)
|
||||
else if (prop === 'heart') heart(ctx, 0, 0, 17 + i * 3)
|
||||
else if (prop === 'question') { ctx.font = 'bold 144px sans-serif'; ctx.textAlign = 'center'; ctx.fillText('?', 0, 48) }
|
||||
else if (prop === 'sweat') drop(ctx, 0, 0, 38)
|
||||
else if (prop === 'crown') { ctx.beginPath(); ctx.moveTo(-78, 42); ctx.lineTo(-94, -45); ctx.lineTo(-35, -3); ctx.lineTo(0, -67); ctx.lineTo(35, -3); ctx.lineTo(94, -45); ctx.lineTo(78, 42); ctx.closePath(); ctx.fill() }
|
||||
ctx.restore()
|
||||
}
|
||||
}
|
||||
|
||||
export class CharacterRenderer {
|
||||
readonly canvas: HTMLCanvasElement | null
|
||||
readonly gl: THREE.WebGLRenderer | null
|
||||
private scene = new THREE.Scene()
|
||||
private camera = new THREE.OrthographicCamera(-1, 1, 1, -1, .1, 30)
|
||||
private root = new THREE.Group()
|
||||
private body: THREE.Mesh<THREE.BufferGeometry, THREE.ShaderMaterial>
|
||||
private lightFill: THREE.Mesh<THREE.BufferGeometry, THREE.ShaderMaterial>
|
||||
private face: THREE.Mesh<THREE.PlaneGeometry, THREE.MeshBasicMaterial>
|
||||
private faceCanvas: HTMLCanvasElement | null = null
|
||||
private faceCtx: CanvasRenderingContext2D | null = null
|
||||
private faceTexture: THREE.CanvasTexture | null = null
|
||||
private propCanvas: HTMLCanvasElement | null = null
|
||||
private propCtx: CanvasRenderingContext2D | null = null
|
||||
private propTexture: THREE.CanvasTexture | null = null
|
||||
private prop: THREE.Sprite
|
||||
private shadow: THREE.Mesh<THREE.CircleGeometry, THREE.MeshBasicMaterial>
|
||||
private shape: Shape = 'capsule'
|
||||
private squareBottom = false
|
||||
private lastFaceKey = ''
|
||||
private resolvedEyes: EyeGazes | undefined
|
||||
private lastProp = ''
|
||||
private disposed = false
|
||||
private options: RenderOptions
|
||||
private captured?: { character: Character; sample: Sample; gaze: Gaze; simpleEyes: boolean }
|
||||
constructor(canvas: HTMLCanvasElement | null, options: RenderOptions) {
|
||||
this.canvas = canvas; this.options = options
|
||||
this.gl = null
|
||||
if (canvas) {
|
||||
this.faceCanvas = document.createElement('canvas')
|
||||
this.propCanvas = document.createElement('canvas')
|
||||
this.gl = new THREE.WebGLRenderer({ canvas, antialias: true, alpha: true, preserveDrawingBuffer: true, powerPreference: 'low-power' })
|
||||
this.gl.outputColorSpace = THREE.SRGBColorSpace
|
||||
this.gl.setPixelRatio(options.pixelRatio ?? Math.min(Math.max(window.devicePixelRatio || 1, 1) * 2, 4))
|
||||
this.faceCanvas.width = this.faceCanvas.height = 1024
|
||||
this.faceCtx = this.faceCanvas.getContext('2d')!
|
||||
this.faceCtx.scale(2, 2)
|
||||
this.faceTexture = new THREE.CanvasTexture(this.faceCanvas); this.faceTexture.colorSpace = THREE.SRGBColorSpace
|
||||
this.faceTexture.generateMipmaps = true; this.faceTexture.minFilter = THREE.LinearMipmapLinearFilter
|
||||
this.propCanvas.width = this.propCanvas.height = 512; this.propCtx = this.propCanvas.getContext('2d')!; this.propCtx.scale(2, 2)
|
||||
this.propTexture = new THREE.CanvasTexture(this.propCanvas); this.propTexture.colorSpace = THREE.SRGBColorSpace
|
||||
}
|
||||
const material = new THREE.ShaderMaterial({
|
||||
uniforms: { colorA: { value: new THREE.Color() }, colorB: { value: new THREE.Color() }, gradientOn: { value: 1 }, toonOn: { value: 1 }, insetFill: { value: 0 }, fillScale: { value: new THREE.Vector3(1, 1, 1) }, fillOffset: { value: new THREE.Vector3() }, angle: { value: 0 }, bodyHeight: { value: 2 } }, vertexShader, fragmentShader
|
||||
})
|
||||
this.body = new THREE.Mesh(geometryFor('capsule'), material)
|
||||
const fillMaterial = material.clone(); fillMaterial.depthTest = false; fillMaterial.depthWrite = false; fillMaterial.uniforms.insetFill!.value = 1
|
||||
this.lightFill = new THREE.Mesh(new THREE.CapsuleGeometry(.42, .94, 24, 80), fillMaterial); this.lightFill.renderOrder = 1
|
||||
this.face = new THREE.Mesh(new THREE.PlaneGeometry(1, 1, 56, 40), new THREE.MeshBasicMaterial({ map: this.faceTexture, transparent: true, alphaTest: .008, depthWrite: false, side: THREE.FrontSide, toneMapped: false }))
|
||||
this.face.geometry.setAttribute('faceValid', new THREE.BufferAttribute(new Float32Array(this.face.geometry.attributes.position!.count).fill(1), 1))
|
||||
this.face.material.onBeforeCompile = shader => {
|
||||
shader.vertexShader = 'attribute float faceValid; varying float vFaceValid;\n' + shader.vertexShader.replace('#include <begin_vertex>', '#include <begin_vertex>\nvFaceValid = faceValid;')
|
||||
shader.fragmentShader = 'varying float vFaceValid;\n' + shader.fragmentShader.replace('void main() {', 'void main() {\nif (vFaceValid < 0.99) discard;')
|
||||
}
|
||||
this.prop = new THREE.Sprite(new THREE.SpriteMaterial({ map: this.propTexture, transparent: true, depthWrite: false, toneMapped: false }))
|
||||
this.shadow = new THREE.Mesh(new THREE.CircleGeometry(.35, 64), new THREE.MeshBasicMaterial({ color: '#809299', transparent: true, opacity: .2, depthWrite: false }))
|
||||
this.root.add(this.body, this.lightFill, this.face, this.prop); this.scene.add(this.root, this.shadow)
|
||||
this.camera.position.set(0, 0, 8); this.camera.lookAt(0, 0, 0)
|
||||
this.resize(options.width, options.height, options.displaySize)
|
||||
}
|
||||
resize(width: number, height: number, displaySize?: number) {
|
||||
this.options = { ...this.options, width: Math.max(1, width), height: Math.max(1, height), displaySize: displaySize ?? Math.min(width, height) }
|
||||
this.gl?.setSize(this.options.width, this.options.height, false)
|
||||
}
|
||||
render(character: Character, sample: Sample, options: Partial<RenderOptions> = {}, gaze = { x: 0, y: 0 }) {
|
||||
if (this.disposed) return
|
||||
this.options = { ...this.options, ...options }
|
||||
const displaySize = this.options.displaySize ?? Math.min(this.options.width, this.options.height)
|
||||
const appearance = faceForSize(character, sample, displaySize)
|
||||
character = appearance.character; sample = appearance.sample
|
||||
const squareBottom = character.shape === 'cap' && displaySize <= 24
|
||||
if (character.shape !== this.shape || squareBottom !== this.squareBottom) { this.shape = character.shape; this.squareBottom = squareBottom; this.body.geometry.dispose(); this.body.geometry = geometryFor(this.shape, squareBottom); this.lightFill.geometry.dispose(); this.lightFill.geometry = this.shape === 'capsule' ? new THREE.CapsuleGeometry(.42, .94, 24, 80) : geometryFor(this.shape) }
|
||||
const { pose } = sample
|
||||
const height = bodyHeight(this.shape)
|
||||
const detail = detailAt(displaySize)
|
||||
const u = this.body.material.uniforms
|
||||
;(u.colorA!.value as THREE.Color).set(character.color); (u.colorB!.value as THREE.Color).set(character.color2)
|
||||
u.gradientOn!.value = character.gradient ? 1 : (sample.gradientMix ?? (sample.gradientRotation !== undefined ? 1 : 0)); u.toonOn!.value = character.toon ? 1 : 0
|
||||
u.angle!.value = (character.gradientAngle + (sample.gradientRotation ?? 0)) * Math.PI / 180; u.bodyHeight!.value = height
|
||||
const rotation = characterRotation(this.options.reducedMotion ? { ...character, followRotation: false } : character, pose, this.options.rotation, this.options.cursor)
|
||||
this.root.rotation.set(rotation.x * Math.PI / 180, rotation.y * Math.PI / 180, rotation.z * Math.PI / 180, 'YXZ')
|
||||
this.root.position.y = character.lockPosition ? 0 : sample.bob * .025 * height
|
||||
const stretch = pose.squash + (character.lockPosition ? 0 : sample.breathe * .008)
|
||||
this.root.scale.set(1 / Math.sqrt(stretch), stretch, 1 / Math.sqrt(stretch))
|
||||
this.lightFill.visible = character.toon
|
||||
const fillScale = this.shape === 'capsule' ? 1 : .85
|
||||
this.lightFill.scale.setScalar(fillScale)
|
||||
this.root.updateMatrixWorld(true)
|
||||
// Shift the inset toward the pointer in the camera plane, not the tilted
|
||||
// body's axes. Bound travel by the narrowest stretch to retain a dark rim.
|
||||
const lightTarget = toonLightOffset(character, this.options.cursor, this.options.reducedMotion)
|
||||
const lightScale = Math.min(this.root.scale.x, this.root.scale.y, this.root.scale.z)
|
||||
const lightOffset = new THREE.Vector3(lightTarget.x * lightScale, lightTarget.y * lightScale, 0).applyMatrix4(new THREE.Matrix4().copy(this.root.matrixWorld).invert())
|
||||
.sub(new THREE.Vector3().applyMatrix4(new THREE.Matrix4().copy(this.root.matrixWorld).invert()))
|
||||
this.lightFill.position.copy(lightOffset)
|
||||
const fu = this.lightFill.material.uniforms
|
||||
for (const name of ['colorA', 'colorB']) (fu[name]!.value as THREE.Color).copy(u[name]!.value as THREE.Color)
|
||||
for (const name of ['gradientOn', 'toonOn', 'angle', 'bodyHeight']) fu[name]!.value = u[name]!.value
|
||||
;(fu.fillScale!.value as THREE.Vector3).setScalar(fillScale); (fu.fillOffset!.value as THREE.Vector3).copy(lightOffset)
|
||||
const zoom = displaySize <= 128 ? 1 : this.options.zoom ?? 1
|
||||
const aspect = this.options.width / this.options.height
|
||||
// Small assets use a tight body fit instead of the companion preview padding.
|
||||
const padding = this.options.framing === 'character' ? .82 : displaySize <= 128 ? .55 : .72
|
||||
let half = Math.max(height * padding, padding / aspect) / zoom
|
||||
if (displaySize === 48 || displaySize === 96) {
|
||||
// A sphere bounds every rotation, avoiding angle-dependent zoom changes.
|
||||
if (!this.body.geometry.boundingSphere) this.body.geometry.computeBoundingSphere()
|
||||
const sphere = this.body.geometry.boundingSphere!
|
||||
const radius = (sphere.radius + sphere.center.length()) * Math.max(1, this.root.scale.x, this.root.scale.y, this.root.scale.z)
|
||||
const safeHalf = (radius + this.root.position.length()) / .9
|
||||
half = Math.max(half, safeHalf, safeHalf / aspect)
|
||||
}
|
||||
this.camera.left = -half * aspect; this.camera.right = half * aspect; this.camera.top = half; this.camera.bottom = -half; this.camera.updateProjectionMatrix()
|
||||
this.camera.updateMatrixWorld(true)
|
||||
this.face.visible = detail !== 'body'
|
||||
// Pointer directions stay relative to the screen when the character rolls or turns.
|
||||
const reduced = !!this.options.reducedMotion
|
||||
const screenGaze = reduced ? { x: 0, y: 0 } : gaze
|
||||
const turnGaze = !reduced && !character.followCursor && character.followRotation && !character.trueFront ? this.options.cursor ?? { x: 0, y: 0 } : { x: 0, y: 0 }
|
||||
const local = character.iris ? new THREE.Vector3(screenGaze.x + turnGaze.x * .5, screenGaze.y + turnGaze.y * .5, 0).applyQuaternion(this.root.quaternion.clone().invert()) : screenGaze
|
||||
const faceGaze = displaySize === 24 ? { x: 0, y: 0 } : irisRestGaze(character, local, reduced)
|
||||
this.resolvedEyes = !appearance.simpleEyes && detail === 'full' && (character.iris || pose.eye === 'pupil' || sample.faceLayers?.some(l => l.traits.eye === 'pupil')) ? (!reduced ? this.options.eyeGazes : undefined) ?? resolveEyeGazes(character, reduced ? { ...pose, gazeX: 0, gazeY: 0 } : pose, sample.blink, this.root.matrixWorld, this.camera, faceGaze, !reduced && character.followCursor ? this.options.pointerLook : undefined, sample.faceLayers) : undefined
|
||||
const key = JSON.stringify([pose, sample.faceLayers, character.eyeColor, character.iris, appearance.simpleEyes, sample.blink.toFixed(3), pose.tears ? sample.effectPhase?.toFixed(2) : 0, sample.tearAmount, detail, faceGaze.x.toFixed(3), faceGaze.y.toFixed(3), this.resolvedEyes])
|
||||
if (this.faceCtx && this.faceTexture && key !== this.lastFaceKey) { drawFace(this.faceCtx, pose, character, sample.blink, detail, faceGaze, { phase: sample.effectPhase, tearAmount: sample.tearAmount, eyeGazes: this.resolvedEyes, faceLayers: sample.faceLayers, simpleEyes: appearance.simpleEyes }); this.faceTexture.needsUpdate = true; this.lastFaceKey = key }
|
||||
const vertices = this.face.geometry.attributes.position!
|
||||
const uv = this.face.geometry.attributes.uv!
|
||||
const valid = this.face.geometry.attributes.faceValid!
|
||||
const shell = character.elevated ? 1 + character.elevation : 1.003
|
||||
const scale = pose.faceScale
|
||||
const offset = (this.shape === 'capsule' ? -.14 : -.025) + pose.faceY
|
||||
for (let i = 0; i < vertices.count; i++) {
|
||||
const x = (uv.getX(i) - .5) * .76 * scale
|
||||
const y = (uv.getY(i) - .5) * .57 * scale + offset
|
||||
const r = shell * radiusAt(this.shape, y / shell)
|
||||
const z = Math.sqrt(Math.max(.001, r * r - x * x))
|
||||
vertices.setXYZ(i, x, y, z)
|
||||
valid.setX(i, x * x < r * r && Math.abs(y) < height / 2 * shell ? 1 : 0)
|
||||
}
|
||||
vertices.needsUpdate = true; valid.needsUpdate = true; this.face.geometry.computeBoundingSphere()
|
||||
this.prop.visible = detail === 'full' && pose.prop !== 'none'
|
||||
this.prop.material.depthTest = pose.prop !== 'zzz'
|
||||
this.prop.renderOrder = pose.prop === 'zzz' ? 2 : 0
|
||||
const propKey = `${pose.prop}:${sample.effectPhase?.toFixed(2) ?? 'still'}`
|
||||
if (this.propCtx && this.propTexture && propKey !== this.lastProp) { drawProp(this.propCtx, pose.prop, pose.prop === 'heart' ? '#ff768c' : pose.prop === 'sweat' ? '#b7e9ff' : '#ffd362', sample.effectPhase); this.propTexture.needsUpdate = true; this.lastProp = propKey }
|
||||
const propSize = this.shape === 'capsule' ? .42 : .32
|
||||
this.prop.scale.setScalar(propSize * (.7 + .3 * (sample.propAmount ?? 1))); this.prop.material.opacity = sample.propAmount ?? 1
|
||||
this.prop.position.set(pose.prop === 'crown' ? 0 : .56, pose.prop === 'crown' ? height / 2 + .08 : height * .29, .15)
|
||||
this.shadow.visible = character.shadow && detail === 'full'
|
||||
this.shadow.position.set(0, -height * .58, -.2)
|
||||
this.shadow.scale.set((1 - (character.lockPosition ? 0 : sample.bob) * .06) * (this.shape === 'capsule' ? 1 : .95), .12, 1)
|
||||
const bg = this.options.background
|
||||
if (bg) this.gl?.setClearColor(bg, 1); else this.gl?.setClearColor(0x000000, 0)
|
||||
this.scene.updateMatrixWorld(true)
|
||||
this.gl?.render(this.scene, this.camera)
|
||||
this.captured = { character, sample, gaze: faceGaze, simpleEyes: appearance.simpleEyes }
|
||||
}
|
||||
snapshotScene() {
|
||||
if (!this.captured) throw new Error('Render the character before taking a snapshot.')
|
||||
return { ...this.captured, options: this.options, camera: this.camera, body: this.body, lightFill: this.lightFill, face: this.face, prop: this.prop, shadow: this.shadow, eyeGazes: this.resolvedEyes }
|
||||
}
|
||||
orientation() { return this.root.quaternion.clone() }
|
||||
eyeGazes() { return this.resolvedEyes ? { left: { ...this.resolvedEyes.left }, right: { ...this.resolvedEyes.right } } : undefined }
|
||||
dispose() {
|
||||
if (this.disposed) return
|
||||
this.disposed = true
|
||||
this.body.geometry.dispose(); this.body.material.dispose(); this.lightFill.geometry.dispose(); this.lightFill.material.dispose(); this.face.geometry.dispose(); this.face.material.dispose()
|
||||
this.faceTexture?.dispose(); this.propTexture?.dispose(); this.prop.material.dispose(); this.shadow.geometry.dispose(); this.shadow.material.dispose()
|
||||
this.gl?.dispose(); this.gl?.forceContextLoss()
|
||||
}
|
||||
}
|
||||
|
||||
let thumbRenderer: CharacterRenderer | undefined
|
||||
const thumbs = new Map<string, string>()
|
||||
export function thumbnail(character: Character, pose: Pose = BASE_POSE, size = 96): string {
|
||||
const key = JSON.stringify([character, pose, size])
|
||||
const cached = thumbs.get(key); if (cached) return cached
|
||||
try {
|
||||
if (!thumbRenderer) thumbRenderer = new CharacterRenderer(document.createElement('canvas'), { width: 192, height: 192, pixelRatio: 1, displaySize: size })
|
||||
thumbRenderer.render({ ...character, shadow: false }, { pose, blink: 0, bob: 0, breathe: 0, expressionId: '', beatIndex: 0, stepIndex: 0 }, { displaySize: size, rotation: { x: -3, y: -8, z: -5 }, zoom: 1.08 })
|
||||
const url = thumbRenderer.canvas!.toDataURL('image/png'); if (thumbs.size > 250) thumbs.clear(); thumbs.set(key, url); return url
|
||||
} catch { return '' }
|
||||
}
|
||||
|
|
@ -0,0 +1,82 @@
|
|||
// ClipLab runtime adapted for Paperclip; geometry/animation model remain upstream.
|
||||
// See PROVENANCE.md and LICENSE.
|
||||
import { CharacterRenderer } from './renderer.js'
|
||||
import { sampleDefinition, animationDuration, type Definition } from './model.js'
|
||||
import { centeredGaze, easeGaze, pointerGaze } from './gaze.js'
|
||||
export interface CharacterOptions {
|
||||
animation?: string; followCursor?: boolean; followRotation?: boolean;
|
||||
trackingRegion?: HTMLElement; trackingScope?: 'region' | 'page'; displaySize?: number; onComplete?: () => void; onError?: () => void;
|
||||
}
|
||||
|
||||
/** One renderer; no frame callbacks or pointer listeners while hidden. */
|
||||
export function createCharacter(target: HTMLElement, definition: Definition, options: CharacterOptions = {}) {
|
||||
const canvas = document.createElement('canvas')
|
||||
canvas.style.cssText = 'display:block;width:100%;height:100%;touch-action:pan-y;'
|
||||
canvas.setAttribute('aria-hidden', 'true')
|
||||
const renderer = new CharacterRenderer(canvas, { width: target.clientWidth || 256, height: target.clientHeight || 256, displaySize: options.displaySize, framing: 'character' })
|
||||
target.appendChild(canvas)
|
||||
let animation = options.animation ?? 'idle', elapsed = 0, last = 0, raf = 0
|
||||
let visible = false, destroyed = false, playing = true, tracking = false
|
||||
let gaze = centeredGaze(), goal = centeredGaze()
|
||||
const region = options.trackingScope === 'page' ? target.ownerDocument.documentElement : options.trackingRegion ?? target
|
||||
const reduced = window.matchMedia('(prefers-reduced-motion: reduce)')
|
||||
const coarse = window.matchMedia('(pointer: coarse)')
|
||||
const canRun = () => !destroyed && visible && !document.hidden && !reduced.matches
|
||||
const render = () => renderer.render({ ...definition.character, followCursor: options.followCursor !== false, followRotation: options.followRotation !== false }, sampleDefinition(definition, animation, elapsed), { cursor: gaze, reducedMotion: reduced.matches }, gaze)
|
||||
const pointer = (event: PointerEvent) => { if (event.pointerType !== 'touch') goal = pointerGaze(event, target.getBoundingClientRect()) }
|
||||
const leave = () => { goal = centeredGaze() }
|
||||
function track(enabled: boolean) {
|
||||
if (tracking === enabled) return
|
||||
tracking = enabled
|
||||
if (enabled) { region.addEventListener('pointermove', pointer, { passive: true }); region.addEventListener('pointerleave', leave) }
|
||||
else { region.removeEventListener('pointermove', pointer); region.removeEventListener('pointerleave', leave); goal = centeredGaze() }
|
||||
}
|
||||
function sync() {
|
||||
cancelAnimationFrame(raf); raf = 0
|
||||
track(canRun() && !coarse.matches && (options.followCursor !== false || options.followRotation !== false))
|
||||
if (canRun() && playing) { last = performance.now(); raf = requestAnimationFrame(frame) }
|
||||
}
|
||||
function frame(now: number) {
|
||||
if (!canRun()) { sync(); return }
|
||||
const seconds = Math.min((now - last) / 1000, .1); last = now
|
||||
elapsed += seconds; gaze = easeGaze(gaze, goal, seconds, false)
|
||||
try { render() } catch { playing = false; sync(); options.onError?.(); return }
|
||||
const selected = definition.animations.find(a => a.id === animation)
|
||||
if (selected && !selected.loop && elapsed * definition.character.speed >= animationDuration(selected)) {
|
||||
animation = 'idle'; elapsed = 0; options.onComplete?.()
|
||||
}
|
||||
raf = requestAnimationFrame(frame)
|
||||
}
|
||||
function renderSafely() {
|
||||
if (destroyed) return
|
||||
try { render() } catch { playing = false; sync(); options.onError?.() }
|
||||
}
|
||||
const resize = new ResizeObserver(() => {
|
||||
try { renderer.resize(target.clientWidth || 256, target.clientHeight || 256, options.displaySize); if (visible) renderSafely() }
|
||||
catch { playing = false; sync(); options.onError?.() }
|
||||
})
|
||||
const intersection = new IntersectionObserver(entries => { visible = entries[0]?.isIntersecting ?? false; sync() })
|
||||
const contextLost = (event: Event) => { event.preventDefault(); playing = false; sync(); options.onError?.() }
|
||||
function destroy() {
|
||||
if (destroyed) return
|
||||
destroyed = true; sync(); resize.disconnect(); intersection.disconnect()
|
||||
document.removeEventListener('visibilitychange', sync); reduced.removeEventListener('change', sync); coarse.removeEventListener('change', sync)
|
||||
canvas.removeEventListener('webglcontextlost', contextLost); renderer.dispose(); canvas.remove()
|
||||
}
|
||||
try {
|
||||
resize.observe(target); intersection.observe(target)
|
||||
document.addEventListener('visibilitychange', sync); reduced.addEventListener('change', sync); coarse.addEventListener('change', sync)
|
||||
canvas.addEventListener('webglcontextlost', contextLost)
|
||||
render()
|
||||
} catch (error) { destroy(); throw error }
|
||||
|
||||
return {
|
||||
setAnimation(id: string) { if (animation === id) return; animation = id; elapsed = 0; renderSafely() },
|
||||
setDefinition(value: Definition) { definition = value; renderSafely() },
|
||||
seek(seconds: number) { elapsed = Math.max(0, seconds); renderSafely() },
|
||||
pause() { playing = false; sync() },
|
||||
play() { playing = true; sync() },
|
||||
destroy,
|
||||
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
import type { AgentAppearance, AgentAvatarSize, CharacterState } from "../agent-appearance.js";
|
||||
import { characterDefinition, characterStill } from "./definition.js";
|
||||
import { CharacterRenderer } from "./renderer.js";
|
||||
import { snapshotSvg } from "./svg-snapshot.js";
|
||||
|
||||
/** Same geometry/face projection as the live character, without DOM or WebGL. */
|
||||
export function renderAgentSvg(appearance: AgentAppearance, size: AgentAvatarSize, scale: 1 | 2, state: CharacterState = "rest", muted = false): string {
|
||||
const definition = characterDefinition(appearance, muted);
|
||||
const renderer = new CharacterRenderer(null, { width: size * scale, height: size * scale, displaySize: size });
|
||||
try {
|
||||
renderer.render({ ...definition.character, trueFront: true, lockPosition: true, followCursor: false, followRotation: false },
|
||||
characterStill(definition, state), { rotation: { x: 0, y: 0, z: 0 } });
|
||||
return snapshotSvg(renderer.snapshotScene(), "agent-");
|
||||
} finally { renderer.dispose(); }
|
||||
}
|
||||
|
|
@ -0,0 +1,109 @@
|
|||
// Vendored from ClipLab a050f724; see PROVENANCE.md and LICENSE.
|
||||
import { flatten, pathData, strokeOutline, type ProjectPoint } from './svg-path.js'
|
||||
// A small vector recorder for the Canvas 2D operations used by ClipLab's faces.
|
||||
// Paths stay editable; no bitmap or external asset is embedded in the SVG.
|
||||
type Matrix = [number, number, number, number, number, number]
|
||||
type Command = { op: string; points: number[] }
|
||||
const identity = (): Matrix => [1, 0, 0, 1, 0, 0]
|
||||
export const xml = (value: unknown) => String(value).replace(/[&<>"']/g, c => ({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' })[c]!)
|
||||
export const number = (n: number) => String(Math.round(n * 100000) / 100000)
|
||||
const point = (m: Matrix, x: number, y: number) => [m[0] * x + m[2] * y + m[4], m[1] * x + m[3] * y + m[5]]
|
||||
const inverse = (m: Matrix): Matrix => {
|
||||
const d = m[0] * m[3] - m[1] * m[2]
|
||||
return d ? [m[3] / d, -m[1] / d, -m[2] / d, m[0] / d, (m[2] * m[5] - m[3] * m[4]) / d, (m[1] * m[4] - m[0] * m[5]) / d] : identity()
|
||||
}
|
||||
export class SvgCanvas {
|
||||
fillStyle = '#000000'; strokeStyle = '#000000'; lineWidth = 1; globalAlpha = 1
|
||||
lineCap = 'butt'; lineJoin = 'miter'; font = '10px sans-serif'; textAlign = 'start'
|
||||
private matrix = identity()
|
||||
private clips: string[] = []
|
||||
private stack: ReturnType<SvgCanvas['state']>[] = []
|
||||
private path: Command[] = []
|
||||
private nodes: string[] = []
|
||||
private definitions: string[] = []
|
||||
bounds = { left: Infinity, top: Infinity, right: -Infinity, bottom: -Infinity }
|
||||
constructor(private prefix: string, private project?: ProjectPoint) {}
|
||||
private state() { return { matrix: [...this.matrix] as Matrix, clips: [...this.clips], fillStyle: this.fillStyle, strokeStyle: this.strokeStyle, lineWidth: this.lineWidth, globalAlpha: this.globalAlpha, lineCap: this.lineCap, lineJoin: this.lineJoin, font: this.font, textAlign: this.textAlign } }
|
||||
save() { this.stack.push(this.state()) }
|
||||
restore() { const state = this.stack.pop(); if (state) Object.assign(this, state) }
|
||||
clearRect() { this.nodes = []; this.definitions = []; this.path = []; this.bounds = { left: Infinity, top: Infinity, right: -Infinity, bottom: -Infinity } }
|
||||
transform(a: number, b: number, c: number, d: number, e: number, f: number) {
|
||||
const m = this.matrix
|
||||
this.matrix = [m[0] * a + m[2] * b, m[1] * a + m[3] * b, m[0] * c + m[2] * d, m[1] * c + m[3] * d, m[0] * e + m[2] * f + m[4], m[1] * e + m[3] * f + m[5]]
|
||||
}
|
||||
translate(x: number, y: number) { this.transform(1, 0, 0, 1, x, y) }
|
||||
scale(x: number, y: number) { this.transform(x, 0, 0, y, 0, 0) }
|
||||
rotate(a: number) { this.transform(Math.cos(a), Math.sin(a), -Math.sin(a), Math.cos(a), 0, 0) }
|
||||
beginPath() { this.path = [] }
|
||||
closePath() { this.path.push({ op: 'Z', points: [] }) }
|
||||
private command(op: string, coords: number[]) {
|
||||
const points = []
|
||||
for (let i = 0; i < coords.length; i += 2) points.push(...point(this.matrix, coords[i]!, coords[i + 1]!))
|
||||
this.path.push({ op, points })
|
||||
}
|
||||
moveTo(x: number, y: number) { this.command('M', [x, y]) }
|
||||
lineTo(x: number, y: number) { this.command(this.path.length ? 'L' : 'M', [x, y]) }
|
||||
bezierCurveTo(...coords: [number, number, number, number, number, number]) { this.command('C', coords) }
|
||||
quadraticCurveTo(...coords: [number, number, number, number]) { this.command('Q', coords) }
|
||||
rect(x: number, y: number, w: number, h: number) { this.moveTo(x, y); this.lineTo(x + w, y); this.lineTo(x + w, y + h); this.lineTo(x, y + h); this.closePath() }
|
||||
roundRect(x: number, y: number, w: number, h: number, radius: number) {
|
||||
const r = Math.min(radius, Math.abs(w) / 2, Math.abs(h) / 2)
|
||||
this.moveTo(x + r, y); this.lineTo(x + w - r, y); this.arc(x + w - r, y + r, r, -Math.PI / 2, 0)
|
||||
this.lineTo(x + w, y + h - r); this.arc(x + w - r, y + h - r, r, 0, Math.PI / 2)
|
||||
this.lineTo(x + r, y + h); this.arc(x + r, y + h - r, r, Math.PI / 2, Math.PI)
|
||||
this.lineTo(x, y + r); this.arc(x + r, y + r, r, Math.PI, Math.PI * 1.5); this.closePath()
|
||||
}
|
||||
arc(x: number, y: number, radius: number, start: number, end: number, ccw = false) { this.ellipse(x, y, radius, radius, 0, start, end, ccw) }
|
||||
ellipse(x: number, y: number, rx: number, ry: number, rotation: number, start: number, end: number, ccw = false) {
|
||||
const tau = Math.PI * 2
|
||||
let delta = end - start
|
||||
if (!ccw) delta = delta >= tau ? tau : ((delta % tau) + tau) % tau
|
||||
else delta = delta <= -tau ? -tau : -(((-delta % tau) + tau) % tau)
|
||||
const at = (a: number) => [x + rx * Math.cos(a) * Math.cos(rotation) - ry * Math.sin(a) * Math.sin(rotation), y + rx * Math.cos(a) * Math.sin(rotation) + ry * Math.sin(a) * Math.cos(rotation)]
|
||||
const tangent = (a: number) => [-rx * Math.sin(a) * Math.cos(rotation) - ry * Math.cos(a) * Math.sin(rotation), -rx * Math.sin(a) * Math.sin(rotation) + ry * Math.cos(a) * Math.cos(rotation)]
|
||||
const first = at(start); this.lineTo(first[0]!, first[1]!)
|
||||
const segments = Math.max(1, Math.ceil(Math.abs(delta) / (Math.PI / 2)))
|
||||
for (let i = 0; i < segments; i++) {
|
||||
const a = start + delta * i / segments, b = start + delta * (i + 1) / segments, k = 4 / 3 * Math.tan((b - a) / 4)
|
||||
const p = at(a), q = at(b), u = tangent(a), v = tangent(b)
|
||||
this.bezierCurveTo(p[0]! + k * u[0]!, p[1]! + k * u[1]!, q[0]! - k * v[0]!, q[1]! - k * v[1]!, q[0]!, q[1]!)
|
||||
}
|
||||
}
|
||||
private data(matrix = identity()) {
|
||||
return this.path.map(c => {
|
||||
const coords = []
|
||||
for (let i = 0; i < c.points.length; i += 2) coords.push(...point(matrix, c.points[i]!, c.points[i + 1]!))
|
||||
return c.op + coords.map(number).join(' ')
|
||||
}).join(' ')
|
||||
}
|
||||
private add(node: string) { this.nodes.push(this.clips.reduceRight((value, id) => `<g clip-path="url(#${id})">${value}</g>`, node)) }
|
||||
private paint(stroke: boolean, rule = 'nonzero') {
|
||||
if (this.globalAlpha <= 0) return
|
||||
const expansion = stroke ? this.lineWidth * Math.max(Math.hypot(this.matrix[0], this.matrix[1]), Math.hypot(this.matrix[2], this.matrix[3])) / 2 : 0
|
||||
for (const c of this.path) for (let i = 0; i < c.points.length; i += 2) {
|
||||
this.bounds.left = Math.min(this.bounds.left, c.points[i]! - expansion); this.bounds.right = Math.max(this.bounds.right, c.points[i]! + expansion)
|
||||
this.bounds.top = Math.min(this.bounds.top, c.points[i + 1]! - expansion); this.bounds.bottom = Math.max(this.bounds.bottom, c.points[i + 1]! + expansion)
|
||||
}
|
||||
if (this.project) {
|
||||
const inv = inverse(this.matrix)
|
||||
const commands = this.path.map(c => ({ op: c.op, points: c.points.flatMap((_, i) => i % 2 ? [] : point(inv, c.points[i]!, c.points[i + 1]!)) }))
|
||||
const contours = flatten(commands)
|
||||
const outlines = stroke ? contours.flatMap(c => strokeOutline(c, this.lineWidth)) : contours.map(c => ({ ...c, closed: true }))
|
||||
const project: ProjectPoint = p => { const [x, y] = point(this.matrix, p.x, p.y); return this.project!({ x: x!, y: y! }) }
|
||||
this.add(`<path d="${pathData(outlines, project)}" opacity="${number(this.globalAlpha)}" fill="${xml(stroke ? this.strokeStyle : this.fillStyle)}" fill-rule="${rule}"/>`)
|
||||
return
|
||||
}
|
||||
this.add(`<path d="${this.data(inverse(this.matrix))}" transform="matrix(${this.matrix.map(number).join(' ')})" opacity="${number(this.globalAlpha)}" ${stroke ? `fill="none" stroke="${xml(this.strokeStyle)}" stroke-width="${number(this.lineWidth)}" stroke-linecap="${xml(this.lineCap)}" stroke-linejoin="${xml(this.lineJoin)}"` : `fill="${xml(this.fillStyle)}" fill-rule="${rule}"`}/>`)
|
||||
}
|
||||
fill(rule = 'nonzero') { this.paint(false, rule) }
|
||||
stroke() { this.paint(true) }
|
||||
clip(rule = 'nonzero') {
|
||||
const id = `${this.prefix}-clip-${this.definitions.length}`
|
||||
this.definitions.push(`<clipPath id="${id}" clipPathUnits="userSpaceOnUse"><path d="${this.project ? pathData(flatten(this.path).map(c => ({ ...c, closed: true })), this.project) : this.data()}" clip-rule="${rule}"/></clipPath>`); this.clips.push(id)
|
||||
}
|
||||
fillText(text: string, x: number, y: number) {
|
||||
const size = Number(this.font.match(/([\d.]+)px/)?.[1] ?? 10)
|
||||
this.add(`<text x="${number(x)}" y="${number(y)}" transform="matrix(${this.matrix.map(number).join(' ')})" font-family="sans-serif" font-size="${size}" font-weight="${this.font.includes('bold') ? 'bold' : 'normal'}" text-anchor="${this.textAlign === 'center' ? 'middle' : 'start'}" fill="${xml(this.fillStyle)}" opacity="${number(this.globalAlpha)}">${xml(text)}</text>`)
|
||||
}
|
||||
markup() { return `<defs>${this.definitions.join('')}</defs>${this.nodes.join('')}` }
|
||||
}
|
||||
|
|
@ -0,0 +1,105 @@
|
|||
// Vendored from ClipLab a050f724; see PROVENANCE.md and LICENSE.
|
||||
/** Compact projected vector paths. Curves are flattened below a display pixel,
|
||||
* then simplified after projection; strokes become filled outlines before warping. */
|
||||
export type Point = { x: number; y: number }
|
||||
export type PathCommand = { op: string; points: number[] }
|
||||
export type Contour = { points: Point[]; closed: boolean }
|
||||
export type ProjectPoint = (point: Point) => Point
|
||||
const distance = (a: Point, b: Point) => Math.hypot(a.x - b.x, a.y - b.y)
|
||||
const middle = (a: Point, b: Point) => ({ x: (a.x + b.x) / 2, y: (a.y + b.y) / 2 })
|
||||
function lineDistance(p: Point, a: Point, b: Point) {
|
||||
const dx = b.x - a.x, dy = b.y - a.y, t = Math.max(0, Math.min(1, ((p.x - a.x) * dx + (p.y - a.y) * dy) / (dx * dx + dy * dy || 1)))
|
||||
return distance(p, { x: a.x + dx * t, y: a.y + dy * t })
|
||||
}
|
||||
export function simplify(points: Point[], tolerance = .06): Point[] {
|
||||
if (points.length <= 2) return points
|
||||
let max = tolerance, index = -1
|
||||
for (let i = 1; i < points.length - 1; i++) { const d = lineDistance(points[i]!, points[0]!, points.at(-1)!); if (d > max) { max = d; index = i } }
|
||||
return index < 0 ? [points[0]!, points.at(-1)!] : [...simplify(points.slice(0, index + 1), tolerance).slice(0, -1), ...simplify(points.slice(index), tolerance)]
|
||||
}
|
||||
export function flatten(commands: PathCommand[], tolerance = .08): Contour[] {
|
||||
const contours: Contour[] = []
|
||||
let current: Contour | undefined
|
||||
const append = (p: Point) => { if (!current) { current = { points: [], closed: false }; contours.push(current) } current.points.push(p) }
|
||||
function cubic(a: Point, b: Point, c: Point, d: Point, depth = 0) {
|
||||
if (depth >= 12 || Math.max(lineDistance(b, a, d), lineDistance(c, a, d)) <= tolerance) { append(d); return }
|
||||
const ab = middle(a, b), bc = middle(b, c), cd = middle(c, d), abc = middle(ab, bc), bcd = middle(bc, cd), mid = middle(abc, bcd)
|
||||
cubic(a, ab, abc, mid, depth + 1); cubic(mid, bcd, cd, d, depth + 1)
|
||||
}
|
||||
for (const command of commands) {
|
||||
const p = command.points, end = { x: p.at(-2)!, y: p.at(-1)! }, start = current?.points.at(-1) ?? end
|
||||
if (command.op === 'M') { current = undefined; append(end) }
|
||||
else if (command.op === 'Z') { if (current) current.closed = true }
|
||||
else if (command.op === 'C') cubic(start, { x: p[0]!, y: p[1]! }, { x: p[2]!, y: p[3]! }, end)
|
||||
else if (command.op === 'Q') cubic(start, { x: start.x + (p[0]! - start.x) * 2 / 3, y: start.y + (p[1]! - start.y) * 2 / 3 }, { x: end.x + (p[0]! - end.x) * 2 / 3, y: end.y + (p[1]! - end.y) * 2 / 3 }, end)
|
||||
else append(end)
|
||||
}
|
||||
return contours
|
||||
}
|
||||
function arc(center: Point, start: number, delta: number, radius: number) {
|
||||
const count = Math.max(1, Math.ceil(Math.abs(delta) / .18))
|
||||
return Array.from({ length: count + 1 }, (_, i) => ({ x: center.x + Math.cos(start + delta * i / count) * radius, y: center.y + Math.sin(start + delta * i / count) * radius }))
|
||||
}
|
||||
/** Round joins/caps, matching all line artwork in the character renderer. */
|
||||
export function strokeOutline(contour: Contour, width: number): Contour[] {
|
||||
let points = contour.points.filter((p, i, all) => !i || distance(p, all[i - 1]!) > 1e-7)
|
||||
if (points.length > 1 && distance(points[0]!, points.at(-1)!) < 1e-7) points = points.slice(0, -1)
|
||||
if (points.length < 2) return []
|
||||
const r = width / 2, len = points.length
|
||||
const direction = (a: Point, b: Point) => { const l = distance(a, b); return { x: (b.x - a.x) / l, y: (b.y - a.y) / l } }
|
||||
const side = (sign: number) => points.flatMap((p, i) => {
|
||||
const before = direction(points[(i - 1 + len) % len]!, p), after = direction(p, points[(i + 1) % len]!)
|
||||
if (!contour.closed && (!i || i === len - 1)) { const d = i ? before : after; return [{ x: p.x - d.y * r * sign, y: p.y + d.x * r * sign }] }
|
||||
const turn = Math.atan2(before.x * after.y - before.y * after.x, before.x * after.x + before.y * after.y)
|
||||
if (turn * sign < -.001) return arc(p, Math.atan2(before.x * sign, -before.y * sign), turn, r)
|
||||
const denom = Math.max(.05, 1 + before.x * after.x + before.y * after.y)
|
||||
return [{ x: p.x - (before.y + after.y) * r * sign / denom, y: p.y + (before.x + after.x) * r * sign / denom }]
|
||||
})
|
||||
const left = side(1), right = side(-1)
|
||||
if (contour.closed) return [{ points: left, closed: true }, { points: right.reverse(), closed: true }]
|
||||
const first = direction(points[0]!, points[1]!), last = direction(points[len - 2]!, points[len - 1]!)
|
||||
return [{ closed: true, points: [...left, ...arc(points.at(-1)!, Math.atan2(last.x, -last.y), -Math.PI, r), ...right.reverse(), ...arc(points[0]!, Math.atan2(-first.x, first.y), -Math.PI, r)] }]
|
||||
}
|
||||
export function pathData(contours: Contour[], project: ProjectPoint = p => p, tolerance = .06) {
|
||||
const n = (value: number) => String(Math.round(value * 1000) / 1000)
|
||||
return contours.map(contour => {
|
||||
// Subdivide straight art-space edges too: the face shell is curved.
|
||||
const points: Point[] = []
|
||||
const all = contour.closed ? [...contour.points, contour.points[0]!] : contour.points
|
||||
function segment(a: Point, b: Point, depth = 0) {
|
||||
const pa = project(a), pb = project(b), mid = middle(a, b), pm = project(mid)
|
||||
if (depth < 10 && (lineDistance(pm, pa, pb) > tolerance || distance(a, b) > 12)) { segment(a, mid, depth + 1); segment(mid, b, depth + 1) }
|
||||
else points.push(pb)
|
||||
}
|
||||
if (!all.length) return ''
|
||||
points.push(project(all[0]!)); for (let i = 1; i < all.length; i++) segment(all[i - 1]!, all[i]!)
|
||||
return simplify(points, tolerance).map((p, i) => `${i ? 'L' : 'M'}${n(p.x)} ${n(p.y)}`).join('') + (contour.closed ? 'Z' : '')
|
||||
}).join('')
|
||||
}
|
||||
/** Union a triangle/polygon tessellation by cancelling shared edges, keeping only
|
||||
* the boundary loops. The output has no interior mesh edges or per-triangle layers. */
|
||||
export function boundaryContours(polygons: Point[][]): Contour[] {
|
||||
const key = (p: Point) => `${Math.round(p.x * 1e5)},${Math.round(p.y * 1e5)}`
|
||||
const edges = new Map<string, { a: Point; b: Point; start: string; end: string }>()
|
||||
for (const poly of polygons) for (let i = 0; i < poly.length; i++) {
|
||||
const a = poly[i]!, b = poly[(i + 1) % poly.length]!, start = key(a), end = key(b)
|
||||
if (start === end) continue
|
||||
const reverse = `${end}/${start}`, forward = `${start}/${end}`
|
||||
if (edges.has(reverse)) edges.delete(reverse); else edges.set(forward, { a, b, start, end })
|
||||
}
|
||||
const starts = new Map<string, Set<string>>()
|
||||
for (const [id, e] of edges) { const set = starts.get(e.start) ?? new Set<string>(); set.add(id); starts.set(e.start, set) }
|
||||
const contours: Contour[] = []
|
||||
while (edges.size) {
|
||||
const first = edges.values().next().value!, points = [first.a]
|
||||
let next: string | undefined = `${first.start}/${first.end}`
|
||||
while (next) {
|
||||
const e = edges.get(next); if (!e) break
|
||||
edges.delete(next); starts.get(e.start)!.delete(next); points.push(e.b)
|
||||
if (e.end === first.start) break
|
||||
next = starts.get(e.end)?.values().next().value
|
||||
}
|
||||
if (points.length > 2) contours.push({ points, closed: true })
|
||||
}
|
||||
return contours
|
||||
}
|
||||
|
|
@ -0,0 +1,145 @@
|
|||
// Vendored from ClipLab a050f724; see PROVENANCE.md and LICENSE.
|
||||
import * as THREE from 'three'
|
||||
import { drawFace, drawProp, type CharacterRenderer } from './renderer.js'
|
||||
import { detailAt } from './model.js'
|
||||
import { SvgCanvas, number as n, xml } from './svg-canvas.js'
|
||||
import { boundaryContours, pathData, type Point, type ProjectPoint } from './svg-path.js'
|
||||
|
||||
type Snapshot = ReturnType<CharacterRenderer['snapshotScene']>
|
||||
type Vertex = { x: number; y: number; z: number; t: number }
|
||||
type Triangle = { points: Vertex[]; indices: number[] }
|
||||
const cross = (a: Pick<Vertex, 'x' | 'y'>, b: Pick<Vertex, 'x' | 'y'>, c: Pick<Vertex, 'x' | 'y'>) => (b.x - a.x) * (c.y - a.y) - (b.y - a.y) * (c.x - a.x)
|
||||
function hull(points: Vertex[]) {
|
||||
const sorted = [...points].sort((a, b) => a.x - b.x || a.y - b.y)
|
||||
const half = (items: Vertex[]) => { const out: Vertex[] = []; for (const p of items) { while (out.length > 1 && cross(out.at(-2)!, out.at(-1)!, p) <= 0) out.pop(); out.push(p) } return out }
|
||||
return [...half(sorted).slice(0, -1), ...half([...sorted].reverse()).slice(0, -1)]
|
||||
}
|
||||
function clip(points: Vertex[], threshold: number, above: boolean) {
|
||||
const out: Vertex[] = []
|
||||
for (let i = 0; i < points.length; i++) {
|
||||
const a = points[i]!, b = points[(i + 1) % points.length]!, insideA = above ? a.z >= threshold : a.z <= threshold, insideB = above ? b.z >= threshold : b.z <= threshold
|
||||
if (insideA) out.push(a)
|
||||
if (insideA !== insideB) {
|
||||
const f = (threshold - a.z) / (b.z - a.z)
|
||||
out.push({ x: a.x + (b.x - a.x) * f, y: a.y + (b.y - a.y) * f, z: a.z + (b.z - a.z) * f, t: a.t + (b.t - a.t) * f })
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
/** Best-fit planar gradient, weighted by visible projected area. A curved 3D
|
||||
* surface is not exactly representable by one SVG linear gradient; this keeps
|
||||
* the authored colors and a compact, editable fill at every camera angle. */
|
||||
export function fitGradient(triangles: { points: Vertex[] }[]) {
|
||||
let weight = 0, x = 0, y = 0, t = 0
|
||||
const samples: { p: Vertex; w: number }[] = []
|
||||
for (const { points } of triangles) {
|
||||
const w = Math.abs(cross(points[0]!, points[1]!, points[2]!)) / 6
|
||||
for (const p of points) { samples.push({ p, w }); weight += w; x += p.x * w; y += p.y * w; t += p.t * w }
|
||||
}
|
||||
if (!weight) return { x: 0, y: 0, t: .5, gx: 0, gy: 0 }
|
||||
x /= weight; y /= weight; t /= weight
|
||||
let xx = 0, xy = 0, yy = 0, xt = 0, yt = 0
|
||||
for (const { p, w } of samples) { const dx = p.x - x, dy = p.y - y, dt = p.t - t; xx += dx * dx * w; xy += dx * dy * w; yy += dy * dy * w; xt += dx * dt * w; yt += dy * dt * w }
|
||||
const det = xx * yy - xy * xy
|
||||
return { x, y, t, gx: det ? (xt * yy - yt * xy) / det : 0, gy: det ? (yt * xx - xt * xy) / det : 0 }
|
||||
}
|
||||
/** Warp each facial outline through the same UV mesh as the live renderer.
|
||||
* This emits the artwork once instead of repeating it in every mesh triangle. */
|
||||
export function faceProjector(face: Snapshot['face'], project: (point: THREE.Vector3) => Point): ProjectPoint {
|
||||
const { widthSegments: nx, heightSegments: ny } = face.geometry.parameters
|
||||
const position = face.geometry.getAttribute('position')
|
||||
return p => {
|
||||
const u = p.x / 512 * nx, v = p.y / 512 * ny
|
||||
const ix = Math.max(0, Math.min(nx - 1, Math.floor(u))), iy = Math.max(0, Math.min(ny - 1, Math.floor(v))), fx = u - ix, fy = v - iy
|
||||
const a = ix + iy * (nx + 1), b = a + nx + 1, c = b + 1, d = a + 1
|
||||
const indices = fx + fy <= 1 ? [a, d, b] : [c, b, d], weights = fx + fy <= 1 ? [1 - fx - fy, fx, fy] : [fx + fy - 1, 1 - fx, 1 - fy]
|
||||
const value = new THREE.Vector3()
|
||||
indices.forEach((index, i) => value.addScaledVector(new THREE.Vector3().fromBufferAttribute(position, index), weights[i]!))
|
||||
return project(value.applyMatrix4(face.matrixWorld))
|
||||
}
|
||||
}
|
||||
|
||||
export function snapshotSvg(snapshot: Snapshot, prefix = `cliplab-${crypto.randomUUID()}-`): string {
|
||||
const { character, sample, camera, options, body, lightFill, face, prop, shadow } = snapshot
|
||||
const { width, height } = options
|
||||
const defs: string[] = [], contents: string[] = []
|
||||
const project = (p: THREE.Vector3) => { const v = p.clone().project(camera); return { x: (v.x + 1) * width / 2, y: (1 - v.y) * height / 2, z: v.z, t: 0 } }
|
||||
const triangles = (mesh: THREE.Mesh, shading = false): { triangles: Triangle[]; vertices: Vertex[] } => {
|
||||
const geometry = mesh.geometry, position = geometry.getAttribute('position'), index = geometry.index
|
||||
const material = mesh.material as THREE.ShaderMaterial, uniforms = shading ? material.uniforms : undefined
|
||||
const angle = uniforms?.angle?.value ?? 0, bodyHeight = uniforms?.bodyHeight?.value ?? 1
|
||||
const scale = uniforms?.fillScale?.value as THREE.Vector3 | undefined, offset = uniforms?.fillOffset?.value as THREE.Vector3 | undefined
|
||||
const vertices: Vertex[] = []
|
||||
for (let i = 0; i < position.count; i++) {
|
||||
const local = new THREE.Vector3().fromBufferAttribute(position, i), v = project(local.clone().applyMatrix4(mesh.matrixWorld))
|
||||
if (shading) {
|
||||
local.multiply(scale ?? new THREE.Vector3(1, 1, 1)).add(offset ?? new THREE.Vector3())
|
||||
v.t = .5 + local.y / bodyHeight * Math.cos(angle) + local.x * Math.sin(angle)
|
||||
}
|
||||
vertices.push(v)
|
||||
}
|
||||
const triangles: Triangle[] = []
|
||||
for (let i = 0; i < (index?.count ?? position.count); i += 3) {
|
||||
const indices = [0, 1, 2].map(j => index ? index.getX(i + j) : i + j), points = indices.map(i => vertices[i]!)
|
||||
if (cross(points[0]!, points[1]!, points[2]!) < -1e-8) triangles.push({ points, indices })
|
||||
}
|
||||
return { triangles, vertices }
|
||||
}
|
||||
const shadedColor = (color: THREE.Color, shade: number) => '#' + color.clone().multiplyScalar(shade).getHexString()
|
||||
const outline = (vertices: Vertex[]) => pathData([{ points: hull(vertices), closed: true }])
|
||||
const surface = (mesh: THREE.Mesh<THREE.BufferGeometry, THREE.ShaderMaterial>, label: string) => {
|
||||
const data = triangles(mesh, true), u = mesh.material.uniforms
|
||||
const colorA = u.colorA!.value as THREE.Color, colorB = (u.colorB!.value as THREE.Color).clone().lerp(colorA, 1 - u.gradientOn!.value)
|
||||
const shade = u.toonOn!.value && !u.insetFill!.value ? .8 : 1
|
||||
const gradient = fitGradient(data.triangles), { x, y, t, gx, gy } = gradient, length = gx * gx + gy * gy
|
||||
let fill = shadedColor(colorA, shade)
|
||||
if (u.gradientOn!.value > 0 && !colorA.equals(colorB)) {
|
||||
if (length < 1e-18) fill = shadedColor(colorB.clone().lerp(colorA, Math.max(0, Math.min(1, t))), shade)
|
||||
else {
|
||||
const id = `${label}-gradient`
|
||||
defs.push(`<linearGradient id="${id}" gradientUnits="userSpaceOnUse" color-interpolation="linearRGB" x1="${n(x - gx * t / length)}" y1="${n(y - gy * t / length)}" x2="${n(x + gx * (1 - t) / length)}" y2="${n(y + gy * (1 - t) / length)}"><stop stop-color="${shadedColor(colorB, shade)}"/><stop offset="1" stop-color="${shadedColor(colorA, shade)}"/></linearGradient>`)
|
||||
fill = `url(#${id})`
|
||||
}
|
||||
}
|
||||
contents.push(`<g id="${label}" data-name="${label === 'body' ? 'Outer body' : 'Inner body'}"><path fill="${fill}" d="${outline(data.vertices)}"/></g>`)
|
||||
return data
|
||||
}
|
||||
if (options.background) contents.push(`<rect width="${width}" height="${height}" fill="${xml(options.background)}"/>`)
|
||||
if (shadow.visible) {
|
||||
const material = shadow.material, data = triangles(shadow)
|
||||
contents.push(`<g id="ground-shadow" data-name="Shadow"><path d="${outline(data.vertices)}" fill="#${material.color.getHexString()}" opacity="${n(material.opacity)}"/></g>`)
|
||||
}
|
||||
const bodyData = surface(body, 'body')
|
||||
if (lightFill.visible) surface(lightFill, 'candle-light')
|
||||
const transparent: { z: number; markup: string }[] = []
|
||||
if (face.visible) {
|
||||
const art = new SvgCanvas('face', faceProjector(face, project))
|
||||
drawFace(art as unknown as CanvasRenderingContext2D, sample.pose, character, sample.blink, detailAt(options.displaySize ?? Math.min(width, height)), snapshot.gaze, { phase: sample.effectPhase, tearAmount: sample.tearAmount, eyeGazes: snapshot.eyeGazes, faceLayers: sample.faceLayers, simpleEyes: snapshot.simpleEyes })
|
||||
const valid = face.geometry.getAttribute('faceValid'), data = triangles(face)
|
||||
const visible = data.triangles.filter(t => t.indices.every(i => valid.getX(i) >= .99))
|
||||
defs.push(`<clipPath id="face-visible"><path d="${pathData(boundaryContours(visible.map(t => t.points)))}"/></clipPath>`)
|
||||
if (!face.geometry.boundingSphere) face.geometry.computeBoundingSphere()
|
||||
const center = project(face.geometry.boundingSphere!.center.clone().applyMatrix4(face.matrixWorld))
|
||||
transparent.push({ z: center.z, markup: `<g clip-path="url(#face-visible)">${art.markup()}</g>` })
|
||||
}
|
||||
if (prop.visible && prop.material.opacity > 0) {
|
||||
const art = new SvgCanvas('prop'), name = sample.pose.prop
|
||||
drawProp(art as unknown as CanvasRenderingContext2D, name, name === 'heart' ? '#ff768c' : name === 'sweat' ? '#b7e9ff' : '#ffd362', sample.effectPhase)
|
||||
const center = project(prop.getWorldPosition(new THREE.Vector3())), scale = prop.getWorldScale(new THREE.Vector3())
|
||||
const w = scale.x * width / (camera.right - camera.left), h = scale.y * height / (camera.top - camera.bottom)
|
||||
const foreground = !prop.material.depthTest
|
||||
if (!foreground) {
|
||||
const occlusion = pathData(boundaryContours(bodyData.triangles.map(t => clip(t.points, center.z, false)).filter(p => p.length >= 3)))
|
||||
defs.push(`<mask id="prop-occlusion" maskUnits="userSpaceOnUse" x="0" y="0" width="${width}" height="${height}"><rect width="${width}" height="${height}" fill="white"/><path d="${occlusion}" fill="black"/></mask>`)
|
||||
}
|
||||
transparent.push({ z: foreground ? -Infinity : center.z, markup: `<g id="supporting-elements" data-name="Supporting elements"${foreground ? '' : ' mask="url(#prop-occlusion)"'} opacity="${n(prop.material.opacity)}"><g transform="translate(${n(center.x - w / 2)} ${n(center.y - h / 2)}) scale(${n(w / 256)} ${n(h / 256)})">${art.markup()}</g></g>` })
|
||||
}
|
||||
if (transparent.length) contents.push(`<g id="face" data-name="Facial expression">${transparent.sort((a, b) => b.z - a.z).map(layer => layer.markup).join('')}</g>`)
|
||||
const metadata = { format: 'cliplab-svg-snapshot', gradientProjection: 'planar-fit', character, pose: sample.pose, gradientRotation: sample.gradientRotation ?? 0, effectiveGradientAngle: character.gradientAngle + (sample.gradientRotation ?? 0), rotation: options.rotation, cursor: options.cursor }
|
||||
// Multiple downloaded characters can safely be placed inline in the same page.
|
||||
const artwork = (`<defs>${defs.join('')}</defs>${contents.join('')}`)
|
||||
.replace(/\bid="([^"]+)"/g, (_, id: string) => `id="${prefix}${id}"`)
|
||||
.replace(/url\(#([^)]+)\)/g, (_, id: string) => `url(#${prefix}${id})`)
|
||||
.replace(/href="#([^"]+)"/g, (_, id: string) => `href="#${prefix}${id}"`)
|
||||
return `<svg xmlns="http://www.w3.org/2000/svg" width="${width}" height="${height}" viewBox="0 0 ${width} ${height}" role="img"><title>${xml(character.name)} — ClipLab snapshot</title><metadata>${xml(JSON.stringify(metadata))}</metadata>${artwork}</svg>`
|
||||
}
|
||||
|
|
@ -2761,3 +2761,5 @@ export type { ExecutionContinuationEnvelope } from "./types/execution-continuati
|
|||
export type { ExecutionProjection, ExecutionReconciliation, ExecutionBlocker } from "./types/execution-projection.js";
|
||||
|
||||
export { EXECUTION_RECONCILIATION_CAUSES, requiresExecutionReconciliation } from "./types/execution-projection.js";
|
||||
|
||||
export * from "./agent-appearance.js";
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import type { AgentAppearance } from "../agent-appearance.js";
|
||||
import type {
|
||||
AgentAdapterType,
|
||||
PauseReason,
|
||||
|
|
@ -78,6 +79,8 @@ export interface Agent {
|
|||
role: AgentRole;
|
||||
title: string | null;
|
||||
icon: string | null;
|
||||
appearance?: AgentAppearance | null;
|
||||
avatarUrl?: string;
|
||||
status: AgentStatus;
|
||||
reportsTo: string | null;
|
||||
capabilities: string | null;
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import type { AgentAppearance } from "../agent-appearance.js";
|
||||
import type { AgentEnvConfig } from "./secrets.js";
|
||||
import type { RoutineVariable } from "./routine.js";
|
||||
import type { IssueCommentAuthorType, PermissionKey } from "../constants.js";
|
||||
|
|
@ -217,6 +218,7 @@ export interface CompanyPortabilityAgentManifestEntry {
|
|||
role: string;
|
||||
title: string | null;
|
||||
icon: string | null;
|
||||
appearance?: AgentAppearance | null;
|
||||
capabilities: string | null;
|
||||
reportsToSlug: string | null;
|
||||
reportsToExistingAgentId: string | null;
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import type { AgentAppearance } from "../agent-appearance.js";
|
||||
import type { BillingType, CostStatus } from "../constants.js";
|
||||
|
||||
export interface CostEvent {
|
||||
|
|
@ -47,6 +48,8 @@ export interface IssueCostSummary {
|
|||
export interface CostByAgent {
|
||||
agentId: string;
|
||||
agentName: string | null;
|
||||
agentAppearance?: AgentAppearance | null;
|
||||
avatarUrl?: string;
|
||||
agentStatus: string | null;
|
||||
costCents: number;
|
||||
inputTokens: number;
|
||||
|
|
@ -94,6 +97,8 @@ export interface CostByBiller {
|
|||
export interface CostByAgentModel {
|
||||
agentId: string;
|
||||
agentName: string | null;
|
||||
agentAppearance?: AgentAppearance | null;
|
||||
avatarUrl?: string;
|
||||
provider: string;
|
||||
biller: string;
|
||||
billingType: BillingType;
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import type { AgentAppearance } from "../agent-appearance.js";
|
||||
import type {
|
||||
SecretAccessOutcome,
|
||||
SecretBindingTargetType,
|
||||
|
|
@ -364,6 +365,8 @@ export type SecretProposalStatus =
|
|||
|
||||
/** Minimal agent reference surfaced on a proposal (proposer / binding target). */
|
||||
export interface SecretProposalAgentRef {
|
||||
appearance?: AgentAppearance | null;
|
||||
avatarUrl?: string;
|
||||
id: string;
|
||||
name: string;
|
||||
/** lucide icon slug, if the agent has one. */
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import type { AgentAppearance } from "../agent-appearance.js";
|
||||
/**
|
||||
* Work Timeline (Gantt) types — shared between the aggregation service
|
||||
* (`server/src/services/work-timeline.ts`) and the UI page
|
||||
|
|
@ -15,6 +16,8 @@ export interface WorkTimelineActor {
|
|||
type: TimelineActorType;
|
||||
name: string;
|
||||
avatar?: string | null;
|
||||
appearance?: AgentAppearance | null;
|
||||
avatarUrl?: string;
|
||||
}
|
||||
|
||||
export interface WorkTimelineSpan {
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import { agentAppearanceSchema } from "../agent-appearance.js";
|
||||
import { z } from "zod";
|
||||
import {
|
||||
AGENT_ICON_NAMES,
|
||||
|
|
@ -78,6 +79,7 @@ export const createAgentSchema = z.object({
|
|||
role: z.enum(AGENT_ROLES).optional().default("general"),
|
||||
title: z.string().optional().nullable(),
|
||||
icon: z.enum(AGENT_ICON_NAMES).optional().nullable(),
|
||||
appearance: agentAppearanceSchema.optional(),
|
||||
reportsTo: z.string().guid().optional().nullable(),
|
||||
capabilities: z.string().optional().nullable(),
|
||||
desiredSkills: z.array(agentDesiredSkillSelectionSchema).optional(),
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import { agentAppearanceSchema } from "../agent-appearance.js";
|
||||
import { z } from "zod";
|
||||
import { PERMISSION_KEYS } from "../constants.js";
|
||||
import {
|
||||
|
|
@ -77,6 +78,7 @@ export const portabilityAgentManifestEntrySchema = z.object({
|
|||
role: z.string().min(1),
|
||||
title: z.string().nullable(),
|
||||
icon: z.string().nullable(),
|
||||
appearance: agentAppearanceSchema.nullable().optional(),
|
||||
capabilities: z.string().nullable(),
|
||||
reportsToSlug: z.string().min(1).nullable(),
|
||||
adapterType: z.string().min(1),
|
||||
|
|
|
|||
|
|
@ -0,0 +1,13 @@
|
|||
import fs from "node:fs";
|
||||
import { fileURLToPath } from "node:url";
|
||||
const css = fs.readFileSync(new URL("../ui/src/index.css", import.meta.url), "utf8");
|
||||
const values = {};
|
||||
for (const [, name, channel, color] of css.matchAll(/--agent-cap-v1-([a-z-]+)-(a|b):\s*(#[0-9a-f]{6});/g)) {
|
||||
(values[name] ??= {})[channel] = color;
|
||||
}
|
||||
if (Object.keys(values).length !== 18 || Object.values(values).some(v => !v.a || !v.b)) throw new Error("Incomplete cap-v1 palette tokens");
|
||||
const output = "// Generated from ui/src/index.css by scripts/sync-agent-palette-tokens.mjs.\nexport const CAP_V1_COLORS = " + JSON.stringify(values, null, 2) + " as const;\n";
|
||||
const target = new URL("../packages/shared/src/cliplab/palette-tokens.ts", import.meta.url);
|
||||
if (process.argv.includes("--check")) {
|
||||
if (fs.readFileSync(target, "utf8") !== output) throw new Error("Run node scripts/sync-agent-palette-tokens.mjs");
|
||||
} else fs.writeFileSync(target, output);
|
||||
|
|
@ -0,0 +1,102 @@
|
|||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { mkdtemp, rm } from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import { Readable } from "node:stream";
|
||||
import { S3Client, PutObjectCommand, HeadObjectCommand, GetObjectCommand, DeleteObjectCommand } from "@aws-sdk/client-s3";
|
||||
import { createS3StorageProvider } from "../storage/s3-provider.js";
|
||||
import path from "node:path";
|
||||
import express from "express";
|
||||
import type { Server } from "node:http";
|
||||
import sharp from "sharp";
|
||||
import { appearanceForPalette } from "@paperclipai/shared";
|
||||
import { createLocalDiskStorageProvider } from "../storage/local-disk-provider.js";
|
||||
import { createAgentAvatarService, avatarCacheKey, type AgentAvatarRequest } from "../services/agent-avatars.js";
|
||||
import { createAgentAvatarPool } from "../services/agent-avatar-pool.js";
|
||||
import { agentAvatarRoutes } from "../routes/agent-avatars.js";
|
||||
|
||||
const request: AgentAvatarRequest = { appearance: appearanceForPalette("arctic-blue"), size: 24, scale: 2, pose: "rest", muted: false };
|
||||
const cleanups: Array<() => Promise<unknown>> = [];
|
||||
afterEach(async () => { await Promise.all(cleanups.splice(0).map(fn => fn())); });
|
||||
async function storage() {
|
||||
const dir = await mkdtemp(path.join(os.tmpdir(), "agent-avatar-test-"));
|
||||
cleanups.push(() => rm(dir, { recursive: true, force: true }));
|
||||
return createLocalDiskStorageProvider(dir);
|
||||
}
|
||||
async function serve(service: ReturnType<typeof createAgentAvatarService>) {
|
||||
const app = express(); app.use("/api", agentAvatarRoutes(service));
|
||||
const server = await new Promise<Server>(resolve => { const running = app.listen(0, "127.0.0.1", () => resolve(running)); });
|
||||
cleanups.push(() => new Promise<void>((resolve, reject) => server.close(error => error ? reject(error) : resolve())));
|
||||
return `http://127.0.0.1:${(server.address() as { port: number }).port}/api/agent-avatars/cap-v1/arctic-blue/rest.png`;
|
||||
}
|
||||
describe("on-demand agent avatars", () => {
|
||||
it("coalesces cold requests, survives restarts and regenerates deleted cache entries", async () => {
|
||||
const provider = await storage();
|
||||
const render = vi.fn(async () => Buffer.from("png-bytes"));
|
||||
const service = createAgentAvatarService(provider, render);
|
||||
const results = await Promise.all(Array.from({ length: 10 }, () => service.get(request)));
|
||||
expect(render).toHaveBeenCalledTimes(1);
|
||||
expect(new Set(results.map(result => result.etag)).size).toBe(1);
|
||||
await Promise.all(results.map(async result => { for await (const _ of result.stream) { /* consume */ } }));
|
||||
(await createAgentAvatarService(provider, render).get(request)).stream.destroy();
|
||||
expect(render).toHaveBeenCalledTimes(1);
|
||||
await provider.deleteObject({ objectKey: avatarCacheKey(request) });
|
||||
(await service.get(request)).stream.destroy();
|
||||
expect(render).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
it("uses the configured S3 prefix and reuses bytes across service instances", async () => {
|
||||
const objects = new Map<string, Buffer>();
|
||||
const send = vi.spyOn(S3Client.prototype, "send").mockImplementation(async (command: any) => {
|
||||
const key = command.input.Key as string;
|
||||
expect(command.input.Bucket).toBe("avatar-test");
|
||||
expect(key).toMatch(/^paperclip\/generated-agent-avatars\/cap-v1\//);
|
||||
if (command instanceof PutObjectCommand) { objects.set(key, Buffer.from(command.input.Body as Uint8Array)); return {}; }
|
||||
if (command instanceof DeleteObjectCommand) { objects.delete(key); return {}; }
|
||||
const bytes = objects.get(key);
|
||||
if (!bytes) throw Object.assign(new Error("missing"), { name: "NoSuchKey" });
|
||||
if (command instanceof HeadObjectCommand) return { ContentLength: bytes.length };
|
||||
if (command instanceof GetObjectCommand) return { Body: Readable.from(bytes), ContentLength: bytes.length };
|
||||
throw new Error("Unexpected S3 command");
|
||||
});
|
||||
try {
|
||||
const provider = createS3StorageProvider({ bucket: "avatar-test", region: "us-east-1", prefix: "paperclip" });
|
||||
const render = vi.fn(async () => Buffer.from("s3-avatar"));
|
||||
const first = await createAgentAvatarService(provider, render).get(request);
|
||||
const second = await createAgentAvatarService(provider, render).get(request);
|
||||
expect(second.etag).toEqual(first.etag);
|
||||
const firstBytes: Buffer[] = [], secondBytes: Buffer[] = [];
|
||||
for await (const chunk of first.stream) firstBytes.push(Buffer.from(chunk));
|
||||
for await (const chunk of second.stream) secondBytes.push(Buffer.from(chunk));
|
||||
expect(Buffer.concat(secondBytes)).toEqual(Buffer.concat(firstBytes));
|
||||
expect(render).toHaveBeenCalledTimes(1);
|
||||
await provider.deleteObject({ objectKey: avatarCacheKey(request) });
|
||||
(await createAgentAvatarService(provider, render).get(request)).stream.destroy();
|
||||
expect(render).toHaveBeenCalledTimes(2);
|
||||
} finally { send.mockRestore(); }
|
||||
});
|
||||
it("renders a PNG in an isolated worker without DOM or WebGL", async () => {
|
||||
const pool = createAgentAvatarPool(1); cleanups.push(() => pool.close());
|
||||
const png = await pool.render(request);
|
||||
expect(await sharp(png).metadata()).toMatchObject({ width: 48, height: 48, format: "png", hasAlpha: true });
|
||||
}, 20_000);
|
||||
it("serves public images with content ETags and validates the finite request space", async () => {
|
||||
const render = vi.fn(async () => Buffer.from("png-bytes"));
|
||||
const url = await serve(createAgentAvatarService(await storage(), render));
|
||||
const first = await fetch(url + "?size=24&scale=2");
|
||||
expect(first.status).toBe(200);
|
||||
expect(first.headers.get("content-type")).toContain("image/png");
|
||||
expect(first.headers.get("cache-control")).toContain("immutable");
|
||||
await first.arrayBuffer();
|
||||
const second = await fetch(url + "?size=24&scale=2", { headers: { "If-None-Match": first.headers.get("etag")! } });
|
||||
expect(second.status).toBe(304);
|
||||
for (const suffix of ["?size=99999", "?scale=3", "?color=red", "?size=24&size=32"]) {
|
||||
const invalid = await fetch(url + suffix); expect(invalid.status).toBe(400); await invalid.text();
|
||||
}
|
||||
expect(render).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
it("does not cache rendering failures and permits retry", async () => {
|
||||
const render = vi.fn().mockRejectedValueOnce(new Error("unavailable")).mockResolvedValue(Buffer.from("png"));
|
||||
const url = await serve(createAgentAvatarService(await storage(), render));
|
||||
const first = await fetch(url); expect(first.status).toBe(503); expect(first.headers.get("cache-control")).toBe("no-store"); await first.text();
|
||||
const retry = await fetch(url); expect(retry.status).toBe(200); await retry.arrayBuffer();
|
||||
});
|
||||
});
|
||||
|
|
@ -136,6 +136,8 @@ describeEmbeddedPostgres("agent hire idempotency within a run", () => {
|
|||
expect(second.status, JSON.stringify(second.body)).toBe(200);
|
||||
expect(second.body.idempotent).toBe(true);
|
||||
expect(second.body.agent?.id).toBe(createdId);
|
||||
expect(first.body.agent?.appearance).toMatchObject({ schemaVersion: 1, characterVersion: "cap-v1" });
|
||||
expect(second.body.agent?.appearance).toEqual(first.body.agent?.appearance);
|
||||
// The retry must not have auto-renamed a duplicate to "Sam 2".
|
||||
expect(second.body.agent?.name).toBe("Sam");
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,53 @@
|
|||
import { randomUUID } from "node:crypto";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import { agents, companies, createDb } from "@paperclipai/db";
|
||||
import { agentAppearanceSchema, appearanceForPalette, legacyAgentAppearance } from "@paperclipai/shared";
|
||||
import { startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
|
||||
import { agentService } from "../services/agents.js";
|
||||
|
||||
describe("persisted agent personas", () => {
|
||||
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
|
||||
let db: ReturnType<typeof createDb>;
|
||||
let companyId: string;
|
||||
beforeAll(async () => {
|
||||
database = await startEmbeddedPostgresTestDatabase("agent-persona-persistence-");
|
||||
db = createDb(database.connectionString);
|
||||
companyId = randomUUID();
|
||||
await db.insert(companies).values({ id: companyId, name: "Persona test", issuePrefix: "CAP" });
|
||||
}, 30_000);
|
||||
afterAll(async () => { await database?.cleanup(); });
|
||||
it("randomizes once, accepts a saved draft and preserves identity across edits and revision rollback", async () => {
|
||||
const service = agentService(db);
|
||||
const random = await service.create(companyId, { name: "Random", role: "engineer", adapterType: "process" });
|
||||
expect(agentAppearanceSchema.safeParse(random.appearance).success).toBe(true);
|
||||
const appearance = appearanceForPalette("arctic-blue");
|
||||
const original = await service.create(companyId, { name: "Draft", role: "engineer", adapterType: "process", appearance });
|
||||
expect(original.appearance).toEqual(appearance);
|
||||
const updated = await service.update(original.id, { name: "Renamed" }, { recordRevision: { source: "test" } });
|
||||
expect(updated?.appearance).toEqual(appearance);
|
||||
const [revision] = await service.listConfigRevisions(original.id);
|
||||
await service.update(original.id, { name: "Changed again", status: "paused" });
|
||||
const restored = await service.rollbackConfigRevision(original.id, revision.id, {});
|
||||
expect(restored?.appearance).toEqual(appearance);
|
||||
expect((await agentService(db).getById(original.id))?.avatarUrl).toContain("/arctic-blue/rest.png?size=512");
|
||||
const [row] = await db.select().from(agents).where(eq(agents.id, original.id));
|
||||
expect(row.appearance).toEqual(appearance);
|
||||
});
|
||||
it("backfills legacy IDs with exactly the same persisted identity as the runtime fallback", async () => {
|
||||
const ids = Array.from({ length: 20 }, () => randomUUID());
|
||||
await db.insert(agents).values(ids.map((id, i) => ({ id, companyId, name: `Legacy ${i}`, role: "engineer", appearance: null, icon: "bot" })));
|
||||
const migration = await readFile(new URL("../../../packages/db/src/migrations/0272_marvelous_madame_web.sql", import.meta.url), "utf8");
|
||||
// Replaying the entire migration must preserve saved choices and tolerate
|
||||
// an already-created column from an earlier worktree migration number.
|
||||
for (let replay = 0; replay < 2; replay++) {
|
||||
for (const statement of migration.split("--> statement-breakpoint")) await db.execute(sql.raw(statement));
|
||||
}
|
||||
for (const id of ids) {
|
||||
const [row] = await db.select().from(agents).where(eq(agents.id, id));
|
||||
expect(row.appearance).toEqual(legacyAgentAppearance(id));
|
||||
expect(row.icon).toBe("bot");
|
||||
}
|
||||
});
|
||||
});
|
||||
|
|
@ -99,6 +99,7 @@ describeEmbeddedPostgres("pending approval agent config integrity", () => {
|
|||
budgetMonthlyCents: 1234,
|
||||
metadata: { source: "hire-form" },
|
||||
agentId: pending.id,
|
||||
appearance: pending.appearance,
|
||||
},
|
||||
decisionNote: null,
|
||||
decidedByUserId: null,
|
||||
|
|
@ -143,6 +144,7 @@ describeEmbeddedPostgres("pending approval agent config integrity", () => {
|
|||
|
||||
await expect(agentSvc.getById(pending.id)).resolves.toMatchObject({
|
||||
status: "idle",
|
||||
appearance: pending.appearance,
|
||||
name: "Pending Coder",
|
||||
role: "engineer",
|
||||
title: "Software Engineer",
|
||||
|
|
|
|||
|
|
@ -1727,6 +1727,22 @@ describe("company portability", () => {
|
|||
expect(exported.warnings.filter((warning) => warning.includes("could not be exported portably"))).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("round-trips the saved visual persona through the portable bundle", async () => {
|
||||
const appearance = { schemaVersion: 1, characterVersion: "cap-v1", paletteId: "arctic-blue" };
|
||||
const source = await agentSvc.list();
|
||||
agentSvc.list.mockResolvedValue(source.map((agent: Record<string, unknown>) => ({ ...agent, appearance })));
|
||||
const portability = companyPortabilityService({} as any);
|
||||
const include = { company: true, agents: true, projects: false, issues: false, skills: false };
|
||||
const exported = await portability.exportBundle("company-1", { include });
|
||||
expect(asTextFile(exported.files[".paperclip.yaml"])).toContain("arctic-blue");
|
||||
agentSvc.list.mockResolvedValue([]);
|
||||
agentSvc.create.mockImplementation(async (_companyId: string, input: Record<string, unknown>) => ({ ...input, id: `imported-${input.name}` }));
|
||||
await portability.importBundle({ source: { type: "inline", files: exported.files, rootPath: exported.rootPath }, include,
|
||||
target: { mode: "new_company", newCompanyName: "Imported personas" }, collisionStrategy: "rename", agents: "all" }, "user-1");
|
||||
expect(agentSvc.create).toHaveBeenCalled();
|
||||
for (const [, input] of agentSvc.create.mock.calls) expect(input).toMatchObject({ appearance });
|
||||
});
|
||||
|
||||
it("reads env inputs back from .paperclip.yaml during preview import", async () => {
|
||||
const portability = companyPortabilityService({} as any);
|
||||
|
||||
|
|
|
|||
|
|
@ -19,6 +19,7 @@ const apiPrefixes: Record<string, string> = {
|
|||
"activity.ts": "/api",
|
||||
"adapters.ts": "/api",
|
||||
"agents.ts": "/api",
|
||||
"agent-avatars.ts": "/api",
|
||||
"attention.ts": "/api",
|
||||
"approvals.ts": "/api",
|
||||
"assets.ts": "/api",
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import { agentAvatarRoutes } from "./routes/agent-avatars.js";
|
||||
import { toolActionDeliveryService } from "./services/tool-action-delivery.js";
|
||||
import express, { Router, type Request as ExpressRequest } from "express";
|
||||
import {
|
||||
|
|
@ -613,6 +614,7 @@ export async function createApp(
|
|||
|
||||
// Mount API routes
|
||||
const api = Router();
|
||||
api.use(agentAvatarRoutes());
|
||||
api.use(boardMutationGuard());
|
||||
api.use(
|
||||
"/health",
|
||||
|
|
|
|||
|
|
@ -0,0 +1,52 @@
|
|||
import { pipeline } from "node:stream/promises";
|
||||
import { Router } from "express";
|
||||
import { z } from "zod";
|
||||
import { AGENT_PALETTE_IDS, AGENT_AVATAR_SIZES, CHARACTER_STATES, appearanceForPalette, type AgentAvatarSize } from "@paperclipai/shared";
|
||||
import { createAgentAvatarService } from "../services/agent-avatars.js";
|
||||
import { createStorageProviderFromConfig } from "../storage/provider-registry.js";
|
||||
import { loadConfig } from "../config.js";
|
||||
import { logger } from "../middleware/logger.js";
|
||||
|
||||
const requestSchema = z.object({
|
||||
version: z.literal("cap-v1"),
|
||||
palette: z.enum([...AGENT_PALETTE_IDS, "muted-dream"]),
|
||||
pose: z.enum(CHARACTER_STATES),
|
||||
size: z.string().regex(/^\d+$/).default("512").transform(Number).refine(n => (AGENT_AVATAR_SIZES as readonly number[]).includes(n)),
|
||||
scale: z.enum(["1", "2"]).default("1"),
|
||||
}).strict();
|
||||
|
||||
/** Public preset artwork only. No agent lookup or tenant data is exposed. */
|
||||
export function agentAvatarRoutes(injected?: ReturnType<typeof createAgentAvatarService>) {
|
||||
const router = Router();
|
||||
let service = injected;
|
||||
router.get("/agent-avatars/:version/:palette/:file", async (req, res) => {
|
||||
const file = String(req.params.file);
|
||||
const parsed = requestSchema.safeParse({ ...req.query, version: req.params.version, palette: req.params.palette, pose: file.endsWith(".png") ? file.slice(0, -4) : file });
|
||||
if (!file.endsWith(".png") || !parsed.success || Object.keys(req.query).some(key => key !== "size" && key !== "scale")) {
|
||||
res.setHeader("Cache-Control", "no-store");
|
||||
res.status(400).json({ error: "Unsupported avatar version, palette, pose, size, or scale" }); return;
|
||||
}
|
||||
const { palette, pose, size, scale } = parsed.data;
|
||||
try {
|
||||
service ??= createAgentAvatarService(createStorageProviderFromConfig(loadConfig()));
|
||||
const { stream, byteSize, etag } = await service.get({ appearance: appearanceForPalette(palette === "muted-dream" ? AGENT_PALETTE_IDS[0] : palette), muted: palette === "muted-dream", pose, size: size as AgentAvatarSize, scale: Number(scale) as 1 | 2 });
|
||||
res.setHeader("Cache-Control", "public, max-age=31536000, immutable");
|
||||
res.setHeader("ETag", etag);
|
||||
res.setHeader("X-Content-Type-Options", "nosniff");
|
||||
res.type("png");
|
||||
const validators = req.get("if-none-match")?.split(",").map(value => value.trim().replace(/^W\//, ""));
|
||||
if (validators?.some(value => value === "*" || value === etag)) { stream.destroy(); res.status(304).end(); return; }
|
||||
res.setHeader("Content-Length", byteSize);
|
||||
await pipeline(stream, res);
|
||||
} catch (error) {
|
||||
logger.warn({ err: error }, "Could not render agent avatar");
|
||||
if (res.headersSent || res.destroyed) { res.destroy(); return; }
|
||||
res.removeHeader("Content-Length");
|
||||
res.removeHeader("ETag");
|
||||
res.setHeader("Cache-Control", "no-store");
|
||||
res.setHeader("Retry-After", "5");
|
||||
res.status(503).json({ error: "Avatar temporarily unavailable" });
|
||||
}
|
||||
});
|
||||
return router;
|
||||
}
|
||||
|
|
@ -1,3 +1,4 @@
|
|||
import { resolveAgentAppearance, agentAvatarUrl } from "@paperclipai/shared";
|
||||
import { paperclipRunnerTransitionConfig, normalizeLegacyRunnerProvider, isPaperclipRunnerProvider } from "@paperclipai/adapter-utils";
|
||||
import { executionProjectionForRun, executionProjectionsForRuns } from "../services/execution-projection.js";
|
||||
import { Router, type NextFunction, type Request, type Response } from "express";
|
||||
|
|
@ -3730,6 +3731,7 @@ export function agentRoutes(
|
|||
id: agentsTable.id,
|
||||
companyId: agentsTable.companyId,
|
||||
agentName: agentsTable.name,
|
||||
agentAppearance: agentsTable.appearance,
|
||||
role: agentsTable.role,
|
||||
title: agentsTable.title,
|
||||
status: agentsTable.status,
|
||||
|
|
@ -4305,6 +4307,7 @@ export function agentRoutes(
|
|||
role: normalizedHireInput.role,
|
||||
title: normalizedHireInput.title ?? null,
|
||||
icon: normalizedHireInput.icon ?? null,
|
||||
appearance: agent.appearance,
|
||||
reportsTo: normalizedHireInput.reportsTo ?? null,
|
||||
capabilities: normalizedHireInput.capabilities ?? null,
|
||||
adapterType: requestedAdapterType,
|
||||
|
|
@ -6278,6 +6281,7 @@ export function agentRoutes(
|
|||
createdAt: heartbeatRuns.createdAt,
|
||||
agentId: heartbeatRuns.agentId,
|
||||
agentName: agentsTable.name,
|
||||
agentAppearance: agentsTable.appearance,
|
||||
adapterType: agentsTable.adapterType,
|
||||
logBytes: heartbeatRuns.logBytes,
|
||||
livenessState: heartbeatRuns.livenessState,
|
||||
|
|
@ -6328,6 +6332,8 @@ export function agentRoutes(
|
|||
const projections = await executionProjectionsForRuns(db, companyId, rows.map(run => run.id));
|
||||
res.json(await Promise.all(rows.map(async (run) => runRedactions.redactForRun(companyId, run.id, {
|
||||
...heartbeat.decorateActiveRunStatus(run),
|
||||
agentAppearance: resolveAgentAppearance(run.agentAppearance, run.agentId),
|
||||
avatarUrl: agentAvatarUrl(resolveAgentAppearance(run.agentAppearance, run.agentId), 512),
|
||||
execution: projections.get(run.id) ?? null,
|
||||
outputSilence: await heartbeat.buildRunOutputSilence(run),
|
||||
}))));
|
||||
|
|
@ -6337,6 +6343,8 @@ export function agentRoutes(
|
|||
const projections = await executionProjectionsForRuns(db, companyId, liveRuns.map(run => run.id));
|
||||
res.json(await Promise.all(liveRuns.map(async (run) => runRedactions.redactForRun(companyId, run.id, {
|
||||
...heartbeat.decorateActiveRunStatus(run),
|
||||
agentAppearance: resolveAgentAppearance(run.agentAppearance, run.agentId),
|
||||
avatarUrl: agentAvatarUrl(resolveAgentAppearance(run.agentAppearance, run.agentId), 512),
|
||||
execution: projections.get(run.id) ?? null,
|
||||
outputSilence: await heartbeat.buildRunOutputSilence(run),
|
||||
}))));
|
||||
|
|
@ -6907,6 +6915,7 @@ export function agentRoutes(
|
|||
createdAt: heartbeatRuns.createdAt,
|
||||
agentId: heartbeatRuns.agentId,
|
||||
agentName: agentsTable.name,
|
||||
agentAppearance: agentsTable.appearance,
|
||||
adapterType: agentsTable.adapterType,
|
||||
logBytes: heartbeatRuns.logBytes,
|
||||
livenessState: heartbeatRuns.livenessState,
|
||||
|
|
@ -6934,6 +6943,8 @@ export function agentRoutes(
|
|||
const projections = await executionProjectionsForRuns(db, issue.companyId, liveRuns.map(run => run.id));
|
||||
res.json(await Promise.all(liveRuns.map(async (run) => ({
|
||||
...heartbeat.decorateActiveRunStatus(run, { companyId: issue.companyId, issueId: issue.id }),
|
||||
agentAppearance: resolveAgentAppearance(run.agentAppearance, run.agentId),
|
||||
avatarUrl: agentAvatarUrl(resolveAgentAppearance(run.agentAppearance, run.agentId), 512),
|
||||
execution: projections.get(run.id) ?? null,
|
||||
outputSilence: await heartbeat.buildRunOutputSilence({ ...run, companyId: issue.companyId }),
|
||||
}))));
|
||||
|
|
@ -6996,6 +7007,8 @@ export function agentRoutes(
|
|||
execution: await executionProjectionForRun(db, issue.companyId, run.id),
|
||||
agentId: agent.id,
|
||||
agentName: agent.name,
|
||||
agentAppearance: agent.appearance,
|
||||
avatarUrl: agent.avatarUrl,
|
||||
adapterType: agent.adapterType,
|
||||
outputSilence: await heartbeat.buildRunOutputSilence({ ...run, companyId: issue.companyId }),
|
||||
});
|
||||
|
|
|
|||
|
|
@ -7,6 +7,10 @@ import { Router } from "express";
|
|||
import { z } from "zod";
|
||||
import {
|
||||
// Agent
|
||||
AGENT_PALETTE_IDS,
|
||||
AGENT_AVATAR_SIZES,
|
||||
CHARACTER_STATES,
|
||||
agentAppearanceSchema,
|
||||
createAgentSchema,
|
||||
createAgentHireSchema,
|
||||
updateAgentSchema,
|
||||
|
|
@ -1237,6 +1241,7 @@ const RUNTIME_TOOLS_OPERATIONS = new Set([
|
|||
]);
|
||||
|
||||
const PUBLIC_OPERATIONS = new Set([
|
||||
"GET /api/agent-avatars/{version}/{palette}/{file}",
|
||||
"GET /api/health",
|
||||
"GET /api/openapi.json",
|
||||
"GET /api/board-claim/{token}",
|
||||
|
|
@ -2517,6 +2522,32 @@ for (const route of [
|
|||
|
||||
// ─── Agents ──────────────────────────────────────────────────────────────────
|
||||
|
||||
registry.register("AgentAppearance", agentAppearanceSchema);
|
||||
registry.registerPath({
|
||||
method: "get",
|
||||
path: "/api/agent-avatars/{version}/{palette}/{file}",
|
||||
tags: ["agents"],
|
||||
summary: "Render or retrieve a public preset agent portrait",
|
||||
description: "On-demand PNG artwork; no agent or company lookup. Logical size determines face detail independently of density. Successful URLs are immutable for one year and return a content-derived ETag. Cache entries regenerate after deletion.",
|
||||
request: {
|
||||
params: z.object({
|
||||
version: z.literal("cap-v1"),
|
||||
palette: z.enum([...AGENT_PALETTE_IDS, "muted-dream"]),
|
||||
file: z.enum(CHARACTER_STATES.map(pose => `${pose}.png`)),
|
||||
}),
|
||||
query: z.object({
|
||||
size: z.enum(AGENT_AVATAR_SIZES.map(String)).optional().default("512"),
|
||||
scale: z.enum(["1", "2"]).optional().default("1"),
|
||||
}).strict(),
|
||||
},
|
||||
responses: {
|
||||
200: { description: "PNG portrait; Cache-Control: public, max-age=31536000, immutable; ETag: SHA-256 of PNG bytes", content: { "image/png": { schema: { type: "string", format: "binary" } } } },
|
||||
304: { description: "If-None-Match matches the cached content ETag" },
|
||||
400: r.badRequest,
|
||||
503: { description: "Retryable rendering/storage failure; Cache-Control: no-store; Retry-After: 5" },
|
||||
},
|
||||
});
|
||||
|
||||
registry.registerPath({
|
||||
method: "get",
|
||||
path: "/api/companies/{companyId}/built-in-agents",
|
||||
|
|
|
|||
|
|
@ -0,0 +1,79 @@
|
|||
import { Worker } from "node:worker_threads";
|
||||
import type { AgentAvatarRequest } from "./agent-avatars.js";
|
||||
|
||||
type Job = { request: AgentAvatarRequest; resolve: (png: Buffer) => void; reject: (error: Error) => void };
|
||||
/** Lazy, bounded workers isolate geometry/rasterization from the API event loop. */
|
||||
export function createAgentAvatarPool(concurrency = 2, maxQueue = 64) {
|
||||
const queue: Job[] = [];
|
||||
let active = 0;
|
||||
const idle: Worker[] = [];
|
||||
const timers = new Map<Worker, ReturnType<typeof setTimeout>>();
|
||||
const workers = new Set<Worker>();
|
||||
let closed = false;
|
||||
function spawn() {
|
||||
const source = import.meta.url.endsWith(".ts");
|
||||
const url = new URL(source ? "./agent-avatar-worker.ts" : "./agent-avatar-worker.js", import.meta.url);
|
||||
const worker = source
|
||||
? new Worker(`import(${JSON.stringify(import.meta.resolve('tsx/esm/api'))}).then(({tsImport}) => tsImport(${JSON.stringify(url.href)}, ${JSON.stringify(import.meta.url)}));`, { eval: true })
|
||||
: new Worker(url);
|
||||
workers.add(worker);
|
||||
// Idle worker failures must not become uncaught events or leave dead slots.
|
||||
worker.on("error", () => {});
|
||||
worker.on("exit", () => {
|
||||
const index = idle.indexOf(worker);
|
||||
if (index >= 0) idle.splice(index, 1);
|
||||
clearTimeout(timers.get(worker)); timers.delete(worker); workers.delete(worker);
|
||||
});
|
||||
return worker;
|
||||
}
|
||||
function drain() {
|
||||
while (!closed && active < concurrency && queue.length) {
|
||||
const job = queue.shift()!;
|
||||
let worker: Worker;
|
||||
try { worker = idle.pop() ?? spawn(); }
|
||||
catch (error) { job.reject(error instanceof Error ? error : new Error(String(error))); continue; }
|
||||
clearTimeout(timers.get(worker)); timers.delete(worker);
|
||||
worker.ref(); active++;
|
||||
let finished = false;
|
||||
const timeout = setTimeout(() => finish(new Error("Avatar rendering timed out")), 15_000);
|
||||
const onError = (error: Error) => finish(error);
|
||||
const onExit = () => finish(new Error("Avatar worker exited"));
|
||||
const onMessage = (result: { png?: Uint8Array; error?: string }) => {
|
||||
finish(result.png ? undefined : new Error(result.error ?? "Avatar rendering failed"), result.png);
|
||||
};
|
||||
function finish(error?: Error, png?: Uint8Array) {
|
||||
if (finished) return;
|
||||
finished = true; clearTimeout(timeout); active--;
|
||||
worker.off("message", onMessage); worker.off("error", onError); worker.off("exit", onExit);
|
||||
if (error || closed) {
|
||||
workers.delete(worker); void worker.terminate();
|
||||
job.reject(error ?? new Error("Avatar pool closed"));
|
||||
} else {
|
||||
job.resolve(Buffer.from(png!));
|
||||
worker.unref(); idle.push(worker);
|
||||
const timer = setTimeout(() => {
|
||||
const index = idle.indexOf(worker);
|
||||
if (index >= 0) idle.splice(index, 1);
|
||||
timers.delete(worker); workers.delete(worker); void worker.terminate();
|
||||
}, 30_000);
|
||||
timer.unref(); timers.set(worker, timer);
|
||||
}
|
||||
drain();
|
||||
}
|
||||
worker.once("message", onMessage); worker.once("error", onError); worker.once("exit", onExit);
|
||||
try { worker.postMessage(job.request); } catch (error) { finish(error instanceof Error ? error : new Error(String(error))); }
|
||||
}
|
||||
}
|
||||
return {
|
||||
render(request: AgentAvatarRequest): Promise<Buffer> {
|
||||
if (closed || queue.length >= maxQueue) return Promise.reject(new Error("Avatar renderer is busy"));
|
||||
return new Promise((resolve, reject) => { queue.push({ request, resolve, reject }); drain(); });
|
||||
},
|
||||
async close() {
|
||||
closed = true;
|
||||
for (const job of queue.splice(0)) job.reject(new Error("Avatar pool closed"));
|
||||
for (const timer of timers.values()) clearTimeout(timer);
|
||||
await Promise.all([...workers].map(worker => worker.terminate()));
|
||||
},
|
||||
};
|
||||
}
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
import { parentPort } from "node:worker_threads";
|
||||
import sharp from "sharp";
|
||||
import { renderAgentSvg } from "@paperclipai/shared/cliplab/static";
|
||||
import type { AgentAvatarRequest } from "./agent-avatars.js";
|
||||
|
||||
parentPort!.on("message", async (request: AgentAvatarRequest) => {
|
||||
try {
|
||||
const svg = renderAgentSvg(request.appearance, request.size, request.scale, request.pose, request.muted);
|
||||
const png = await sharp(Buffer.from(svg)).png().toBuffer();
|
||||
parentPort!.postMessage({ png });
|
||||
} catch (error) {
|
||||
parentPort!.postMessage({ error: error instanceof Error ? error.message : "Avatar rendering failed" });
|
||||
}
|
||||
});
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
import { createHash } from "node:crypto";
|
||||
import type { AgentAppearance, AgentAvatarSize, CharacterState } from "@paperclipai/shared";
|
||||
import type { StorageProvider } from "../storage/types.js";
|
||||
import { createAgentAvatarPool } from "./agent-avatar-pool.js";
|
||||
|
||||
export interface AgentAvatarRequest {
|
||||
appearance: AgentAppearance;
|
||||
size: AgentAvatarSize;
|
||||
scale: 1 | 2;
|
||||
pose: CharacterState;
|
||||
muted: boolean;
|
||||
}
|
||||
export function avatarCacheKey(request: AgentAvatarRequest) {
|
||||
const { appearance, size, scale, pose, muted } = request;
|
||||
return `generated-agent-avatars/${appearance.characterVersion}/${muted ? "muted-dream" : appearance.paletteId}/${pose}-${size}-${scale}.png`;
|
||||
}
|
||||
type CacheMetadata = { sha256: string; byteSize: number };
|
||||
export function createAgentAvatarService(storage: StorageProvider, render?: (request: AgentAvatarRequest) => Promise<Buffer>) {
|
||||
const pool = render ? undefined : createAgentAvatarPool();
|
||||
const pending = new Map<string, Promise<CacheMetadata>>();
|
||||
async function ensure(request: AgentAvatarRequest, key: string): Promise<CacheMetadata> {
|
||||
const metadataKey = `${key}.json`;
|
||||
const [image, metadata] = await Promise.all([
|
||||
storage.headObject({ objectKey: key }), storage.headObject({ objectKey: metadataKey }),
|
||||
]);
|
||||
if (image.exists && metadata.exists) {
|
||||
const object = await storage.getObject({ objectKey: metadataKey });
|
||||
const chunks: Buffer[] = [];
|
||||
for await (const chunk of object.stream) chunks.push(Buffer.from(chunk));
|
||||
try {
|
||||
const cached = JSON.parse(Buffer.concat(chunks).toString()) as CacheMetadata;
|
||||
if (/^[a-f0-9]{64}$/.test(cached.sha256) && cached.byteSize > 0 && cached.byteSize === image.contentLength) return cached;
|
||||
} catch { /* Disposable metadata: regenerate a corrupt or old cache entry. */ }
|
||||
}
|
||||
const bytes = await (render ?? pool!.render)(request);
|
||||
const result = { sha256: createHash("sha256").update(bytes).digest("hex"), byteSize: bytes.length };
|
||||
// Both providers publish whole objects atomically. Publish metadata last so
|
||||
// readers never consider an unfinished image a completed cache entry.
|
||||
await storage.putObject({ objectKey: key, body: bytes, contentLength: bytes.length, contentType: "image/png" });
|
||||
const encoded = Buffer.from(JSON.stringify(result));
|
||||
await storage.putObject({ objectKey: metadataKey, body: encoded, contentLength: encoded.length, contentType: "application/json" });
|
||||
return result;
|
||||
}
|
||||
return {
|
||||
async get(request: AgentAvatarRequest) {
|
||||
const key = avatarCacheKey(request);
|
||||
let result = pending.get(key);
|
||||
if (!result) {
|
||||
result = ensure(request, key).finally(() => pending.delete(key));
|
||||
pending.set(key, result);
|
||||
}
|
||||
const metadata = await result;
|
||||
const object = await storage.getObject({ objectKey: key });
|
||||
return { stream: object.stream, byteSize: metadata.byteSize, etag: `"${metadata.sha256}"` };
|
||||
},
|
||||
async close() { await pool?.close(); },
|
||||
};
|
||||
}
|
||||
|
|
@ -1,3 +1,4 @@
|
|||
import { agentAppearanceSchema, randomAgentAppearance, resolveAgentAppearance, agentAvatarUrl } from "@paperclipai/shared";
|
||||
import { createHash, randomBytes } from "node:crypto";
|
||||
import { and, desc, eq, gte, inArray, lt, ne, or, sql } from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
|
@ -65,6 +66,7 @@ const CONFIG_REVISION_FIELDS = [
|
|||
"role",
|
||||
"title",
|
||||
"icon",
|
||||
"appearance",
|
||||
"reportsTo",
|
||||
"capabilities",
|
||||
"adapterType",
|
||||
|
|
@ -155,6 +157,7 @@ function buildConfigSnapshot(
|
|||
role: row.role,
|
||||
title: row.title,
|
||||
icon: row.icon,
|
||||
appearance: row.appearance,
|
||||
reportsTo: row.reportsTo,
|
||||
capabilities: row.capabilities,
|
||||
adapterType: row.adapterType,
|
||||
|
|
@ -190,6 +193,7 @@ function configPatchFromApprovalPayload(payload: Record<string, unknown>) {
|
|||
const patch: Partial<typeof agents.$inferInsert> = {};
|
||||
if (typeof payload.name === "string") patch.name = payload.name;
|
||||
if (typeof payload.role === "string") patch.role = payload.role;
|
||||
if (payload.appearance != null) patch.appearance = agentAppearanceSchema.parse(payload.appearance);
|
||||
if (Object.prototype.hasOwnProperty.call(payload, "title")) {
|
||||
patch.title = typeof payload.title === "string" ? payload.title : null;
|
||||
}
|
||||
|
|
@ -362,8 +366,11 @@ export function agentService(db: Db) {
|
|||
const eligibilityAgents = allCompanyRows.map(toEligibilityAgent);
|
||||
return rows.map((row) => {
|
||||
const base = normalizeAgentBaseRow(row);
|
||||
const appearance = resolveAgentAppearance(row.appearance, row.id);
|
||||
return {
|
||||
...base,
|
||||
appearance,
|
||||
avatarUrl: agentAvatarUrl(appearance),
|
||||
orgChainHealth: getAgentWorkEligibility({
|
||||
agent: toEligibilityAgent(row),
|
||||
agents: eligibilityAgents,
|
||||
|
|
@ -848,6 +855,7 @@ export function agentService(db: Db) {
|
|||
.values({
|
||||
...data,
|
||||
name: uniqueName,
|
||||
appearance: data.appearance == null ? randomAgentAppearance() : agentAppearanceSchema.parse(data.appearance),
|
||||
companyId,
|
||||
role,
|
||||
adapterType,
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import { agentAppearanceSchema } from "@paperclipai/shared";
|
||||
import { and, asc, eq, inArray, sql } from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { approvalComments, approvals } from "@paperclipai/db";
|
||||
|
|
@ -160,6 +161,7 @@ export function approvalService(db: Db) {
|
|||
} else {
|
||||
const created = await agentsSvc.create(updated.companyId, {
|
||||
name: String(payload.name ?? "New Agent"),
|
||||
appearance: payload.appearance == null ? undefined : agentAppearanceSchema.parse(payload.appearance),
|
||||
role: String(payload.role ?? "general"),
|
||||
title: typeof payload.title === "string" ? payload.title : null,
|
||||
reportsTo: typeof payload.reportsTo === "string" ? payload.reportsTo : null,
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import { agentAppearanceSchema } from "@paperclipai/shared";
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import { promises as fs } from "node:fs";
|
||||
import { execFile } from "node:child_process";
|
||||
|
|
@ -702,6 +703,7 @@ type ProjectLike = {
|
|||
targetDate: string | null;
|
||||
color: string | null;
|
||||
icon: string | null;
|
||||
appearance?: import("@paperclipai/shared").AgentAppearance | null;
|
||||
status: string;
|
||||
env: Record<string, unknown> | null;
|
||||
executionWorkspacePolicy: Record<string, unknown> | null;
|
||||
|
|
@ -3238,6 +3240,7 @@ function buildManifestFromPackageFiles(
|
|||
role: asString(extension.role) ?? asString(frontmatter.role) ?? "agent",
|
||||
title,
|
||||
icon: asString(extension.icon),
|
||||
appearance: extension.appearance == null ? undefined : agentAppearanceSchema.parse(extension.appearance),
|
||||
capabilities: asString(extension.capabilities),
|
||||
reportsToSlug: asString(frontmatter.reportsTo) ?? asString(extension.reportsTo),
|
||||
reportsToExistingAgentId: asString(extension.reportsToExistingAgentId),
|
||||
|
|
@ -4254,6 +4257,7 @@ export function companyPortabilityService(db: Db, storage?: StorageService) {
|
|||
const extension = stripEmptyValues({
|
||||
role: agent.role !== "agent" ? agent.role : undefined,
|
||||
icon: agent.icon ?? null,
|
||||
appearance: agent.appearance,
|
||||
capabilities: agent.capabilities ?? null,
|
||||
adapter: {
|
||||
type: agent.adapterType,
|
||||
|
|
@ -5592,6 +5596,7 @@ export function companyPortabilityService(db: Db, storage?: StorageService) {
|
|||
role: manifestAgent.role,
|
||||
title: manifestAgent.title,
|
||||
icon: manifestAgent.icon,
|
||||
...(manifestAgent.appearance ? { appearance: manifestAgent.appearance } : {}),
|
||||
capabilities: manifestAgent.capabilities,
|
||||
reportsTo: null,
|
||||
adapterType: normalizedAdapter.adapterType,
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import { agentAvatarUrl, resolveAgentAppearance } from "@paperclipai/shared";
|
||||
import { and, desc, eq, gte, isNotNull, isNull, lt, lte, sql } from "drizzle-orm";
|
||||
import { alias } from "drizzle-orm/pg-core";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
|
@ -282,10 +283,11 @@ export function costService(db: Db, budgetHooks: BudgetServiceHooks = {}) {
|
|||
if (range?.from) conditions.push(gte(costEvents.occurredAt, range.from));
|
||||
if (range?.to) conditions.push(lte(costEvents.occurredAt, range.to));
|
||||
|
||||
return db
|
||||
const rows = await db
|
||||
.select({
|
||||
agentId: costEvents.agentId,
|
||||
agentName: agents.name,
|
||||
agentAppearance: agents.appearance,
|
||||
agentStatus: agents.status,
|
||||
costCents: sumAsNumber(costEvents.costCents),
|
||||
inputTokens: sumAsNumber(costEvents.inputTokens),
|
||||
|
|
@ -305,8 +307,12 @@ export function costService(db: Db, budgetHooks: BudgetServiceHooks = {}) {
|
|||
.from(costEvents)
|
||||
.leftJoin(agents, eq(costEvents.agentId, agents.id))
|
||||
.where(and(...conditions))
|
||||
.groupBy(costEvents.agentId, agents.name, agents.status)
|
||||
.groupBy(costEvents.agentId, agents.name, agents.appearance, agents.status)
|
||||
.orderBy(desc(sumAsNumber(costEvents.costCents)));
|
||||
return rows.map(row => {
|
||||
const appearance = resolveAgentAppearance(row.agentAppearance, row.agentId);
|
||||
return { ...row, agentAppearance: appearance, avatarUrl: agentAvatarUrl(appearance, 512) };
|
||||
});
|
||||
},
|
||||
|
||||
byProvider: async (companyId: string, range?: CostDateRange) => {
|
||||
|
|
@ -431,10 +437,11 @@ export function costService(db: Db, budgetHooks: BudgetServiceHooks = {}) {
|
|||
// the (companyId, agentId, occurredAt) composite index covers this well.
|
||||
// order by provider + model for stable db-level ordering; cost-desc sort
|
||||
// within each agent's sub-rows is done client-side in the ui memo.
|
||||
return db
|
||||
const rows = await db
|
||||
.select({
|
||||
agentId: costEvents.agentId,
|
||||
agentName: agents.name,
|
||||
agentAppearance: agents.appearance,
|
||||
provider: costEvents.provider,
|
||||
biller: costEvents.biller,
|
||||
billingType: costEvents.billingType,
|
||||
|
|
@ -450,12 +457,17 @@ export function costService(db: Db, budgetHooks: BudgetServiceHooks = {}) {
|
|||
.groupBy(
|
||||
costEvents.agentId,
|
||||
agents.name,
|
||||
agents.appearance,
|
||||
costEvents.provider,
|
||||
costEvents.biller,
|
||||
costEvents.billingType,
|
||||
costEvents.model,
|
||||
)
|
||||
.orderBy(costEvents.provider, costEvents.biller, costEvents.billingType, costEvents.model);
|
||||
return rows.map(row => {
|
||||
const appearance = resolveAgentAppearance(row.agentAppearance, row.agentId);
|
||||
return { ...row, agentAppearance: appearance, avatarUrl: agentAvatarUrl(appearance, 512) };
|
||||
});
|
||||
},
|
||||
|
||||
byProject: async (companyId: string, range?: CostDateRange) => {
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import { withAgentAppearance } from "@paperclipai/shared";
|
||||
import { and, count, desc, eq, gte, inArray, lte, or, sql } from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import {
|
||||
|
|
@ -479,10 +480,10 @@ export function createSecretProposalsService(db: Db) {
|
|||
|
||||
async function enrich(proposal: Proposal) {
|
||||
const [proposer, target, originIssue, secret, secretProposal] = await Promise.all([
|
||||
db.select({ id: agents.id, name: agents.name, icon: agents.icon }).from(agents)
|
||||
db.select({ id: agents.id, name: agents.name, icon: agents.icon, appearance: agents.appearance }).from(agents)
|
||||
.where(eq(agents.id, proposal.proposedByAgentId)).then((rows) => rows[0] ?? null),
|
||||
proposal.targetId
|
||||
? db.select({ id: agents.id, name: agents.name, icon: agents.icon }).from(agents)
|
||||
? db.select({ id: agents.id, name: agents.name, icon: agents.icon, appearance: agents.appearance }).from(agents)
|
||||
.where(eq(agents.id, proposal.targetId)).then((rows) => rows[0] ?? null)
|
||||
: Promise.resolve(null),
|
||||
proposal.originIssueId
|
||||
|
|
@ -510,8 +511,8 @@ export function createSecretProposalsService(db: Db) {
|
|||
...safe,
|
||||
secretName: secret?.name ?? null,
|
||||
secretProposalName: secretProposal?.proposedName ?? null,
|
||||
proposedBy: proposer,
|
||||
target,
|
||||
proposedBy: withAgentAppearance(proposer),
|
||||
target: target ? withAgentAppearance(target) : null,
|
||||
originIssue,
|
||||
};
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import { withAgentAppearance } from "@paperclipai/shared";
|
||||
import { and, asc, desc, eq, gte, inArray, isNull, lte, or, sql } from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import {
|
||||
|
|
@ -430,7 +431,7 @@ export function workTimelineService(db: Db) {
|
|||
const [agentRows, userRows] = await Promise.all([
|
||||
agentIds.length > 0
|
||||
? db
|
||||
.select({ id: agents.id, name: agents.name, icon: agents.icon })
|
||||
.select({ id: agents.id, name: agents.name, icon: agents.icon, appearance: agents.appearance })
|
||||
.from(agents)
|
||||
.where(and(eq(agents.companyId, companyId), inArray(agents.id, maybeUuidList(agentIds))))
|
||||
: [],
|
||||
|
|
@ -763,7 +764,8 @@ export function workTimelineService(db: Db) {
|
|||
const [type, rawId] = id.split(":", 2) as [TimelineActorType, string];
|
||||
if (type === "agent") {
|
||||
const agent = actorMaps.agents.get(rawId);
|
||||
return { id, type, name: agent?.name ?? "Unknown agent", avatar: agent?.icon ?? null };
|
||||
const identity = withAgentAppearance(agent ?? { id: rawId });
|
||||
return { id, type, name: agent?.name ?? "Unknown agent", avatar: identity.avatarUrl, appearance: identity.appearance, avatarUrl: identity.avatarUrl };
|
||||
}
|
||||
if (type === "user") {
|
||||
const user = actorMaps.users.get(rawId);
|
||||
|
|
|
|||
|
|
@ -0,0 +1,33 @@
|
|||
import { useState } from "react";
|
||||
import { agentAvatarUrl, resolveAgentAppearance, type AgentAppearance, type AgentAvatarSize, type CharacterState } from "@paperclipai/shared";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { deriveInitials } from "./Identity";
|
||||
|
||||
export type AvatarAgent = { id?: string; name?: string; appearance?: AgentAppearance | null; avatarUrl?: string | null };
|
||||
export const avatarSizeClasses: Record<AgentAvatarSize, string> = {
|
||||
16: "size-4", 20: "size-5", 24: "size-6", 32: "size-8", 40: "size-10", 48: "size-12",
|
||||
64: "size-16", 96: "size-24", 128: "size-32", 256: "size-64", 512: "size-128",
|
||||
};
|
||||
export interface AgentAvatarProps {
|
||||
agent?: AvatarAgent | null;
|
||||
appearance?: AgentAppearance | null;
|
||||
size?: AgentAvatarSize;
|
||||
name?: string;
|
||||
label?: string;
|
||||
pose?: CharacterState;
|
||||
muted?: boolean;
|
||||
className?: string;
|
||||
}
|
||||
export function AgentAvatar({ agent, appearance, size = 24, name, label, pose = "rest", muted = false, className }: AgentAvatarProps) {
|
||||
const identity = resolveAgentAppearance(appearance ?? agent?.appearance, agent?.id);
|
||||
const src = agentAvatarUrl(identity, size, 1, pose, muted);
|
||||
const [failedUrl, setFailedUrl] = useState<string | null>(null);
|
||||
return (
|
||||
<span data-slot="agent-avatar" className={cn("relative inline-flex shrink-0 items-center justify-center align-middle", avatarSizeClasses[size], className)}
|
||||
role={label ? "img" : undefined} aria-label={label} aria-hidden={label ? undefined : true}>
|
||||
{failedUrl === src ? <span className="text-xs text-muted-foreground">{deriveInitials(name ?? agent?.name ?? "Agent")}</span> :
|
||||
<img src={src} srcSet={`${agentAvatarUrl(identity, size, 2, pose, muted)} 2x`} alt="" width={size} height={size}
|
||||
decoding="async" loading="lazy" className="size-full object-contain" onError={() => setFailedUrl(src)} />}
|
||||
</span>
|
||||
);
|
||||
}
|
||||
|
|
@ -0,0 +1,63 @@
|
|||
import { useEffect, useMemo, useRef, useState, useSyncExternalStore, type RefObject } from "react";
|
||||
import { resolveAgentAppearance, type CharacterState } from "@paperclipai/shared";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { characterSlot } from "@/lib/agent-character-slot";
|
||||
import { AgentAvatar, avatarSizeClasses, type AgentAvatarProps } from "./AgentAvatar";
|
||||
import type { createCharacter } from "@paperclipai/shared/cliplab/runtime";
|
||||
|
||||
type Player = ReturnType<typeof createCharacter>;
|
||||
export interface AgentCharacterProps extends Omit<AgentAvatarProps, "pose"> {
|
||||
state?: CharacterState;
|
||||
motion?: "auto" | "still";
|
||||
trackingRegion?: RefObject<HTMLElement | null>;
|
||||
trackingScope?: "region" | "page";
|
||||
followCursor?: boolean;
|
||||
followRotation?: boolean;
|
||||
}
|
||||
export function AgentCharacter({ agent, appearance, size = 256, state = "idle", muted = false, motion = "auto", trackingRegion, trackingScope = "region", followCursor = true, followRotation = true, className, label, name }: AgentCharacterProps) {
|
||||
const identity = useMemo(() => resolveAgentAppearance(appearance ?? agent?.appearance, agent?.id), [appearance, agent?.appearance, agent?.id]);
|
||||
const root = useRef<HTMLSpanElement>(null), host = useRef<HTMLSpanElement>(null), player = useRef<Player | null>(null);
|
||||
const slotId = useRef(Symbol("agent-character"));
|
||||
const owner = useSyncExternalStore(characterSlot.subscribe, characterSlot.getSnapshot, () => null);
|
||||
const [visible, setVisible] = useState(false), [reduced, setReduced] = useState(true), [failed, setFailed] = useState(false), [ready, setReady] = useState(false);
|
||||
const active = visible && !reduced && !failed && motion === "auto" && state !== "rest";
|
||||
useEffect(() => {
|
||||
if (typeof matchMedia !== "function" || typeof IntersectionObserver !== "function") return;
|
||||
const media = matchMedia("(prefers-reduced-motion: reduce)");
|
||||
const change = () => setReduced(media.matches); change(); media.addEventListener("change", change);
|
||||
const observer = new IntersectionObserver(entries => setVisible(entries[0]?.isIntersecting ?? false));
|
||||
if (root.current) observer.observe(root.current);
|
||||
return () => { observer.disconnect(); media.removeEventListener("change", change); };
|
||||
}, []);
|
||||
useEffect(() => {
|
||||
if (active && owner === null) characterSlot.acquire(slotId.current);
|
||||
if (!active) characterSlot.release(slotId.current);
|
||||
}, [active, owner]);
|
||||
useEffect(() => () => characterSlot.release(slotId.current), []);
|
||||
useEffect(() => {
|
||||
if (!active || owner !== slotId.current) return;
|
||||
let disposed = false;
|
||||
setReady(false);
|
||||
void Promise.all([import("@paperclipai/shared/cliplab/runtime"), import("@paperclipai/shared/cliplab/definition")]).then(([runtime, library]) => {
|
||||
if (disposed || !host.current) return;
|
||||
player.current = runtime.createCharacter(host.current, library.characterDefinition(identity, muted), {
|
||||
animation: library.animationId(state), trackingRegion: trackingRegion?.current ?? root.current ?? undefined,
|
||||
followCursor, followRotation, trackingScope, displaySize: size, onError: () => setFailed(true),
|
||||
});
|
||||
setReady(true);
|
||||
}).catch(() => { if (!disposed) setFailed(true); });
|
||||
return () => { disposed = true; player.current?.destroy(); player.current = null; setReady(false); };
|
||||
}, [active, owner, trackingRegion, trackingScope, size, followCursor, followRotation]);
|
||||
useEffect(() => {
|
||||
if (!player.current) return;
|
||||
void import("@paperclipai/shared/cliplab/definition").then(library => {
|
||||
player.current?.setDefinition(library.characterDefinition(identity, muted));
|
||||
player.current?.setAnimation(library.animationId(state));
|
||||
});
|
||||
}, [identity, muted, state, ready]);
|
||||
return <span ref={root} role={label ? "img" : undefined} aria-label={label} aria-hidden={label ? undefined : true}
|
||||
className={cn("relative inline-block shrink-0", avatarSizeClasses[size], className)}>
|
||||
<AgentAvatar agent={agent} appearance={identity} size={size < 256 ? 256 : size} name={name} pose={state} muted={muted} className={cn("size-full", ready && "invisible")} />
|
||||
<span ref={host} className="absolute inset-0" />
|
||||
</span>;
|
||||
}
|
||||
|
|
@ -0,0 +1,85 @@
|
|||
// @vitest-environment jsdom
|
||||
import { afterEach, beforeEach, expect, it, vi } from "vitest";
|
||||
import { createCharacter } from "@paperclipai/shared/cliplab/runtime";
|
||||
import { characterDefinition } from "@paperclipai/shared/cliplab/definition";
|
||||
import { appearanceForPalette } from "@paperclipai/shared";
|
||||
const renderer = vi.hoisted(() => ({ render: vi.fn(), resize: vi.fn(), dispose: vi.fn() }));
|
||||
vi.mock("@paperclipai/shared/cliplab/renderer", () => ({ CharacterRenderer: class { render = renderer.render; resize = renderer.resize; dispose = renderer.dispose; } }));
|
||||
let intersect: (entries: any[]) => void;
|
||||
let hidden = false;
|
||||
let frames: Map<number, FrameRequestCallback>;
|
||||
let reduced: MediaQueryList;
|
||||
let coarse: MediaQueryList;
|
||||
let target: HTMLDivElement;
|
||||
let region: HTMLDivElement;
|
||||
const players: Array<ReturnType<typeof createCharacter>> = [];
|
||||
function media() { return Object.assign(new EventTarget(), { matches: false }) as unknown as MediaQueryList; }
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks(); renderer.render.mockReset(); hidden = false; frames = new Map(); reduced = media(); coarse = media();
|
||||
let id = 0;
|
||||
vi.stubGlobal("requestAnimationFrame", (callback: FrameRequestCallback) => { frames.set(++id, callback); return id; });
|
||||
vi.stubGlobal("cancelAnimationFrame", (key: number) => frames.delete(key));
|
||||
vi.stubGlobal("matchMedia", (query: string) => query.includes("reduced-motion") ? reduced : coarse);
|
||||
vi.spyOn(document, "hidden", "get").mockImplementation(() => hidden);
|
||||
vi.stubGlobal("IntersectionObserver", class { constructor(callback: typeof intersect) { intersect = callback; } observe() {} disconnect() {} });
|
||||
vi.stubGlobal("ResizeObserver", class { observe() {} disconnect() {} });
|
||||
target = document.createElement("div"); region = document.createElement("div"); region.appendChild(target); document.body.appendChild(region);
|
||||
});
|
||||
afterEach(() => { players.splice(0).forEach(player => player.destroy()); region.remove(); vi.restoreAllMocks(); vi.unstubAllGlobals(); });
|
||||
function create() {
|
||||
const player = createCharacter(target, characterDefinition(appearanceForPalette("arctic-blue")), { trackingRegion: region });
|
||||
players.push(player); return player;
|
||||
}
|
||||
it("stops scheduling offscreen, when hidden, for reduced motion, and after disposal", () => {
|
||||
const player = create(); expect(frames.size).toBe(0);
|
||||
intersect([{ isIntersecting: true }]); expect(frames.size).toBe(1);
|
||||
hidden = true; document.dispatchEvent(new Event("visibilitychange")); expect(frames.size).toBe(0);
|
||||
hidden = false; document.dispatchEvent(new Event("visibilitychange")); expect(frames.size).toBe(1);
|
||||
Object.assign(reduced, { matches: true }); reduced.dispatchEvent(new Event("change")); expect(frames.size).toBe(0);
|
||||
Object.assign(reduced, { matches: false }); reduced.dispatchEvent(new Event("change")); expect(frames.size).toBe(1);
|
||||
intersect([{ isIntersecting: false }]); expect(frames.size).toBe(0);
|
||||
intersect([{ isIntersecting: true }]); player.destroy(); player.destroy();
|
||||
expect(frames.size).toBe(0); expect(renderer.dispose).toHaveBeenCalledTimes(1); expect(target.children).toHaveLength(0);
|
||||
});
|
||||
it("tracks only its region, ignores touch, and removes listeners for coarse pointers", () => {
|
||||
const add = vi.spyOn(region, "addEventListener"), remove = vi.spyOn(region, "removeEventListener");
|
||||
const documentAdd = vi.spyOn(document, "addEventListener");
|
||||
create(); intersect([{ isIntersecting: true }]);
|
||||
expect(add.mock.calls.some(([event]) => event === "pointermove")).toBe(true);
|
||||
expect(documentAdd.mock.calls.some(([event]) => event === "pointermove")).toBe(false);
|
||||
const before = renderer.render.mock.calls.at(-1)?.[3];
|
||||
region.dispatchEvent(Object.assign(new Event("pointermove"), { pointerType: "touch", clientX: 100, clientY: 100 }));
|
||||
const [id, frame] = [...frames][0]; frames.delete(id); frame(performance.now() + 16);
|
||||
expect(renderer.render.mock.calls.at(-1)?.[3]).toEqual(before);
|
||||
Object.assign(coarse, { matches: true }); coarse.dispatchEvent(new Event("change"));
|
||||
expect(remove.mock.calls.some(([event]) => event === "pointermove")).toBe(true);
|
||||
});
|
||||
it("disposes the canvas and subscriptions when the first render fails", () => {
|
||||
renderer.render.mockImplementationOnce(() => { throw new Error("lost context"); });
|
||||
expect(() => create()).toThrow("lost context");
|
||||
expect(renderer.dispose).toHaveBeenCalledTimes(1); expect(target.children).toHaveLength(0); expect(frames.size).toBe(0);
|
||||
});
|
||||
|
||||
it("falls back when a later expression render fails", () => {
|
||||
const onError = vi.fn();
|
||||
const player = createCharacter(target, characterDefinition(appearanceForPalette("arctic-blue")), { onError });
|
||||
players.push(player); intersect([{ isIntersecting: true }]);
|
||||
renderer.render.mockImplementationOnce(() => { throw new Error("lost context"); });
|
||||
expect(() => player.setAnimation("success")).not.toThrow();
|
||||
expect(onError).toHaveBeenCalledOnce(); expect(frames.size).toBe(0);
|
||||
});
|
||||
|
||||
it("can follow the whole page while measuring gaze from the character and cleaning up", () => {
|
||||
const page = document.documentElement;
|
||||
const remove = vi.spyOn(page, "removeEventListener");
|
||||
vi.spyOn(target, "getBoundingClientRect").mockReturnValue({ left: 100, top: 100, width: 100, height: 100 } as DOMRect);
|
||||
const player = createCharacter(target, characterDefinition(appearanceForPalette("arctic-blue")), { trackingScope: "page", displaySize: 256 });
|
||||
players.push(player); intersect([{ isIntersecting: true }]);
|
||||
// Outside the character and its parent region, but still on the page.
|
||||
page.dispatchEvent(Object.assign(new Event("pointermove"), { pointerType: "mouse", clientX: 900, clientY: 150 }));
|
||||
const [id, frame] = [...frames][0]; frames.delete(id); frame(performance.now() + 50);
|
||||
const gaze = renderer.render.mock.calls.at(-1)?.[3];
|
||||
expect(gaze.x).toBeGreaterThan(0); expect(gaze.y).toBe(0);
|
||||
player.destroy();
|
||||
expect(remove.mock.calls.some(([event]) => event === "pointermove")).toBe(true);
|
||||
});
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
import { AgentAvatar, type AvatarAgent } from "./AgentAvatar";
|
||||
import { cn } from "@/lib/utils";
|
||||
export function AgentIdentity({ agent, size = "default", className }: { agent: AvatarAgent; size?: "xs" | "sm" | "default" | "lg"; className?: string }) {
|
||||
const pixels = { xs: 20, sm: 24, default: 32, lg: 40 } as const;
|
||||
return <span title={agent.name} className={cn("inline-flex min-w-0 items-center gap-1.5", className)}>
|
||||
<AgentAvatar agent={agent} size={pixels[size]} />
|
||||
<span className={cn("truncate", size === "sm" ? "text-xs" : "text-sm")}>{agent.name ?? "Agent"}</span>
|
||||
</span>;
|
||||
}
|
||||
|
|
@ -0,0 +1,78 @@
|
|||
// @vitest-environment jsdom
|
||||
import { act } from "react";
|
||||
import { createRoot, type Root } from "react-dom/client";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { appearanceForPalette } from "@paperclipai/shared";
|
||||
import { AgentAvatar } from "./AgentAvatar";
|
||||
import { AgentCharacter } from "./AgentCharacter";
|
||||
import { useAgentAppearanceDraft } from "../hooks/useAgentAppearanceDraft";
|
||||
|
||||
const renderer = vi.hoisted(() => ({ destroy: vi.fn(), setDefinition: vi.fn(), setAnimation: vi.fn() }));
|
||||
const createCharacter = vi.hoisted(() => vi.fn(() => renderer));
|
||||
vi.mock("@paperclipai/shared/cliplab/runtime", () => ({ createCharacter }));
|
||||
vi.mock("@paperclipai/shared/cliplab/definition", () => ({ characterDefinition: vi.fn(value => value), animationId: vi.fn(value => value) }));
|
||||
(globalThis as any).IS_REACT_ACT_ENVIRONMENT = true;
|
||||
let root: Root;
|
||||
let host: HTMLDivElement;
|
||||
let observers: Array<(entries: any[]) => void>;
|
||||
let reduced = false;
|
||||
const appearance = appearanceForPalette("bubblegum-sky");
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks(); observers = []; reduced = false;
|
||||
sessionStorage.clear();
|
||||
host = document.createElement("div"); document.body.appendChild(host); root = createRoot(host);
|
||||
vi.stubGlobal("IntersectionObserver", class {
|
||||
constructor(callback: (entries: any[]) => void) { observers.push(callback); }
|
||||
observe() {} disconnect() {}
|
||||
});
|
||||
vi.stubGlobal("matchMedia", () => ({ matches: reduced, addEventListener() {}, removeEventListener() {} }));
|
||||
});
|
||||
afterEach(async () => { await act(async () => root.unmount()); host.remove(); vi.unstubAllGlobals(); });
|
||||
async function show() { await act(async () => { for (const observer of observers) observer([{ isIntersecting: true }]); }); }
|
||||
describe("agent persona presentation", () => {
|
||||
it("renders 500 static images without initializing a live renderer", async () => {
|
||||
await act(async () => root.render(<>{Array.from({ length: 500 }, (_, i) => <AgentAvatar key={i} appearance={appearance} size={24} />)}</>));
|
||||
expect(host.querySelectorAll("img")).toHaveLength(500);
|
||||
expect(host.querySelector("canvas")).toBeNull();
|
||||
expect(createCharacter).not.toHaveBeenCalled();
|
||||
expect(host.querySelector("img")?.getAttribute("srcset")).toContain("size=24&scale=2");
|
||||
expect(host.querySelector("img")?.getAttribute("width")).toBe("24");
|
||||
});
|
||||
it("keeps reserved image dimensions and falls back to initials after an image error", async () => {
|
||||
await act(async () => root.render(<AgentAvatar appearance={appearance} name="Chief of Staff" size={48} label="Chief of Staff" />));
|
||||
expect(host.querySelector("img")?.height).toBe(48);
|
||||
await act(async () => { host.querySelector("img")!.dispatchEvent(new Event("error")); });
|
||||
expect(host.textContent).toBe("CS");
|
||||
expect(host.querySelector('[role="img"]')?.getAttribute("aria-label")).toBe("Chief of Staff");
|
||||
});
|
||||
it("allows only one live character, releases it offscreen and disposes on unmount", async () => {
|
||||
await act(async () => root.render(<><AgentCharacter appearance={appearance} /><AgentCharacter appearance={appearance} /></>));
|
||||
expect(createCharacter).not.toHaveBeenCalled();
|
||||
await show();
|
||||
expect(createCharacter).toHaveBeenCalledTimes(1);
|
||||
await act(async () => { observers[0]([{ isIntersecting: false }]); });
|
||||
expect(renderer.destroy).toHaveBeenCalledTimes(1);
|
||||
expect(createCharacter).toHaveBeenCalledTimes(2);
|
||||
await act(async () => root.render(null));
|
||||
expect(renderer.destroy).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
it("uses only static images for reduced motion and an explicit still policy", async () => {
|
||||
reduced = true;
|
||||
await act(async () => root.render(<><AgentCharacter appearance={appearance} /><AgentCharacter appearance={appearance} motion="still" /></>));
|
||||
await show();
|
||||
expect(createCharacter).not.toHaveBeenCalled();
|
||||
expect(host.querySelectorAll("img")).toHaveLength(2);
|
||||
});
|
||||
it("retains the draft assignment across remounts and clears it only after creation", async () => {
|
||||
let draft!: ReturnType<typeof useAgentAppearanceDraft>;
|
||||
function Draft() { draft = useAgentAppearanceDraft("company:new-agent"); return null; }
|
||||
await act(async () => root.render(<Draft />));
|
||||
const first = draft.appearance;
|
||||
await act(async () => root.render(null));
|
||||
await act(async () => root.render(<Draft />));
|
||||
expect(draft.appearance).toEqual(first);
|
||||
expect(JSON.parse(sessionStorage.getItem("paperclip.agent-appearance.company:new-agent")!)).toEqual(first);
|
||||
draft.clear();
|
||||
expect(sessionStorage.getItem("paperclip.agent-appearance.company:new-agent")).toBeNull();
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
import { useState } from "react";
|
||||
import { agentAppearanceSchema, randomAgentAppearance } from "@paperclipai/shared";
|
||||
|
||||
/** Non-secret visual identity only. The caller remounts when its draft key changes. */
|
||||
export function useAgentAppearanceDraft(draftKey: string) {
|
||||
const key = `paperclip.agent-appearance.${draftKey}`;
|
||||
const [appearance] = useState(() => {
|
||||
try {
|
||||
const stored = agentAppearanceSchema.safeParse(JSON.parse(sessionStorage.getItem(key) ?? "null"));
|
||||
if (stored.success) return stored.data;
|
||||
} catch { /* Storage can be unavailable; retain the in-memory assignment. */ }
|
||||
const value = randomAgentAppearance();
|
||||
try { sessionStorage.setItem(key, JSON.stringify(value)); } catch { /* In-memory draft still works. */ }
|
||||
return value;
|
||||
});
|
||||
return { appearance, clear() { try { sessionStorage.removeItem(key); } catch { /* Best effort. */ } } };
|
||||
}
|
||||
|
|
@ -2912,3 +2912,43 @@ span.paperclip-mention-chip[data-mention-kind="external-object"] {
|
|||
.task-chat-loading-shell .animate-pulse {
|
||||
animation: none;
|
||||
}
|
||||
|
||||
/* Immutable ClipLab cap-v1 artwork tokens. New artwork gets a new version. */
|
||||
:root {
|
||||
--agent-cap-v1-bubblegum-sky-a: #8bd4ff;
|
||||
--agent-cap-v1-bubblegum-sky-b: #ff51bf;
|
||||
--agent-cap-v1-pink-lemonade-a: #ff26a8;
|
||||
--agent-cap-v1-pink-lemonade-b: #fff78a;
|
||||
--agent-cap-v1-orchid-peach-a: #e771ff;
|
||||
--agent-cap-v1-orchid-peach-b: #ffd87c;
|
||||
--agent-cap-v1-coral-mint-a: #beffe8;
|
||||
--agent-cap-v1-coral-mint-b: #ff797b;
|
||||
--agent-cap-v1-lime-lagoon-a: #b4ffa4;
|
||||
--agent-cap-v1-lime-lagoon-b: #26dfff;
|
||||
--agent-cap-v1-arctic-blue-a: #97fff3;
|
||||
--agent-cap-v1-arctic-blue-b: #0084ff;
|
||||
--agent-cap-v1-solar-flare-a: #ecca5c;
|
||||
--agent-cap-v1-solar-flare-b: #fe3c3f;
|
||||
--agent-cap-v1-violet-ember-a: #ff5c43;
|
||||
--agent-cap-v1-violet-ember-b: #6262ff;
|
||||
--agent-cap-v1-deep-tide-a: #003d60;
|
||||
--agent-cap-v1-deep-tide-b: #32fffc;
|
||||
--agent-cap-v1-coral-current-a: #26e2ff;
|
||||
--agent-cap-v1-coral-current-b: #ff6666;
|
||||
--agent-cap-v1-golden-hour-a: #ffcd4f;
|
||||
--agent-cap-v1-golden-hour-b: #fde5ba;
|
||||
--agent-cap-v1-tangerine-cobalt-a: #ffad32;
|
||||
--agent-cap-v1-tangerine-cobalt-b: #4169ff;
|
||||
--agent-cap-v1-electric-grove-a: #b4ff32;
|
||||
--agent-cap-v1-electric-grove-b: #008d58;
|
||||
--agent-cap-v1-flamingo-jade-a: #ff4f9a;
|
||||
--agent-cap-v1-flamingo-jade-b: #36edaa;
|
||||
--agent-cap-v1-cherry-pop-a: #ff69db;
|
||||
--agent-cap-v1-cherry-pop-b: #d51c46;
|
||||
--agent-cap-v1-turquoise-cherry-a: #37f0db;
|
||||
--agent-cap-v1-turquoise-cherry-b: #ff4f64;
|
||||
--agent-cap-v1-ultraviolet-tide-a: #7516cf;
|
||||
--agent-cap-v1-ultraviolet-tide-b: #00e8d1;
|
||||
--agent-cap-v1-muted-dream-a: #a6aaad;
|
||||
--agent-cap-v1-muted-dream-b: #44464a;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,9 @@
|
|||
// The app gets one live hero. Other placements retain their static image.
|
||||
let owner: symbol | null = null;
|
||||
const listeners = new Set<() => void>();
|
||||
export const characterSlot = {
|
||||
subscribe(listener: () => void) { listeners.add(listener); return () => { listeners.delete(listener); }; },
|
||||
getSnapshot() { return owner; },
|
||||
acquire(id: symbol) { if (owner) return false; owner = id; for (const listener of listeners) listener(); return true; },
|
||||
release(id: symbol) { if (owner !== id) return; owner = null; for (const listener of listeners) listener(); },
|
||||
};
|
||||
|
|
@ -47,6 +47,11 @@ const baseAgent: AgentDetail = {
|
|||
};
|
||||
|
||||
describe("duplicate agent payload", () => {
|
||||
it("omits the source persona so central creation makes a fresh assignment", () => {
|
||||
const source = { ...baseAgent, appearance: { schemaVersion: 1, characterVersion: "cap-v1", paletteId: "arctic-blue" } } as AgentDetail;
|
||||
expect(buildDuplicateAgentPayload(source)).not.toHaveProperty("appearance");
|
||||
});
|
||||
|
||||
it("suffixes duplicate names", () => {
|
||||
expect(duplicateAgentName("Senior Product Engineer")).toBe("Senior Product Engineer Copy");
|
||||
expect(duplicateAgentName(" ")).toBe("Agent Copy");
|
||||
|
|
|
|||
|
|
@ -1,4 +1,7 @@
|
|||
import { SavedProviderKeySelect } from "../components/onboarding/SavedProviderKeySelect";
|
||||
import { AgentAvatar } from "@/components/AgentAvatar";
|
||||
import { AgentCharacter } from "@/components/AgentCharacter";
|
||||
import { AGENT_PALETTE_IDS, appearanceForPalette } from "@paperclipai/shared";
|
||||
import { RepositoryEditor } from "@/components/RepositoryEditor";
|
||||
import { TaskChatMarker } from "@/components/task-chat/TaskChatMarker";
|
||||
import { TaskChatComposer } from "@/components/task-chat/TaskChatComposer";
|
||||
|
|
@ -1452,20 +1455,29 @@ export function DesignGuide() {
|
|||
{/* ============================================================ */}
|
||||
{/* IDENTITY */}
|
||||
{/* ============================================================ */}
|
||||
<Section title="Identity">
|
||||
<Section title="Agent personas">
|
||||
<SubSection title="Stable palette identities">
|
||||
<div className="flex flex-wrap gap-3">{AGENT_PALETTE_IDS.map(palette => <AgentAvatar key={palette} appearance={appearanceForPalette(palette)} size={48} label={palette} />)}</div>
|
||||
</SubSection>
|
||||
<SubSection title="Onboarding and live character">
|
||||
<p className="text-sm text-muted-foreground">Place one live character beside the agent name. Onboarding uses a larger padded frame. Onboarding and agent headers follow the pointer across the page; other placements track within their region. Full-page examples are in Storybook under Agents / Personas / Full pages.</p>
|
||||
<div className="flex gap-4"><AgentCharacter muted state="sleepy" motion="still" size={128} /><AgentCharacter size={128} /></div>
|
||||
</SubSection>
|
||||
</Section>
|
||||
<Section title="Human identity">
|
||||
<SubSection title="Sizes">
|
||||
<div className="flex items-center gap-6">
|
||||
<Identity name="Agent Alpha" size="sm" />
|
||||
<Identity name="Agent Alpha" />
|
||||
<Identity name="Agent Alpha" size="lg" />
|
||||
<Identity name="Alex Morgan" size="sm" />
|
||||
<Identity name="Alex Morgan" />
|
||||
<Identity name="Alex Morgan" size="lg" />
|
||||
</div>
|
||||
</SubSection>
|
||||
|
||||
<SubSection title="Initials derivation">
|
||||
<div className="flex flex-col gap-2">
|
||||
<Identity name="CEO Agent" size="sm" />
|
||||
<Identity name="Casey Jordan" size="sm" />
|
||||
<Identity name="Alpha" size="sm" />
|
||||
<Identity name="Quality Assurance Lead" size="sm" />
|
||||
<Identity name="Quinn Lee" size="sm" />
|
||||
</div>
|
||||
</SubSection>
|
||||
|
||||
|
|
|
|||
|
|
@ -20,6 +20,7 @@ const config: StorybookConfig = {
|
|||
viteFinal: async (baseConfig) =>
|
||||
mergeConfig(baseConfig, {
|
||||
plugins: [tailwindcss()],
|
||||
server: { proxy: { "/api/agent-avatars": { target: process.env.PAPERCLIP_STORYBOOK_API_URL ?? "http://localhost:3100", changeOrigin: true } } },
|
||||
optimizeDeps: { include: ["motion/react", "react", "react-dom"] },
|
||||
resolve: {
|
||||
// Storybook's core and the react-vite builder each resolve their own
|
||||
|
|
|
|||
|
|
@ -0,0 +1,130 @@
|
|||
import { agentAvatarUrl } from "@paperclipai/shared";
|
||||
import { expect, waitFor } from "storybook/test";
|
||||
import { useEffect, useRef, useState } from "react";
|
||||
import type { Meta, StoryObj } from "@storybook/react-vite";
|
||||
import { AGENT_PALETTE_IDS, AGENT_AVATAR_SIZES, CHARACTER_STATES, appearanceForPalette, type CharacterState } from "@paperclipai/shared";
|
||||
import { AgentAvatar, avatarSizeClasses } from "../../src/components/AgentAvatar";
|
||||
import { AgentCharacter } from "../../src/components/AgentCharacter";
|
||||
import { AgentIdentity } from "../../src/components/AgentIdentity";
|
||||
import { Button } from "../../src/components/ui/button";
|
||||
|
||||
const appearance = appearanceForPalette("bubblegum-sky");
|
||||
const agent = { id: "storybook-agent", name: "Chief of Staff", appearance };
|
||||
const meta = {
|
||||
title: "Agents/Personas",
|
||||
component: AgentCharacter,
|
||||
args: { appearance, size: 256, state: "listening", label: "Chief of Staff" },
|
||||
parameters: { docs: { description: { component: "Persistent cap-v1 identities. Avatars request on-demand PNGs from Paperclip; the hero alone loads ClipLab. Run Paperclip locally, or set PAPERCLIP_STORYBOOK_API_URL to an isolated API when starting Storybook. Static Storybook hosting must route /api/agent-avatars to Paperclip. No images are baked into Storybook." } } },
|
||||
argTypes: {
|
||||
state: { control: "select", options: CHARACTER_STATES },
|
||||
size: { control: "select", options: AGENT_AVATAR_SIZES },
|
||||
motion: { control: "select", options: ["auto", "still"] },
|
||||
trackingScope: { control: "select", options: ["region", "page"] },
|
||||
},
|
||||
} satisfies Meta<typeof AgentCharacter>;
|
||||
export default meta;
|
||||
type Story = StoryObj<typeof meta>;
|
||||
export const LiveCharacter: Story = {};
|
||||
export const ReducedMotion: Story = { args: { motion: "still" } };
|
||||
export const GrayBeforeConnection: Story = { args: { muted: true, state: "sleepy" } };
|
||||
export const Palettes: Story = {
|
||||
render: () => <div className="grid grid-cols-4 gap-6">{AGENT_PALETTE_IDS.map(palette => <div key={palette} className="flex flex-col items-center gap-2">
|
||||
<AgentAvatar appearance={appearanceForPalette(palette)} size={96} label={palette} /><span className="text-xs">{palette}</span>
|
||||
</div>)}</div>,
|
||||
};
|
||||
export const Sizes: Story = {
|
||||
render: () => <div className="flex flex-wrap items-end gap-4">{AGENT_AVATAR_SIZES.map(size => <div key={size} className="flex flex-col items-center gap-2">
|
||||
<AgentAvatar agent={agent} size={size} /><span className="text-xs">{size} px</span>
|
||||
</div>)}</div>,
|
||||
};
|
||||
export const Expressions: Story = {
|
||||
render: () => <div className="grid grid-cols-3 gap-4">{CHARACTER_STATES.map(state => <div key={state} className="flex flex-col items-center gap-2">
|
||||
<AgentCharacter appearance={appearance} state={state} motion="still" size={128} /><span className="text-xs">{state}</span>
|
||||
</div>)}</div>,
|
||||
};
|
||||
export const LightAndDark: Story = {
|
||||
render: () => <div className="flex gap-4">{["light", "dark"].map(theme => <div key={theme} className={`${theme} bg-background p-6 text-foreground`}>
|
||||
<AgentIdentity agent={agent} /><AgentAvatar agent={agent} size={128} />
|
||||
</div>)}</div>,
|
||||
};
|
||||
export const StaticAndLive: Story = {
|
||||
render: () => <div className="flex items-center gap-8"><AgentAvatar agent={agent} pose="listening" size={256} /><AgentCharacter agent={agent} state="listening" /></div>,
|
||||
};
|
||||
function Onboarding() {
|
||||
const [state, setState] = useState<CharacterState>("sleepy");
|
||||
const region = useRef<HTMLDivElement>(null);
|
||||
return <div ref={region} className="flex max-w-lg flex-col items-center gap-4 p-6">
|
||||
<AgentCharacter appearance={appearance} state={state} muted={state === "sleepy" || state === "loading"} trackingRegion={region} />
|
||||
<h2 className="text-lg font-semibold">{state === "success" ? "Ready to work" : "Connect your agent"}</h2>
|
||||
<div className="flex gap-2"><Button onClick={() => setState("loading")}>Connecting</Button><Button onClick={() => setState("success")}>Connection succeeded</Button><Button variant="outline" onClick={() => setState("sleepy")}>Reset</Button></div>
|
||||
</div>;
|
||||
}
|
||||
export const OnboardingTransition: Story = { render: () => <Onboarding /> };
|
||||
export const AppPlacements: Story = {
|
||||
render: () => <div className="grid max-w-3xl gap-6">
|
||||
<section className="space-y-3"><h2 className="text-lg font-semibold">Agents</h2>{AGENT_PALETTE_IDS.slice(0, 4).map((palette, i) => <div className="flex items-center justify-between gap-4" key={palette}>
|
||||
<AgentIdentity agent={{ id: palette, name: ["Chief of Staff", "Researcher", "Designer", "Engineer"][i], appearance: appearanceForPalette(palette) }} /><span className="text-xs text-muted-foreground">Idle</span>
|
||||
</div>)}</section>
|
||||
<section className="space-y-2"><h2 className="text-lg font-semibold">Task conversation</h2><AgentIdentity agent={agent} size="sm" /><p className="text-sm">The launch brief is ready for review.</p></section>
|
||||
<section className="flex items-center justify-between gap-4"><span className="text-sm">Prepare the launch brief</span><AgentAvatar agent={agent} size={20} label="Assigned to Chief of Staff" /></section>
|
||||
<section className="flex items-center gap-6"><AgentCharacter agent={agent} size={128} /><div><h2 className="text-lg font-semibold">Agent configuration</h2><p className="text-sm text-muted-foreground">Chief of Staff · Connected</p></div></section>
|
||||
</div>,
|
||||
};
|
||||
export const FiveHundredStaticAvatars: Story = {
|
||||
render: () => <div className="grid grid-cols-10 gap-2">{Array.from({ length: 500 }, (_, i) => <AgentAvatar key={i} appearance={appearanceForPalette(AGENT_PALETTE_IDS[i % AGENT_PALETTE_IDS.length])} size={24} />)}</div>,
|
||||
};
|
||||
export const LegacyIdentity: Story = { render: () => <AgentIdentity agent={{ id: "legacy-agent", name: "Existing agent" }} /> };
|
||||
function Lifecycle() {
|
||||
const [mounted, setMounted] = useState(true);
|
||||
return <div className="space-y-4"><Button onClick={() => setMounted(value => !value)}>Toggle character</Button>{mounted && <AgentCharacter agent={agent} />}</div>;
|
||||
}
|
||||
export const MountAndUnmount: Story = { render: () => <Lifecycle /> };
|
||||
export const OneLiveRenderer: Story = { render: () => <div className="flex gap-6"><AgentCharacter agent={agent} /><AgentCharacter appearance={appearanceForPalette("arctic-blue")} /></div> };
|
||||
export const ImageFailure: Story = {
|
||||
render: () => <AgentAvatar appearance={appearance} name="Chief of Staff" size={64} />,
|
||||
play: async ({ canvasElement }) => {
|
||||
// Exercise the actual image error handler without a custom product URL API.
|
||||
canvasElement.querySelector("img")?.dispatchEvent(new Event("error"));
|
||||
await waitFor(() => expect(canvasElement.textContent).toContain("CS"));
|
||||
},
|
||||
};
|
||||
export const CacheMissLoading: Story = {
|
||||
render: () => <div className="flex items-center gap-4"><AgentAvatar agent={agent} size={64} /><span className="text-sm">The image slot keeps its dimensions while Paperclip renders a cold cache entry.</span></div>,
|
||||
};
|
||||
function WebGLFailure() {
|
||||
const region = useRef<HTMLDivElement>(null);
|
||||
return <div ref={region} className="space-y-4"><AgentCharacter agent={agent} />
|
||||
<Button onClick={() => region.current?.querySelector("canvas")?.dispatchEvent(new Event("webglcontextlost", { cancelable: true }))}>Simulate WebGL loss</Button>
|
||||
</div>;
|
||||
}
|
||||
export const RenderFailure: Story = { render: () => <WebGLFailure /> };
|
||||
export const PointerScope: Story = {
|
||||
render: () => <div className="flex items-start gap-8"><Onboarding /><p className="text-sm">Pointer movement outside the connection panel does not move the character. Touch and reduced motion disable tracking.</p></div>,
|
||||
};
|
||||
|
||||
// A seeked WebGL frame lets Linux visual tests compare the identical front-facing
|
||||
// sample with the server SVG/raster path. It schedules no animation frames.
|
||||
type SnapshotPairProps = { size?: typeof AGENT_AVATAR_SIZES[number]; state?: CharacterState; density?: 1 | 2 };
|
||||
function SnapshotPair({ size = 256, state = "rest", density = 2 }: SnapshotPairProps) {
|
||||
const host = useRef<HTMLSpanElement>(null);
|
||||
useEffect(() => {
|
||||
let disposed = false;
|
||||
let cleanup: (() => void) | undefined;
|
||||
void Promise.all([import("@paperclipai/shared/cliplab/renderer"), import("@paperclipai/shared/cliplab/definition")]).then(([{ CharacterRenderer }, library]) => {
|
||||
if (disposed || !host.current) return;
|
||||
const canvas = document.createElement("canvas");
|
||||
canvas.className = "size-full";
|
||||
const renderer = new CharacterRenderer(canvas, { width: size, height: size, displaySize: size, pixelRatio: density });
|
||||
const definition = library.characterDefinition(appearance);
|
||||
renderer.render({ ...definition.character, trueFront: true, lockPosition: true, followCursor: false, followRotation: false }, library.characterStill(definition, state), { rotation: { x: 0, y: 0, z: 0 } });
|
||||
host.current.appendChild(canvas);
|
||||
cleanup = () => { renderer.dispose(); canvas.remove(); };
|
||||
});
|
||||
return () => { disposed = true; cleanup?.(); };
|
||||
}, [size, state, density]);
|
||||
return <div className="flex items-start gap-8">
|
||||
<div className="space-y-3"><span data-testid="static-frame" className={`inline-block ${avatarSizeClasses[size]}`}><img src={agentAvatarUrl(appearance, size, density, state)} width={size} height={size} className="size-full object-contain" alt="Static PNG portrait" /></span><p className="text-xs text-muted-foreground">PNG · {size * density} × {size * density}</p></div>
|
||||
<div className="space-y-3"><span data-testid="live-frame" ref={host} className={`inline-block ${avatarSizeClasses[size]}`} /><p className="text-xs text-muted-foreground">WebGL · {size * density} × {size * density}</p></div>
|
||||
</div>;
|
||||
}
|
||||
export const SnapshotAgreement = { args: { size: 256, state: "rest", density: 2 }, argTypes: { density: { control: "inline-radio", options: [1, 2] } }, render: (args: SnapshotPairProps) => <SnapshotPair {...args} /> } satisfies StoryObj<SnapshotPairProps>;
|
||||
Loading…
Reference in New Issue