feat: add agent skills system with list_skills and load_skill tools
Adds a skill discovery and loading system that lets the LLM agent autonomously find and follow pre-built editing recipes. Skills are exposed as two new tools: list_skills (search/discover) and load_skill (get full instructions). Built-in skills: - viral-short: high-retention TikTok/Reels/Shorts workflow with hook, retention cuts, zoom effects, captions, and CTA - pitch-video: professional pitch/explainer with 5-act structure (opening, problem, solution, metrics, CTA)
This commit is contained in:
Luis Esteban Acevedo Ladino
parent
bf354642ba
commit
effdc28237
|
|
@ -0,0 +1,218 @@
|
|||
import type { SkillDefinition } from "../types";
|
||||
import { skillRegistry } from "../registry";
|
||||
|
||||
const pitchVideoSkill: SkillDefinition = {
|
||||
id: "pitch-video",
|
||||
name: "Pitch Video",
|
||||
description:
|
||||
"Creates a professional pitch or explainer video with clean structure: problem statement, solution, key benefits with text cards, and call-to-action. Ideal for startups, product demos, and investor presentations.",
|
||||
keywords: [
|
||||
"pitch",
|
||||
"investor",
|
||||
"presentation",
|
||||
"startup",
|
||||
"explainer",
|
||||
"product demo",
|
||||
"elevator pitch",
|
||||
"pitch deck",
|
||||
"proposal",
|
||||
],
|
||||
author: "system",
|
||||
instructions: `You are operating in PITCH VIDEO mode. Your goal is to transform raw footage into a polished, professional pitch or explainer video with clean structure, elegant text overlays, and a clear narrative arc.
|
||||
|
||||
## MANDATORY PRE-FLIGHT
|
||||
|
||||
Before making ANY edit, you MUST:
|
||||
1. Call list_project_assets to discover available media and its duration
|
||||
2. Call list_timeline to understand the current state
|
||||
3. If the user references visual content, call load_context to analyze the footage
|
||||
|
||||
## STRUCTURE
|
||||
|
||||
A pitch video follows this EXACT 5-act structure. Do NOT skip any section:
|
||||
|
||||
### ACT 1: OPENING (0-5 seconds)
|
||||
Professional cold open that immediately establishes credibility.
|
||||
|
||||
**Steps:**
|
||||
1. If the raw footage has a natural opening, use it. If not, find the most professional-looking segment
|
||||
2. Add opening title text with add_text:
|
||||
- text: The project/product/company name (ask the user if not clear from context)
|
||||
- style: "hook"
|
||||
- position: "center"
|
||||
- start: 0.5
|
||||
- end: 3.5
|
||||
- color: "#FFFFFF"
|
||||
- background: { enabled: true, color: "#1a1a2e", cornerRadius: 10, padding: 16 }
|
||||
3. Add a tagline or one-liner below:
|
||||
- text: One sentence that captures the value proposition
|
||||
- style: "label"
|
||||
- position: "bottom"
|
||||
- start: 1
|
||||
- end: 4
|
||||
- color: "#E0E0E0"
|
||||
- background: { enabled: false }
|
||||
|
||||
### ACT 2: THE PROBLEM (5s - 20s)
|
||||
Clearly articulate the pain point. This builds empathy and urgency.
|
||||
|
||||
**Steps:**
|
||||
1. Select footage that shows the problem or the user speaking about it
|
||||
2. Split at appropriate boundaries to isolate this segment (5s to 20s)
|
||||
3. Add a "THE PROBLEM" section header:
|
||||
- text: "THE PROBLEM" (or user's preferred phrasing)
|
||||
- style: "label"
|
||||
- position: "top"
|
||||
- start: 5
|
||||
- end: 7
|
||||
- color: "#FF6B6B"
|
||||
- background: { enabled: true, color: "#1a1a2e", cornerRadius: 6, padding: 10 }
|
||||
4. Add supporting text cards that highlight key pain points (1 card per 3-4 seconds):
|
||||
- Each card: max 8 words, one key statistic or pain point per card
|
||||
- style: "subtitle"
|
||||
- position: "bottom"
|
||||
- color: "#FFFFFF"
|
||||
- background: { enabled: true, color: "rgba(0,0,0,0.7)", cornerRadius: 6, padding: 10 }
|
||||
|
||||
### ACT 3: THE SOLUTION (20s - 40s)
|
||||
Present the product/solution with clarity and confidence.
|
||||
|
||||
**Steps:**
|
||||
1. Transition to solution footage (split at 20s boundary)
|
||||
2. Add a "THE SOLUTION" section header:
|
||||
- text: "THE SOLUTION" (or "INTRODUCING [product name]")
|
||||
- style: "hook"
|
||||
- position: "center"
|
||||
- start: 20
|
||||
- end: 23
|
||||
- color: "#FFFFFF"
|
||||
- background: { enabled: true, color: "#0f3460", cornerRadius: 10, padding: 14 }
|
||||
3. Add 2-3 benefit text cards (one every 5-7 seconds):
|
||||
- Each card states ONE clear benefit, max 8 words
|
||||
- style: "label"
|
||||
- position: "bottom"
|
||||
- color: "#FFFFFF"
|
||||
- background: { enabled: true, color: "rgba(15,52,96,0.85)", cornerRadius: 6, padding: 10 }
|
||||
4. For each benefit card, add a subtle scale animation via upsert_keyframe:
|
||||
- propertyPath: "transform.scaleX"
|
||||
- time 0: value 0.95, interpolation "bezier"
|
||||
- time 0.3: value 1.0, interpolation "linear"
|
||||
- Repeat for "transform.scaleY" with same values
|
||||
This creates a professional "pop-in" effect for each text card
|
||||
|
||||
### ACT 4: KEY NUMBERS (40s - 50s)
|
||||
Social proof and data build trust.
|
||||
|
||||
**Steps:**
|
||||
1. Add 1-2 text cards with impressive metrics (user should provide these, or infer from context):
|
||||
- Examples: "10x faster", "50% cost reduction", "Used by 10,000+ teams"
|
||||
- style: "hook"
|
||||
- position: "center"
|
||||
- color: "#4ECDC4"
|
||||
- background: { enabled: true, color: "#1a1a2e", cornerRadius: 10, padding: 14 }
|
||||
- Each metric card: 3-4 seconds
|
||||
2. Add scale-up animation on these cards via upsert_keyframe:
|
||||
- propertyPath: "transform.scaleX"
|
||||
- time 0: value 0.8
|
||||
- time 0.4: value 1.05
|
||||
- time 0.5: value 1.0
|
||||
- interpolation: "bezier"
|
||||
- Repeat for "transform.scaleY"
|
||||
|
||||
### ACT 5: THE CTA (last 5 seconds)
|
||||
End with a clear, confident call-to-action.
|
||||
|
||||
**Steps:**
|
||||
1. Split 5 seconds before the end
|
||||
2. Add CTA text:
|
||||
- text: "Let's talk" or "Book a demo" or "Invest in [name]"
|
||||
- style: "hook"
|
||||
- position: "center"
|
||||
- start: (end - 4.5)
|
||||
- end: (end - 0.5)
|
||||
- color: "#FFFFFF"
|
||||
- background: { enabled: true, color: "#0f3460", cornerRadius: 10, padding: 14 }
|
||||
3. Add contact info text below:
|
||||
- text: Website URL or email (ask user if not provided)
|
||||
- style: "plain"
|
||||
- position: "bottom"
|
||||
- start: (end - 4)
|
||||
- end: (end - 0.5)
|
||||
- color: "#CCCCCC"
|
||||
- background: { enabled: false }
|
||||
|
||||
## TIMING RULES
|
||||
|
||||
- Ideal total duration: 60-90 seconds (can extend to 120s for investor pitches)
|
||||
- Opening: 0-5s
|
||||
- Problem: 5-20s (15 seconds max)
|
||||
- Solution: 20-40s (20 seconds max)
|
||||
- Key Numbers: 40-50s (10 seconds)
|
||||
- CTA: last 5 seconds
|
||||
- Each text card: 3-5 seconds
|
||||
- No gaps — every second should have either footage or a text element
|
||||
|
||||
## TEXT STYLE GUIDE (professional, NOT flashy)
|
||||
|
||||
This is a PITCH. Clean, confident, professional. No gimmicks.
|
||||
|
||||
- Section headers ("THE PROBLEM", "THE SOLUTION"): style "label", position "top" or "center", dark navy background (#1a1a2e or #0f3460)
|
||||
- Benefit/statistic cards: style "label" or "hook" (for emphasis), position "bottom" or "center"
|
||||
- CTA: style "hook", center, confident navy background
|
||||
- Color palette:
|
||||
- Primary text: #FFFFFF (white)
|
||||
- Emphasis text: #4ECDC4 (teal) for key numbers
|
||||
- Problem text: #FF6B6B (coral) for pain points only
|
||||
- Backgrounds: #1a1a2e (dark navy), #0f3460 (navy blue), rgba(0,0,0,0.7) (semi-transparent black)
|
||||
- Subtitle text: #E0E0E0 (light gray) for secondary info
|
||||
- ALL text must have a background. NO floating text without a backing shape
|
||||
- Font weight: "bold" for headers and numbers, "normal" for subtitle lines
|
||||
- NEVER use more than 10 words per text card
|
||||
- NEVER use flashy colors (no neon, no yellow, no red except for problem section)
|
||||
|
||||
## EFFECTS USAGE (minimal and professional)
|
||||
|
||||
Pitch videos use effects to ENHANCE clarity, never to distract:
|
||||
|
||||
- sharpen: Apply to the FULL video duration for crisp visuals, params: { amount: 0.25 }
|
||||
- vignette: Optional, apply once during the key numbers section for subtle focus, params: { intensity: 0.2 }
|
||||
- brightness-contrast: If footage looks flat, apply across the full duration, params: { brightness: 5, contrast: 10 }
|
||||
|
||||
NEVER use:
|
||||
- glitch, pixelate, wave, chromatic-aberration (too chaotic for professional context)
|
||||
- glow, halftone, scanlines (too flashy)
|
||||
- sepia, duotone, cross-process (changes color perception — misleading for pitch)
|
||||
|
||||
## TRANSITION PATTERN
|
||||
|
||||
Between each act, use a subtle opacity fade on the section header:
|
||||
- upsert_keyframe on the text element's "opacity" propertyPath:
|
||||
- time 0: value 0 (invisible)
|
||||
- time 0.3: value 1 (fully visible)
|
||||
- time (duration - 0.3): value 1 (still visible)
|
||||
- time (duration): value 0 (faded out)
|
||||
- interpolation: "bezier"
|
||||
|
||||
This gives a professional fade-in/fade-out on each text card.
|
||||
|
||||
## QUALITY CHECKLIST
|
||||
|
||||
Before finishing, verify:
|
||||
- [ ] Opening title with product/project name at 0.5-3.5s
|
||||
- [ ] "THE PROBLEM" section header exists with coral accent
|
||||
- [ ] At least 2 pain point text cards in the problem section
|
||||
- [ ] "THE SOLUTION" section header with navy background
|
||||
- [ ] At least 2 benefit text cards in the solution section
|
||||
- [ ] At least 1 key metric/number with teal emphasis
|
||||
- [ ] CTA text in the last 5 seconds with navy background
|
||||
- [ ] Contact info text below CTA
|
||||
- [ ] All text has backgrounds (no floating text)
|
||||
- [ ] Effects are limited to sharpen/vignette/brightness only
|
||||
- [ ] Fade-in/fade-out animations on section headers
|
||||
- [ ] Total duration is between 60-120 seconds
|
||||
- [ ] Color palette is consistent (navy backgrounds, white/teal/coral text)
|
||||
|
||||
If any checklist item fails, fix it before responding to the user.`,
|
||||
};
|
||||
|
||||
skillRegistry.register(pitchVideoSkill.id, pitchVideoSkill);
|
||||
|
|
@ -0,0 +1,152 @@
|
|||
import type { SkillDefinition } from "../types";
|
||||
import { skillRegistry } from "../registry";
|
||||
|
||||
const viralShortSkill: SkillDefinition = {
|
||||
id: "viral-short",
|
||||
name: "Viral Short",
|
||||
description:
|
||||
"Edits a viral-style short video (TikTok, Reels, Shorts) with hook text, retention cuts, zoom effects, captions, and CTA. Optimized for maximum engagement under 60 seconds.",
|
||||
keywords: [
|
||||
"viral",
|
||||
"short",
|
||||
"tiktok",
|
||||
"reels",
|
||||
"shorts",
|
||||
"hook",
|
||||
"trending",
|
||||
"engagement",
|
||||
"viral video",
|
||||
"short video",
|
||||
],
|
||||
author: "system",
|
||||
instructions: `You are operating in VIRAL SHORT mode. Your goal is to transform raw footage into a high-retention short-form video optimized for TikTok, Instagram Reels, and YouTube Shorts.
|
||||
|
||||
## MANDATORY PRE-FLIGHT
|
||||
|
||||
Before making ANY edit, you MUST:
|
||||
1. Call list_project_assets to discover available media and its duration
|
||||
2. Call list_timeline to understand the current state
|
||||
3. If the user references visual content, call load_context to analyze the footage
|
||||
|
||||
## STRUCTURE
|
||||
|
||||
Every viral short follows this EXACT structure. Do NOT skip any section:
|
||||
|
||||
### SECTION 1: THE HOOK (0-3 seconds)
|
||||
This is the most critical part. If the viewer scrolls past 3 seconds, the video is dead.
|
||||
|
||||
**Steps:**
|
||||
1. Identify the most visually striking or surprising moment in the footage
|
||||
2. If the raw video starts slow, call split at 3 seconds from the best moment, then delete_timeline_elements for everything before it, then move_timeline_elements so the hook starts at 0
|
||||
3. Add hook text with add_text:
|
||||
- text: A punchy, curiosity-driving phrase (max 6 words). Examples: "Wait for it...", "Nobody talks about this", "This changed everything"
|
||||
- style: "hook" (bold, large font)
|
||||
- position: "center"
|
||||
- start: 0
|
||||
- end: 2.5 (do NOT exceed 3 seconds)
|
||||
- color: "#FFFFFF"
|
||||
- background: { enabled: true, color: "#000000", cornerRadius: 8, padding: 12 }
|
||||
|
||||
### SECTION 2: THE CONTENT (3s to end-3s)
|
||||
This is the meat. Keep it FAST and TIGHT.
|
||||
|
||||
**Step 1: Remove dead air**
|
||||
- Split the video at every point where there is silence or filler content
|
||||
- Delete those segments using split + delete_timeline_elements
|
||||
- Aim for average shot length of 3-5 seconds. If a clip is longer than 6 seconds without visual change, cut it
|
||||
|
||||
**Step 2: Add retention captions**
|
||||
- For each spoken segment, add caption text at the bottom:
|
||||
- style: "subtitle"
|
||||
- position: "bottom"
|
||||
- color: "#FFFFFF"
|
||||
- background: { enabled: true, color: "#000000", cornerRadius: 6, padding: 8 }
|
||||
- Each caption should be 2-3 seconds long, max 5 words per card
|
||||
- If a sentence is longer, split it across multiple add_text calls with sequential timing
|
||||
|
||||
**Step 3: Retention zoom effects (CRITICAL for algorithm)**
|
||||
Every 4-6 seconds, add a subtle zoom to prevent scrolling:
|
||||
- Call upsert_keyframe on the current main clip element:
|
||||
- propertyPath: "transform.scaleX"
|
||||
- At the start of a segment: value 1.0
|
||||
- At the end of a segment: value 1.05 (subtle, not disorienting)
|
||||
- interpolation: "bezier" (smooth)
|
||||
- Repeat for "transform.scaleY" with the same values
|
||||
|
||||
**Step 4: Visual emphasis effects**
|
||||
Add these effects at key moments (not everywhere — 2-3 times max):
|
||||
- For "wow" moments: apply_effect with effectType "glow" at that specific time range, params: { intensity: 0.3, radius: 5 }
|
||||
- For transitions between topics: apply_effect with effectType "pixelate" for 0.3s as a transition, params: { size: 8 }
|
||||
- For dramatic pauses: apply_effect with effectType "vignette", params: { intensity: 0.4 }
|
||||
|
||||
### SECTION 3: THE CTA (last 2-3 seconds)
|
||||
End with a call-to-action that drives engagement.
|
||||
|
||||
**Steps:**
|
||||
1. Split the video 2.5 seconds before the end
|
||||
2. Add CTA text with add_text:
|
||||
- text: "Follow for more" or "Like if this helped" or topic-relevant CTA
|
||||
- style: "hook"
|
||||
- position: "center"
|
||||
- start: (end - 2.5)
|
||||
- end: (video end)
|
||||
- color: "#FFFFFF"
|
||||
- background: { enabled: true, color: "#000000", cornerRadius: 8, padding: 12 }
|
||||
3. Add a final zoom pulse via upsert_keyframe:
|
||||
- propertyPath: "transform.scaleX" — go from 1.0 to 1.08 over the CTA duration
|
||||
- Same for "transform.scaleY"
|
||||
|
||||
## TIMING RULES
|
||||
|
||||
- Total duration MUST be under 60 seconds (ideal: 30-45 seconds)
|
||||
- Hook: 0-3s, no exceptions
|
||||
- Each caption card: 2-3 seconds
|
||||
- CTA: last 2-3 seconds
|
||||
- No gaps between elements — everything must be continuous
|
||||
- If the raw footage is over 60s, aggressively cut. Keep only the best 45s of content
|
||||
|
||||
## TEXT STYLE GUIDE
|
||||
|
||||
- Hook text: style "hook", center, white on black rounded background
|
||||
- Captions: style "subtitle", bottom, white on semi-transparent black
|
||||
- CTA text: style "hook", center, white on black
|
||||
- NEVER use plain text without background for any on-screen text
|
||||
- Font weight must always be "bold" for all text elements
|
||||
- NEVER add more than 6 words per text element
|
||||
|
||||
## EFFECTS USAGE GUIDE
|
||||
|
||||
Use effects SPARINGLY. A viral short should feel dynamic, not chaotic:
|
||||
- glow: Use 1-2 times for emphasis moments (0.3s each)
|
||||
- vignette: Use once for a dramatic moment (2-3s)
|
||||
- pixelate: Use once as a transition between sections (0.3s)
|
||||
- sharpen: Optionally apply to the full video for clarity (params: { amount: 0.3 })
|
||||
- NEVER use: blur (makes it unreadable), invert, posterize (too distracting for short form)
|
||||
|
||||
## ZOOM PATTERN (the secret sauce)
|
||||
|
||||
This is what makes shorts feel dynamic without jump cuts:
|
||||
|
||||
For each content segment (every 4-6 seconds):
|
||||
1. upsert_keyframe: propertyPath "transform.scaleX", time at segment start, value 1.0, interpolation "linear"
|
||||
2. upsert_keyframe: propertyPath "transform.scaleX", time at segment end, value 1.04-1.06, interpolation "bezier"
|
||||
3. Repeat identically for "transform.scaleY"
|
||||
|
||||
The zoom should be imperceptible to the conscious eye but felt by the viewer. Total zoom range: NEVER exceed 1.10.
|
||||
|
||||
## QUALITY CHECKLIST
|
||||
|
||||
Before finishing, verify:
|
||||
- [ ] Hook text is present at 0-2.5s with bold style and background
|
||||
- [ ] Total duration is under 60 seconds
|
||||
- [ ] At least 2 retention zooms exist on the main content
|
||||
- [ ] Captions are present for spoken content (subtitle style, bottom)
|
||||
- [ ] CTA text is present in the last 2-3 seconds
|
||||
- [ ] No gaps exist between timeline elements
|
||||
- [ ] No single text element exceeds 6 words
|
||||
- [ ] Effects are used sparingly (max 3 total)
|
||||
|
||||
If any checklist item fails, fix it before responding to the user.`,
|
||||
};
|
||||
|
||||
skillRegistry.register(viralShortSkill.id, viralShortSkill);
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
/**
|
||||
* Client-only barrel for agent skills. Importing this file registers every
|
||||
* built-in skill with `skillRegistry` via side effects.
|
||||
*
|
||||
* To add a new skill: create a `*.skill.ts` file in builtin/, then add
|
||||
* a single side-effect import below.
|
||||
*/
|
||||
|
||||
import "@/agent/skills/builtin/viral-short.skill";
|
||||
import "@/agent/skills/builtin/pitch-video.skill";
|
||||
|
|
@ -0,0 +1,47 @@
|
|||
import type { AgentContext, ToolDefinition } from "@/agent/types";
|
||||
import { toolRegistry } from "@/agent/tools/registry";
|
||||
import { listSkillsSchema } from "@/agent/tools/schemas";
|
||||
import { skillRegistry } from "@/agent/skills/registry";
|
||||
import "@/agent/skills";
|
||||
|
||||
type ListSkillsResult = {
|
||||
skills: Array<{
|
||||
id: string;
|
||||
name: string;
|
||||
description: string;
|
||||
keywords: string[];
|
||||
}>;
|
||||
};
|
||||
|
||||
const listSkillsTool: ToolDefinition = {
|
||||
...listSkillsSchema,
|
||||
execute: async (
|
||||
args: Record<string, unknown>,
|
||||
_context: AgentContext,
|
||||
): Promise<ListSkillsResult> => {
|
||||
const allSkills = skillRegistry.getAll();
|
||||
const query =
|
||||
typeof args.query === "string" ? args.query.trim().toLowerCase() : "";
|
||||
|
||||
const skills = allSkills
|
||||
.filter((skill) => {
|
||||
if (!query) return true;
|
||||
const nameMatch = skill.name.toLowerCase().includes(query);
|
||||
const descMatch = skill.description.toLowerCase().includes(query);
|
||||
const keywordMatch = skill.keywords.some((kw) =>
|
||||
kw.toLowerCase().includes(query),
|
||||
);
|
||||
return nameMatch || descMatch || keywordMatch;
|
||||
})
|
||||
.map((skill) => ({
|
||||
id: skill.id,
|
||||
name: skill.name,
|
||||
description: skill.description,
|
||||
keywords: skill.keywords,
|
||||
}));
|
||||
|
||||
return { skills };
|
||||
},
|
||||
};
|
||||
|
||||
toolRegistry.register(listSkillsSchema.name, listSkillsTool);
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
import type { AgentContext, ToolDefinition } from "@/agent/types";
|
||||
import { toolRegistry } from "@/agent/tools/registry";
|
||||
import { loadSkillSchema } from "@/agent/tools/schemas";
|
||||
import { skillRegistry } from "@/agent/skills/registry";
|
||||
import "@/agent/skills";
|
||||
|
||||
type LoadSkillResult =
|
||||
| {
|
||||
skillId: string;
|
||||
name: string;
|
||||
instructions: string;
|
||||
}
|
||||
| { error: string };
|
||||
|
||||
const loadSkillTool: ToolDefinition = {
|
||||
...loadSkillSchema,
|
||||
execute: async (
|
||||
args: Record<string, unknown>,
|
||||
_context: AgentContext,
|
||||
): Promise<LoadSkillResult> => {
|
||||
const skillId = args.skillId;
|
||||
if (typeof skillId !== "string" || !skillId.trim()) {
|
||||
return { error: "skillId must be a non-empty string" };
|
||||
}
|
||||
|
||||
if (!skillRegistry.has(skillId)) {
|
||||
const available = skillRegistry
|
||||
.getAll()
|
||||
.map((s) => s.id)
|
||||
.join(", ");
|
||||
return {
|
||||
error: `Skill not found: ${skillId}. Available skills: ${available}`,
|
||||
};
|
||||
}
|
||||
|
||||
const skill = skillRegistry.get(skillId);
|
||||
return {
|
||||
skillId: skill.id,
|
||||
name: skill.name,
|
||||
instructions: skill.instructions,
|
||||
};
|
||||
},
|
||||
};
|
||||
|
||||
toolRegistry.register(loadSkillSchema.name, loadSkillTool);
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
import { DefinitionRegistry } from "@/lib/registry";
|
||||
import type { SkillDefinition } from "./types";
|
||||
|
||||
export const skillRegistry = new DefinitionRegistry<string, SkillDefinition>(
|
||||
"skill",
|
||||
);
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
export interface SkillDefinition {
|
||||
id: string;
|
||||
name: string;
|
||||
description: string;
|
||||
keywords: string[];
|
||||
instructions: string;
|
||||
author: "system" | "user";
|
||||
}
|
||||
|
|
@ -45,6 +45,7 @@ export function buildSystemPrompt(
|
|||
"Only claim edits that were actually performed by tool calls in this conversation. Do not say you added, removed, cleaned, or updated text/subtitles unless the relevant add_text, update_text, delete_timeline_elements, or update_timeline_element_timing tool call succeeded.",
|
||||
"When the user asks to add titles, hooks, labels, captions, subtitles, or visible text, call add_text. Do not add text proactively for unrelated edit requests such as cutting silence unless the user asks for text.",
|
||||
"When you need to perform an action matching one of these tools, call the appropriate tool. For all other requests, respond directly in plain text.",
|
||||
"SKILLS: You have editing skills available — pre-built recipe workflows for common editing patterns. When the user asks for a complex edit (viral video, pitch, specific style, etc.), call list_skills to discover relevant skills, then call load_skill with the matching skillId to get full step-by-step instructions. Follow those instructions precisely using the standard editing tools. If the user's request doesn't match any skill, proceed with your own editing approach.",
|
||||
);
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -36,3 +36,5 @@ import "@/agent/tools/upsert-keyframe.tool";
|
|||
import "@/agent/tools/remove-keyframe.tool";
|
||||
import "@/agent/tools/update-keyframe-curve.tool";
|
||||
import "@/agent/tools/list-animatable-properties.tool";
|
||||
import "@/agent/skills/list-skills.tool";
|
||||
import "@/agent/skills/load-skill.tool";
|
||||
|
|
|
|||
|
|
@ -291,6 +291,20 @@ export const listAnimatablePropertiesSchema: ToolSchema = {
|
|||
parameters: [{ key: "elementId", type: "string", required: true }],
|
||||
};
|
||||
|
||||
export const listSkillsSchema: ToolSchema = {
|
||||
name: "list_skills",
|
||||
description:
|
||||
"Lists available editing skill recipes — pre-built workflows for common editing patterns like viral shorts, pitch videos, and more. Each skill contains a structured recipe with specific tool sequences, timing, and styling rules. Returns skill id, name, and short description. Use this to discover relevant skills, then call load_skill to get full instructions and apply them.",
|
||||
parameters: [{ key: "query", type: "string", required: false }],
|
||||
};
|
||||
|
||||
export const loadSkillSchema: ToolSchema = {
|
||||
name: "load_skill",
|
||||
description:
|
||||
"Loads the full instructions for a specific skill by its id. After loading, follow the skill's instructions to accomplish the user's request using the standard editing tools (split, add_text, apply_effect, upsert_keyframe, etc.). Each skill contains a complete recipe with sections, timing rules, text styles, effect parameters, and a quality checklist. Always call list_skills first to discover available skills.",
|
||||
parameters: [{ key: "skillId", type: "string", required: true }],
|
||||
};
|
||||
|
||||
/**
|
||||
* The exact list of schemas exposed to the LLM.
|
||||
* Excludes internal-only tools (transcribe_video, mock).
|
||||
|
|
@ -322,4 +336,6 @@ export const providerToolSchemas: ToolSchema[] = [
|
|||
removeKeyframeSchema,
|
||||
updateKeyframeCurveSchema,
|
||||
listAnimatablePropertiesSchema,
|
||||
listSkillsSchema,
|
||||
loadSkillSchema,
|
||||
];
|
||||
|
|
|
|||
Loading…
Reference in New Issue