| 1 | # RFC-03：Agent Skill 设计 |
| 2 | |
| 3 | > **Status**: Draft v0.1 |
| 4 | > **Date**: 2026-05-26 |
| 5 | > **Depends on**: [RFC-01](./2026-05-26-spec-01-engine-adapter.md), [RFC-02](./2026-05-26-spec-02-template-metadata.md) |
| 6 | > **Scope**: 本地 coding agent（Claude Code / Cursor / Codex / Gemini / OpenCode）通过 html-video 做视频的协议、CLI、决策逻辑 |
| 7 | |
| 8 | --- |
| 9 | |
| 10 | ## 顶层架构 |
| 11 | |
| 12 | ``` |
| 13 | ┌─────────────────────────────────────────────┐ |
| 14 | │ coding agent (Claude Code / Cursor / ...) │ |
| 15 | │ 读 SKILL.md → 调 html-video CLI │ |
| 16 | └──────────────────┬──────────────────────────┘ |
| 17 | │ stdio / shell |
| 18 | ┌──────────────────▼──────────────────────────┐ |
| 19 | │ html-video CLI │ |
| 20 | │ ┌────────────────────────────────────┐ │ |
| 21 | │ │ Command router │ │ |
| 22 | │ │ - search-templates │ │ |
| 23 | │ │ - inspect-template │ │ |
| 24 | │ │ - render │ │ |
| 25 | │ │ - preview │ │ |
| 26 | │ │ - list-engines │ │ |
| 27 | │ └────────────┬────────────────────────┘ │ |
| 28 | └────────────────┼──────────────────────────────┘ |
| 29 | │ in-process call |
| 30 | ┌────────────────▼──────────────────────────────┐ |
| 31 | │ @html-video/core │ |
| 32 | │ - Template registry │ |
| 33 | │ - Engine selector (capabilities-based) │ |
| 34 | │ - Render orchestrator │ |
| 35 | └────────────────┬──────────────────────────────┘ |
| 36 | │ dynamic import |
| 37 | ┌────────────────▼──────────────────────────────┐ |
| 38 | │ adapter-hyperframes / adapter-remotion / │ |
| 39 | │ adapter-motion-canvas / adapter-revideo │ |
| 40 | └────────────────────────────────────────────────┘ |
| 41 | ``` |
| 42 | |
| 43 | **关键约束**：agent 只跟 CLI 打交道，**不**直接 import 任何 npm 包，**不**直接调 engine adapter。所有跨语言隔离 + 决策逻辑收敛在 CLI 层。 |
| 44 | |
| 45 | --- |
| 46 | |
| 47 | ## CLI Contract（v0.1） |
| 48 | |
| 49 | ### 全局 flags |
| 50 | |
| 51 | ``` |
| 52 | --json 输出 JSON（agent 默认带） |
| 53 | --no-color 关 ANSI（agent 默认带） |
| 54 | --cwd <path> 项目工作目录（默认 process.cwd()） |
| 55 | --config <path> html-video.config.json 路径（覆盖自动发现） |
| 56 | --verbose 打印决策依据（debug） |
| 57 | ``` |
| 58 | |
| 59 | ### `html-video list-engines` |
| 60 | |
| 61 | 列出本机可用 adapters + 各自 capabilities。 |
| 62 | |
| 63 | ```bash |
| 64 | html-video list-engines --json |
| 65 | ``` |
| 66 | |
| 67 | 输出（节选）： |
| 68 | |
| 69 | ```json |
| 70 | { |
| 71 | "engines": [ |
| 72 | { |
| 73 | "id": "hyperframes", |
| 74 | "name": "Hyperframes", |
| 75 | "installed": true, |
| 76 | "version": "0.4.2", |
| 77 | "capabilities": { ... } |
| 78 | }, |
| 79 | { |
| 80 | "id": "remotion", |
| 81 | "installed": false, |
| 82 | "install_hint": "pnpm add -D remotion @remotion/renderer" |
| 83 | } |
| 84 | ] |
| 85 | } |
| 86 | ``` |
| 87 | |
| 88 | ### `html-video search-templates` |
| 89 | |
| 90 | 按 intent / tags / category 检索。输出已经按 RFC-02 流程排好序。 |
| 91 | |
| 92 | ```bash |
| 93 | html-video search-templates \ |
| 94 | --intent "show our github stars growth over the past 24 months" \ |
| 95 | --aspect 16:9 \ |
| 96 | --license-allow Apache-2.0,MIT,CC0-1.0,CC-BY-4.0 \ |
| 97 | --top 5 \ |
| 98 | --json |
| 99 | ``` |
| 100 | |
| 101 | 输出： |
| 102 | |
| 103 | ```json |
| 104 | { |
| 105 | "matches": [ |
| 106 | { |
| 107 | "id": "data-bar-chart-race", |
| 108 | "name": "Bar Chart Race", |
| 109 | "engine": "motion-canvas", |
| 110 | "engine_installed": true, |
| 111 | "score": 0.91, |
| 112 | "score_reason": "tags=chart,race,data-driven match intent; aspect 16:9 supported; Apache-2.0 ok", |
| 113 | "preview_poster": "/path/to/preview.png", |
| 114 | "best_for": ["GitHub stars race", "Sales ranking", "..."] |
| 115 | }, |
| 116 | { ... } |
| 117 | ], |
| 118 | "filtered_out": [ |
| 119 | { |
| 120 | "id": "logo-reveal-cinematic", |
| 121 | "reason": "category=intro-outro doesn't match intent 'data growth'" |
| 122 | } |
| 123 | ] |
| 124 | } |
| 125 | ``` |
| 126 | |
| 127 | ### `html-video inspect-template` |
| 128 | |
| 129 | 返回某个 template 的完整 metadata，agent 借此了解需要哪些 inputs。 |
| 130 | |
| 131 | ```bash |
| 132 | html-video inspect-template data-bar-chart-race --json |
| 133 | ``` |
| 134 | |
| 135 | 输出 = RFC-02 的 metadata + 一些 runtime info（resolved source path, cached preview URL...） |
| 136 | |
| 137 | ### `html-video render` |
| 138 | |
| 139 | 实际渲染。 |
| 140 | |
| 141 | ```bash |
| 142 | html-video render \ |
| 143 | --template data-bar-chart-race \ |
| 144 | --vars-file vars.json \ |
| 145 | --format mp4 \ |
| 146 | --resolution 1920x1080 \ |
| 147 | --fps 60 \ |
| 148 | --duration auto \ |
| 149 | --output /tmp/out.mp4 \ |
| 150 | --json |
| 151 | ``` |
| 152 | |
| 153 | 输出（streaming JSON 或 final summary）： |
| 154 | |
| 155 | ```json |
| 156 | { |
| 157 | "status": "ok", |
| 158 | "output_path": "/tmp/out.mp4", |
| 159 | "engine": "motion-canvas", |
| 160 | "duration_sec": 14.5, |
| 161 | "render_wall_clock_sec": 28.3, |
| 162 | "file_size_mb": 12.4, |
| 163 | "diagnostics": [] |
| 164 | } |
| 165 | ``` |
| 166 | |
| 167 | 如果 streaming 模式（`--stream-progress`），每秒 emit 一行 NDJSON： |
| 168 | |
| 169 | ```ndjson |
| 170 | {"type":"progress","stage":"preparing","pct":5} |
| 171 | {"type":"progress","stage":"preparing","pct":10} |
| 172 | {"type":"progress","stage":"rendering","pct":12,"frame":108,"total_frames":870} |
| 173 | ... |
| 174 | {"type":"progress","stage":"muxing","pct":97} |
| 175 | {"type":"done","status":"ok","output_path":"/tmp/out.mp4",...} |
| 176 | ``` |
| 177 | |
| 178 | ### `html-video preview` |
| 179 | |
| 180 | 启 dev server 让用户在浏览器调（agent 调起后把 URL 推给用户）。 |
| 181 | |
| 182 | ```bash |
| 183 | html-video preview --template data-bar-chart-race --vars-file vars.json --json |
| 184 | ``` |
| 185 | |
| 186 | ```json |
| 187 | { |
| 188 | "url": "http://127.0.0.1:53219", |
| 189 | "engine": "motion-canvas", |
| 190 | "pid": 87234, |
| 191 | "stop_command": "html-video preview-stop --pid 87234" |
| 192 | } |
| 193 | ``` |
| 194 | |
| 195 | ### `html-video doctor` |
| 196 | |
| 197 | 诊断本机环境。Agent 在用户首次启动 html-video 时**应该先跑这个**，把缺的依赖补全提示给用户。 |
| 198 | |
| 199 | ```bash |
| 200 | html-video doctor --json |
| 201 | ``` |
| 202 | |
| 203 | 输出： |
| 204 | |
| 205 | ```json |
| 206 | { |
| 207 | "status": "warning", |
| 208 | "checks": [ |
| 209 | { "name": "node-version", "status": "ok", "value": "v20.10.0" }, |
| 210 | { "name": "ffmpeg", "status": "ok", "value": "ffmpeg version 7.0" }, |
| 211 | { "name": "chromium", "status": "ok" }, |
| 212 | { "name": "adapter-hyperframes", "status": "ok", "version": "0.4.2" }, |
| 213 | { "name": "adapter-remotion", "status": "missing", "install_hint": "pnpm add -D @html-video/adapter-remotion" } |
| 214 | ] |
| 215 | } |
| 216 | ``` |
| 217 | |
| 218 | --- |
| 219 | |
| 220 | ## SKILL.md（Claude Code 版） |
| 221 | |
| 222 | 放置位置：`@html-video/agent-skill-claude-code/SKILL.md`，用户 install 后 symlink 到 `~/.claude/skills/html-video/`。 |
| 223 | |
| 224 | ```markdown |
| 225 | --- |
| 226 | name: html-video |
| 227 | description: | |
| 228 | Generate HTML videos by orchestrating multiple rendering engines (Hyperframes, |
| 229 | Remotion, Motion Canvas, Revideo) through a unified CLI. Pick the right engine |
| 230 | per use case, fill template variables, render to MP4/WebM. Use when user asks |
| 231 | to "create video", "render video", "make a video", "数据动画", "演示视频", |
| 232 | "social short", or provides data/intent that fits a video format. |
| 233 | --- |
| 234 | |
| 235 | # html-video skill |
| 236 | |
| 237 | You orchestrate the `html-video` CLI to generate videos. Never call engine |
| 238 | adapters directly — always go through the CLI. |
| 239 | |
| 240 | ## Initial check (run once per session) |
| 241 | |
| 242 | ```bash |
| 243 | html-video doctor --json |
| 244 | ``` |
| 245 | |
| 246 | If `status` is `error` or critical adapters missing, surface install hints to |
| 247 | the user and stop. If `warning`, note the gap but proceed if user's intent |
| 248 | doesn't need the missing piece. |
| 249 | |
| 250 | ## Standard workflow |
| 251 | |
| 252 | ### 1. Understand intent |
| 253 | |
| 254 | When user asks for a video, extract: |
| 255 | - **purpose** (data viz / social short / explainer / promo / ...) |
| 256 | - **input data** (if any — table, list, JSON, narrative) |
| 257 | - **aspect ratio** preference (16:9 / 9:16 / 1:1) |
| 258 | - **duration** target (if mentioned) |
| 259 | - **license needs** (commercial vs personal) |
| 260 | |
| 261 | If any is unclear, ask **one batched question** before searching templates. |
| 262 | |
| 263 | ### 2. Search templates |
| 264 | |
| 265 | ```bash |
| 266 | html-video search-templates \ |
| 267 | --intent "<extracted purpose+data summary>" \ |
| 268 | --aspect <ratio> \ |
| 269 | --license-allow <comma-list> \ |
| 270 | --top 3 --json |
| 271 | ``` |
| 272 | |
| 273 | Show the user **top 3** results with poster image and score reason. Let them pick. |
| 274 | |
| 275 | ### 3. Inspect chosen template |
| 276 | |
| 277 | ```bash |
| 278 | html-video inspect-template <id> --json |
| 279 | ``` |
| 280 | |
| 281 | Read the `inputs.schema`. Compare with the data the user has provided. |
| 282 | |
| 283 | - If user data already covers the schema → fill `vars.json`, skip to render |
| 284 | - If gaps → ask user **the missing fields only** (don't re-confirm what's |
| 285 | already known) |
| 286 | - If user data needs reshaping (e.g. user gave CSV, schema wants JSON |
| 287 | array-of-objects) → reshape silently, show before rendering |
| 288 | |
| 289 | ### 4. Optional preview |
| 290 | |
| 291 | If template's `performance.reference_render` suggests render will take |
| 292 | > 30 seconds, offer a preview first: |
| 293 | |
| 294 | ```bash |
| 295 | html-video preview --template <id> --vars-file vars.json --json |
| 296 | ``` |
| 297 | |
| 298 | Open the URL in the user's browser. After their OK, render. |
| 299 | |
| 300 | ### 5. Render |
| 301 | |
| 302 | ```bash |
| 303 | html-video render --template <id> --vars-file vars.json \ |
| 304 | --format mp4 --resolution 1920x1080 --fps 60 \ |
| 305 | --output ~/Desktop/<descriptive-name>.mp4 \ |
| 306 | --stream-progress --json |
| 307 | ``` |
| 308 | |
| 309 | Surface progress to user every 25% (preparing / rendering 25% / 50% / 75% / |
| 310 | muxing / done). On done, paste the absolute output path. |
| 311 | |
| 312 | ### 6. On errors |
| 313 | |
| 314 | - `engine-not-installed` → run `html-video doctor` and surface install hints |
| 315 | - `template-invalid` → re-inspect template, check vars match schema |
| 316 | - `render-failed` → read `diagnostics` array, propose fix, ask user before retry |
| 317 | - `render-timeout` → ask user if lower resolution / shorter duration is OK |
| 318 | |
| 319 | ## Anti-patterns (don't do these) |
| 320 | |
| 321 | - ❌ Don't pick a template silently — always show top 3 with reasons |
| 322 | - ❌ Don't render a 60-second video without offering preview first |
| 323 | - ❌ Don't re-ask for vars the user already provided |
| 324 | - ❌ Don't fall back to a different engine on render failure without telling |
| 325 | the user (the user might have license / aesthetic reasons for the choice) |
| 326 | - ❌ Don't write to engine-native files directly — only edit `vars.json` |
| 327 | |
| 328 | ## Quick reference |
| 329 | |
| 330 | | Task | Command | |
| 331 | |---|---| |
| 332 | | Health check | `html-video doctor --json` | |
| 333 | | Find templates | `html-video search-templates --intent "..." --json` | |
| 334 | | Read template | `html-video inspect-template <id> --json` | |
| 335 | | Render | `html-video render --template <id> --vars-file vars.json --output <path> --json` | |
| 336 | | Preview | `html-video preview --template <id> --vars-file vars.json --json` | |
| 337 | | List engines | `html-video list-engines --json` | |
| 338 | ``` |
| 339 | |
| 340 | --- |
| 341 | |
| 342 | ## Cursor / Codex / Gemini variants |
| 343 | |
| 344 | 不重写一遍。用 [`@html-video/agent-skill-claude-code`](#) 作为 master，**自动 transpile** 成： |
| 345 | |
| 346 | - `@html-video/agent-skill-cursor` —— `.cursor/rules/html-video.mdc` 形式 |
| 347 | - `@html-video/agent-skill-codex` —— `~/.codex/skills/html-video.md` |
| 348 | - `@html-video/agent-skill-gemini` —— `~/.gemini/agents/html-video.toml` |
| 349 | - `@html-video/agent-skill-opencode` —— OD 项目内 `.opencode/skills/html-video/` |
| 350 | |
| 351 | transpile 工具藏在 `tools/skill-transpile/`，CI 跑。所有变体共享 SKILL.md 的 prose，只换 frontmatter / file 命名。 |
| 352 | |
| 353 | --- |
| 354 | |
| 355 | ## Engine 选择逻辑（agent 不必懂细节，但放这里供 review） |
| 356 | |
| 357 | CLI 内部决策伪码： |
| 358 | |
| 359 | ```ts |
| 360 | function selectEngineForTemplate(template, userPrefs, installedEngines) { |
| 361 | // Stage 0: template 已经声明 engine，必须用这个 engine 的 adapter |
| 362 | const targetEngine = template.engine; |
| 363 | const adapter = installedEngines.find(e => e.id === targetEngine); |
| 364 | |
| 365 | if (!adapter) { |
| 366 | return { |
| 367 | ok: false, |
| 368 | reason: `Template requires ${targetEngine} but it's not installed`, |
| 369 | install_hint: `pnpm add -D @html-video/adapter-${targetEngine}` |
| 370 | }; |
| 371 | } |
| 372 | |
| 373 | // Stage 1: validate template against adapter capabilities |
| 374 | const validation = adapter.validate(template); |
| 375 | if (!validation.ok) { |
| 376 | return { ok: false, reason: validation.errors[0].message }; |
| 377 | } |
| 378 | |
| 379 | // Stage 2: license filter |
| 380 | if (userPrefs.commercial && template.license.commercial_use === false) { |
| 381 | return { ok: false, reason: 'Template not licensed for commercial use' }; |
| 382 | } |
| 383 | |
| 384 | // Stage 3: aspect / format / fps 兼容 |
| 385 | if (userPrefs.format && !adapter.capabilities.outputFormats.includes(userPrefs.format)) { |
| 386 | return { ok: false, reason: `Engine ${targetEngine} doesn't support ${userPrefs.format}` }; |
| 387 | } |
| 388 | |
| 389 | return { ok: true, adapter }; |
| 390 | } |
| 391 | ``` |
| 392 | |
| 393 | 注意：**v0.1 不做 cross-engine fallback**（一个 template 只对应一个 engine）。如果 agent 想"换 engine 试试"，让它重新 search-templates 选另一个 template id。 |
| 394 | |
| 395 | --- |
| 396 | |
| 397 | ## 监控 / 遥测（可选） |
| 398 | |
| 399 | `html-video.config.json` 可加： |
| 400 | |
| 401 | ```json |
| 402 | { |
| 403 | "telemetry": { |
| 404 | "enabled": false, |
| 405 | "endpoint": "https://html-video.nexu.io/telemetry" |
| 406 | } |
| 407 | } |
| 408 | ``` |
| 409 | |
| 410 | 默认 **off**。开了后只发匿名渲染统计（template id, engine, duration, success/fail），**不**发用户 vars 内容。Apache-2.0 + 显式 opt-in 是底线。 |
| 411 | |
| 412 | --- |
| 413 | |
| 414 | ## 用户交互范例（实际对话） |
| 415 | |
| 416 | ``` |
| 417 | User: 帮我做个视频，展示 OD 仓库 GitHub stars 25 天涨 5 万的曲线 |
| 418 | |
| 419 | Agent (内部跑 search): |
| 420 | → html-video search-templates --intent "github stars growth over time" --aspect 16:9 --top 3 --json |
| 421 | |
| 422 | Agent (回应用户): |
| 423 | 我找到 3 个匹配的模板，你想用哪个？ |
| 424 | |
| 425 | 1. **Bar Chart Race** ★★★ (Motion Canvas) |
| 426 | [poster.png] |
| 427 | 适合：rankings 对比 / 多 entity 演变 |
| 428 | 时长：5-60s 可调 |
| 429 | License: Apache-2.0 |
| 430 | |
| 431 | 2. **Line Chart Animation** ★★ (Hyperframes) |
| 432 | [poster.png] |
| 433 | 适合：单指标随时间演变（你这个场景） |
| 434 | 时长：3-30s |
| 435 | License: Apache-2.0 |
| 436 | |
| 437 | 3. **Number Counter Reveal** ★ (Hyperframes) |
| 438 | [poster.png] |
| 439 | 适合：单数字爆炸式 reveal |
| 440 | 时长：3-10s |
| 441 | |
| 442 | User: 第二个 |
| 443 | |
| 444 | Agent: 我看了 Line Chart 模板需要这些数据： |
| 445 | - data: 时间序列 (date + value) |
| 446 | - title: 标题（≤80字） |
| 447 | - y_axis_label: Y 轴标签 |
| 448 | |
| 449 | 你的 25 天数据我能帮你拉，但 GitHub API 翻页限制可能不全，要不要我用 growth-dashboard 已 backfill 的数据？ |
| 450 | |
| 451 | User: 用 dashboard 的 |
| 452 | |
| 453 | Agent (调 growth-dashboard 的 SQLite 拉数据 → 整成 JSON → 渲染): |
| 454 | 好。Title 我用 "Open Design GitHub Stars · 25 Days to 50K"， |
| 455 | y 轴 "Stars (cumulative)"。开始渲染... |
| 456 | |
| 457 | [progress 25% / 50% / 75%] |
| 458 | |
| 459 | ✅ 渲染完成 |
| 460 | /Users/joey/Desktop/od-stars-50k-race.mp4 (1920×1080, 14.5s, 12 MB) |
| 461 | ``` |
| 462 | |
| 463 | --- |
| 464 | |
| 465 | ## Open Questions（v0.2 待定） |
| 466 | |
| 467 | 1. **Skill 之间的协作**：html-video skill 想调 growth-dashboard 的数据 → 是直接读 SQLite 还是走 cross-skill MCP？v0.1 让 agent 自己 bridge |
| 468 | 2. **Multi-shot 工作流**：用户说"做个视频，60 秒，前 10 秒 logo reveal、中间 40 秒 chart race、最后 10 秒 CTA" → 需要 timeline composer skill。v0.2+ 才考虑 |
| 469 | 3. **缓存层**：相同 template + 相同 vars 多次 render 是否复用？倾向加 content-hash cache，但 v0.1 先不做 |
| 470 | 4. **OpenCode 集成**：OD 用 opencode 跑 agent；html-video 是否也注入 opencode skill registry？等 OD client 那边的 skill 接入接口稳定再做 |
| 471 |