| 1 | <div align="center"> |
| 2 | <img src="./assets/vimax.png"> |
| 3 | <br> |
| 4 | <br> |
| 5 | <h1 align="center">ViMax: Agentic Video Generation</h1> |
| 6 | <p align="center"> |
| 7 | <a href="https://trendshift.io/repositories/15299" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/repositories/15299" alt="HKUDS%2FViMax | Trendshift" width="250" height="55"/></a> |
| 8 | <a href="https://trendshift.io/repositories/15299?utm_source=trendshift-badge&utm_medium=badge&utm_campaign=badge-trendshift-15299" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/trendshift/repositories/15299/weekly?language=Python" alt="HKUDS%2FViMax | Trendshift" width="250" height="55"/></a> |
| 9 | </p> |
| 10 | |
| 11 | <div align="center"> |
| 12 | </div> |
| 13 | |
| 14 | <p align="center"> |
| 15 | <img src="https://img.shields.io/badge/Python-3.12-00d9ff?style=flat-square&logo=python&logoColor=white&labelColor=1a1a2e"> |
| 16 | <a href="https://github.com/astral-sh/uv"><img src="https://img.shields.io/badge/uv-Ready-ff6b6b?style=flat-square&logo=uv&logoColor=white&labelColor=1a1a2e"></a> |
| 17 | <img src="https://img.shields.io/badge/License-MIT-4ecdc4?style=flat-square&logo=opensourceinitiative&logoColor=white" alt="MIT License"> |
| 18 | <a href="https://github.com/HKUDS/ViMax/releases/tag/v1.2.0"><img src="https://img.shields.io/badge/Version-v1.2.0-2563eb?style=flat-square&labelColor=1a1a2e" alt="ViMax v1.2.0"></a> |
| 19 | <a href='https://www.youtube.com/@AI-Creator-is-here'><img src='https://img.shields.io/badge/YouTube-ff0000?style=flat-square&logo=youtube&logoColor=white&labelColor=1a1a2e' /></a> |
| 20 | <a href='https://arxiv.org/abs/2606.07649'><img src='https://img.shields.io/badge/arXiv-2606.07649-b31b1b?style=flat-square&logo=arxiv&logoColor=white&labelColor=1a1a2e' /></a> |
| 21 | </p> |
| 22 | |
| 23 | <p align="center"> |
| 24 | <a href="./Communication.md"><img src="https://img.shields.io/badge/Feishu-Group-07c160?style=flat-square&logo=lark&logoColor=white&labelColor=1a1a2e"></a> |
| 25 | <a href="./Communication.md"><img src="https://img.shields.io/badge/WeChat-Group-07c160?style=flat-square&logo=wechat&logoColor=white&labelColor=1a1a2e"></a> |
| 26 | <a href="readme.md"><img src="https://img.shields.io/badge/English-1a1a2e?style=flat-square"></a> |
| 27 | <a href="README_ZH.md"><img src="https://img.shields.io/badge/中文版-1a1a2e?style=flat-square"></a> |
| 28 | <a href="#quick-start"><img src="https://img.shields.io/badge/Quick%20Start-Get%20Started%20Now-FFC107?style=flat-square&logo=rocket&logoColor=white&labelColor=1a1a2e"></a> |
| 29 | </p> |
| 30 | |
| 31 | </div> |
| 32 | |
| 33 | --- |
| 34 | |
| 35 | <div align="center"> |
| 36 | |
| 37 | |
| 38 | |
| 39 | https://github.com/user-attachments/assets/5bad46b2-8276-4e1d-9480-3522640744b2 |
| 40 | |
| 41 | |
| 42 | |
| 43 | |
| 44 | </div> |
| 45 | |
| 46 | --- |
| 47 | |
| 48 | ### 📰 **动态** |
| 49 | |
| 50 | - **2026-07-20** 🚀 **ViMax v1.2.0** 发布 Web UI，支持命名项目、Agent Loop 对话、产物与分镜预览、渲染检查点、文件上传、供应商设置和深色模式。 |
| 51 | - **2026-07-17** 🎬 新增 OpenRouter GPT Image 2 图像生成与 Seedance 2.0 Fast 视频生成支持。 |
| 52 | |
| 53 | --- |
| 54 | |
| 55 | |
| 56 | ## 📑 目录 |
| 57 | |
| 58 | - [✨ 核心特性](#核心特性) |
| 59 | - [🔮 演示示例](#演示示例) |
| 60 | - [🚀 快速开始](#quick-start) |
| 61 | |
| 62 | --- |
| 63 | ## ✨ 核心特性 |
| 64 | |
| 65 | ViMax 是一个智能体驱动的视频创作框架，在统一且可扩展的工作流中串联叙事规划、视觉一致性、图像生成、视频生成与成片组装。 |
| 66 | |
| 67 | - **Idea2Video** — 将简短创意扩展为结构化故事、角色、剧本、分镜、镜头设计与最终视频。 |
| 68 | - **Script2Video** — 将明确剧本转化为可控的多场景、多镜头视频，同时保留原有创作意图。 |
| 69 | - **Novel2Video** — 通过叙事压缩、角色追踪与场景规划，将长篇小说改编为分集视觉内容。 |
| 70 | - **AutoCameo** — 根据参考照片将人物或宠物融入生成故事，并保持外观一致性。 |
| 71 | - **Agent Loop 与 TUI** — 在统一交互工作区中讨论创意、修改规划、恢复 session、检查文本产物并控制渲染。 |
| 72 | - **Web UI** — 在浏览器中管理命名项目、与 ViMax Agent 协作、上传源文件、查看产物和分镜进度、预览渲染并配置供应商。 |
| 73 | - **一致性制作流程** — 端到端协调参考图、首帧、机位连续性与最终组装。 |
| 74 | - **并行加速生成** — 并发生成可并行处理的镜头与媒体资产，缩短多镜头视频制作时间。 |
| 75 | |
| 76 | --- |
| 77 | <table> |
| 78 | <tr> |
| 79 | |
| 80 | <td align="center" width="33%"> |
| 81 | <video src="https://github.com/user-attachments/assets/c2fb27b0-218c-4976-b3d6-2abf8ea06be7" controls width="100%"></video> |
| 82 | </td> |
| 83 | <td align="center" width="33%"> |
| 84 | <video src="https://github.com/user-attachments/assets/bfa566a8-688d-4d53-a9e2-6cedeb4a399d" controls width="100%"></video> |
| 85 | </td> |
| 86 | <td align="center" width="33%"> |
| 87 | <video src="https://github.com/user-attachments/assets/49f61134-4f78-4285-9a9e-bb5e3e0c4abf" controls width="100%"></video> |
| 88 | </td> |
| 89 | </tr> |
| 90 | <tr> |
| 91 | <td align="center" width="33%"> |
| 92 | <video src="https://github.com/user-attachments/assets/a950f449-a15c-449b-a1b8-c393951aa9be" controls width="100%"></video> |
| 93 | </td> |
| 94 | <td align="center" width="33%"> |
| 95 | <video src="https://github.com/user-attachments/assets/bb3ff0fd-9433-4806-886a-3f77b61d06ec" controls width="100%"></video> |
| 96 | </td> |
| 97 | <td align="center" width="33%"> |
| 98 | <video src="https://github.com/user-attachments/assets/2624a3f0-9f66-4fa4-b527-45c0ea0353fc" controls width="100%"></video> |
| 99 | </td> |
| 100 | </tr> |
| 101 | |
| 102 | <tr> |
| 103 | <td align="center" width="33%"> |
| 104 | <video src="https://github.com/user-attachments/assets/5dbb80f7-aff0-4211-940c-a898f91fb80c" controls width="100%"></video> |
| 105 | </td> |
| 106 | <td align="center" width="33%"> |
| 107 | <video src="https://github.com/user-attachments/assets/cc0b0bcd-e7db-4839-950b-0b03949637bd" controls width="100%"></video> |
| 108 | </td> |
| 109 | <td align="center" width="33%"> |
| 110 | <video src="https://github.com/user-attachments/assets/85919b59-80f0-461a-af7e-a93d3fb412fc" controls width="100%"></video> |
| 111 | </td> |
| 112 | </tr> |
| 113 | |
| 114 | |
| 115 | |
| 116 | |
| 117 | |
| 118 | |
| 119 | |
| 120 | |
| 121 | |
| 122 | |
| 123 | |
| 124 | |
| 125 | |
| 126 | |
| 127 | |
| 128 | |
| 129 | |
| 130 | |
| 131 | |
| 132 | </table> |
| 133 | |
| 134 | |
| 135 | |
| 136 | --- |
| 137 | |
| 138 | ### 🖥️ **ViMax Web UI** |
| 139 | |
| 140 | <div align="center"> |
| 141 | <img src="assets/vimax-web-ui.png" width="100%" alt="ViMax Web UI 工作区"> |
| 142 | </div> |
| 143 | |
| 144 | Web UI 将 Agent 对话、项目产物、分镜预览与渲染进度集中在同一个可视化工作区中。 |
| 145 | |
| 146 | --- |
| 147 | |
| 148 | ### 🎯 **端到端视频创作引擎** |
| 149 | |
| 150 | **面临的挑战**： |
| 151 | |
| 152 | - 🌅 **参考图像**：获取、整理并精准对齐能准确表达角色、物体、位置与环境的参考帧，耗时费力。 |
| 153 | |
| 154 | - 🫠 **一致性校验**：即使提供了正确的角色、位置、环境参考图与提示词，图像生成器有时仍会产出不可用图像。 |
| 155 | |
| 156 | - 📄 **剧本生成**：专业高质量视频需要高信息密度与结构化设计。 |
| 157 | |
| 158 | - 📝 **分镜设计**：将故事转化为视觉叙事，需要摄影、构图与视觉叙事的专业知识，而大多数创作者并不具备。 |
| 159 | |
| 160 | - 🎬 **镜头设计**：在复杂场景中保持叙事连贯性的同时，设计合理的镜头角度、转场与节奏。 |
| 161 | |
| 162 | - 🎨 **风格一致性**：在长视频中确保数百个镜头的角色外观、环境与艺术风格保持一致。 |
| 163 | |
| 164 | - ⏱️ **制作效率**：传统视频制作依赖多个专业人员与冗长流程，阻碍了独立创作者与快速原型开发。 |
| 165 | |
| 166 | - 🎥 **AI视频扩展性**：AI生成视频通常仅几秒，而分钟级甚至小时级的高质量长视频需要复杂的跨场景连续性与多分镜协同处理能力。 |
| 167 | |
| 168 | **ViMAX**：通过自动化从叙事输入到最终视频输出的完整流程，彻底消除上述制作瓶颈。 |
| 169 | |
| 170 | --- |
| 171 | |
| 172 | |
| 173 | ### 🔥 **为什么选择 ViMax？** |
| 174 | |
| 175 | | 🧠 **一键生成** | 🚀 **完全创作自由** | 🔊 **音画同步** | 🎨 **专业品质** | 🤩 **互动视频** |
| 176 | |:---:|:---:|:---:|:---:|:---:| |
| 177 | | 一句话生成完整视频 | 任何叙事皆可成真 | 音画完美融合 | 电影级输出 | 生成你的专属客串视频 |
| 178 | | 无需技术细节——只需描述你的创意，ViMax 自动完成剧本生成、分镜设计、镜头规划、参考管理与一致性验证 | 创意无边界——无论是预告片、短篇故事、小说章节还是原创概念，ViMax 都能智能构建叙事并设计镜头语言，将任何想法变为现实 | 无缝融合角色语音与音效，打造沉浸式视听体验 | 自动质量控制确保角色一致性、场景构图合理、每帧画面均达专业水准 | 上传你的照片即可在自己的故事中互动出演——ViMax 智能将你作为角色融入视频，保持外观一致并实现自然交互 |
| 179 | |
| 180 | |
| 181 | |
| 182 | --- |
| 183 | |
| 184 | ### ☄️ **路线图** |
| 185 | |
| 186 | - ✅ 🖥️ **支持产物、分镜与渲染预览的 Web 前端工作区** |
| 187 | - ✅ 🤖 **Agent Loop + TUI 交互式工作流** |
| 188 | - ✅ 🎬 **Seedance 2.0 Fast 视频生成支持** |
| 189 | - ✅ 🖼️ **GPT Image 2 图像生成支持** |
| 190 | |
| 191 | --- |
| 192 | |
| 193 | |
| 194 | |
| 195 | ## 🚀Quick Start |
| 196 | |
| 197 | ### 🖥️ **Environment** |
| 198 | |
| 199 | ``` |
| 200 | OS: Linux, Windows |
| 201 | ``` |
| 202 | |
| 203 | ### 📥 **Clone and Install** |
| 204 | We use uv to manage the environment. For uv installation, please refer to the https://docs.astral.sh/uv/getting-started/installation/. |
| 205 | ```bash |
| 206 | git clone https://github.com/HKUDS/ViMax.git |
| 207 | cd ViMax |
| 208 | uv sync |
| 209 | ``` |
| 210 | |
| 211 | |
| 212 | <details> |
| 213 | <summary><strong>Agent TUI / Agents Loop</strong></summary> |
| 214 | |
| 215 | ViMax 还提供用于交互式 Agent 视频创作的最小 TUI。先从受 Git 跟踪的示例创建私有本地配置： |
| 216 | |
| 217 | ```bash |
| 218 | cp configs/agent.example.yaml configs/agent.local.yaml |
| 219 | ``` |
| 220 | |
| 221 | 随后在 `configs/agent.local.yaml` 中配置 LLM、图像和视频供应商，并从 ViMax 根目录启动 TUI。 |
| 222 | ```bash |
| 223 | vimax tui |
| 224 | ``` |
| 225 | |
| 226 | Start a new session or resume an existing one: |
| 227 | ```bash |
| 228 | vimax tui new |
| 229 | vimax tui resume |
| 230 | vimax tui resume <session_id> |
| 231 | ``` |
| 232 | |
| 233 | </details> |
| 234 | |
| 235 | <details> |
| 236 | <summary><strong>Web UI</strong></summary> |
| 237 | |
| 238 | Web UI 与 TUI 共用同一套 ViMax agent runtime、session、tools 和私有的 `configs/agent.local.yaml` 配置。运行 Web UI 需要 Node.js 18 或更高版本。 |
| 239 | |
| 240 | 在 `ViMax` 仓库根目录中，首次使用时安装前端依赖，然后启动本地服务： |
| 241 | |
| 242 | ```bash |
| 243 | cd web |
| 244 | npm install |
| 245 | npm run dev |
| 246 | ``` |
| 247 | |
| 248 | 在浏览器中打开 [http://127.0.0.1:4173](http://127.0.0.1:4173)。Web UI 支持命名项目、Agent 对话、斜杠命令、产物与渲染查看、分镜预览、文件上传和供应商设置。 |
| 249 | |
| 250 | 服务默认只监听 `127.0.0.1`。如果 ViMax 运行在远程服务器上，请在本地电脑建立 SSH 端口转发： |
| 251 | |
| 252 | ```bash |
| 253 | ssh -N -L 4173:127.0.0.1:4173 <user>@<server> |
| 254 | ``` |
| 255 | |
| 256 | 需要改用其他端口时，设置 `VIMAX_WEB_PORT`： |
| 257 | |
| 258 | ```bash |
| 259 | VIMAX_WEB_PORT=4174 npm run dev |
| 260 | ``` |
| 261 | |
| 262 | </details> |
| 263 | |
| 264 | <details> |
| 265 | <summary><strong>Usage</strong></summary> |
| 266 | |
| 267 | main_idea2video.py is used to convert your ideas into videos. |
| 268 | You need to configure the model and API key information in the configs/idea2video.yaml file, including three parts—the chat model, the image generator, and the video generator, as shown below |
| 269 | ```yaml |
| 270 | chat_model: |
| 271 | init_args: |
| 272 | model: google/gemini-2.5-flash-lite-preview-09-2025 |
| 273 | model_provider: openai |
| 274 | api_key: <YOUR_API_KEY> |
| 275 | base_url: https://openrouter.ai/api/v1 |
| 276 | |
| 277 | image_generator: |
| 278 | class_path: tools.ImageGeneratorNanobananaGoogleAPI |
| 279 | init_args: |
| 280 | api_key: <YOUR_API_KEY> |
| 281 | |
| 282 | video_generator: |
| 283 | class_path: tools.VideoGeneratorVeoGoogleAPI |
| 284 | init_args: |
| 285 | api_key: <YOUR_API_KEY> |
| 286 | |
| 287 | working_dir: .working_dir/idea2video |
| 288 | ``` |
| 289 | |
| 290 | Then, provide a simple yet thoughtful idea and the corresponding creative requirements in main_idea2video.py. |
| 291 | ```bash |
| 292 | idea = \ |
| 293 | """ |
| 294 | If a cat and a dog are best friends, what would happen when they meet a new cat? |
| 295 | """ |
| 296 | user_requirement = \ |
| 297 | """ |
| 298 | For children, do not exceed 3 scenes. |
| 299 | """ |
| 300 | style = "Cartoon" |
| 301 | ``` |
| 302 | |
| 303 | main_script2video.py generates a video based on a specific script. |
| 304 | You similarly need to set up the API configuration in configs/script2video.yaml file. Then, provide a scene script and the corresponding creative requirements in main_script2video.py, as shown below. |
| 305 | ```python |
| 306 | script = \ |
| 307 | """ |
| 308 | EXT. SCHOOL GYM - DAY |
| 309 | A group of students are practicing basketball in the gym. The gym is large and open, with a basketball hoop at one end and a large crowd of spectators at the other end. John (18, male, tall, athletic) is the star player, and he is practicing his dribble and shot. Jane (17, female, short, athletic) is the assistant coach, and she is helping John with his practice. The other students are watching the practice and cheering for John. |
| 310 | John: (dribbling the ball) I'm going to score a basket! |
| 311 | Jane: (smiling) Good job, John! |
| 312 | John: (shooting the ball) Yes! |
| 313 | ... |
| 314 | """ |
| 315 | user_requirement = \ |
| 316 | """ |
| 317 | Fast-paced with no more than 20 shots. |
| 318 | """ |
| 319 | style = "Animate Style" |
| 320 | ``` |
| 321 | |
| 322 | </details> |
| 323 | |
| 324 | --- |
| 325 |