Topic · 主题全部主题 →

AI 视频

AI 视频生成与理解:文生视频模型、视频编辑工具与影视创作变革的追踪。

2,070条收录
135条精选

精选归档 · 第 7 页

121135 条 · 共 135

3月18日

星期三 · 1 条
05:07
Hao AI Lab@haoailab精选
AI 评分 65/100
FastVideo推出Dreamverse原型,实现"氛围导演"式实时视频生成http://x.com/i/article/2034009793598464000Into the DreamVerseTL;DR: Our new real-time inference stack in FastVideo enables Dreamverse, a prototype for a new interface where users can vibe direct their own “multiverse” of videos.AI video generation is already good enough to make a convincing clip. But real creative work is not about getting a clip in one shot. It’s about iteration. An idea appears, you test it: keep the subject, change the camera angle, continue the scene, and try again. The problem is that ideas move faster than generations. If every attempt takes minutes, the creative loop breaks; your imagination moves on before the video does.We think there is a better interface for AI video generation, which is why we created Dreamverse, an interface that enables a new workflow called vibe directing.Vibe directing is to video what vibe coding is to software. Instead of rewriting giant prompts from scratch, you talk to the system in natural language and steer the video through fast revision. Keep the subject, change the background, slow the camera, or anything else! Rather than jamming everything into a single prompt, iterate with multiple simple prompts.This kind of workflow is only possible when video generation is done in real-time. Current video generation models like Sora take 1-2 minutes to generate a 5s 1080p clip. We can do it in ~4.55 seconds on a single GPU. In other words, our inference stack in FastVideo can generate a clip faster than you can watch it. This capability completely changes the feel of video generation inference; it stops feeling like a passive experience and starts feeling like directing your own scenes. This allows us to create a longer 30-second scene that unfolds as a chain of these 5-second clips, while keeping a chat window open so you can keep directing in real time.This matters because serious video creation is almost never perfect on the first try. A shot may look off. Motion may break halfway through. Characters may drift between frames. In addition, creators may have multiple versions of a scene and want to play them out to determine which is better. In practice, creators are constantly making small adjustments and trying again. When revisions are slow, it’s much more difficult to explore many ideas. However, when the next result comes back almost immediately, it becomes possible to quickly try many ideas rather than just one. Better creative work comes from a faster feedback loop, not just a better model.We think this is where video generation is going: a way to direct the video as it unfolds. The best systems will not just generate impressive clips. They will let people explore ideas at the speed of their imagination.That is what vibe directing is all about. Step into the Dreamverse today with our demo.The TeamCore contributors: Will Lin*, Matthew Noto*, Junda Su*, Yechen Xu*, Peiyuan Zhang* (* equal contribution) Contributors: Shao Duan, Minshen Zhang, Loay Rashid, Kevin Lin UI: Tina Mai Tech leads: Will Lin, Hao Zhang Advisors: Hao Zhang (corresponding), Danyang Zhuo, Eric Xing, Zhengzhong LiuLearn More• FastVideo Documentation• FastVideo Roadmap for 26Q1FastVideo团队发布Dreamverse原型界面,引入创新的"氛围导演"工作流。该模式允许用户通过自然语言实时、迭代地引导视频生成,如更换背景或调整运镜,无需编写复杂的长提示词。其核心是全新的实时推理栈,能在单GPU上以约4.55秒生成5秒1080p视频,速度快于观看时间,从而将生成过程从被动等待转变为实时导演体验。团队认为,视频生成的未来在于让创作速度跟上想象速度,快速的反馈循环比单纯追求模型性能更能催生优质作品。
推荐理由:视频生成从「等一分钟看结果」变成「边看边改」,这个交互范式转变比模型本身更值得关注。做内容创作工具的产品人,这个 demo 值得花五分钟体验一下实时迭代的手感。

3月9日

星期一 · 1 条
00:00
Runway:News(网页)精选
Runway 推出 Characters:单图实时生成可对话虚拟角色 API

Runway 推出 Characters API,基于 GWM-1 世界模型,支持用单张图片零微调生成实时可对话虚拟角色。支持自定义外观风格、声音、性格及知识库,具备自然表情、眼神、口型同步和手势。面向客户支持、培训教育和品牌营销等企业场景,已获 BBC 等采用。开发者可通过 API 集成,消费者也可在网页端体验预设角色。


推荐理由:Runway推出实时视频Agent,单图生成可对话数字人,拓展AI交互形态

2月27日

星期五 · 1 条
04:15
Saining Xie@sainingxie精选
世界建模绝非像素渲染:Project Solaris发布多智能体视频世界模型world modeling is never about rendering pixels.rendering is local. world state is global. as soon as more than one agent exists, the only thing that truly matters is the shared representation beneath individual views. that shared representation is what scales into collective capability.this is why I'm super excited to share project Solaris -- our new work focused on building a multiplayer video world model in minecraft.This release includes three main pieces. 1⃣Solaris Engine, a fully featured multiplayer data collection system with built in visuals. the team put a huge amount of work into this since nothing like it really exists yet. https://github.com/solaris-wm/solaris-engine 2⃣Solaris Model, a multiplayer DiT with a new memory efficient self forcing design, trained on 12.6M frames of coordinated Minecraft gameplay. https://github.com/solaris-wm/solaris 3⃣Solaris Eval, which uses a VLM as a judge to evaluate different multiplayer capabilities.read the full technical breakdown by @ojmichel4, and start building with Solaris.https://solaris-wm.github.io/Project Solaris提出世界建模的本质在于全局共享状态而非局部像素渲染,推出基于Minecraft的多人在线视频世界模型。该系统突破单智能体视角局限,支持任意数量智能体随时介入交互,实现持久化世界状态演化。核心包含三大组件:Solaris Engine多人数据收集系统、基于DiT架构的Solaris Model(采用新型内存高效自强制设计,训练于1260万帧协调游戏数据)、以及使用VLM评判的Solaris Eval评估体系。这一范式转变为构建神经MMORPG服务器奠定基础。

Oscar Michel: 📢Current world models aren't really modeling the world; they're modeling one agent's view of it. Partial observations ≠...


推荐理由:从单视角到共享全局状态,开源多智能体世界模型或改变AI训练范式

2月12日

星期四 · 1 条
00:00
字节 Seed:Research Feed(网页内嵌数据)精选
Seedance 2.0 正式发布

Seedance 2.0 视频生成模型正式发布,综合性能达业界 SOTA 水平。新版本支持多模态输入,可同时参考 9 张图片、3 段视频、3 段音频及文本指令,精准迁移构图、动作、运镜与音效。模型支持 15 秒高质量多镜头生成,集成双声道立体声技术,并新增视频延长与编辑功能,可定向修改片段、角色及剧情。目前已在即梦 AI、豆包及火山方舟平台上线,适用于影视、广告、电商等工业级内容创作场景。

另有 1 家信源报道字节 Seed:Research Papers(网页内嵌数据)
推荐理由:字节Seedance 2.0发布,支持多模态输入与物理稳定生成,已上线豆包即梦

1月28日

星期三 · 1 条
08:00
xAI:News(网页)精选
xAI发布Grok Imagine API

xAI推出Grok Imagine API,提供文本/图像转视频及高精度编辑功能,支持物体增删、风格转换与原生音频生成。在Artificial Analysis和LMArena基准测试中排名首位,720p视频生成延迟与成本均低于Veo 3和Sora 2;在IVEBench评测中整体表现超越Kling o1与Runway Aleph。API已开放xAI及第三方平台接入。


推荐理由:xAI发布Grok Imagine视频生成API,性能宣称超越Sora与Veo,支持端到端视频创作工作流

1月20日

星期二 · 1 条
08:00
Hugging Face:Blog(RSS)精选
AI 评分 73/100
Overworld发布实时交互式视频扩散模型Waypoint-1

Overworld推出实时交互式视频扩散模型Waypoint-1,用户可通过文本、鼠标和键盘实时控制生成可步入的虚拟世界。该模型基于帧因果校正流变换器架构,在1万小时游戏视频及对应控制数据上训练,从一开始就专注于交互体验,支持零延迟的自由操控。其配套的高性能推理库WorldEngine在消费级硬件上可实现流畅运行,例如Waypoint-1-Small在RTX 5090上能以30 FPS(4步去噪)或60 FPS(2步去噪)生成画面。模型采用扩散强制预训练和自我强制后训练来确保生成长序列的稳定性。


推荐理由:零延迟交互式视频生成,游戏和创意应用开发者的福音。

10月28日

星期二 · 1 条
00:00
MiniMax:Blog(网页)精选
MiniMax 发布 Hailuo 2.3 / 2.3 Fast 视频模型

MiniMax 推出 Hailuo 2.3 视频生成模型,在物理动作流畅度、艺术风格化(支持动漫、水墨、游戏 CG)及角色微表情方面显著提升,维持 Hailuo 02 原价,Fast 版本批量创作成本降低 50%。Hailuo Video Agent 同步升级为 Media Agent,支持多模态一键视频生成与分步自定义创作,已全平台上线并开放免费试用。

另有 1 家信源报道MiniMax:Blog(网页)
推荐理由:MiniMax 发布 Hailuo 2.3 视频模型及 Media Agent,支持多模态一键生成

10月10日

星期五 · 1 条
10:20
蚂蚁 inclusionAI:GitHub 新仓库精选
AI 评分 66/100
Ming-VideoMAR:基于连续令牌的自回归视频生成模型

Ming-VideoMAR 是一款仅解码器的自回归图像到视频生成模型,采用连续令牌统一视觉表征。它首次将时间因果性与空间双向性作为视频自回归核心原则,并提出了整合掩码生成的下一帧扩散损失。该模型首次实现了视频生成的零样本分辨率缩放,能灵活生成远超训练分辨率的视频。其在训练与推理效率上表现突出,参数量、训练数据量和GPU消耗仅为之前最佳模型Cosmos的极小比例(9.3%、0.5%和0.2%),同时在定量与定性评估中均实现超越。模型代码与检查点已开源,论文已被NeurIPS 2025接收。


推荐理由:蚂蚁把自回归视频生成的训练成本砍到 Cosmos 的 0.2% 还能赢,这个效率信号比分数本身更值得关注,做视频生成的团队该认真看看它的课程学习和渐进分辨率策略。

10月1日

星期三 · 1 条
01:13
Sam Altman:Blog(RSS)精选
Sora 2

OpenAI发布Sora应用,集成Sora 2模型,支持快速创作、分享和观看视频,团队称其为"创意领域的ChatGPT时刻"。核心功能包括cameo客串特性,可保持角色一致性并将用户及朋友置入视频。团队同时表达对成瘾性和低质内容("slop feed")风险的担忧,提出四项产品原则:优化长期用户满意度、赋予用户信息流控制权、优先鼓励创作、帮助实现长期目标,并配备深度伪造防护和情绪健康监测等安全措施。


推荐理由:OpenAI 正式发布 Sora 2 应用,定位「创意领域的 ChatGPT时刻」

8月5日

星期二 · 2 条
23:57
Jim Fan@DrJimFan精选
NVIDIA推出DreamGen引擎:让机器人在视频生成模型中"做梦"学习World modeling for robotics is incredibly hard because (1) control of humanoid robots & 5-finger hands is wayyy harder than ⬆️⬅️⬇️➡️ in games (Genie 3); and (2) object interaction is much more diverse than FSD, which needs to *avoid* coming into contact. Our GR00T Dreams work was a first attempt at building high-fidelity world simulator for humanoid robots. It's not only for evaluation but also for large-scale synthetic data generation. Time to move away from the "fossil fuel" of robotics (human teleoperation) and embrace clean energy (nuclear "diffusion")!GR00T Dreams kind of flew under the radar, so bringing it back to life on a cheerful day ;)NVIDIA发布DreamGen引擎(GR00T Dreams),将Sora/Veo等视频生成模型用作神经物理引擎,通过微调模型、模拟并行世界、恢复伪动作、训练基础模型四步流程,为机器人生成大规模合成训练数据。人形机器人仅凭单一拾放任务即可学会倾倒、折叠等22种新行为,在新动词和陌生环境中实现零样本泛化(成功率分别达43%和28%)。相比传统图形引擎,该方法以恒定计算成本处理可变形物体、流体等复杂交互,团队计划数周内完全开源。

Jim Fan: What if robots could dream inside a video generative model? Introducing DreamGen, a new engine that scales up robot lear...


推荐理由:NVIDIA提出用视频生成模型为机器人“造梦”合成训练数据,实现零样本技能泛化
05:25
Hao AI Lab@haoailab精选
AI 评分 67/100
FastWan视频生成模型实现70倍加速,5秒出片Try FastWan at https://fastwan.fastvideo.org/!FastVideo团队推出FastWan系列快速视频生成模型。该模型采用名为"稀疏蒸馏"的新训练方法,能将视频去噪速度提升70倍。在单块H200 GPU上,仅需5秒即可生成一段5秒的视频。团队提供了在线演示,并依据Apache-2.0许可证完全开源了模型、代码和数据。

Hao AI Lab: (1/n) 🚀 With FastVideo, you can now generate a 5-second video in 5 seconds on a single H200 GPU! Introducing FastWan se...


推荐理由:视频生成终于从「等一分钟」进化到「实时出片」,FastWan 用稀疏蒸馏把去噪压了 70 倍,单卡 H200 五秒出五秒视频,做短视频工具和实时交互产品的团队该认真看看这个开源方案。

5月20日

星期二 · 2 条
21:29
Jim Fan@DrJimFan精选
DreamGen:让机器人在视频生成模型中"做梦"合成训练数据What if robots could dream inside a video generative model? Introducing DreamGen, a new engine that scales up robot learning not with fleets of human operators, but with digital dreams in pixels. DreamGen produces massive volumes of neural trajectories - photorealistic robot videos paired with motor action labels - and unlocks strong generalization to new nouns, verbs, and environments. Whether you’re a humanoid (GR1), an industrial arm (Franka), or a cute little robot (HuggingFace SO-100), DreamGen enables you to dream.Video generation models like Sora & Veo are neural physics engines. By compressing billions of internet videos, they learn a multiverse of plausible futures, i.e. superpositions of how the world could unfold from any initial image frame. DreamGen taps into this power with a simple 4-step recipe:1. Fine-tune a SOTA video model on your target robot; 2. Prompt the model with diverse language prompts to simulate parallel worlds: how your robot would have acted in new scenarios. Filter out the bad dreams (ha!) that don’t follow instructions; 3. Recover pseudo-actions using inverse dynamics or latent action models; 4. Train robot foundation models on the massively augmented dataset of neural trajectories.That’s it. Just more data, and plain old supervised learning. Simple, right?What’s remarkable is how far this goes. Starting with just a single-task dataset of pick-and-place, our humanoid robot learns 22 new behaviors, such as pouring, folding, scooping, ironing, and hammering, despite never seeing those verbs before. Better yet, we can take the robot out of the lab and drop it into the NVIDIA HQ Cafe, and let DreamGen work its magic. We show true zero-to-one generalization: from 0% success to over 43% for novel verbs, and 0 -> 28% in unseen environments.Compared to a traditional graphics engine, DreamGen doesn’t care if the scene involves deformable objects, fluids, translucent materials, contact-rich interactions, or crazy lighting. Good luck engineering those by hand. For DreamGen, every world is just a forward pass through a diffusion neural net. No matter how complex the dream is, it takes constant compute time to roll out.Read our blog and paper today! We plan to fully open-source the entire pipeline in the next few weeks. Links in thread:DreamGen让机器人在视频生成模型中"做梦"合成训练数据。通过微调Sora等模型生成海量神经轨迹(逼真视频+动作标签),机器人从单一拾取放置任务泛化到倾倒、折叠等22种新行为。在NVIDIA总部咖啡厅测试中,人形机器人对新动词零样本成功率从0%提升至43%,新环境达28%。相比传统图形引擎,无需手工建模即可处理流体、可变形物体等复杂场景,整个pipeline将于近期完全开源。

推荐理由:NVIDIA 提出 DreamGen:让机器人在视频生成模型中「做梦」合成训练数据,实现强零样本泛化,将开源
17:45

9月18日

星期三 · 1 条
00:00
Runway:News(网页)精选
Runway 与 Lionsgate 达成合作

Runway 与 Lionsgate 达成首创性合作,基于后者超过 20,000 部作品的专有片库定制训练 AI 视频生成模型,供电影制作人在前期和后期流程中增强创作。该模型可生成电影级视频并支持迭代编辑,双方未来计划向个人创作者开放模型授权。


推荐理由:Runway与好莱坞大厂达成首个定制模型合作,标志AI视频正式进入主流影视工业化流程