# VisualClaw：面向物理世界的实时个性化多模态智能体

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-06-15 08:00
- AIHOT 分数：55
- AIHOT 链接：https://aihot.virxact.com/items/cmqg30by70327slspy3ed3rrh
- 原文链接：https://arxiv.org/abs/2606.16295

## AI 摘要

VisualClaw是一个自进化多模态智能体，通过级联门过滤流式帧与热/冷top-k注入技能库，将单问题API成本降至全帧上传的-98%、均匀8帧基线的-25.9%。技能进化模块从失败中学习并更新技能库，在4个视频QA基准上平均准确率提升+3.85%，EgoSchema上Gemini 3 Flash达+15.80%。研究者构建了VisualClawArena（200场景多模态智能体基准），在该基准上结合计算机使用后端使Codex (GPT-5.5)宏观准确率+2.9%、Claude Code (Sonnet 4.6)+3.2%，成本降低-9.5%。级联机制将1小时流媒体从~3,600次API调用降至5-20次，适合边缘部署。

## 正文

Vision language models are serving as general-purpose interfaces for complex multimodal tasks. However, deployment still faces three gaps: VLMs typically incur high latency and cost when processing dense video frames and long prompts, the agent scaffold remains static after deployment, and standard video-QA benchmarks do not test whether agents can use visual evidence inside tool-using workspaces. We present VisualClaw, a self-evolving multimodal agent built around two principles. First, hybrid encoding reduces deployment cost by filtering less informative streaming frames with a cascaded gate and compressing the text skill bank through hot/cold top-k injection. Second, skill evolution lets the agent learn from failures: retrieved memories condition an evolver as direct concatenated context or as guided evidence, producing skill-bank updates that help future questions. Across 4 video-QA benchmarks with 2 VLMs, VisualClaw cuts per-question API cost by an average -98% versus full-frame upload and by -25.9% over the offline uniform 8 frame baseline, while boosting accuracy in most settings, e.g., an average +3.85% and a peak +15.80% on EgoSchema with Gemini 3 Flash. To address the gap, we curate VisualClawArena, a 200-scenario multimodal agentic benchmark built through a strict five-stage pipeline; models must use video evidence, documents, dynamic updates, and executable checks inside a workspace. On VisualClawArena, the same framework with computer-use agent backends improves macro accuracy by +2.9% for Codex (GPT-5.5) and +3.2% for Claude Code (Sonnet 4.6) over no-evolution baselines, with a -9.5% cost reduction compared to the uniform-sampled baseline. These properties make VisualClaw a natural fit for edge applications, where the cascade reduces a 1-hour streaming session from ~3,600 API uploads down to only 5-20 calls and the self-evolution makes it a perfect personalized assistant.