OpenClaw 2.0 vs Hermes experiment by @atomicbot_ai is a good example of why the model alone tells you very little about agent performance.

Outputs: OpenClaw 2.0: ~2.1M tokens, ~$4.5, 10 self-fixes Hermes Agent: ~2.9M tokens, ~$4, 20 self-fixes

The difference was in how they checked themselves.

Atomic Bot lets you run AI agents in the cloud without managing the setup yourself.

In this experiment

• OpenClaw screenshotted frames at specific timestamps, counted pixels, then corrected visual errors such as an inverted coat-tail direction. • Hermes went further and created its own render-checking setup, measuring frame timing, testing whether loop seams were visible, and auditing scenes against the original spec.

That extra ~800K tokens was largely verification work.

We usually talk about agent tokens as inference cost.

This run is a nice reminder that some of those tokens are effectively a QA budget. Once agents can generate their own checks, the important question becomes how much compute you want them spending proving their work before returning it.