In today's regime, token costs can actually be quite misleading. For example, even though Astra and Fable 5.1 have the same sticker price, Astra is often a lot cheaper – sometimes less than half the cost!
This is because Astra often uses far fewer tokens to achieve the same result. The best way to see this is by looking at evals that the community runs – you're giving the models the same set of tasks and can see how much it costs to run.
@PawelHuryn's Bug Hunt Bench below is a good example (60% cheaper!)
Sharing some other good examples below:
So, I finally tested GPT-6 Astra. 2 real repos, 105 bugs, find and fix what you can. Max effort: GPT-6 Astra: 48/105 Fable 5.1: 43/105 GPT-5.6 Sol: 42/105 Gemin...