简而言之：使用前沿模型的 AI 邮件每月成本为 22-130 美元，但通过采用更小模型和本地推理进行成本优化，可将这一成本降低 100 倍。
在昨天的文章（由智能体通过邮件以原始大纲形式推送！）中，我探讨了 AI 邮件的未来。那么，这个未来的成本是多少？
如果你使用的是最先进的模型，每月成本在 22 到 130 美元之间。你愿意为此付费吗？我想，在工作中，很多人会愿意。我们取中间情况，即每月 26 美元的原始成本。
一家追求 75% 毛利率的软件公司，在不包含托管和服务成本的情况下，该产品的年收费大约为 350 美元。那么，我们姑且将其标价定为每年 500 美元，大规模使用时再给予 15% 的折扣。
Google 企业版套餐的价格是每月 11-18 美元。一个完全智能体化的解决方案，其成本大约是前者的两倍。
更小的模型能起到作用。它们能将成本降低 10 到 20 倍，但我们还能做得更好。
通过在本地运行模型，当成本骤降至零时：用户的 GPU 承担了计算工作。
正是我在此粗略估算的这种成本优化，我认为将定义未来 12 到 24 个月的 AI 软件：即确定哪些组件可以确定性执行（例如邮件过滤器，它们只是规则），其次是让模型与工作负载相匹配。
借助一些基本的启发式规则和技术，我们可以将总成本降低 100 倍。鉴于 GPU 的严重短缺，这种推理的分层是不可避免的。
In short : AI email using frontier models costs $22-130/month, but cost optimization through smaller models and local inference can reduce this by 100x.
In yesterday’s post (which an agent pushed in raw outline form via email!), I wrote about the future of AI email. What does that future cost?
If you are using state-of-the-art model ranging, it costs between $22 to $130 per month. Would you pay for that? At work, I imagine, many would. Let’s take the middle case of $26/month raw cost.
A software company seeking 75% gross margin would charge about $350 per year for that product excluding hosting & serving costs. So let’s call it a $500 per year list with a 15% discount at scale.
A Google Enterprise plan is $11-18/month. A fully agentic solution would then cost about twice as much.
Smaller models help. They cut cost by 10 to 20x, but we can do better.
By running the models locally, when the cost plummets to zero : users’ GPU does the work.
It’s this type of cost optimization that I have done crudely here that I think will define the next 12 to 24 months of AI software : determining which components can be executed deterministically, like the email filters, which are just rules. And the next is matching the model to the workload.
With some basic heuristics and techniques we can drop the overall cost by 100x. Given the tremendous shortage of GPUs, this segmentation of inference is inevitable.