Yuchen Jin@Yuchenj_UW · XAI 摘要
我依然惊叹于 DeepSeek、Kimi 和 Qwen 能够用少得多且经常是降配的 NVIDIA GPU，甚至华为芯片，训练出非常强大的大语言模型。 DeepSeek V4 的报告显示，他们发明了新的注意力架构，使训练/推理更加高效。 创造力热爱约束。 我真心希望我们看到有强大的美国开源模型能够参与竞争。
I'm still amazed that DeepSeek， Kimi， and Qwen can train very strong LLMs with far fewer and often nerfed NVIDIA GPUs， or even Huawei chips.
DeepSeek V4 report shows they invent new attention architectures to make training/inference more efficient.
Creativity loves constraints.
I really hope we see strong US open-source models that can compete.