# DeepSeek开源3FS并行文件系统与Smallpond框架

- 来源：DeepSeek (@deepseek_ai)
- 发布时间：2025-02-28 09:06
- AIHOT 链接：https://aihot.virxact.com/items/cmo77611t003aslhe54xfdpez
- 原文链接：https://x.com/deepseek_ai/status/1895279409185390655

## AI 摘要

DeepSeek发布开源并行文件系统3FS（Fire-Flyer File System），专为现代SSD和RDMA网络优化。180节点集群可实现6.6 TiB/s聚合读取吞吐量，25节点GraySort测试达3.66 TiB/min，单节点KVCache查找峰值超40 GiB/s。采用分离式架构与强一致性语义，支持训练数据预处理、检查点存取及V3/R1推理的KVCache查找。同步开源Smallpond数据处理框架。

## 正文

🚀 Day 5 of #OpenSourceWeek： 3FS， Thruster for All DeepSeek Data Access

Fire-Flyer File System （3FS） - a parallel file system that utilizes the full bandwidth of modern SSDs and RDMA networks.

⚡ 6.6 TiB/s aggregate read throughput in a 180-node cluster
⚡ 3.66 TiB/min throughput on GraySort benchmark in a 25-node cluster
⚡ 40+ GiB/s peak throughput per client node for KVCache lookup
🧬 Disaggregated architecture with strong consistency semantics
✅ Training data preprocessing， dataset loading， checkpoint saving/reloading， embedding vector search & KVCache lookups for inference in V3/R1

📥 3FS → https://github.com/deepseek-ai/3FS
⛲ Smallpond - data processing framework on 3FS → https://github.com/deepseek-ai/smallpond