# NVIDIA 开源 srt-slurm 编排推理部署

- 来源：SemiAnalysis (@SemiAnalysis_)
- 发布时间：2026-08-28 04:00
- AIHOT 分数：38
- AIHOT 链接：https://aihot.virxact.com/items/cmtbzck2a01x2rolblzlribty
- 原文链接：https://x.com/SemiAnalysis_/status/2093066214771679716

## AI 摘要

NVIDIA 开源 srt-slurm，为 Slurm 集群上的生产级推理服务提供基于 YAML 的编排层。它协调 sbatch/srun、GPU 分配、网络、就绪检查及清理，支持 prefill/decode 分离、多 worker、Dynamo 前端、KV-cache 感知路由等拓扑，便于在 GPU 集群上快速部署代表性推理系统进行测试或基准评测。

## 正文

Most production inference services run on Kubernetes, where declarative configuration helps coordinate distributed services. Running one inference server in a Slurm job is easy; running a production-style topology across multiple containers and nodes is not.

NVIDIA’s open-source srt-slurm (https://github.com/NVIDIA/srt-slurm) provides a YAML-based orchestration layer for these deployments. It coordinates sbatch and srun, GPU placement, networking, readiness checks, and cleanup for systems involving prefill/decode disaggregation, multiple workers, Dynamo frontends, KV-cache-aware routers, and KV-cache offloading services.

It’s useful for quickly deploying representative inference systems for testing or benchmarking on GPU clusters that already run Slurm.

Great work to @0xishand and the rest of the @NVIDIAAI team on this!