Strands Robots 中 LeRobot 集成的完整介绍——一个智能体循环，从 Hub 数据集到实体机器人，采用相同磁盘格式的仿真到真实数据集，以及只需一个字符串即可切换的策略。
你有一台机器人、一个存放在 Hugging Face Hub 上的演示数据文件夹，以及一项希望它学习的新任务。如今，这需要五个独立的工具：一个用于录制新的演示数据，一个用于训练，一个用于在仿真中测试，还需要自定义代码部署到硬件上，当你有多个机器人时，还需要另一个工具来协调。这些组件各自独立运行，彼此之间无法通信。
Strands Robots 是 AWS 推出的开源 SDK（Apache 2.0 许可证），它将机器人抽象、仿真以及 LeRobot 技术栈暴露为 AgentTools，你可以将它们组合成一个单一的 Strands 智能体。这种集成设计得非常精简：LeRobot 自身的脚本负责硬件录制和校准，而 Strands AgentTools 则负责处理智能体实际编排的部分。仿真工具录制的 LeRobotDatasets 与 LeRobot 在硬件上写入的格式完全相同。GR00T 和 LerobotLocal 在统一接口背后提供策略推理服务，MolmoAct2 检查点则通过 LerobotLocal 路径运行。一个对等网格将智能体扩展到远程机器人上。数据集格式完全保持 LeRobot 写入时的原样；智能体循环就是将它们粘合在一起的胶水。
本文将在单个智能体内引导你完成五个步骤：基于 LeRobot AgentTools 构建智能体，在仿真中将演示数据录制为 LeRobotDataset，在同一台机器人上运行策略，仅更改一个关键字参数就将相同的智能体代码部署到实体 SO-101 机器人上，并通过 Zenoh 网格向整个机器人集群广播指令。最后，你可以从 GitHub 克隆可运行的示例应用，并在笔记本电脑的仿真环境中运行。默认路径无需硬件、无需 GPU、也无需 Hugging Face 凭证。本文配套的可运行代码位于 `examples/lerobot/hub_to_hardware.py` 和 `hub_to_hardware.ipynb`。该笔记本默认仅使用仿真环境和 Mock 策略。
你将构建的内容
Strands Robots SDK 将 LeRobot 技术栈以 AgentTools 的形式暴露出来，你可以将它们组合成一个 Strands 智能体。本文中的示例智能体执行四项任务：在仿真环境中录制新的演示数据、将结果以 LeRobotDataset 格式推送到 Hub、在仿真环境中基于相同格式运行策略，以及仅通过更改一个关键字参数，将相同的智能体代码部署到实体机器人上。当你拥有多个机器人时，该智能体可通过内置的对等网格协调整个机器人集群。在硬件录制和标定方面，LeRobot 自带的 CLI 工具（lerobot-record、lerobot-calibrate）负责初始启动；智能体则在此基础上接管后续工作。
图 1. Robot("so100") 默认返回一个基于 MuJoCo 的仿真环境；mode="real" 则返回由 LeRobot 驱动的硬件机器人。两种模式共享相同的 DatasetRecorder 和相同的策略提供器，因此在仿真环境中采集的数据集与在硬件上采集的数据集使用相同的磁盘存储格式 LeRobotDataset。
两个设计决策确保了这一点。首先，Robot("so100") 默认返回仿真环境（无需硬件，无风险），而 mode="real" 返回由 LeRobot 驱动的硬件机器人。智能体代码在两种模式下完全相同。其次，用于写入 LeRobotDataset 的 DatasetRecorder 在仿真路径和 LeRobot 自身的硬件录制路径之间共享，因此在 MuJoCo 中采集的数据集与从实体 SO-101 机器人采集的数据集采用相同的格式。
整个工作流程仅需五行 Python 代码：
from strands_robots import Robot
from strands import Agent
arm = Robot("so100")
agent = Agent(tools=[arm])
agent("Pick up the red cube")
接下来将逐步说明该调用内部实际发生的过程。
前置条件
最低配置（默认仿真路径）
- Python 3.12 或更高版本，运行于 Linux 或 macOS（Apple Silicon 支持 MuJoCo 后端）。
- 一个与 Strands 兼容的模型提供器，用于智能体的推理。可以是带有 AWS 凭证的 Amazon Bedrock、Anthropic API、OpenAI，或本地运行的 Ollama。
- 使用附加安装选项安装 Strands Robots：uv pip install "strands-robots[sim-mujoco,lerobot,mesh]"
仅此而已。本文中的示例在上述三个条件下即可在笔记本电脑上端到端运行。
高级配置（硬件部署、真实策略、Hub 推送）
- 一个具有写入权限的 Hugging Face 账户和令牌，用于将数据集推送到 Hub 以及从 Hub 拉取策略检查点。
- 对于硬件路径：需要一对 SO-101 跟随者与领导者机械臂，或任何其他 LeRobot 支持的机器人。两台设备都需要在 `~/.cache/huggingface/lerobot/calibration/` 路径下存放标定文件。
- 对于本地 GR00T 推理：需要一块至少 16 GB 显存的 NVIDIA GPU，并安装 Docker。该文章使用了 `gr00t_inference` 工具的 `lifecycle="full"` 动作，该动作会一次性拉取镜像、下载检查点并启动容器。
步骤 1 - 设置示例
安装 Strands Robots 并获取示例文件：
uv pip install "strands-robots[sim-mujoco,lerobot,mesh]"
git clone https://github.com/strands-labs/robots.git
cd robots
如果你希望智能体将数据集推送到 Hub 或从 Hub 拉取策略，请导出你的 Hugging Face token。对于本文中的默认模拟路径，此步骤为可选；该示例使用 Mock 策略端到端运行，并将数据集写入本地缓存，无需访问 Hub。
export HF_TOKEN=hf_...
可运行的示例位于 `strands-labs/robots` 仓库中的 `examples/lerobot/hub_to_hardware.py`（Python 脚本）和 `hub_to_hardware.ipynb`（笔记本），与 MuJoCo 和 LIBERO 示例放在一起。推荐从笔记本开始：在 JupyterLab 中打开它，在模拟模式下从上到下运行单元格，无需连接任何硬件。
步骤 2 - 录制演示并推送到 Hub
模拟工具以 LeRobot 在硬件上录制的相同格式录制 LeRobotDataset。无需硬件。模拟工具的 `start_recording` 动作通过相同的 `DatasetRecorder` 类进行写入：关节状态和动作使用相同的 parquet 模式，每个摄像头使用相同的 MP4 布局。智能体提示词几乎完全相同：
from strands import Agent
from strands_robots import Robot
robot = Robot("so100")
agent = Agent(tools=[robot])
agent(
"Record a demonstration of 'pick the red cube and place it in the box' "
"using the Mock policy provider at FPS 30. Write the dataset to "
"my_user/cube_picking_sim and push to the Hub when done."
)
图 2. MuJoCo 模拟中的录制场景：SO-100 机械臂伸向地面上的红色方块，并录制为 LeRobotDataset。此默认路径无需硬件、无需 GPU、无需 Hugging Face 凭证。
Mock 策略是有意为之的：它会生成占位的联合动作，让工作流无需训练好的检查点即可端到端运行。机器人会执行随机动作而非完成抓取，录制结果在结构上是完整的（有效的联合状态、有效的摄像头帧、格式正确的 LeRobotDataset 片段），但该演示本身并不能用作训练数据。下面的第 3 步会替换为 GR00T 或 LerobotLocal 来实现真实的抓取行为。若要在该步骤中看到实际的方块拾取，请运行 `--policy lerobot_local --checkpoint allenai/MolmoAct2-SO100_101`（一个 MolmoAct2 检查点，会从其 config.json 自动检测并通过 LerobotLocal 路径路由）；提示词、数据集格式和智能体代码保持不变。
接下来的结果就是证明。LeRobot 自身的数据集加载器在读取模拟录制数据时，无需任何 Strands 专属的代码路径：
from lerobot.datasets.lerobot_dataset import LeRobotDataset
dataset = LeRobotDataset("my_user/cube_picking_sim")
print(dataset.features)
这个特征字典在结构上与 Hub 上的任何 LeRobot 数据集完全相同：相同的列名、相同的 parquet+MP4 布局、相同的加载器路径。使用硬件录制数据训练的脚本，无需修改即可直接使用模拟录制数据。如果你愿意，从模拟环境推送的数据集可以与硬件录制数据存放在同一个 Hub 仓库中。
一个来自录制 LeRobotDataset 的单个片段，通过录制器写入的每个摄像头对应的 MP4 文件回放，也就是训练脚本读取的同一份磁盘视频文件。
在硬件上录制
要在物理 SO-101 上录制演示而非在模拟环境中录制，请直接使用 LeRobot 的录制 CLI。Strands 集成并未将该命令封装为 AgentTool，因为 LeRobot 本身已经很好地完成了这项工作：
lerobot-calibrate --robot.type=so101_follower --robot.id=my_follower
lerobot-calibrate --robot.type=so101_leader --robot.id=my_leader
lerobot-record \
--robot.type=so101_follower --robot.id=my_follower \
--teleop.type=so101_leader --teleop.id=my_leader \
--dataset.repo_id=my_user/cube_picking \
--dataset.single_task='Pick up the red cube and place it in the box' \
--dataset.num_episodes=25 \
--dataset.push_to_hub=true
通过此命令上传到 Hub 的数据集，其格式与模拟录制数据相同。要基于该数据集微调策略，请运行 LeRobot 的训练 CLI（lerobot-train）；训练本身不在本文讨论范围内，遵循标准的 LeRobot 工作流即可。从第 3 步开始，智能体可以互换地使用原始检查点或微调后的检查点。关于完整的 SO-101 硬件设置、校准指南和故障排除，请参阅示例文件夹中的 README。
第 3 步 - 在模拟环境中运行策略
数据集上传到 Hub 后，下一步就是运行策略。示例代码使用 `Robot()` 工厂函数，并采用默认的模拟模式，然后附加 `gr00t_inference`，以便智能体能够管理推理容器：
from strands import Agent
from strands_robots import Robot, gr00t_inference
robot = Robot("so100")
agent = Agent(tools=[robot, gr00t_inference])
agent(
"Start GR00T inference on port 5555 with the cube-picking checkpoint "
"from my_user/cube-picker. Then ask the robot to pick up the red cube."
)
在底层，智能体会运行 `gr00t_inference(action="lifecycle", lifecycle="full", ...)` 来拉取 GR00T 容器镜像、从 Hub 下载检查点，并启动推理服务。接着，它会在模拟机器人上以 `policy_provider="groot"` 参数执行 `run_policy` 动作，并通过 `policy_config` 字典传入 GR00T 服务的主机和端口（该容器可通过 5555 端口访问）。模拟过程会按照策略生成的动作块逐步推进，最终结果可通过 `Simulation.render` 获取渲染画面。
图 3. 使用训练好的策略（GR00T 或 MolmoAct2 检查点），智能体驱动 SO-100 在模拟环境中抓取红色方块，这正是 Mock 策略所模拟的行为。
对于偏好进程内推理（无需容器，无需 ZeroMQ (ZMQ)）的开发者，可以将 `gr00t_inference` 替换为从 Hub 仓库加载的 `LerobotLocalPolicy` 实例。该提供程序会将任何属于 `lerobot/` 组织的模型 ID 路由到进程内路径：
from strands_robots.policies import create_policy
policy = create_policy("lerobot/act_aloha_sim_transfer_cube_human")
`LerobotLocalPolicy` 支持 ACT、扩散策略、SmolVLA、π0 和 π0.5，以及任何 LeRobot 自身策略注册表能从 `config.json` 解析出的模型。对于附带 `rtc_config` 的流匹配策略（如 π0、SmolVLA），实时分块功能会自动启用。
英伟达最近发布的 Cosmos 3 也可作为同一接口下的策略提供程序使用，因此无论指向哪个提供程序，智能体代码都保持不变。
注意：`LerobotLocalPolicy` 在加载 Hugging Face 模型时会使用 `trust_remote_code=True`。请设置 `STRANDS_TRUST_REMOTE_CODE=1` 以选择启用，并且只加载来自您信任的组织的检查点。
第 4 步 - 将策略部署到物理硬件
此步骤与第 3 步代码相同，仅更改了一个关键字参数。`Robot` 工厂函数会返回一个由 LeRobot 的 `make_robot_from_config` 驱动的、基于硬件的机器人：
robot = Robot(
"so100",
mode="real",
port="/dev/ttyACM0",
data_config="so100_dualcam",
cameras={
"front": {"type": "opencv", "index_or_path": "/dev/video0", "fps": 30},
"wrist": {"type": "opencv", "index_or_path": "/dev/video2", "fps": 30},
},
)
agent = Agent(tools=[robot, gr00t_inference])
agent(
"Start GR00T inference on port 5555 with the cube-picking checkpoint "
"from my_user/cube-picker. Then ask the robot to pick up the red cube."
)
相同的智能体提示词现在将针对物理机械臂运行。硬件路径使用 LeRobot 的机器人抽象层来处理关节指令和摄像头数据读取，而可通过 5555 端口访问的 GR00T 容器则负责生成动作块。
在执行此操作之前，需要先完成从动臂和主动臂的校准。每个设备需运行一次 LeRobot 的校准命令（lerobot-calibrate）；校准文件会存放在 `~/.cache/huggingface/lerobot/calibration/` 目录下，所有涉及硬件的 Strands 代码路径都会从该目录读取校准数据。如果缺少校准文件，智能体会从 LeRobot 驱动层抛出错误。
步骤 5 - 通过网格协调多台机器人
到目前为止，我们一次只驱动一台机器人。网格（mesh）是 Strands Robots 处理多台机器人的方式。想象一下：你桌上的主动臂远程操控另一房间的从动臂，或者五台 SO-101 并行执行相同的仓库任务，又或者人形机器人与移动底盘协同作业——这些都是网格模式。网格基于开源点对点协议 Zenoh 构建，你无需管理 IP 地址、编写发现代码或选择代理；新机器人一旦上线就会自动出现在网格中，智能体可以同时与所有机器人通信。
每个 `Robot()` 和每个 `Simulation()` 都会自动加入 Zenoh 点对点网格。`robot_mesh` 工具为智能体提供了编队操作所需的指令集，包括发现、结构化命令、广播和紧急停止：
agent = Agent(tools=[robot_mesh])
agent(
"List every robot and simulation on the mesh. "
"Then send 'go to home pose' to each one in parallel."
)
智能体调用 `robot_mesh(action="peers")` 枚举本地及已发现的节点，然后通过 `robot_mesh(action="broadcast", ...)` 向所有节点发送带超时设置的结构化命令。添加 `[mesh-iot]` 扩展包可将此流量路由至 AWS IoT Core，实现跨网络编队。项目文档中 `robot_mesh` 工具的操作参考部分涵盖了完整的指令集：订阅、监视、收件箱以及结构化点对点命令。
默认情况下，每个实际执行物理动作的网格操作在运行前都会暂停，等待人工批准中断：包括面向整个机群的广播和紧急停止，以及针对单个节点的告知、发送和停止。你可以通过 `STRANDS_MESH_HITL_ACTIONS` 环境变量来调整这一集合（可设置为 `all`、`none`，或逗号分隔的子集）。首次运行此示例时，你会在终端看到 `robot_mesh-broadcast-approval` 的提示；输入 `y`（或 `yes` / `approve`）即可授权广播。该批准过程独立于大语言模型的工具参数通道进行，因此试图将批准标志混入命令体中的提示注入攻击无法绕过这一关卡。
传输层可在不触及智能体代码的情况下进行扩展。内置的 Zenoh 网格作为自动回退方案：在局域网内，Zenoh 多播无需代理即可完成节点发现；而添加 `[mesh-iot]` 扩展后，可通过 AWS IoT Core（基于 mTLS 的 MQTT5）为云端机群路由流量，并借助 BridgeTransport 将局域网与云端统一在单一 API 背后（通过 `STRANDS_MESH_BACKEND=bridge` 选择该模式）。
对于生产级机群，Device Connect（与 Arm 合作开发的设备感知网络层）负责处理节点发现、在线状态、结构化 RPC、事件路由及安全机制。同一个 `robot_mesh` 工具在 Device Connect 可用时会通过它进行调度，否则自动回退到内置的 Zenoh 网格，因此本文中的智能体代码在两种情况下均无需修改。有关设置及当前可用性，请参阅 Device Connect 文档。
使用示例应用程序进行尝试
完整示例代码位于 GitHub 上的 strands-labs/robots 仓库的 examples/lerobot/ 文件夹中。它将全部五个步骤打包为一个 CLI 脚本（`hub_to_hardware.py`）和一个笔记本文件（`hub_to_hardware.ipynb`）。CLI 默认使用 Mock 策略在仿真环境中端到端运行。无需 GPU、无需 Docker、无需 Hugging Face 凭证。
uv pip install "strands-robots[sim-mujoco,lerobot,mesh]"
git clone https://github.com/strands-labs/robots.git
cd robots
export STRANDS_MESH_LOCAL_DEV=1
python examples/lerobot/hub_to_hardware.py
录制得到的数据集存放在 `~/.cache/huggingface/lerobot/local/strands-cube-pick/` 目录下。若要将数据推送到 Hugging Face Hub 而非保存在本地，需先导出带有写入权限的 `HF_TOKEN`，然后添加 `--hf-user <your-user>` 参数。在步骤 3 中实现真实抓取行为时，可传入 `--policy groot --checkpoint <hf_repo>`（需要 Docker + NVIDIA GPU）或 `--policy lerobot_local --checkpoint <hf_repo>`（需要 GPU 并设置 `STRANDS_TRUST_REMOTE_CODE=1`）。
该笔记本（`examples/lerobot/hub_to_hardware.ipynb`）以逐单元格方式演示了相同的工作流程，并在每一步之间附有说明文字。在 JupyterLab 中打开它，并在模拟模式下从头到尾运行即可。
安全注意事项
本配置中展示的代码片段仅作为将 Strands 机器人与 HuggingFace 集成的“hello world”示例。对于更严肃、面向生产环境的用例，用户需注意以下重要事项：
提示词注入
向智能体提供不可信数据可能导致提示词注入，即不可信的上下文被当作大语言模型的指令来处理。鉴于这些机器人在物理空间中的实际操控能力，这是一个需要重点关注的风险。为缓解此问题，开发者应谨慎确保仅向机器人输入来自可信来源的数据。如果无法保证所有输入数据都可信，开发者应限制智能体可使用的工具，防止机器人执行涉及安全关键的操作。
机器人网格认证行为
本文代码片段中使用的 `STRANDS_MESH_LOCAL_DEV=1` 设置会在无认证或访问控制的情况下初始化机器人网格。这意味着同一网络上的任何设备都可以向机器人集群发送指令。这在受信任的开发环境中是可以接受的，但不适用于不可信网络或生产环境。对于这些场景，必须使用 `STRANDS_MESH_AUTH_MODE=mtls`。
集群范围操作需经操作员批准
robot_mesh 工具的物理驱动动作会影响网络上的其他节点：广播和紧急停止会到达所有节点，而告知、发送和停止则只到达单个目标节点。为防止智能体自主发出这些命令（或在提示词注入下发出），默认情况下，这五个动作都通过人工介入中断机制进行管控。当智能体调用一个受管控的动作时，Strands 运行时会暂停智能体循环，并要求操作员在大语言模型的工具参数之外进行批准。你可以使用 `STRANDS_MESH_HITL_ACTIONS` 环境变量（可选值为 all、none，或逗号分隔的子集）来调整受管控的动作集合。每个动作都有速率限制、命令验证和审计追踪，与中断机制并行运行。在智能体循环之外（如纯脚本或单元测试中），受管控的动作会默认失败关闭。
清理
上述工作流程会启动一个 GR00T 容器，打开硬件上的串行端口，并写入本地数据集缓存。要将环境恢复到干净状态：
- 停止 GR00T 推理容器：使用 `agent.tool.gr00t_inference(action="stop", port=5555)`，或使用 `lifecycle="teardown"` 来同时移除该容器。
- 释放串行端口：如果你运行了硬件路径，请断开 SO-101 从动端和主控端的连接。
- 可选：移除本地数据集缓存。录制的数据集位于 `~/.cache/huggingface/lerobot/<repo_id>` 目录下。你已推送到 Hub 的数据集不受影响。
各部分如何协同工作
该集成的核心设计理念是，Strands Robots 不会重新实现 LeRobot 已经提供的功能。硬件抽象、校准和数据集格式都保留在上游。Strands 增加了 AgentTool 接口，使得这些功能可以通过自然语言进行组合。
这带来了两个结果。对用户而言，Hub 上的每个数据集都是智能体可以扩展、微调并部署的资产，无需任何转换步骤。对开发者而言，仿真数据和硬件数据共享单一文件格式，因此为一种数据编写的训练脚本可以直接用于另一种数据。仿真与现实之间的界限变成了一个部署细节，而非架构上的鸿沟。
下一步方向
图 4. Strands Robots 目录涵盖了机械臂、人形机器人、四足机器人和机械手，全部在同一个 MuJoCo 模拟环境中，并统一通过 `Robot()` 工厂接口调用。本文中的 SO-100 只是众多受支持的具体形态之一。
Strands Robots 完整文档深入介绍了机器人目录、模拟、策略提供器、网格以及 Device Connect。对于更大规模的工作负载，`strands-labs/robots-sim` 仓库提供了更重的模拟后端，包括 Isaac Sim 和 Newton，以及一个 LIBERO 基准测试示例。这两个后端都接入本文展示的同一 `Robot` 抽象层，因此随着规模扩大，智能体代码保持不变。
欢迎在 Apache 2.0 许可下贡献代码。如果你使用此工作流构建了项目，请提交一个 issue，说明哪些部分有效、哪些无效。当开发者的反馈直接落在需要改进的接口上时，SDK 的改进速度最快。
资源
- Strands Robots（SDK、AgentTools、Robot 工厂）：github.com/strands-labs/robots，Apache 2.0
- Strands Robots 文档（完整文档）：strands-labs.github.io/robots
- Strands Robots Sim（示例、模拟后端）：github.com/strands-labs/robots-sim
- 示例：`examples/lerobot/hub_to_hardware.py` 和 `hub_to_hardware.ipynb`
- 如何构建物理 AI 智能体：面向真实世界机器人的自然语言：直播与博客
- 深入探讨物理 AI | 第 1 季第 4 集 | 使用 NVIDIA NeMo Agent Toolkit 和 Bedrock AgentCore 实现自动化：直播
- LeRobot：github.com/huggingface/lerobot - 数据集、策略、硬件驱动
- Strands Agents SDK：github.com/strands-agents/harness-sdk
- SmolVLA：SmolVLA
- Pi0：Pi0
- NVIDIA Isaac-GR00T N1.7：GR00T N1.7
- NVIDIA Cosmos3 Nano：Cosmos 3 Nano
Cagatay Cali 是 AWS 的研究工程师，专注于智能体 AI 和机器人技术。他设计连接 AI 智能体与物理机器人的接口，使开发者能够通过自然语言控制机器人系统，并让任何技能水平的构建者都能轻松进行智能体和机器人开发。
Sundar Raghavan 是 AWS 智能体 AI 基础团队的高级解决方案架构师。他负责 Amazon Bedrock AgentCore 的开发者体验，主导 SDK 和 CLI 的开发，并推动框架与生态系统集成战略。他专注于开发者如何在 AWS 上构建、部署和扩展生产级 AI 智能体。目前，他正将这一关注点延伸至物理 AI 领域，与 Strands Robots 合作，将同样的智能体开发者体验引入机器人技术。
A walkthrough of the LeRobot integration in Strands Robots - one agent loop, from a Hub dataset to a physical robot, with sim-to-real datasets in the same on-disk format and policies you swap with a string.
You have a robot, a folder of demonstration data on the Hugging Face Hub, and a new task you want it to learn. Today that takes five separate tools: one to record new demonstrations, another to train, a third to test in simulation, custom code to deploy on hardware, and yet another to coordinate when you have more than one robot. The pieces work on their own. They don't talk to each other.
Strands Robots is an open source SDK from AWS (Apache 2.0) that exposes robot abstractions, simulation, and the LeRobot stack as AgentTools that you compose into a single Strands agent. The integration is deliberately thin: LeRobot's own scripts handle hardware recording and calibration, and the Strands AgentTools come in for the parts an agent actually orchestrates. The simulation tool records LeRobotDatasets in the same format LeRobot writes on hardware. GR00T and LerobotLocal serve policy inference behind a common interface, and MolmoAct2 checkpoints run through the LerobotLocal path. A peer mesh fans the agent out to remote robots. The dataset format stays exactly as LeRobot wrote it; the agent loop is the glue.
This post walks you through five steps inside a single agent: build the agent over the LeRobot AgentTools, record a demonstration as a LeRobotDataset in simulation, run a policy on the same robot, deploy the same agent code to a physical SO-101 with one keyword argument change, and broadcast commands across a fleet over the Zenoh mesh. At the end, you can clone the working sample application from GitHub and run it on your laptop in simulation. No hardware, no GPU, no Hugging Face credentials needed for the default path. The runnable companion to this post lives at examples/lerobot/hub_to_hardware.py and hub_to_hardware.ipynb. The notebook is sim-only and Mock-policy by default.
What you'll build
The Strands Robots SDK exposes the LeRobot stack as AgentTools that you compose into one Strands agent. The example agent in this post does four things: record new demonstrations in simulation, push the result to the Hub as a LeRobotDataset, run a policy in simulation against that same format, and deploy the same agent code to a physical robot with one keyword argument change. When you have more than one robot, the agent can coordinate the whole fleet through a built-in peer mesh. For hardware recording and calibration, LeRobot's own CLIs (lerobot-record, lerobot-calibrate) handle the bring-up; the agent picks up from there.
Figure 1. Robot("so100") defaults to a MuJoCo-backed simulation; mode="real" returns a hardware robot driven by LeRobot. Both modes share the same DatasetRecorder and the same policy providers, so a dataset captured in sim and a dataset captured on hardware use the same on-disk LeRobotDataset format.
Two design choices make this work. First, Robot("so100") returns a simulation by default (no hardware, no risk), and mode="real" returns a hardware-backed robot driven by LeRobot. The agent code is identical across both modes. Second, the DatasetRecorder that writes a LeRobotDataset is shared between the simulation path and LeRobot's own hardware recording, so a dataset captured in MuJoCo and one captured from a physical SO-101 are in the same format.
The whole workflow in five lines of Python:
from strands_robots import Robot
from strands import Agent
arm = Robot("so100")
agent = Agent(tools=[arm])
agent("Pick up the red cube")
What follows is what's actually happening inside that call, step by step.
Prerequisites
Minimal (default simulation path)
- Python 3.12+, on Linux or macOS (Apple Silicon supported for the MuJoCo backend).
- A Strands-compatible model provider for the agent's reasoning. Amazon Bedrock with AWS credentials, the Anthropic API, OpenAI, or Ollama running locally.
- Strands Robots installed with the install extras:
uv pip install "strands-robots[sim-mujoco,lerobot,mesh]"
That's it. The example in this post runs end-to-end on a laptop with these three.
Advanced (hardware deployment, real policies, Hub push)
- A Hugging Face account and token with write permission, for pushing datasets and pulling policy checkpoints from the Hub.
- For the hardware path: an SO-101 follower and leader pair, or any other LeRobot-supported robot. Both devices need calibration files under
~/.cache/huggingface/lerobot/calibration/. - For local GR00T inference: an NVIDIA GPU with at least 16 GB of video memory and Docker installed. The post uses the gr00t_inference tool's lifecycle="full" action, which pulls the image, downloads a checkpoint, and starts the container in one call.
Step 1 - Set up the example
Install Strands Robots and get the example files:
uv pip install "strands-robots[sim-mujoco,lerobot,mesh]"
git clone https://github.com/strands-labs/robots.git
cd robots
Export your Hugging Face token if you want the agent to push datasets or pull policies from the Hub. This is optional for the default simulation path in this post; the example runs end-to-end with the Mock policy and writes the dataset to your local cache without needing Hub access.
export HF_TOKEN=hf_...
The runnable example lives at examples/lerobot/hub_to_hardware.py (Python script) and hub_to_hardware.ipynb (notebook), in the strands-labs/robots repository alongside the MuJoCo and LIBERO examples. The notebook is the recommended starting point: open it in JupyterLab and run cells top-to-bottom in simulation mode without any hardware connected.
Step 2 - Record demonstrations and push to the Hub
The simulation tool records LeRobotDatasets in the same format LeRobot writes on hardware. No hardware required. The Simulation tool's start_recording action writes through the same DatasetRecorder class: same parquet schema for joint states and actions, same per-camera MP4 layout. The agent prompt is almost identical:
from strands import Agent
from strands_robots import Robot
robot = Robot("so100")
agent = Agent(tools=[robot])
agent(
"Record a demonstration of 'pick the red cube and place it in the box' "
"using the Mock policy provider at FPS 30. Write the dataset to "
"my_user/cube_picking_sim and push to the Hub when done."
)
Figure 2. The recording scene in MuJoCo simulation: the SO-100 arm reaching toward a red cube on the ground plane, captured to a LeRobotDataset. No hardware, no GPU, no Hugging Face credentials needed for this default path.
The Mock policy is intentional: it generates placeholder joint actions so the workflow runs end-to-end without a trained checkpoint. The robot moves through random motions rather than completing the grasp, and the recording is structurally complete (valid joint states, valid camera frames, a well-formed LeRobotDataset episode), but the demonstration itself isn't useful as training data. Step 3 below swaps in GR00T or LerobotLocal for real grasping behavior. To see actual cube-picking in this step, run --policy lerobot_local --checkpoint allenai/MolmoAct2-SO100_101 (a MolmoAct2 checkpoint, auto-detected from its config.json and routed through the LerobotLocal path); the prompt, dataset format, and agent code stay the same.
The proof is what happens next. LeRobot's own dataset loader reads the sim-recorded data with no Strands-specific code path:
from lerobot.datasets.lerobot_dataset import LeRobotDataset
dataset = LeRobotDataset("my_user/cube_picking_sim")
print(dataset.features)
This features dict is identical in shape to any LeRobot dataset on the Hub: same column names, same parquet+MP4 layout, same loader path. Training scripts that consume hardware-recorded data consume the sim-recorded data without modification. Datasets pushed from sim sit alongside hardware recordings in the same Hub repository if you want them to.
A single episode from a recorded LeRobotDataset, played back from the per-camera MP4 the recorder wrote, the same on-disk video a training script reads.
Recording on hardware
To record demonstrations on a physical SO-101 instead of simulation, use LeRobot's record CLI directly. The Strands integration doesn't wrap that command as an AgentTool because LeRobot already does the job cleanly:
lerobot-calibrate --robot.type=so101_follower --robot.id=my_follower
lerobot-calibrate --robot.type=so101_leader --robot.id=my_leader
lerobot-record \
--robot.type=so101_follower --robot.id=my_follower \
--teleop.type=so101_leader --teleop.id=my_leader \
--dataset.repo_id=my_user/cube_picking \
--dataset.single_task='Pick up the red cube and place it in the box' \
--dataset.num_episodes=25 \
--dataset.push_to_hub=true
The dataset that lands on the Hub from this command is in the same format as the simulation recording. To fine-tune a policy on it, run LeRobot's training CLI (lerobot-train); training itself is out of scope for this post and follows the standard LeRobot workflow. From Step 3 onward, the agent picks up either the original or a fine-tuned checkpoint interchangeably. For full SO-101 hardware setup, calibration walkthroughs, and troubleshooting, see the README in the example folder.
Step 3 - Run a policy in simulation
With the dataset on the Hub, the next step is to run a policy. The example uses the Robot() factory in its default sim mode, then attaches gr00t_inference so the agent can manage the inference container:
from strands import Agent
from strands_robots import Robot, gr00t_inference
robot = Robot("so100")
agent = Agent(tools=[robot, gr00t_inference])
agent(
"Start GR00T inference on port 5555 with the cube-picking checkpoint "
"from my_user/cube-picker. Then ask the robot to pick up the red cube."
)
Under the hood, the agent runs gr00t_inference(action="lifecycle", lifecycle="full", ...) to pull the GR00T container image, download the checkpoint from the Hub, and start the inference service. It then runs a run_policy action on the simulated robot with policy_provider="groot", passing the GR00T service's host and port in the policy_config dict (the container is reachable on port 5555). The simulation steps with the policy's action chunks, and a render of the result is available via Simulation.render.
Figure 3. With a trained policy (a GR00T or MolmoAct2 checkpoint), the agent drives the SO-100 to grasp the red cube in simulation, the behavior the Mock policy stands in for.
For developers who prefer in-process inference (no container, no ZeroMQ (ZMQ)), swap gr00t_inference for a LerobotLocalPolicy instance loaded from a Hub repository. The provider routes any model ID under the lerobot/ organization to the in-process path:
from strands_robots.policies import create_policy
policy = create_policy("lerobot/act_aloha_sim_transfer_cube_human")
LerobotLocalPolicy supports ACT, Diffusion Policy, SmolVLA, π0, and π0.5, anything LeRobot's own policy registry can resolve from a config.json. Real-Time Chunking turns on automatically for flow-matching policies that ship an rtc_config (π0, SmolVLA).
NVIDIA's recently released Cosmos 3 is also available as a policy provider behind the same interface, so the agent code stays the same whichever provider you point it at.
Note: LerobotLocalPolicy loads Hugging Face models with trust_remote_code=True. Set STRANDS_TRUST_REMOTE_CODE=1 to opt in, and only load checkpoints from organizations you trust.
Step 4 - Deploy the policy to physical hardware
This is the same code as Step 3, with one keyword argument changed. The Robot factory returns a hardware-backed robot driven by LeRobot's make_robot_from_config:
robot = Robot(
"so100",
mode="real",
port="/dev/ttyACM0",
data_config="so100_dualcam",
cameras={
"front": {"type": "opencv", "index_or_path": "/dev/video0", "fps": 30},
"wrist": {"type": "opencv", "index_or_path": "/dev/video2", "fps": 30},
},
)
agent = Agent(tools=[robot, gr00t_inference])
agent(
"Start GR00T inference on port 5555 with the cube-picking checkpoint "
"from my_user/cube-picker. Then ask the robot to pick up the red cube."
)
The same agent prompt now runs against a physical arm. The hardware path uses LeRobot's robot abstraction for joint commands and camera reads, and the GR00T container reachable on port 5555 generates the action chunks.
Before this runs against your SO-101, calibration for both follower and leader has to be in place. Run LeRobot's calibration command (lerobot-calibrate) once per device; the files land under ~/.cache/huggingface/lerobot/calibration/ and any Strands code path that touches the hardware reads them from there. If a calibration is missing, the agent surfaces the error from the LeRobot driver layer.
Step 5 - Coordinate multiple robots with the mesh
Up to now we've driven one robot at a time. The mesh is how Strands Robots handles more than one. Picture a leader arm on your desk teleoperating a follower arm in another room, or five SO-101s running the same warehouse task in parallel, or a humanoid coordinating with a mobile base. All of those are mesh patterns. The mesh is built on Zenoh, an open source peer-to-peer protocol, and you don't manage IP addresses, write discovery code, or pick a broker; new robots show up on the mesh the moment they come up, and the agent can talk to all of them at once.
Every Robot() and every Simulation() joins a Zenoh peer mesh automatically. The robot_mesh tool gives the agent a vocabulary for fleet operations such as discovery, structured commands, broadcasts, and emergency stop:
agent = Agent(tools=[robot_mesh])
agent(
"List every robot and simulation on the mesh. "
"Then send 'go to home pose' to each one in parallel."
)
The agent calls robot_mesh(action="peers") to enumerate locals and discovered peers, then robot_mesh(action="broadcast", ...) to send the structured command to every peer with a timeout. Add the [mesh-iot] extra to route this traffic over AWS IoT Core for cross-network fleets. The robot_mesh tool's action reference in the project documentation covers the full vocabulary: subscribe, watch, inbox, and structured peer-to-peer commands.
By default, every physically-actuating mesh action pauses for a human approval interrupt before it runs: the fleet-wide broadcast and emergency_stop, plus the single-peer tell, send, and stop. You can tune this set with the STRANDS_MESH_HITL_ACTIONS environment variable (set it to all, none, or a comma-separated subset). The first time you run this example, you'll see a robot_mesh-broadcast-approval prompt in your terminal; type y (or yes / approve) to authorize the broadcast. The approval is delivered out-of-band of the LLM's tool arguments, so a prompt-injection attempt that tries to slip an approval flag into the command body cannot bypass the gate.
The transport scales without touching agent code. The built-in Zenoh mesh is the automatic fallback: on the LAN, Zenoh multicast handles peer discovery with no broker, and adding the [mesh-iot] extra routes traffic through AWS IoT Core (MQTT5 with mTLS) for cloud fleets, with a BridgeTransport that fans LAN and cloud behind one API (select it with STRANDS_MESH_BACKEND=bridge).
For production fleets, Device Connect, a device-aware networking layer developed in collaboration with Arm, handles discovery, presence, structured RPC, event routing, and safety. The same robot_mesh tool dispatches through Device Connect when it is available and falls back to the built-in Zenoh mesh otherwise, so the agent code in this post is unchanged either way. See the Device Connect documentation for setup and current availability.
Try it using the sample application
The full sample is on GitHub at strands-labs/robots in the examples/lerobot/ folder. It packages all five steps into a single CLI script (hub_to_hardware.py) and a notebook (hub_to_hardware.ipynb). The CLI defaults run end-to-end in simulation with the Mock policy. No GPU, no Docker, no Hugging Face credentials needed.
uv pip install "strands-robots[sim-mujoco,lerobot,mesh]"
git clone https://github.com/strands-labs/robots.git
cd robots
export STRANDS_MESH_LOCAL_DEV=1
python examples/lerobot/hub_to_hardware.py
The recorded dataset lands at ~/.cache/huggingface/lerobot/local/strands-cube-pick/. To push to the Hugging Face Hub instead of keeping it local, pass --hf-user <your-user> after exporting HF_TOKEN with write scope. For real grasping behavior in Step 3, pass --policy groot --checkpoint <hf_repo> (requires Docker + NVIDIA GPU) or --policy lerobot_local --checkpoint <hf_repo> (requires a GPU and STRANDS_TRUST_REMOTE_CODE=1).
The notebook (examples/lerobot/hub_to_hardware.ipynb) walks through the same workflow cell by cell, with narration between each step. Open it in JupyterLab and run top-to-bottom in simulation mode.
Security Considerations
The code snippets shown in this setup represent a “hello world” example of setting up Strands Robots with HuggingFace. For more serious, production-ready use cases there are some important considerations users should be aware of:
Prompt Injection
Supplying untrusted data into agents can lead to prompt injection, where untrustworthy context is treated as LLM instructions. Given the actuation of these robots in physical space, this is an important risk to track. To mitigate this behavior, developers should be careful to feed the robots only data that comes from a trusted source. If not all input data can be trusted, developers should restrict the tools available to the agent to prevent the robots from making safety-critical actions.
Robot Mesh Auth Behavior
The STRANDS_MESH_LOCAL_DEV=1 setting shared in the code snippets in this blog initializes the robot mesh without authentication or access controls. This means that any device on the same network can provide commands to the robot fleet. This is acceptable for trusted development environments, but is not suitable for untrusted networks or production environments. For these use cases, STRANDS_MESH_AUTH_MODE=mtls is required.
Operator approval for fleet-wide actions
The robot_mesh tool's physically-actuating actions affect peers on the network: broadcast and emergency_stop reach every peer, while tell, send, and stop reach a single targeted peer. To prevent an agent from issuing these commands autonomously (or under prompt injection), all five are gated behind a human-in-the-loop interrupt by default. When the agent invokes a gated action, the Strands runtime pauses the agent loop and asks the operator to approve out-of-band of the LLM's tool arguments. You can adjust the gated set with the STRANDS_MESH_HITL_ACTIONS environment variable (all, none, or a comma-separated subset). Per-action rate limits, command validation, and an audit trail run alongside the interrupt. Outside an agent loop (a bare script or unit test), the gated actions fail closed.
Clean up
The preceding workflow starts a GR00T container, opens serial ports on hardware, and writes a local dataset cache. To return your environment to a clean state:
- Stop the GR00T inference container:
agent.tool.gr00t_inference(action="stop", port=5555), or uselifecycle="teardown"to remove the container as well. - Release the serial ports: if you ran the hardware path, disconnect the SO-101 follower and leader.
- Optionally remove the local dataset cache: the recorded dataset lives under
~/.cache/huggingface/lerobot/<repo_id>. Datasets you pushed to the Hub are unaffected.
How this fits together
The integration's central design choice is that Strands Robots doesn't reimplement what LeRobot already provides. Hardware abstraction, calibration, and the dataset format stay upstream. Strands adds the AgentTool surface that makes them composable from natural language.
Two consequences follow. For users, every dataset on the Hub is an asset an agent can extend, fine-tune from, and deploy against with no conversion step. For developers, simulation data and hardware data share a single file format, so training scripts written for one consume the other unchanged. The line between sim and real becomes a deployment detail, not an architectural divide.
Where to go from here
Figure 4. The Strands Robots catalog spans arms, humanoids, quadrupeds, and hands, all in the same MuJoCo simulation and behind the same Robot() factory. The SO-100 in this post is one of many supported embodiments.
The full Strands Robots documentation covers the robot catalog, simulation, policy providers, the mesh, and Device Connect in depth. For larger workloads, the strands-labs/robots-sim repository hosts heavier simulation backends including Isaac Sim and Newton, plus a LIBERO benchmark example. Both backends plug into the same Robot abstraction shown in this post, so the agent code stays the same as you scale up.
Contributions are welcome under Apache 2.0. If you build something with this workflow, open an issue with what worked and what didn't. The SDK improves fastest when developer feedback lands directly on the surface that needs it.
Resources
- Strands Robots (SDK, AgentTools, Robot factory): github.com/strands-labs/robots, Apache 2.0
- Strands Robots docs (full documentation): strands-labs.github.io/robots
- Strands Robots Sim (examples, simulation backends): github.com/strands-labs/robots-sim
- The example: examples/lerobot/hub_to_hardware.py and hub_to_hardware.ipynb
- How to Build Physical AI Agents: Natural Language for Real-World Robotics: Live Stream and Blog
- Diving Deep on Physical AI | S1E4 | Automate with NVIDIA NeMo Agent Toolkit and Bedrock AgentCore: Live Stream
- LeRobot: github.com/huggingface/lerobot - datasets, policies, hardware drivers
- Strands Agents SDK: github.com/strands-agents/harness-sdk
- SmolVLA: SmolVLA
- Pi0: Pi0
- NVIDIA Isaac-GR00T N1.7: GR00T N1.7
- NVIDIA Cosmos3 Nano: Cosmos 3 Nano
Authors
Cagatay Cali is a Research Engineer at AWS focused on Agentic AI and robotics. He designs interfaces that connect AI agents to physical robots, enabling developers to control robotic systems through natural language and making agents and robotics development accessible to builders at any skill level.
Sundar Raghavan is a Sr Solutions Architect at AWS on the Agentic AI Foundations team. He leads the developer experience for Amazon Bedrock AgentCore, owning the SDK and CLI, and drives the framework and ecosystem integrations strategy. He focuses on how developers build, deploy, and scale production AI agents on AWS. He is currently extending that focus into physical AI, collaborating on Strands Robots to bring the same agent developer experience to robotics.