Reachy 2

Reachy 2 is an open-source humanoid robot made by Pollen Robotics, specifically designed for the development of embodied AI and real-world applications. Check out Pollen Robotics website, or access Reachy 2 documentation for more information on the platform!

Teleoperate Reachy 2

Currently, there are two ways to teleoperate Reachy 2:

• Pollen Robotics’ VR teleoperation (not included in LeRobot).
• Robot-to-robot teleoperation (use one Reachy 2 to control another).

Reachy 2 Simulation

(Linux only) You can run Reachy 2 in simulation (Gazebo or MuJoCo) using the provided Docker image.

1. Install Docker Engine.
2. Run (for MuJoCo):

docker run --rm -it \
  --name reachy \
  --privileged \
  --network host \
  --ipc host \
  --device-cgroup-rule='c 189:* rwm' \
  --group-add audio \
  -e ROS_DOMAIN_ID="$ROS_DOMAIN_ID" \
  -e DISPLAY="$DISPLAY" \
  -e RCUTILS_CONSOLE_OUTPUT_FORMAT="[{severity}]: {message}" \
  -e REACHY2_CORE_SERVICE_FAKE="${REACHY2_CORE_SERVICE_FAKE:-true}" \
  -v /dev:/dev \
  -v "$HOME/.reachy_config":/home/reachy/.reachy_config_override \
  -v "$HOME/.reachy.log":/home/reachy/.ros/log \
  -v /usr/lib/x86_64-linux-gnu:/opt/host-libs \
  --entrypoint /package/launch.sh \
  pollenrobotics/reachy2_core:1.7.5.9_deploy \
  start_rviz:=true start_sdk_server:=true mujoco:=true

If MuJoCo runs slowly (low simulation frequency), append -e LD_LIBRARY_PATH="/opt/host-libs:$LD_LIBRARY_PATH" \ to the previous command to improve performance:

Copied

docker run --rm -it \
  --name reachy \
  --privileged \
  --network host \
  --ipc host \
  --device-cgroup-rule='c 189:* rwm' \
  --group-add audio \
  -e ROS_DOMAIN_ID="$ROS_DOMAIN_ID" \
  -e DISPLAY="$DISPLAY" \
  -e RCUTILS_CONSOLE_OUTPUT_FORMAT="[{severity}]: {message}" \
  -e REACHY2_CORE_SERVICE_FAKE="${REACHY2_CORE_SERVICE_FAKE:-true}" \
  -e LD_LIBRARY_PATH="/opt/host-libs:$LD_LIBRARY_PATH" \
  -v /dev:/dev \
  -v "$HOME/.reachy_config":/home/reachy/.reachy_config_override \
  -v "$HOME/.reachy.log":/home/reachy/.ros/log \
  -v /usr/lib/x86_64-linux-gnu:/opt/host-libs \
  --entrypoint /package/launch.sh \
  pollenrobotics/reachy2_core:1.7.5.9_deploy \
  start_rviz:=true start_sdk_server:=true mujoco:=true

Setup

Prerequisites

• On your robot, check the service images meet the minimum versions:
  • reachy2-core >= 1.7.5.2
  • webrtc >= 2.0.1.1

Then, if you want to use VR teleoperation:

• Install the Reachy 2 teleoperation application. Use version >=v1.2.0

We recommend using two computers: one for teleoperation (Windows required) and another for recording with LeRobot.

Install LeRobot

Follow the installation instructions to install LeRobot.

Install LeRobot with Reachy 2 dependencies:

pip install -e ".[reachy2]"

Step 1: Recording

Get Reachy 2 IP address

Before starting teleoperation and data recording, find the robot’s IP address. We strongly recommend connecting all devices (PC and robot) via Ethernet.

Launch recording

Use LeRobot’s lerobot-record command to manage recording sessions. The pollen_data_acquisition_server workflow previously documented here is no longer available.

When using the Reachy 2 VR teleoperation application, choose “Standard session”. The VR application controls the robot’s motion, while lerobot-record manages the dataset and starts and stops episodes.

Record with lerobot-record

Reachy 2 is fully supported by LeRobot’s recording features.

Example: start a recording without the mobile base: First add reachy2 and reachy2_teleoperator to the imports of the record script. Then you can use the following command:

lerobot-record \
    --robot.type=reachy2 \
    --robot.ip_address=192.168.0.200 \
    --robot.id=r2-0000 \
    --robot.use_external_commands=true \
    --robot.with_mobile_base=false \
    --teleop.type=reachy2_teleoperator \
    --teleop.ip_address=192.168.0.200 \
    --teleop.with_mobile_base=false \
    --robot.with_torso_camera=true \
    --dataset.repo_id=pollen_robotics/record_test \
    --dataset.single_task="Reachy 2 recording test" \
    --dataset.num_episodes=1 \
    --dataset.episode_time_s=5 \
    --dataset.fps=15 \
    --dataset.push_to_hub=true \
    --dataset.private=true \
    --dataset.streaming_encoding=true \
    --dataset.encoder_threads=2 \
    # --dataset.rgb_encoder.vcodec=auto \
    --display_data=true

Specific Options

Extended setup overview (all options included):

lerobot-record \
    --robot.type=reachy2 \
    --robot.ip_address=192.168.0.200 \
    --robot.use_external_commands=true \
    --robot.with_mobile_base=true \
    --robot.with_l_arm=true \
    --robot.with_r_arm=true \
    --robot.with_neck=true \
    --robot.with_antennas=true \
    --robot.with_left_teleop_camera=true \
    --robot.with_right_teleop_camera=true \
    --robot.with_torso_camera=false \
    --robot.camera_width=640 \
    --robot.camera_height=480 \
    --robot.disable_torque_on_disconnect=false \
    --robot.max_relative_target=5.0 \
    --teleop.type=reachy2_teleoperator \
    --teleop.ip_address=192.168.0.200 \
    --teleop.use_present_position=false \
    --teleop.with_mobile_base=false \
    --teleop.with_l_arm=true \
    --teleop.with_r_arm=true \
    --teleop.with_neck=true \
    --teleop.with_antennas=true \
    --dataset.repo_id=pollen_robotics/record_test \
    --dataset.single_task="Reachy 2 recording test" \
    --dataset.num_episodes=1 \
    --dataset.episode_time_s=5 \
    --dataset.fps=15 \
    --dataset.push_to_hub=true \
    --dataset.private=true \
    --dataset.streaming_encoding=true \
    --dataset.encoder_threads=2 \
    # --dataset.rgb_encoder.vcodec=auto \
    --display_data=true

--robot.use_external_commands

Determine whether LeRobot robot.send_action() sends commands to the robot. Must be set to false while using the VR teleoperation application, as the app already sends commands.

--teleop.use_present_position

Determine whether the teleoperator reads the goal or present position of the robot. Must be set to true if a compliant Reachy 2 is used to control another one.

Use the relevant parts

From our initial tests, recording all joints when only some are moving can reduce model quality with certain policies. To avoid this, you can exclude specific parts from recording and replay using:

--robot.with_<part>=false

with <part> being one of : mobile_base, l_arm, r_arm",neck,antennas`. It determine whether the corresponding part is recorded in the observations. True if not set.

By default, all parts are recorded.

The same per-part mechanism is available in reachy2_teleoperator as well.

--teleop.with\_<part>

with <part> being one of : mobile_base, l_arm, r_arm",neck,antennas`. Determine whether the corresponding part is recorded in the actions. True if not set.

Important: In a given session, the enabled parts must match on both the robot and the teleoperator. For example, if the robot runs with --robot.with_mobile_base=false, the teleoperator must disable the same part --teleoperator.with_mobile_base=false.

Use the relevant cameras

You can do the same for cameras. Enable or disable each camera with default parameters using:

--robot.with_left_teleop_camera=<true|false> \
--robot.with_right_teleop_camera=<true|false> \
--robot.with_torso_camera=<true|false>

By default, no camera is recorded, all camera arguments are set to false. If you want to, you can use custom width and height parameters for Reachy 2’s cameras using the --robot.camera_width & --robot.camera_height argument:

--robot.camera_width=1920 \
--robot.camera_height=1080

This will change the resolution of all 3 default robot cameras (enabled by the above bool arguments).

If you want, you can add additional cameras other than the ones in the robot as usual with:

--robot.cameras="{ extra: {type: opencv, index_or_path: 42, width: 640, height: 480, fps: 30}}" \

Step 2: Replay

Make sure the robot is configured with the same parts as the dataset:

lerobot-replay \
    --robot.type=reachy2 \
    --robot.ip_address=192.168.0.200 \
    --robot.use_external_commands=false \
    --robot.with_mobile_base=false \
    --dataset.repo_id=pollen_robotics/record_test \
    --dataset.episode=0

Step 3: Train

lerobot-train \
  --dataset.repo_id=pollen_robotics/record_test \
  --policy.type=act \
  --output_dir=outputs/train/reachy2_test \
  --job_name=reachy2 \
  --policy.device=mps \
  --wandb.enable=true \
  --policy.repo_id=pollen_robotics/record_test_policy

Step 4: Evaluate

lerobot-eval \
  --robot.type=reachy2 \
  --robot.ip_address=192.168.0.200 \
  --dataset.repo_id=pollen_robotics/eval_record_test \
  --dataset.single_task="Evaluate reachy2 policy" \
  --dataset.num_episodes=10 \
  --policy.path=outputs/train/reachy2_test/checkpoints/last/pretrained_model

Sections that were moved:

[ (Optional but recommended) Install pollen_data_acquisition_server ]

[ Option 1: Using Pollen data acquisition server (recommended for VR teleop) ]

[ Option 2: Using lerobot.record ]