3. Read IMU Data

1. Camera Introduction

The D-ROBOTICS binocular IMU camera comes with complete calibration parameters, including binocular calibration parameters, IMU intrinsic parameters, and extrinsic parameters between the binocular camera and IMU. Users can use it directly without additional calibration. Using these parameters, high-precision depth maps can be computed through binocular stereo matching algorithms, enabling real-time 3D environment perception. Meanwhile, the camera data can be applied to open-source Visual-Inertial Odometry (VIO) algorithms such as OpenVINS, which can compute camera pose and trajectory information. Suitable for robot navigation, obstacle avoidance and other applications, providing developers with plug-and-play depth perception and visual-inertial fusion capabilities.

MIPI camera code repository: https://github.com/D-Robotics/hobot_mipi_cam

2. Supported Platforms

3. Preparation

3.1. RDK Platform

1. RDK has been flashed with RDK OS system
2. RDK has successfully installed TogetheROS.Bot
3. Confirm that the PC can access RDK through the network

3.2. System and Package Versions

• If the system image version doesn't meet requirements, please refer to the corresponding section for image flashing
• If the package version doesn't meet requirements, execute the following commands to update:

xxxxxxxxxx
sudo apt update
sudo apt install --only-upgrade tros-humble-hobot-stereonet
sudo apt install --only-upgrade tros-humble-mipi-cam

• If the above commands cannot update to the latest version, you need to modify the apt source file to beta source:

xxxxxxxxxx
# Modify to beta source, execute the following command:
sudo echo 'deb [signed-by=/usr/share/keyrings/sunrise.gpg] http://archive.d-robotics.cc/ubuntu-rdk-x5-beta  jammy main' | sudo tee /etc/apt/sources.list.d/sunrise.list
apt update
​
# If you need to switch back to the official source, execute the following command:
sudo echo 'deb [signed-by=/usr/share/keyrings/sunrise.gpg] http://archive.d-robotics.cc/ubuntu-rdk-x5  jammy main' | sudo tee /etc/apt/sources.list.d/sunrise.list
apt update

4. Camera Connection and Preparation

4.1. Binocular IMU Camera

• Note that the camera has two versions: black metal shell and acrylic shell, both with the same functionality (the cable may vary based on what you receive)

4.2. Hardware Connection

1. Set the switch behind the module to EXT mode, not LPWM mode

2. No other pins need to be connected except the black Dupont wire connecting to RDK X5's 37 pin, which is needed for external trigger control of camera and IMU time synchronization

3. Camera 3Pin pin definition

   

   

4. For the motherboard pin definition, please refer to 3.1.1 Pin Definition and Application

4.3. RDK X5 Configuration

1. RDK X5 needs to be configured to read IMU data. First check if the system version meets the requirements, need 3.4.1 or above:

xxxxxxxxxx
cat /etc/version

2. Run the following command on RDK X5 for configuration:

xxxxxxxxxx
srpi-config

3. After reboot, confirm that the directory /sys/bus/iio/devices/ has iio:device1 and iio:device2, which means the configuration is successful:

xxxxxxxxxx
ll /sys/bus/iio/devices/

5. Read IMU Data

5.1 Execute the following commands to start the camera

Switch to root user, username: root, password: root

xxxxxxxxxx
source /opt/tros/humble/setup.bash

xxxxxxxxxx
ros2 launch mipi_cam mipi_cam_dual_channel.launch.py \
mipi_channel:=2 mipi_channel2:=0 \
mipi_lpwm_enable:=True mipi_frame_ts_type:=realtime \
mipi_image_width:=816 mipi_image_height:=960 \
mipi_image_framerate:=10.0 mipi_gdc_enable:=True \
mipi_out_format:=nv12 \
log_level:=info

Parameter explanation:

• mipi_channel:=2 mipi_channel2:=0 Adjust left-right camera stitching order
• mipi_lpwm_enable:=True Enable LPWM hardware synchronization
• mipi_frame_ts_type:=realtime Timestamp uses system time
• mipi_image_width:=816 mipi_image_height:=960 Adjust image resolution, maximum can be set to 1088*1280
• mipi_image_framerate:=10.0 Adjust camera frame rate, maximum can be adjusted to 30.0
• mipi_gdc_enable:=True Enable GDC correction, will publish corrected binocular images, otherwise will publish images with distortion
• mipi_out_format:=nv12 Set image format, supports nv12/bgr8
• log_level:=info Log level, info level can print calibration parameters, if you don't need to display that much information, set to warn

5.2 View IMU Data

View camera topic information

xxxxxxxxxx
ros2 topic list -v

• /image_combine_raw is the up-down stitched binocular image, mipi_channel:=2 mipi_channel2:=0 parameter controls the stitching order
• /image_left_raw /image_right_raw are left-right camera data topics, mipi_channel:=2 mipi_channel2:=0 parameter controls the order
• /imu_data is the IMU data topic, publishing gyroscope and accelerometer data

View IMU topic data

xxxxxxxxxx
ros2 topic echo

You can see the change of IMU data by changing the camera's posture.

Note: In the IMU data topic, angular_velocity unit is rad/s, linear_acceleration unit is m/s², the gravity acceleration value is 9.81