Welcome to ROSMASTER M3 repository

17.Binocular camera depth algorithm

13.Binocular camera depth algorithm

Binocular camera depth algorithm

Binocular camera depth algorithm1. Use of the stereo depth estimation algorithm1.1. Start with the RDK X5 official stereo cameraFunction installation and updateStart the binocular model launch file, which includes the algorithm and the startup of the binocular camera nodeExpected results1.2. Offline local image re-injection1.3. In addition, you can start the node through the component method2. Interface descriptionSubscribe to topicPost a topicParameter3. Notes

The stereo depth estimation algorithm is a StereoNet model trained on the SceneFlow dataset using the horizon [OpenExplorer]

The algorithm input is the stereo image data, which are the left and right views. The algorithm output is the depth of the left view.

This example uses the mipi stereo camera as the image data input source, uses the BPU for algorithm reasoning, publishes a topic message containing the left image of the stereo image and the perception result, and renders the algorithm result on the PC rviz2.

1. Use of the stereo depth estimation algorithm

1.1. Start with the RDK X5 official stereo camera

Connect the power supply
Connect the binocular mipi camera to the RDK X5, and note that the blue side is facing the HDMI port. Do not connect the wires in reverse, which will cause the left and right images to be swapped and the binocular algorithm to run incorrectly.

Connect the development board and the display via an HDMI cable
Log in to RDK X5 via MobaXterm or other ssh tools (please log in as the root user. Root login username: root password: root)

Function installation and update

Before running the binocular depth algorithm, you need to ensure that the tros-humble-mipi-cam function package is above version 2.3.6 and the tros-humble-hobot-stereonet function package is above version 2.3.3. The command to query the function package version is as follows:


apt list | grep tros-humble-mipi-cam


xxxxxxxxxx
apt list | grep tros-humble-hobot-stereonet

If the version does not meet the requirements, you need to update it. Run the following command in the terminal of the RDK system to update it.


xxxxxxxxxx
apt update


xxxxxxxxxx
apt upgrade

Note: In some old image versions, there may be a tros-humble-stereonet-model function package. You need to uninstall it before you can update the function package. Run the following command in the terminal of the RDK system:

Confirm that the system has the tros-humble-stereonet-model function package


xxxxxxxxxx
apt list | grep tros-humble-hobot-stereonet-model

Uninstall


xxxxxxxxxx
sudo apt-get remove tros-humble-stereonet-model

If the uninstallation fails, execute the forced deletion command (optional)


 Use MIPI camera
YOLO environment construction