1. ROS+opencv application

ROS transmits images in its own sensor_msgs/Image message format and cannot directly perform image processing, but the provided [CvBridge] can perfectly convert and be converted image data formats. [CvBridge] is a ROS library, equivalent to the bridge between ROS and Opencv.

Opencv and ROS image data conversion is shown in the figure below:

This lesson uses two cases to show how to use CvBridge for data conversion.

1.1. Camera topic data

In the previous section, we have set up the camera driver environment and the color images, depth images and infrared IR images that can be viewed. We can first check what topics are published and what the content of the image data is after driving the camera. Enter the following command in the terminal to start the camera,

xxxxxxxxxx
#astrapropluscamera
ros2 launch orbbec_camera astra.launch.xml
#gemini2camera
ros2 launch orbbec_camera gemini2.launch.py

Then, view the topic data list through the following command,

xxxxxxxxxx
ros2 topic list

Among them, /camera/color/image_raw and /camera/depth/image_raw are the data of color image and depth image. You can use the following command to view the data content of a certain frame.

xxxxxxxxxx
#View RGB image topic data content
ros2 topic echo /camera/color/image_raw
#View Depth image topic data content
ros2 topic echo /camera/depth/image_raw

Color image:

Depth map:

There is an important value here: encoding, which represents the encoding format of the image and needs to be referenced when converting image data later.

1.2. Subscribe to color image topic data and display color images

1.2.1. Run command

xxxxxxxxxx
ros2 launch orbbec_camera gemini2.launch.py
ros2 run yahboomcar_visual astra_rgb_image

View topic communication between nodes, terminal input,

xxxxxxxxxx
ros2 run rqt_graph rqt_graph

1.2.2. Core code analysis

Code reference path:

xxxxxxxxxx
~/orbbec_ws/src/yahboomcar_visual/yahboomcar_visual/astra_rgb_image.py

The /get_astra_rgb_node node subscribes to the topic of /camera/color/image_raw, and then uses data conversion to convert the topic data into image data and publish it. The code is as follows,

xxxxxxxxxx
#Import opecv library and cv_bridge library
import cv2 as cv
from cv_bridge import CvBridge
#Create CvBridge object
self.bridge = CvBridge()
#Define a subscriber to subscribe to the RGB color image topic data published by the depth camera node
self.sub_img
=self.create_subscription(Image,'/camera/color/image_raw',self.handleTopic,100)
#msg is converted into image data, where bgr8 is the image encoding format
frame = self.bridge.imgmsg_to_cv2(msg, "bgr8")

1.3. Subscribe to depth image topic information and display depth images

1.3.1. Run command

xxxxxxxxxx
ros2 launch orbbec_camera gemini2.launch.py
ros2 run yahboomcar_visual astra_depth_image

View topic communication between nodes, terminal input,

xxxxxxxxxx
ros2 run rqt_graph rqt_graph

1.3.2. Core code analysis

Code reference path:

xxxxxxxxxx
~/orbbec_ws/src/yahboomcar_visual/yahboomcar_visual/astra_depth_image.py

The basic implementation process is the same as RGB color image display. It subscribes to the topic data of /camera/depth/image_raw published by the depth camera node, and then converts it into image data through data conversion. The code is as follows,

xxxxxxxxxx
#Import opecv library and cv_bridge library
import cv2 as cv
from cv_bridge import CvBridge
#Create CvBridge object
self.bridge = CvBridge()
#Define a subscriber to subscribe to the Depth depth image topic data published by the depth camera node
self.sub_img
=self.create_subscription(Image,'/camera/depth/image_raw',self.handleTopic,10)
#msg is converted into image data, where 32FC1 is the image encoding format
frame = self.bridge.imgmsg_to_cv2(msg, "32FC1")