Welcome to RDK X3 Robot repository
3. ROS+OpenCV Foundation
3. ROS+OpenCV Foundation
3. ROS+OpenCV Foundation3.1 Description of program function3.2 Application code reference path3.3 Start the camera3.4 Subscribe to RGB image topics and publish the converted image data3.4.1. Node Startup3.4.2. View the node communication graph3.4.3. Core source code parsing3.5 Subscribe to Depth image topics and publish the transformed image data3.5.1. Node Startup3.5.2. View the node communication graph3.5.3. Core Source Code Resolution
3.1 Description of program function
ROS delivers images in its own sensor_msgs/Image message format, which cannot be directly processed, but provides [CvBridge] that can perfectly convert and be converted image data formats. [CvBridge] is a ROS library that acts as a bridge between ROS and OpenCV.
OpenCV and ROS image data conversion is shown in the figure below:
The following takes the Astra pro camera as an example, using two examples to show how to use CvBridge for data conversion, and using ROSboard web side to display topic data.
3.2 Application code reference path
After SSH connection car, the location of the function source code is located at,
xxxxxxxxxx#RGB图像示例#RGB image example/userdata/yahboomcar_ws/src/yahboomcar_astra/yahboomcar_imshow/yahboomcar_imshow/astra_rgb_image.py#深度图像示例# Depth image example/userdata/yahboomcar_ws/src/yahboomcar_astra/yahboomcar_imshow/yahboomcar_imshow/astra_depth_image.py3.3 Start the camera
After SSH connects to the car, terminal input,
xxxxxxxxxxros2 launch astra_camera astra_pro.launch.xml
To view the topic, run the following command on the VM terminal:
xxxxxxxxxxros2 topic list
Mainly look at the image data topic, here only analyze the RGB color image and Depth image topic information, use the following command to view the respective data information, virtual machine terminal input,
xxxxxxxxxx#查看RGB图像话题数据内容# View the RGB image topic data contentros2 topic echo /camera/color/image_rawCan capture a frame of RGB color image information,
This describes the basic information of the image, an important value, encoding, the value here is rgb8, indicating that the encoding format of this frame image is rgb8, which needs to be referred to when doing data conversion in the following.
Virtual machine terminal input,
xxxxxxxxxx#查看Depth图像话题数据内容# View Depth image topic data contentros2 topic echo /camera/depth/image_rawIntercept depth image a frame of image data information,
The encoding value here is 16UC1.
3.4 Subscribe to RGB image topics and publish the converted image data
3.4.1. Node Startup
After SSH connects to the car and starts the camera, terminal type,
xxxxxxxxxx#RGB彩色图像显示节点#RGB color image display noderos2 run yahboomcar_imshow astra_rgb_imageCar terminal input,
xxxxxxxxxx#启动ROSboard# Start ROSboardros2 run rosboard rosboard_nodeOpen the browser on the PC side (note that the computer and the Rising Sun network must be in the same LAN), enter the URL: car IP:9999, for example, my car IP is 192.168.2.67, enter the URL in the browser on the virtual machine side to open the ROSboard webpage:
xxxxxxxxxx192.168.2.67:9999
Select /camera/color/image_raw and /cv_image/rgb topics, you can see the following screen,
The two images are the same and keep in sync, only the /cv_image/rgb image has more "cv_rgb" characters.
3.4.2. View the node communication graph
Virtual machine terminal input,
xxxxxxxxxxros2 run rqt_graph rqt_graph
3.4.3. Core source code parsing
Code reference path,
xxxxxxxxxx/userdata/yahboomcar_ws/src/yahboomcar_astra/yahboomcar_imshow/yahboomcar_imshow/astra_rgb_image.py
As can be seen from 3.2.2, /get_astra_rgb_node subscribes to the topic of /camera/color/image_raw, and then converts ROS topic data into opencv image data through CvBridge. After opencv processing, Then the processed image data is converted into ROS topic data through CvBridge and released, that is, ROS topic data ->opencv image data ->ROS topic data. The code is as follows,
x# 导入opecv库以及cv_bridge库# Import opecv library and cv_bridge libraryimport cv2 as cvfrom cv_bridge import CvBridge# 创建CvBridge对象# Create CvBridge objectsself.bridge = CvBridge()# 定义一个订阅者订阅深度相机节点发布的RGB彩色图像话题数据# Define a subscriber to subscribe to the RGB color image topic data published by the Depth camera nodeself.sub_img = self.create_subscription(Image,'/camera/color/image_raw',self.handleTopic,1)# 定义一个发布者发布经处理后的图像话题数据# Define a publisher to publish processed image topic dataself.pub_img = self.create_publisher(Image,'/cv_image/rgb',10)# msg转换成cv图像数据，这里的bgr8是图像编码格式# msg converts to cv image data, where bgr8 is the image encoding formatframe = self.bridge.imgmsg_to_cv2(msg, "bgr8")# 规范输入图像大小# Standardize the input image sizeframe = cv.resize(frame, (640, 480))# 为图像添加字符# Adds characters to the imagecv.putText(frame, "cv_rgb", (280, 30), cv.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2)# 将处理后的cv图像数据转换成msg数据# Convert the processed cv image data into msg datamsg = self.bridge.cv2_to_imgmsg(frame, "bgr8")# 发布转换后的图像# Post the converted imageself.pub_img.publish(msg)3.5 Subscribe to Depth image topics and publish the transformed image data
3.5.1. Node Startup
After SSH connects to the car and starts the camera, terminal type,
xxxxxxxxxx#深度图像显示节点# Depth image display noderos2 run yahboomcar_imshow astra_depth_imageCar terminal input,
xxxxxxxxxx#启动ROSboard# Start ROSboardros2 run rosboard rosboard_nodeOpen the browser on the PC side (note that the computer and the Rising Sun network must be in the same LAN), enter the URL: car IP:9999, for example, my car IP is 192.168.2.67, enter the URL in the browser on the virtual machine side to open the ROSboard webpage:
xxxxxxxxxx192.168.2.67:9999
Select the /camera/depth/image_raw and /cv_image/depth topics to see the following screen,
The two images are the same and synchronized, except that the /cv_image/depth image has the "cv_depth" character.
3.5.2. View the node communication graph
Virtual machine terminal input,
xxxxxxxxxxros2 run rqt_graph rqt_graph
3.5.3. Core Source Code Resolution
Code reference path,
xxxxxxxxxx/userdata/yahboomcar_ws/src/yahboomcar_astra/yahboomcar_imshow/yahboomcar_imshow/astra_depth_image.py
The basic implementation process is the same as the RGB color image display, subscribe to the topic data of /camera/depth/image_raw published by the depth camera node, and then complete the conversion of ROS topic data ->opencv image data ->ROS topic data through CvBridge. The code is as follows,
xxxxxxxxxx# 导入opecv库以及cv_bridge库# Import opecv library and cv_bridge libraryimport cv2 as cvfrom cv_bridge import CvBridge# 创建CvBridge对象# Create CvBridge objectsself.bridge = CvBridge()# 定义一个订阅者订阅深度相机节点发布的深度图像话题数据# Define a subscriber to subscribe to the depth image topic data published by the Depth Camera nodeself.sub_img = self.create_subscription(Image,'/camera/depth/image_raw',self.handleTopic,1)# 定义一个发布者发布经处理后的图像话题数据# Define a publisher to publish processed image topic dataself.pub_img = self.create_publisher(Image,'/cv_image/depth',10)# msg转换成cv图像数据，这里的16UC1是图像编码格式# msg converts to cv image data, where 16UC1 is the image encoding formatframe = self.bridge.imgmsg_to_cv2(msg, "16UC1")# 规范输入图像大小# Standardize the input image sizeframe = cv.resize(frame, (640, 480))# 为图像添加字符# Adds characters to the imagecv.putText(frame, "cv_depth", (280, 30), cv.FONT_HERSHEY_SIMPLEX, 1, (65536, 65536, 65536), 2)# 将处理后的cv图像数据转换成msg数据# Convert the processed cv image data into msg datamsg = self.bridge.cv2_to_imgmsg(frame, "16UC1")# 发布转换后的图像# Post the converted imageself.pub_img.publish(msg)