3. ROS+Opencv basics

This lesson uses the Gemini2 camera as an example, which is similar to a normal camera.

3.1. Overview

Wiki: http://wiki.ros.org/cv_bridge/

Teaching: http://wiki.ros.org/cv_bridge/Tutorials

Source code: https://github.com/ros-perception/vision_opencv.git

Feature package location: ~/astra_ws/src/astra_visual

ROS has been integrated with Opencv3.0 and above during the installation process, so the installation and configuration hardly need to be considered. ROS in their own

message format Image, not directly to Image processing, However, the provided [CvBridge] can perfectly convert and be converted image data formats. [CvBridge] is a ROS library that acts as a bridge between ROS and Opencv.

Opencv and ROS image data conversion is shown in the figure below:

Although the installation configuration does not need to be much thought, but the use of the environment still need to be configured, mainly 【package.xml】 and 【CMakeLists.txt】 these two files. The function pack not only uses [CvBridge], but also needs [Opencv] and [PCL], so it is configured together.

• package.xml

Add the following

​x
    <build_depend>sensor_msgs</build_depend>
    <build_export_depend>sensor_msgs</build_export_depend>
    <exec_depend>sensor_msgs</exec_depend>
​
    <build_depend>std_msgs</build_depend>
    <build_export_depend>std_msgs</build_export_depend>
    <exec_depend>std_msgs</exec_depend>
​
    <build_depend>cv_bridge</build_depend>
    <build_export_depend>cv_bridge</build_export_depend>
    <exec_depend>cv_bridge</exec_depend>
​
    <exec_depend>image_transport</exec_depend>

[cv_bridge] : Image conversion dependency package.

【transbot_msgs】 : Custom message dependency package.

• CMakeLists.txt

This file contains many configurations. For details, see the source file.

3.2 Gemini2

3.2.1. Start the Gemini2 camera

xxxxxxxxxx
roslaunch orbbec_camera gemini2.launch

View topic

xxxxxxxxxx
rostopic list

As you can see, there are many topics, but only a few are commonly used in this section

View the topic encoding format: rostopic echo + [topic] +encoding, for example

xxxxxxxxxx
rostopic echo /camera/rgb/image_raw/encoding
rostopic echo /camera/depth/image_raw/encoding

3.2.2. Start the color graph subscription node

xxxxxxxxxx
roslaunch orbbec_camera gemini2.launch          #启动相机launch# Start camera launch
roslaunch astra_visual astra_get_rgb.launch     #启动功能launch# start launch function

Select 1 of the following two nodes to start the launch file.

xxxxxxxxxx
astra_rgb_image.py         # py 
astra_rgb_image            # C++

View node diagram

xxxxxxxxxx
rqt_graph

• py Code analysis

Create a subscriber: Subscribe to the topic ["/camera/rgb/image_raw"], data type [Image], callback function [topic()]

xxxxxxxxxx
sub = rospy.Subscriber("/camera/rgb/image_raw", Image, topic)

Use [CvBridge] for data conversion, here we should pay attention to the encoding format, the encoding format is wrong, the converted image will have problems.

xxxxxxxxxx
frame = bridge.imgmsg_to_cv2(msg, "bgr8")

• c++ code analysis

Similar to py code

xxxxxxxxxx
//创建一个接收者.// Create a recipient.
ros::Subscriber subscriber = n.subscribe<sensor_msgs::Image>("/camera/rgb/image_raw", 10, RGB_Callback);
// 创建cv_bridge示例 // Create a cv_bridge example
cv_bridge::CvImagePtr cv_ptr;
// 数据转换 // Data conversion
cv_ptr = cv_bridge::toCvCopy(msg, sensor_msgs::image_encodings::BGR8);

3.2.3. Start the depth map subscription node

xxxxxxxxxx
roslaunch orbbec_camera gemini2.launch          #启动相机launch# Start camera launch
roslaunch astra_visual astra_get_depth.launch    # launch

Select 1 of the following two nodes to start the launch file.

xxxxxxxxxx
astra_depth_image.py         # py 
astra_depth_image            # C++

View node diagram

xxxxxxxxxx
rqt_graph

• py Code analysis

Create a subscriber: Subscribe to the topic ["/camera/depth/image_raw"], data type [Image], callback function [topic()]

xxxxxxxxxx
sub = rospy.Subscriber("/camera/depth/image_raw", Image, topic)

Use [CvBridge] for data conversion, here we should pay attention to the encoding format, the encoding format is wrong, the converted image will have problems.

xxxxxxxxxx
# 编码格式# Encoding format
encoding = ['16UC1', '32FC1']
# 可以切换不同编码格式测试效果# You can switch between different encoding formats
frame = bridge.imgmsg_to_cv2(msg, encoding[1])

• c++ code analysis

Similar to py code

xxxxxxxxxx
//创建一个接收者.// Create a recipient.
ros::Subscriber subscriber = n.subscribe<sensor_msgs::Image>("/camera/depth/image_raw", 10, depth_Callback);
// 创建cv_bridge示例 // Create a cv_bridge example
cv_bridge::CvImagePtr cv_ptr;
// 数据转换 // Data conversion
cv_ptr = cv_bridge::toCvCopy(msg, sensor_msgs::image_encodings::TYPE_16UC1);

3.2.4 Start color image inversion

xxxxxxxxxx
roslaunch orbbec_camera gemini2.launch            #开启相机launch# Start camera launch
roslaunch astra_visual astra_image_flip.launch    #开启功能launch# Enable function launch

Image viewing

xxxxxxxxxx
rqt_image_view      #开启第一个# Open the first one
rqt_image_view      #开启第二个# Open the second

• py Code analysis

Two subscribers and two publishers are created here, one for general image data and one for compressed image data.

1) Create subscribers

Subscribe to the topic ["/camera/rgb/image_raw"], data type [Image], callback function [topic()].

2) Create a publisher

The topics published are ["/camera/rgb/image_flip"], data type [Image], queue size [10].

xxxxxxxxxx
sub_img = rospy.Subscriber("/camera/rgb/image_raw", Image, topic)
pub_img = rospy.Publisher("/camera/rgb/image_flip", Image, queue_size=10)

3) Callback function

xxxxxxxxxx
# 正常图像传输处理# Normal image transmission processing
def topic(msg):
    if not isinstance(msg, Image):
        return
    bridge = CvBridge()
    frame = bridge.imgmsg_to_cv2(msg, "bgr8")
    # Opencv处理图像# Opencv processes images
    frame = cv.resize(frame, (640, 480))
    frame = cv.flip(frame, 1)
    # opencv mat ->  ros msg
    msg = bridge.cv2_to_imgmsg(frame, "bgr8")
    # 图像处理完毕，直接发布# Image processing is complete, release directly
    pub_img.publish(msg)