Welcome to ROSMASTER R2 repository

8. ROS+Opencv foundation

This lesson takes the Astra camera as an example, which is similar to ordinary cameras.

8.1. Overview

Wiki： http://wiki.ros.org/cv_bridge/

Teaching: http://wiki.ros.org/cv_bridge/Tutorials

Source code: https://github.com/ros-perception/vision_opencv.git

Feature pack location: ~/yahboomcar_ws/src/yahboomcar_visual

ROS has already integrated versions above Opencv3.0 during the installation process, so the installation configuration hardly needs to be considered too much. ROS transmits images in its own sensor_msgs/Image message format and cannot directly process images, but the provided [CvBridge] ] Can perfectly convert and be converted image data formats. [CvBridge] is a ROS library, equivalent to a bridge between ROS and Opencv.

Opencv and ROS image data conversion is shown in the following figure:

Although the installation configuration does not need to be considered too much, the use environment still needs to be configured, mainly the two files [package.xml] and [CMakeLists.txt]. This function package not only uses [CvBridge], but also needs [Opencv] and [PCL], so it is configured together.

• package.xml

Add the following

​x
  < build_depend > sensor_msgs </ build_depend > 
  < build_export_depend > sensor_msgs </ build_export_depend > 
  < exec_depend > sensor_msgs </ exec_depend > 

  < build_depend > std_msgs </ build_depend > 
  < build_export_depend > std_msgs </ build_export_depend > 
  < exec_depend > std_msgs </ exec_depend > 
  < build_depend > cv_bridge </ build_depend > 
  < build_export_depend > cv_bridge </ build_export_depend > 
  < exec_depend > cv_bridge </ exec_depend > 
  < exec_depend > image_transport </ exec_depend > 

[cv_bridge]: Image conversion dependency package.

• CMakeLists.txt

There are many configuration contents in this file. For details, please refer to the source file.

8.2. Astra

8.2.1. Start Astra Camera

xxxxxxxxxx
#Raspberry Pi 5 master needs to enter docker first, please perform this step
#If running the script into docker fails, please refer to ROS/07, Docker tutorial
~/run_docker.sh
#Multiple ros commands require multiple terminals to be executed in the same docker container. Please refer to the tutorials in Sections 07/5 and 5.8.

xxxxxxxxxx
roslaunch astra_camera astrapro.launch 

View threads

xxxxxxxxxx
rostopic list 

You can see a lot of topics, just a few commonly used in this section

Check the encoding format of the topic: rostopic echo +[topic]+encoding, for example

xxxxxxxxxx
rostopic echo /camera/rgb/image_raw/encoding 
rostopic echo /camera/depth/image_raw/encoding 

The topic with [compressed] or [compressedDepth] after the topic is a compressed topic. When ROS transmits images, data packets may be lost due to factors such as the network, the running speed of the host, the running memory of the host, and the huge amount of video stream data. off topic. So there is no way, I can only subscribe to the compressed topic. Open two images at the same time to subscribe to different topics for testing. If the device performance is good and the network is also good, there will be no change. Otherwise, you will find that the topics after image compression will be much smoother.

8.2.2. Start the color map subscription node

xxxxxxxxxx
roslaunch yahboomcar_visual astra_get_rgb.launch version:=cpp 

• version parameter: optional [py, cpp] different codes have the same effect.

View Node Graph

xxxxxxxxxx
rqt_graph 

When opening the node graph, there will be dense nodes and relationships between nodes. At this time, we use the part linked to the topic [/camera/rgb/image_raw], and [/astra_rgb_Image_cpp] is the node we wrote.

• py code analysis

Create a subscriber: The topic of subscription is ["/camera/rgb/image_raw"], the data type is [Image], and the callback function [topic()]

xxxxxxxxxx
sub = rospy.Subscriber("/camera/rgb/image_raw", Image, topic) 

Use [CvBridge] for data conversion. What should be paid attention to here is the encoding format. If the encoding format is incorrect, the converted image will have problems.

xxxxxxxxxx
frame = bridge.imgmsg_to_cv2(msg, "bgr8") 

• c++ code analysis

similar to py code

xxxxxxxxxx
//Create a receiver.
ros::Subscriber subscriber = n.subscribe<sensor_msgs::Image>("/camera/rgb/image_raw", 10, RGB_Callback);
// create cv_bridge example 
cv_bridge::CvImagePtr cv_ptr;
// data conversion 
cv_ptr = cv_bridge::toCvCopy(msg, sensor_msgs::image_encodings::BGR8);

8.2.3. Start the depth graph subscription node

xxxxxxxxxx
roslaunch yahboomcar_visual astra_get_depth.launch version:=cpp 

View Node Graph

xxxxxxxxxx
rqt_graph 

Opening the node graph will show dense nodes and relationships between nodes. At this time, we use the part linked to the [/camera/depth/image_raw] topic. , [/astra_depth_Image_cpp] is the node we wrote.

• py code analysis

Create a subscriber: The topic of subscription is ["/camera/depth/image_raw"], the data type is [Image], and the callback function [topic()]

xxxxxxxxxx
sub = rospy.Subscriber("/camera/depth/image_raw", Image, topic)

Use [CvBridge] for data conversion. What should be paid attention to here is the encoding format. If the encoding format is incorrect, the converted image will have problems.

xxxxxxxxxx
# Encoding format 
encoding = ['16UC1', '32FC1']
# You can switch between different encoding formats to test the effect 
frame = bridge.imgmsg_to_cv2(msg, encoding[1])

• c++ code analysis

similar to py code

xxxxxxxxxx
//Create a receiver. 
ros::Subscriber subscriber = n.subscribe<sensor_msgs::Image>
("/camera/depth/image_raw", 10, depth_Callback); 
// create cv_bridge example 
cv_bridge::CvImagePtr cv_ptr;
// data conversion 
cv_ptr = cv_bridge::toCvCopy(msg, sensor_msgs::image_encodings::TYPE_16UC1);

8.2.4. Start color image inversion

xxxxxxxxxx
roslaunch yahboomcar_visual astra_image_flip.launch    

image view

xxxxxxxxxx
rqt_image_view 

• py code analysis

Two subscribers and two publishers are created here, one for general image data and one for compressed image data.

1. Create subscribers

The subscribed topic is ["/camera/rgb/image_raw"], the data type is [Image], and the callback function [topic()].

The topic of subscription is ["/camera/rgb/image_raw/compressed"], data type [CompressedImage], and callback function [compressed_topic()].

2. Create a publisher

The published topic is ["/camera/rgb/image_flip"], data type [Image], queue size [10].

The posted topic is ["/camera/rgb/image_flip/compressed"], data type [CompressedImage], queue size [10].

xxxxxxxxxx
sub_img = rospy.Subscriber("/camera/rgb/image_raw", Image, topic)
pub_img = rospy.Publisher("/camera/rgb/image_flip", Image, queue_size=10)
sub_comimg = rospy.Subscriber("/camera/rgb/image_raw/compressed", CompressedImage, compressed_topic)
pub_comimg = rospy.Publisher("/camera/rgb/image_flip/compressed", CompressedImage, queue_size=10)

3. Callback function

xxxxxxxxxx
# Normal image transfer processing 
def topic(msg):
    if not isinstance(msg, Image):
        return
    bridge = CvBridge()
    frame = bridge.imgmsg_to_cv2(msg, "bgr8")
    frame = cv.resize(frame, (640, 480))
    frame = cv.flip(frame, 1)
    # opencv mat ->  ros msg
    msg = bridge.cv2_to_imgmsg(frame, "bgr8")
    pub_img.publish(msg)
    
# Compressed image transmission processing 
def compressed_topic(msg):
    if not isinstance(msg, CompressedImage): return
    bridge = CvBridge()
    frame = bridge.compressed_imgmsg_to_cv2(msg, "bgr8")
    frame = cv.resize(frame, (640, 480))
    frame = cv.flip(frame, 1)
    # Create CompressedIamge 
    msg = CompressedImage()
    msg.header.stamp = rospy.Time.now()
    msg.data = np.array(cv.imencode('.jpg', frame)[1]).tostring()
    pub_comimg.publish(msg)