Welcome to Transbot SE repository
9. ROS+Opencv application
9. ROS+Opencv application
9. ROS+Opencv application9.1. Overview9.2. Use9.2.1. Start9.2.2. Display method9.2.3. Effect display9.3. Node9.3.1. Edge detection algorithm9.3.2. Contour moment9.3.3. Face recognition
9.1. Overview
wiki: http://wiki.ros.org/opencv_apps
Source code: https://github.com/ros-perception/opencv_apps.git
Function package: ~/software/transbot_library/src/opencv_apps
Most of the code was originally taken from https://github.com/Itseez/opencv/tree/master/samples/cpp
The opencv_apps program provides various nodes that internally run the functions of opencv and publish the results to a ROS topic. When using the opencv_apps program, you only need to run a launch file according to your own business needs, so you don't have to write program codes for these functions.
ROS Wiki has related node analysis, topic subscription and topic publishing of the corresponding node, introduction of related parameters, etc. See the ROS WiKi for details.
9.2. Use
9.2.1. Start
Step 1: Start the camera
jetson motherboard/Raspberry Pi 4B
roslaunch transbot_visual opencv_apps.launch camDevice:=USBCamRaspberry Pi 5
Before running, please confirm that the large program has been permanently closed
Enter docker
Note: If there is a terminal that automatically starts docker, or there is a docker terminal that has been opened, you can directly enter the docker terminal to run the command, and there is no need to manually start docker
Start docker manually
xxxxxxxxxx./run_docker.sh
xxxxxxxxxxroslaunch transbot_visual opencv_apps.launch camDevice:=USBCam
[camDevice]: The high frame rate camera is [USBCam];
If the web page cannot be viewed, check whether there is no [web_video_server] node. If not, run the following command
jetson motherboard/Raspberry Pi 4B
xxxxxxxxxxrosrun web_video_server web_video_serverRaspberry Pi 5
Enter the same docker from multiple terminals
Keep the program of the previous docker terminal running and open a new terminal
Enter the following command
xxxxxxxxxxdocker ps
Enter the same docker and use the following 18870bc3dc00 to modify the ID displayed on the actual terminal.
xxxxxxxxxxdocker exec -it 18870bc3dc00 /bin/bash
Step 2: Start the function of Opencv_apps
jetson motherboard/Raspberry Pi 4B
xxxxxxxxxxroslaunch opencv_apps face_recognition.launch # Face recognitionroslaunch opencv_apps corner_harris.launch # harris corner detectionroslaunch opencv_apps camshift.launch # Target tracking algorithmroslaunch opencv_apps contour_moments.launch # Contour momentsroslaunch opencv_apps convex_hull.launch # Polygon outlineroslaunch opencv_apps discrete_fourier_transform.launch # Discrete Fourier transform algorithmroslaunch opencv_apps edge_detection.launch # Edge detection algorithmroslaunch opencv_apps face_detection.launch # Face detection algorithmroslaunch opencv_apps fback_flow.launch # Optical flow detection algorithmroslaunch opencv_apps find_contours.launch # Contour detectionroslaunch opencv_apps general_contours.launch # General contour detectionroslaunch opencv_apps goodfeature_track.launch # Feature point trackingroslaunch opencv_apps hls_color_filter.launch # HLS color filteringroslaunch opencv_apps hough_circles.launch # Hough circle detectionroslaunch opencv_apps hough_lines.launch # Hough line detectionroslaunch opencv_apps hsv_color_filter.launch # HSV color filteringroslaunch opencv_apps lk_flow.launch # LK optical flow algorithmroslaunch opencv_apps people_detect.launch # Human detection algorithmroslaunch opencv_apps phase_corr.launch # Phase-related displacement detectionroslaunch opencv_apps pyramids.launch # Image pyramid sampling algorithmroslaunch opencv_apps rgb_color_filter.launch # RGB color filteringroslaunch opencv_apps segment_objects.launch # Clear background detection algorithmroslaunch opencv_apps simple_flow.launch # Simplified optical flow algorithmroslaunch opencv_apps smoothing.launch # Simple filterroslaunch opencv_apps threshold.launch # Threshold image processingroslaunch opencv_apps watershed_segmentation.launch # Watershed segmentation algorithmRaspberry Pi 5
Enter the same docker from multiple terminals
Keep the program of the previous docker terminal running and open a new terminal
Enter the following command
xxxxxxxxxxdocker ps
Enter the same docker and use the following 18870bc3dc00 to modify the ID displayed on the actual terminal.
xxxxxxxxxxdocker exec -it 18870bc3dc00 /bin/bash
xxxxxxxxxxroslaunch opencv_apps face_recognition.launch # Face recognitionroslaunch opencv_apps corner_harris.launch # harris corner detectionroslaunch opencv_apps camshift.launch # Target tracking algorithmroslaunch opencv_apps contour_moments.launch # Contour momentsroslaunch opencv_apps convex_hull.launch # Polygon outlineroslaunch opencv_apps discrete_fourier_transform.launch # Discrete Fourier transform algorithmroslaunch opencv_apps edge_detection.launch # Edge detection algorithmroslaunch opencv_apps face_detection.launch # Face detection algorithmroslaunch opencv_apps fback_flow.launch # Optical flow detection algorithmroslaunch opencv_apps find_contours.launch # Contour detectionroslaunch opencv_apps general_contours.launch # General contour detectionroslaunch opencv_apps goodfeature_track.launch # Feature point trackingroslaunch opencv_apps hls_color_filter.launch # HLS color filteringroslaunch opencv_apps hough_circles.launch # Hough circle detectionroslaunch opencv_apps hough_lines.launch # Hough line detectionroslaunch opencv_apps hsv_color_filter.launch # HSV color filteringroslaunch opencv_apps lk_flow.launch # LK optical flow algorithmroslaunch opencv_apps people_detect.launch # Human detection algorithmroslaunch opencv_apps phase_corr.launch # Phase-related displacement detectionroslaunch opencv_apps pyramids.launch # Image pyramid sampling algorithmroslaunch opencv_apps rgb_color_filter.launch # RGB color filteringroslaunch opencv_apps segment_objects.launch # Clear background detection algorithmroslaunch opencv_apps simple_flow.launch # Simplified optical flow algorithmroslaunch opencv_apps smoothing.launch # Simple filterroslaunch opencv_apps threshold.launch # Threshold image processingroslaunch opencv_apps watershed_segmentation.launch # Watershed segmentation algorithm
Almost every functional case will have a parameter [debug_view], Boolean type, whether to use Opencv to display pictures, which is displayed by default.
If no display is required, set it to [False], for example
xxxxxxxxxxroslaunch opencv_apps contour_moments.launch debug_view:=False
However, after starting in this way, some cases cannot be displayed in other ways, because in the source code, some [debug_view] is set to [False], which will turn off image processing.
9.2.2. Display method
- rqt_image_view
Enter the following command to select the corresponding topic
jetson motherboard/Raspberry Pi 4B
xxxxxxxxxxrqt_image_view
Raspberry Pi 5
Enter the same docker from multiple terminals
Keep the program of the previous docker terminal running and open a new terminal
Enter the following command
xxxxxxxxxxdocker ps
Enter the same docker and use the following 18870bc3dc00 to modify the ID displayed on the actual terminal.
xxxxxxxxxxdocker exec -it 18870bc3dc00 /bin/bash
xxxxxxxxxxrqt_image_view
- opencv
The system displays it by default, no need to do anything.
- Web viewing
(Same as under LAN) Enter IP+port in the browser, for example:
xxxxxxxxxx192.168.2.102:8080
9.2.3. Effect display
- Optical flow detection algorithm
Move the screen and observe the phenomenon.
- Feature point tracking
- Hough circle detection
- Hough linear detection
The lower the threshold, the more lines there are and the easier it is for the picture to freeze.
- Phase dependent displacement detection
The faster the camera moves, the larger the radius of the circle.
- Watershed segmentation algorithm
Use the mouse to select different objects and the system will automatically distinguish them.
9.3. Node
Each case in this section will have a topic for subscribing images and publishing images.
9.3.1. Edge detection algorithm
| Parameters | Type | Default | Parse |
|---|---|---|---|
| ~use_camera_info | bool | true | Subscribe to the topic [camera_info] and get the default coordinate system ID, otherwise use the image information directly. |
| ~debug_view | bool | false | Whether to create a window to display the node image |
| ~edge_type | int | 0 | Specify the edge detection method: 0: Sobel operator, 1: Laplacian operator, 2: Canny edge detection |
| ~canny_threshold1 | int | 100 | Specify the second canny threshold |
| ~canny_threshold2 | int | 200 | Specify the first canny threshold |
| ~apertureSize | int | 3 | The aperture size of the Sobel operator. |
| ~apply_blur_pre | bool | True | Whether to apply blur() to the input image |
| ~postBlurSize | double | 3.2 | Input image aperture size |
| ~apply_blur_post | bool | False | Whether to apply GaussianBlur() to the input image |
| ~L2gradient | bool | False | canny's parameters |
| ~queue_size | int | 3 | Queue size |
9.3.2. Contour moment
| Parameters | Type | Default | Parse |
|---|---|---|---|
| ~use_camera_info | bool | true | Subscribe to the topic [camera_info] and get the default coordinate system ID, otherwise use the image information directly. |
| ~debug_view | bool | false | Whether to create a window to display the node image |
| ~canny_low_threshold | int | 0 | Canny edge detection low threshold |
| ~queue_size | int | 3 | Queue size |
9.3.3. Face recognition
This case involves autonomous training and real-time recognition by collecting images of people in real time, and the steps are slightly complicated.
| Parameters | Type | Default | Parse |
|---|---|---|---|
| ~approximate_sync | bool | false | Subscribe to the topic [camera_info] and get the default coordinate system ID, otherwise use the image information directly. |
| ~queue_size | int | 100 | The queue size of the subscription topic |
| ~model_method | string | "eigen" | Face recognition method: "eigen", "fisher" or "LBPH" |
| ~use_saved_data | bool | true | Load training data from ~data_dir path |
| ~save_train_data | bool | true | Save the training data to the ~data_dir path for retraining |
| ~data_dir | string | "~/opencv_apps/face_data" | Save training data path |
| ~face_model_width | int | 190 | Width of training face image |
| ~face_model_height | int | 90 | Height of training face image |
| ~face_padding | double | 0.1 | The padding ratio of each face |
| ~model_num_components | int | 0 | Number of components for the face recognizer model (0 is considered unlimited) |
| ~model_threshold | double | 8000.0 | Face recognition model threshold |
| ~lbph_radius | int | 1 | Radius parameter (only for LBPH method) |
| ~lbph_neighbors | int | 8 | Neighborhood parameters (only for LBPH method) |
| ~lbph_grid_x | int | 8 | Grid x parameter (only for LBPH method) |
| ~lbph_grid_y | int | 8 | Grid y parameter (only for LBPH method) |
| ~queue_size | int | 100 | Image subscriber queue size |
Steps:
- First, enter the character’s name after the colon in the picture below: Yahboom
- Confirm name: y
- Then place the face in the center of the image and click OK.
- Add a photo in the loop: y, click to confirm.
- To end image collection, enter: n and click Confirm.
- Close the launch file and restart.
If you need to enter the recognized identifications, cycle through steps 1 to 5 until all identified persons have been entered, and then proceed to step six.
Step 3: Ensure that faces can be recognized
final recognition effect
