Welcome to Transbot SE repository
4.HD camera object tracking
4、HD camera object tracking
4、HD camera object tracking4.1、Introduction4.2. Operation steps4.2.1. Start4.2.2. Identification4.2.3. PID adjustment4.2.4. Target tracking
4.1、Introduction
Website：https://learnopencv.com/object-tracking-using-opencv-cpp-python/#opencv-tracking-api
Object tracking is to locate an object in consecutive video frames.
- Comparison of OpenCV algorithms
| Algorithm | Speed | Accuracy | Description |
|---|---|---|---|
| BOOSTING | Slow | Low | It is the same as the machine learning algorithm behind Haar casades (AdaBoost), but it has been born for more than ten years, a veteran algorithm. |
| MIL | Slow | Low | It is more accurate than BOOSTING, but the failure rate is higher. |
| KCF | Fast | High | Faster than BOOSTING and MIL, but it is not effective when there is occlusion |
| TLD | Middle | Middle | There are a lot of erro |
| MEDIANFLOW | Middle+ | Middle | The model will fail for fast-jumping or fast-moving objects. |
| GOTURN | Middle | Middle | A deep learning-based object detector requires additional models to run. |
| MOSSE | Fastest | High | The speed is really fast, but not as high as the accuracy of CSRT and KCF. If you are looking for speed, you can choose it. |
| CSRT | Fast - | Higher | Slightly more accurate than KCF, but not as fast as KCF. |
4.2. Operation steps
Note: [R2] on the remote control handle has the [pause/start] function for all gameplays.
4.2.1. Start
jetson motherboard/Raspberry Pi 4B
Start the underlying driver control, which can also be placed in other launch files. (robot side)
roslaunch transbot_bringup bringup.launchRaspberry Pi 5
Before running, please confirm that the large program has been permanently closed
Enter docker
Note: If there is a terminal that automatically starts docker, or there is a docker terminal that has been opened, you can directly enter the docker terminal to run the command, and there is no need to manually start docker
Start docker manually
xxxxxxxxxx./run_docker.sh
Start low-level driver control
xxxxxxxxxxroslaunch transbot_bringup bringup.launch
method one
jetson motherboard/Raspberry Pi 4B
Start the monocular camera (robot side)
xxxxxxxxxxroslaunch usb_cam usb_cam-test.launchStart monocular target tracking control (virtual machine)
xxxxxxxxxxroslaunch transbot_mono mono_tracker.launch VideoSwitch:=false tracker_type:=KCFRaspberry Pi 5
Enter the same docker from multiple terminals
Keep the program of the previous docker terminal running and open a new terminal
Enter the following command
xxxxxxxxxxdocker ps
Enter the same docker and use the following 18870bc3dc00 to modify the ID displayed on the actual terminal.
xxxxxxxxxxdocker exec -it 18870bc3dc00 /bin/bash
Start the monocular camera (robot side)
xxxxxxxxxxroslaunch usb_cam usb_cam-test.launch
Start monocular target tracking control (virtual machine)
xxxxxxxxxxroslaunch transbot_mono mono_tracker.launch VideoSwitch:=false tracker_type:=KCF
Method 2
Note: [q] key to exit.
jetson motherboard/Raspberry Pi 4B
xxxxxxxxxxpython3 ~/transbot_ws/src/transbot_mono/scripts/mono_Tracker.pyRaspberry Pi 5
Enter the same docker from multiple terminals
Keep the program of the previous docker terminal running and open a new terminal
Enter the following command
xxxxxxxxxxdocker ps
Enter the same docker and use the following 18870bc3dc00 to modify the ID displayed on the actual terminal.
xxxxxxxxxxdocker exec -it 18870bc3dc00 /bin/bash
xxxxxxxxxxpython3 ~/transbot_ws/src/transbot_mono/scripts/mono_Tracker.py
This method can only be started in the main control connected to the camera
- VideoSwitch parameter: whether to use the camera function package to launch; for example: to launch usb_cam-test.launch, this parameter must be set to true; otherwise, it is false.
- tracker_type parameter: OpenCV Tracking API; optional: ['BOOSTING', 'MIL', 'KCF', 'TLD', 'MEDIANFLOW', 'MOSSE', 'CSRT']
Set parameters according to needs, or modify the launch file directly, so there is no need to attach parameters when starting.
4.2.2. Identification
After starting, enter the selection mode, use the mouse to select the target location, as shown in the figure below, release it to start recognition.
Keyboard key control:
[r]: Select mode, you can use the mouse to select the area to identify the target, as shown in the picture above.
[f]: Switching algorithm; ['BOOSTING', 'MIL', 'KCF', 'TLD', 'MEDIANFLOW', 'MOSSE', 'CSRT'].
[q]: Exit the program.
[Spacebar]: Target tracking; just move the target slowly while following. If you move too fast, you will lose the target.
4.2.3. PID adjustment
Dynamic parameter tuning
jetson motherboard/Raspberry Pi 4B
xxxxxxxxxxrosrun rqt_reconfigure rqt_reconfigure
Raspberry Pi 5
Enter the same docker from multiple terminals
Keep the program of the previous docker terminal running and open a new terminal
Enter the following command
xxxxxxxxxxdocker ps
Enter the same docker and use the following 18870bc3dc00 to modify the ID displayed on the actual terminal.
xxxxxxxxxxdocker exec -it 18870bc3dc00 /bin/bash
xxxxxxxxxxrosrun rqt_reconfigure rqt_reconfigure
Select the [mono_Tracker] node, and the six parameters [Hmin], [Smin], [Vmin], [Hmax], [Smax], and [Vmax] do not need to be adjusted. The slide bar is always in a dragging state and data will not be transferred to the system until it is released; you can also select a row and then slide the mouse wheel.
Parameter analysis:
[Kp], [Ki], [Kd]: PID control during car driving.
[scale]: PID scaling.
4.2.4. Target tracking
After the recognition is correct, click the [space bar] on the keyboard to execute the object following program.
- Node view
jetson motherboard/Raspberry Pi 4B
xxxxxxxxxxrqt_graph
Raspberry Pi 5
Enter the same docker from multiple terminals
Keep the program of the previous docker terminal running and open a new terminal
Enter the following command
xxxxxxxxxxdocker ps
Enter the same docker and use the following 18870bc3dc00 to modify the ID displayed on the actual terminal.
xxxxxxxxxxdocker exec -it 18870bc3dc00 /bin/bash
xxxxxxxxxxrqt_graph
When starting method one, the node [mono_Tracker]
Subscribe to image topics; publish gimbal and servo topics
- When starting method one, the node [mono_Tracker]
Post the gimbal servo topic and control the gimbal to follow.