11. Human tracking

1. Program function description

After the program starts, stand in front of the mipi camera, let the car recognize the whole body, slowly move the body, the car will follow the human movement.

You can view the identification result in the browser on the PC.

2. Introduction to principle

The human body following function is to control the robot to follow the human body, which consists of MIPI image acquisition, human body detection and tracking, human body following strategy, image encoding and decoding, and WEB display terminal. The process is as follows:

Human body detection and tracking algorithm example Subscribe pictures, use BPU for algorithmic reasoning, publish msg, including human body, human head, face, human hand frame and human key point detection results, and through multi-target tracking (MOT) function, realize the tracking of detection frame. The websocket package is used to render and display the pictures published by the camera and the corresponding algorithm results on the browser of the PC side.

The detection categories supported by the algorithm and the corresponding data types of different categories in the algorithm msg are as follows:

Human key point algorithm result index is as follows:

3. Program reference path

After SSH connection car, the function source code is located in,

xxxxxxxxxx
#不识别手势
# Don't recognize gestures
/userdata/yahboomcar_ws/src/yahboomcar_deeplearning/body_tracking/launch/hobot_body_tracking_without_gesture.launch.py
#识别手势
# Recognize gestures
/userdata/yahboomcar_ws/src/yahboomcar_deeplearning/body_tracking/launch/hobot_body_tracking.launch.py

4. The program starts

After SSH connects to the car, the terminal runs,

xxxxxxxxxx
# 从TogetherROS的安装路径中拷贝出运行示例需要的配置文件
# Copy the configuration files needed to run the example from TogetherROS
cp -r /opt/tros/lib/mono2d_body_detection/config/ .
​
#启动launch文件
# Start the launch file
#不识别手势
# Don't recognize gestures
ros2 launch body_tracking hobot_body_tracking_without_gesture.launch.py
#识别手势
# Recognize gestures
ros2 launch body_tracking hobot_body_tracking.launch.py

The output log shows that the program runs successfully, the algorithm input and output frame rate is 30fps when reasoning, and the statistical frame rate is refreshed once every second.

Open the browser on the PC side (note that the computer and the Rising Sun network must be in the same LAN), enter the URL: car IP:8000, for example, my car IP is 192.168.2.67, enter the URL in the browser on the virtual machine side,

xxxxxxxxxx
192.168.2.67:8000

Click to enter the Web display terminal, the display screen is as follows:

Do not recognize gestures:

Recognize gestures:

Adjust the camera Angle, after recognizing the human body, the car will follow the human body, and maintain a certain distance.