Human tracking

Note: This case needs to be run as root user, and administrator privileges are required to call the MIPI camera!

Switch to root user

xxxxxxxxxx
su root

Password: yahboom

1. Program function description

After the program is started, stand in front of the mipi camera, let the robot recognize the whole body, move the body slowly, and the robot will follow the human body.

You can view the recognition results in the PC browser.

Note: This case needs to be switched to the root user to run, and administrator privileges are required to call the MIPI camera!

2. Principle introduction

The human body following function is to control the robot to follow the human body. It consists of MIPI image acquisition, human body detection and tracking, human body following strategy, image encoding and decoding, and WEB display end. The process is as follows:

Human body detection and tracking algorithm example subscribes to pictures, uses BPU for algorithm reasoning, publishes msg containing human body, head, face, hand frame and human key point detection results, and realizes the tracking of detection frame through multi-target tracking (MOT) function. The websocket package is used to render and display the images released by the camera and the corresponding algorithm results on the PC browser.

The detection categories supported by the algorithm and the corresponding data types of different categories in the algorithm msg are as follows:

The index of the human key point algorithm result is as follows:

3. Program reference path

After SSH connects to the car, the source code of this function is located at,

#Do not recognize gestures
/home/sunrise/yahboomcar_ws/src/yahboomcar_deeplearning/body_tracking/launch/hobot_body_tracking_without_gesture.launch.py
#Recognize gestures
/home/sunrise/yahboomcar_ws/src/yahboomcar_deeplearning/body_tracking/launch/hobot_body_tracking.launch.py

4. Program startup

After SSH connects to the car, the terminal runs,

​x
# Copy the configuration files needed to run the example from the installation path of TogetherROS
cp -r /opt/tros/${TROS_DISTRO}/lib/mono2d_body_detection/config/ .
cp -r /opt/tros/${TROS_DISTRO}/lib/hand_lmk_detection/config/ .
cp -r /opt/tros/${TROS_DISTRO}/lib/hand_gesture_detection/config/ .

#Start launch file
#Do not recognize gestures
ros2 launch body_tracking hobot_body_tracking_without_gesture.launch.py
#Recognize gestures
ros2 launch body_tracking hobot_body_tracking.launch.py

The output log shows that the program runs successfully. The input and output frame rates of the algorithm during inference are 30fps, and the statistical frame rate is refreshed every second.

Open the browser on the PC (note that the computer and the Xuripai network must be in the same LAN), enter the URL: car IP:8000, for example, my car IP is 192.168.2.67, enter the URL in the browser on the virtual machine,

xxxxxxxxxx
192.168.2.67:8000

Click to enter the Web display terminal, the display screen is as follows,

Do not recognize gestures:

Recognize gestures:

Adjust the camera angle, after recognizing the human body, the car will move with the human body and keep a certain distance.