Welcome to MicroROS V2 Robot Car repository
7. Mediapipe Human Pose Tracking
Mediapipe Human Pose Tracking (No-PTZ)
1. Experiment Objective
MediaPipe Pose is used to detect 33 keypoints of the human body, and the car rotates to follow so that the target person stays at the center of the frame. It supports Tracking mode (rotation only) and Follow mode (rotation + distance keeping).
2. Experiment Principle
1. Keyword Explanation
| Keyword | Description |
|---|---|
| MediaPipe Pose | A lightweight human pose estimation model provided by Google that can detect the coordinates of 33 human keypoints in real time |
| 33 Pose Keypoints | The human skeleton nodes defined by MediaPipe, including nose, eyes, shoulders, elbows, wrists, hips, knees, ankles, etc. |
| Body Center of Gravity Calculation | Combines keypoints such as the hip center and shoulder center to compute the reference position of the human body in the frame |
| Pose Skeleton Connections | Connects the 33 keypoints according to the human topology to form a skeleton |
| PD Control | Proportional-derivative controller that computes the body angular velocity based on the body center deviation and its rate of change |
| QoS | best_effort (image input + cmd_vel downlink), depth=1 |
2. Technical Architecture
/camera/image_raw (BEST_EFFORT)→ pose_tracker_node├── Image processing (15Hz): image conversion → flip → MediaPipe Pose.process() → 33 keypoints → center of gravity calculation└── Body control (10Hz): pixel error → PD angular velocity + bounding box width distance following → /cmd_vel (BEST_EFFORT)
Requires a MediaPipe Python environment (ROS 2 and workspace environment).
3. Core Topic Quick Reference
Firmware Downlink
| Topic | Message Type | QoS | Field | Range | Direction |
|---|---|---|---|---|---|
/cmd_vel | geometry_msgs/Twist | best_effort + keep_last(1) | linear.x | -0.30~0.30 m/s | += forward, −= backward |
angular.z | -0.60~0.60 rad/s | += turn left, −= turn right |
3. Experiment Steps
1. Hardware Preparation
| Hardware | Requirement |
|---|---|
| PTZ version | ❌ |
| No-PTZ version | ✅ |
| Required peripheral | ESP32 camera (bracket-mounted) |
2. Start the Agent + Camera + Joint Model (Terminal 1)
xxxxxxxxxxros2 launch microros_v2_bringup car_base.launch.py
3. Start Human Pose Tracking (Terminal 2)
Tracking mode:
xxxxxxxxxxros2 launch microros_v2_no_ptz_visual_control_pkg pose_tracker.launch.py
6. Operation Instructions
| Key | Function |
|---|---|
| Space | Toggle tracking/standby state |
| i / I | Switch to the standby (IDLE) state |
| q / Q / ESC | Exit the program |
4. Experiment Phenomenon
- When the person moves left or right, the car automatically rotates to track and keep them at the center of the frame
- The 33 human keypoints and skeleton connections are displayed in real time
- Follow mode: the car backs away when the person moves closer and moves forward when they move away
- With multiple people, the tracking target may switch
5. Code Explanation
Source path:
- Launch (track):
~/microros_ws/src/microros_v2_no_ptz_visual_control_pkg/launch/pose_tracker.launch.py- Node source:
~/microros_ws/src/microros_v2_no_ptz_visual_control_pkg/microros_v2_no_ptz_visual_control_pkg/pose_tracker_node.py- Pose detector:
~/microros_ws/src/microros_v2_no_ptz_visual_control_pkg/microros_v2_no_ptz_visual_control_pkg/pose_detector.py
1. Core Node Analysis
pose_tracker_node uses PoseDetector to wrap the MediaPipe Pose model. Core workflow:
- Image processing (15Hz): raw Image frame conversion → flip →
PoseDetector.findPose()→ extract 33 keypoints - Center of gravity calculation: takes keypoints such as the hip center (midpoint of the left and right hip joints) plus the shoulder center to comprehensively compute the human reference position
- Body control (10Hz): deviation between the center x-coordinate and the image center → PD computes angular velocity; bounding box width → PD computes linear velocity
2. Key Parameters
Parameter definition file:
- Launch:
~/microros_ws/src/microros_v2_no_ptz_visual_control_pkg/launch/pose_tracker.launch.py- Node:
~/microros_ws/src/microros_v2_no_ptz_visual_control_pkg/microros_v2_no_ptz_visual_control_pkg/pose_tracker_node.py
| Parameter | Type | Default | Description |
|---|---|---|---|
pose_detection_confidence | float | 0.5 | Pose detection confidence threshold |
pose_tracking_confidence | float | 0.5 | Pose tracking confidence threshold |
deadband_x_pixel | float | 60.0 px | Horizontal deadband |
angular_kp | float | 1.8 | Body angle PD proportional coefficient |
angular_kd | float | 0.8 | Body angle PD derivative coefficient |
max_angular_z | float | 0.6 rad/s | Maximum body steering speed |
enable_distance_follow | bool | false | Whether to enable distance following |
target_pose_size_px | float | 320.0 px | Target human bounding box width for distance following |
distance_tolerance_pose_size_px | float | 10.0 px | Distance-following deadband |
distance_kp | float | 0.02 | Distance PD proportional coefficient |
max_linear_x | float | 0.30 m/s | Maximum forward/backward speed |
3. Node Description
| Node | Package | Description |
|---|---|---|
pose_tracker_node | microros_v2_no_ptz_visual_control_pkg | MediaPipe human pose tracking with 33 keypoints + PD body control |
6. Common Problems
| Problem | Cause | Solution |
|---|---|---|
| Tracking lag | Pose detection is computationally heavy and the frame rate is low | Lower image_processing_rate_hz or reduce the processing resolution |
| Person is not detected | Distance too far or part of the body is out of the frame | Keep the whole body in the frame and move closer to the camera |
| Multiple-person interference | The tracking target switches when several people are in the frame | Ensure only the target person is in the frame |