Welcome to MicroROS V2 Robot Car repository
6. Mediapipe Human Posture Follow
Mediapipe Human Pose Follow (No-PTZ)
1. Experiment Objective
Use MediaPipe Pose to detect 33 human keypoints, with body rotation tracking + distance following keeping the target person centered in the frame and maintaining a constant distance.
2. Experiment Principle
1. Key Terms
| Term | Description |
|---|---|
| MediaPipe Pose | Google's lightweight human pose estimation that detects 33 human keypoints in real time |
| Body Center of Gravity | Computes the body reference position from keypoints such as the hip center and shoulder center |
| Distance Follow | Estimates distance from the human bounding box width, with PD control maintaining the desired distance |
| Pose Skeleton | 33 keypoints connected according to the human topology for visualization |
| QoS | best_effort (image input + cmd_vel downlink), depth=1 |
2. Technical Architecture
/camera/image_raw (BEST_EFFORT)→ pose_tracker_node (enable_distance_follow:=true)├── MediaPipe Pose → 33 keypoints → center of gravity + bounding box width└── PD angular velocity + bounding box width distance following → /cmd_vel
A MediaPipe Python environment is required (for both ROS2 and the workspace environment).
3. Core Topics Quick Reference
Firmware Downlink
| Topic | Message Type | QoS | Field | Range | Direction |
|---|---|---|---|---|---|
/cmd_vel | geometry_msgs/Twist | best_effort + keep_last(1) | linear.x | -0.30~0.30 m/s | += forward, −= backward |
angular.z | -0.60~0.60 rad/s | += left turn, −= right turn |
3. Experiment Steps
1. Hardware Preparation
| Hardware | Requirement |
|---|---|
| PTZ Version | ❌ |
| No-PTZ Version | ✅ |
| Required Peripherals | ESP32 camera (mounted on bracket) |
2. Start Agent + Camera + Joint Model (Terminal 1)
xxxxxxxxxxros2 launch microros_v2_bringup car_base.launch.py
3. Start Human Pose Follow (Terminal 2)
xxxxxxxxxxros2 launch microros_v2_no_ptz_visual_control_pkg pose_follow.launch.py
6. Operation Instructions
| Key | Function |
|---|---|
| Space | Toggle follow/standby |
| i / I | Standby (IDLE) |
| q / ESC | Exit |
4. Experiment Result
- Human approaches → the car moves backward; moves away → the car moves forward to follow
- 33 keypoints and skeleton connections displayed in real time
- Human moves out of the frame → stops moving
- With multiple people, the tracking target may switch
5. Code Explanation
Source code paths:
- Launch:
~/microros_ws/src/microros_v2_no_ptz_visual_control_pkg/launch/pose_follow.launch.py- Node source:
~/microros_ws/src/microros_v2_no_ptz_visual_control_pkg/microros_v2_no_ptz_visual_control_pkg/pose_tracker_node.py- Pose detector:
~/microros_ws/src/microros_v2_no_ptz_visual_control_pkg/microros_v2_no_ptz_visual_control_pkg/pose_detector.py
1. Core Node Explanation
pose_follow.launch.py launches pose_tracker_node, with enable_distance_follow defaulting to true. Pose detection is computationally heavy (33 keypoints), so image processing defaults to 15 Hz to ensure real-time performance. Distance following is based on the human bounding box width (not radius/size): the wider the bounding box (the closer) → move backward; the narrower (the farther) → move forward.
2. Key Parameters
Parameter definition files:
- Launch:
~/microros_ws/src/microros_v2_no_ptz_visual_control_pkg/launch/pose_follow.launch.py- Node:
~/microros_ws/src/microros_v2_no_ptz_visual_control_pkg/microros_v2_no_ptz_visual_control_pkg/pose_tracker_node.py
| Parameter | Type | Default | Description |
|---|---|---|---|
pose_detection_confidence | float | 0.5 | Pose detection confidence threshold |
pose_tracking_confidence | float | 0.5 | Pose tracking confidence threshold |
deadband_x_pixel | float | 60.0 px | Horizontal deadband |
angular_kp / angular_kd | float | 1.8 / 0.8 | Angle PD gains |
max_angular_z | float | 0.6 rad/s | Maximum turning speed |
enable_distance_follow | bool | true | Enable distance following |
target_pose_size_px | float | 320.0 px | Desired human bounding box width |
distance_tolerance_pose_size_px | float | 10.0 px | Distance tolerance |
distance_kp / distance_kd | float | 0.02 / 0.01 | Distance PD gains |
max_linear_x | float | 0.30 m/s | Maximum forward/backward speed |
3. Node Description
| Node | Package | Description |
|---|---|---|
pose_tracker_node | microros_v2_no_ptz_visual_control_pkg | MediaPipe human pose tracking + following, 33 keypoints + dual PD |
6. Common Issues
| Issue | Cause | Solution |
|---|---|---|
| Tracking lag | High Pose detection computation | Lower the image processing resolution or rate |
| Person not detected | Too far away or part of the body outside the frame | Keep the whole body within the frame |
| Interference from multiple people | Multiple people in the frame | Make sure only the target person is in the frame |