Welcome to MicroROS V2 Robot Car repository
4. Mediapipe Palm Follow
Mediapipe Palm Follow (No-PTZ)
1. Experiment Objective
Use MediaPipe to detect the palm, with body rotation tracking + distance following keeping the palm centered in the frame and maintaining a constant distance.
2. Experiment Principle
1. Key Terms
| Term | Description |
|---|---|
| MediaPipe Hands | Google's lightweight hand keypoint detection that outputs 21 hand skeleton points |
| Palm center (5th keypoint) | The MCP joint at the base of the index finger (index 5), used as the reference center for following |
| Distance Follow | Estimates distance from the palm bounding box size, with PD control maintaining the desired distance |
| Hand Skeleton | 21 keypoints connected according to the hand topology, visualized in real time |
| QoS | best_effort (image input + cmd_vel downlink), depth=1 |
2. Technical Architecture
/camera/image_raw (BEST_EFFORT)→ hand_tracker_node (enable_distance_follow:=true)├── MediaPipe Hands → 21 keypoints → palm center + bounding box size└── PD angular velocity + bounding box size distance following → /cmd_vel
A MediaPipe Python environment is required (for both ROS2 and the workspace environment).
3. Core Topics Quick Reference
Firmware Downlink
| Topic | Message Type | QoS | Field | Range | Direction |
|---|---|---|---|---|---|
/cmd_vel | geometry_msgs/Twist | best_effort + keep_last(1) | linear.x | -0.20~0.20 m/s | += forward, −= backward |
angular.z | -0.60~0.60 rad/s | += left turn, −= right turn |
3. Experiment Steps
1. Hardware Preparation
| Hardware | Requirement |
|---|---|
| PTZ Version | ❌ |
| No-PTZ Version | ✅ |
| Required Peripherals | ESP32 camera (mounted on bracket) |
2. Start Agent + Camera + Joint Model (Terminal 1)
xxxxxxxxxxros2 launch microros_v2_bringup car_base.launch.py
3. Start Palm Follow (Terminal 2)
xxxxxxxxxxros2 launch microros_v2_no_ptz_visual_control_pkg hand_follow.launch.py 
6. Operation Instructions
| Key | Function |
|---|---|
| Space | Toggle follow/standby |
| q / ESC | Exit |
Distance Follow Behavior: Palm approaches (bounding box grows) → move backward; moves away (bounding box shrinks) → move forward; size within tolerance → stay still.
4. Experiment Result
- Palm approaches → the car moves backward to keep the distance
- Palm moves away → the car moves forward to follow
- 21 keypoints and skeleton connections displayed in real time
- Palm moves out of the frame → stops moving
5. Code Explanation
Source code paths:
- Launch:
~/microros_ws/src/microros_v2_no_ptz_visual_control_pkg/launch/hand_follow.launch.py- Node source:
~/microros_ws/src/microros_v2_no_ptz_visual_control_pkg/microros_v2_no_ptz_visual_control_pkg/hand_tracker_node.py- Hand detector:
~/microros_ws/src/microros_v2_no_ptz_visual_control_pkg/microros_v2_no_ptz_visual_control_pkg/hand_detector.py
1. Core Node Explanation
hand_follow.launch.py launches hand_tracker_node, with enable_distance_follow defaulting to true.
Distance Following: The bounding box size is computed from the 21 keypoints (the larger of max(x)-min(x) and max(y)-min(y)) and compared with the desired size target_hand_size_px (90 px). Once the deviation exceeds the distance_tolerance_hand_size_px (12 px) deadband, the PD computes the linear.x output.
2. Key Parameters
Parameter definition files:
- Launch:
~/microros_ws/src/microros_v2_no_ptz_visual_control_pkg/launch/hand_follow.launch.py- Node:
~/microros_ws/src/microros_v2_no_ptz_visual_control_pkg/microros_v2_no_ptz_visual_control_pkg/hand_tracker_node.py
| Parameter | Type | Default | Description |
|---|---|---|---|
deadband_x_pixel | float | 50.0 px | Horizontal deadband |
angular_kp / angular_kd | float | 1.2 / 0.55 | Angle PD gains |
max_angular_z | float | 0.6 rad/s | Maximum turning speed |
enable_distance_follow | bool | true | Enable distance following |
target_hand_size_px | float | 90.0 px | Desired palm bounding box size |
distance_tolerance_hand_size_px | float | 12.0 px | Distance tolerance deadband |
distance_kp / distance_kd | float | 0.05 / 0.03 | Distance PD gains |
max_linear_x | float | 0.2 m/s | Maximum forward/backward speed |
lost_frame_threshold | int | 5 | Lost-target tolerance frame count |
3. Node Description
| Node | Package | Description |
|---|---|---|
hand_tracker_node | microros_v2_no_ptz_visual_control_pkg | MediaPipe palm tracking + following, 21 keypoints + dual PD |
6. Common Issues
| Issue | Cause | Solution |
|---|---|---|
| Frequent palm target loss | Palm turned over or moving quickly | Keep the palm plane facing the camera |
| Distance following overshoot | PD gains too large | Lower distance_kp |
| Palm not detected | Too far away or insufficient lighting | Move closer to the camera |