Holistic Detection

1. Experiment Objective

Use MediaPipe Holistic to simultaneously detect the face (468 keypoints), both hands (21 keypoints each) and body pose (33 keypoints) in the camera image.

2. Experiment Principle

1. Key Terms

2. Technical Architecture

Terminal 1: start agent + camera + joint models
Terminal 2: start holistic detection

3. Experiment Steps

1. Hardware Preparation

2. Start the agent + camera + joint models (Terminal 1)

xxxxxxxxxx
ros2 launch microros_v2_bringup car_base.launch.py

3. Start holistic detection (Terminal 2)

xxxxxxxxxx
ros2 launch microros_v2_mediapipe_pkg holistic.launch.py

4. Operating Instructions

Press q to exit.

5. How to Stop

Press Ctrl+C in each terminal, one by one, to stop.

4. Observed Results

• The face is finely covered by a 468-point white mesh; each hand has 21 red dots + a skeleton; the body has a 33-point skeleton overlaid
• All three sets of keypoints render simultaneously without occluding each other

5. Code Walkthrough

Source code paths:

• Launch: ~/microros_ws/src/microros_v2_mediapipe_pkg/launch/holistic.launch.py
• Node: ~/microros_ws/src/microros_v2_mediapipe_pkg/microros_v2_mediapipe_pkg/Holistic.py

1. Core Node Analysis

​x
# Source file: ~/microros_ws/src/microros_v2_mediapipe_pkg/microros_v2_mediapipe_pkg/Holistic.py
class Holistic(Node):
    def __init__(self):
        super().__init__("holistic_node")
        self.mp_holistic = mp.solutions.holistic
        self.holistic = self.mp_holistic.Holistic(
            static_image_mode=False,
            min_detection_confidence=0.5,
            min_tracking_confidence=0.5)
​
    def image_callback(self, msg):
        frame = self.bridge.imgmsg_to_cv2(msg, 'bgr8')
        results = self.holistic.process(cv.cvtColor(frame, cv.COLOR_BGR2RGB))
        if results.pose_landmarks:
            self.mp_draw.draw_landmarks(frame, results.pose_landmarks,
                self.mp_holistic.POSE_CONNECTIONS)
        if results.face_landmarks:
            self.mp_draw.draw_landmarks(frame, results.face_landmarks,
                self.mp_holistic.FACEMESH_TESSELATION, ...)
        if results.left_hand_landmarks:
            self.mp_draw.draw_landmarks(frame, results.left_hand_landmarks,
                self.mp_holistic.HAND_CONNECTIONS)
        if results.right_hand_landmarks:
            self.mp_draw.draw_landmarks(frame, results.right_hand_landmarks,
                self.mp_holistic.HAND_CONNECTIONS)
        cv.imshow(self.window_name, frame)

Key logic:

• mp.solutions.holistic.Holistic(): three-in-one detector that runs the Pose, Face Mesh and Hands sub-models simultaneously
• The four landmark sets are checked independently: face (468 points), pose (33 points), left hand (21 points) and right hand (21 points) may not all be present at the same time
• FACEMESH_TESSELATION: triangular face mesh, finer than simple skeleton connections

2. Key Parameters

Parameter definition files:

• Launch: ~/microros_ws/src/microros_v2_mediapipe_pkg/launch/holistic.launch.py
• Node: ~/microros_ws/src/microros_v2_mediapipe_pkg/microros_v2_mediapipe_pkg/Holistic.py

3. Node Description

6. Frequently Asked Questions