Palm Target Localization

1. Experiment Objective

Use MediaPipe Hands to detect the position of the palm in the camera image and output the center coordinates of the palm's bounding box.

2. Experiment Principle

1. Key Terms

2. Technical Architecture

Terminal 1: start agent + camera + joint models
Terminal 2: start palm localization

3. Experiment Steps

1. Hardware Preparation

2. Start the agent + camera + joint models (Terminal 1)

xxxxxxxxxx
ros2 launch microros_v2_bringup car_base.launch.py

3. Start palm localization (Terminal 2)

xxxxxxxxxx
ros2 launch microros_v2_mediapipe_pkg find_hand.launch.py

4. Operating Instructions

Place your palm in the camera's field of view. The screen shows the palm bounding box and a red center dot (indexX, indexY). Press q to exit.

5. How to Stop

Press q to exit.

4. Observed Results

• The palm is enclosed by a blue rectangular bounding box
• A red dot at the center marks the (indexX, indexY) coordinates
• As the palm moves across the frame, the coordinate values update in real time

5. Code Walkthrough

Source code paths:

• Launch: ~/microros_ws/src/microros_v2_mediapipe_pkg/launch/find_hand.launch.py
• Node: ~/microros_ws/src/microros_v2_mediapipe_pkg/microros_v2_mediapipe_pkg/FindHand.py

1. Core Node Analysis

​x
# Source file: ~/microros_ws/src/microros_v2_mediapipe_pkg/microros_v2_mediapipe_pkg/FindHand.py
class FindHand(Node):
    def __init__(self):
        super().__init__("find_hand_node")
        self.hands = mp.solutions.hands.Hands(
            static_image_mode=False, max_num_hands=1,
            min_detection_confidence=0.5, min_tracking_confidence=0.5)
​
    def image_callback(self, msg):
        frame = self.bridge.imgmsg_to_cv2(msg, 'bgr8')
        results = self.hands.process(cv.cvtColor(frame, cv.COLOR_BGR2RGB))
        if results.multi_hand_landmarks:
            for hand_landmarks in results.multi_hand_landmarks:
                h, w, _ = frame.shape
                # Compute the palm bounding box from the 21 landmarks
                xs = [lm.x for lm in hand_landmarks.landmark]
                ys = [lm.y for lm in hand_landmarks.landmark]
                x_min, x_max = int(min(xs)*w), int(max(xs)*w)
                y_min, y_max = int(min(ys)*h), int(max(ys)*h)
                # Center coordinates (normalized)
                indexX = (x_min + x_max) / 2 / w
                indexY = (y_min + y_max) / 2 / h
                cv.rectangle(frame, (x_min, y_min), (x_max, y_max), (255,0,0), 2)
                cv.circle(frame, (int(indexX*w), int(indexY*h)), 8, (0,0,255), -1)
                cv.putText(frame, f'({indexX:.2f},{indexY:.2f})', ...)
        cv.imshow(self.window_name, frame)

Key logic:

• The bounding box = the min/max of the 21 landmarks' x/y coordinates, not relying on MediaPipe's built-in rectangle (more flexible)
• Center coordinates normalized to 0~1: (x_min+x_max)/2/w, so downstream control nodes can directly use the ratio value
• A red solid dot (radius 8px) + (indexX, indexY) text marks the palm position

2. Key Parameters

Parameter definition files:

• Launch: ~/microros_ws/src/microros_v2_mediapipe_pkg/launch/find_hand.launch.py
• Node: ~/microros_ws/src/microros_v2_mediapipe_pkg/microros_v2_mediapipe_pkg/FindHand.py

3. Node Description

6. Frequently Asked Questions