Pose Estimation

1. Experiment Objective

Use YOLO for human pose estimation — detect 17 keypoints (joint positions) of the human body and draw skeleton connections. It supports simultaneous detection of multiple people, as well as three input sources: images, videos, and a real-time camera feed.

2. Experiment Principle

1. Glossary

2. Technical Architecture

YOLO pose estimation starts inference nodes through ROS2 launch files, supporting three input sources:

• Image/Video mode: The node reads local files for offline inference; it does not require micro_ros_agent or the state estimation layer.
• Real-time camera mode: The node subscribes to the /camera/image_raw topic to get real-time camera frames. It requires three processes to work together: micro_ros_agent → state estimation layer → YOLO pose estimation node.

Real-time mode data flow:

ESP32 camera → micro_ros_agent → /camera/image_raw → YOLO pose estimation node → /yolo_pose/annotated_image + /yolo_pose/status

YOLO inference depends on the ultralytics + PyTorch libraries installed in the current Python environment; the ROS2 and workspace environment must be sourced before starting.

3. Core Topic Quick Reference

ROS2 Internal Topics

3. Experiment Steps

1. Hardware Requirements

2. Start the YOLO Environment

xxxxxxxxxx
yolo_env

3. Image Mode (Single Image Inference)

xxxxxxxxxx
ros2 launch microros_v2_yolo_pkg yolo_pose_image.launch.py

4. Video Mode (Frame-by-Frame Inference on a Video File)

xxxxxxxxxx
ros2 launch microros_v2_yolo_pkg yolo_pose_video.launch.py

5. Real-time Camera Mode

Launch the proxy + camera + joint model (Terminal 1)

xxxxxxxxxx
ros2 launch microros_v2_bringup car_base.launch.py

Terminal 2 — Launch YOLO real-time pose estimation:

xxxxxxxxxx
ros2 launch microros_v2_yolo_pkg yolo_pose_realtime.launch.py

6. Operating Instructions

• Image mode: The program reads a single image, detects each person, and annotates 17 keypoints (red circles) and skeleton connections (orange line segments). Press q or ESC to close the window and exit.
• Video mode: Frame-by-frame inference; each frame detects the people in the frame and draws the skeletons. Tracking is enabled by default (use_track=true), keeping the ID of the same person stable across frames. The node has built-in IoU deduplication (dedup_iou=0.5) that removes duplicate boxes the tracker may produce for the same person. Press q or ESC to stop playback and exit.
• Real-time camera mode: The human skeletons in the live camera feed update instantly as people move. The top-left corner of the frame shows the model name, FPS, number of detected people, and keypoint count. Press q or ESC to exit.

7. How to Stop

Press Ctrl+C in each terminal, or press q or ESC to exit.

4. Experimental Phenomena

1. Image Mode

• Each person shows 17 keypoints (colored dots) and orange skeleton connections.
• The person's bounding box is annotated in green, with the tracking ID and confidence shown above the box.
• Multiple people are detected simultaneously, and each person is annotated independently.

2. Video Mode

• Human poses update in real time, and the skeleton changes naturally with body movement.
• In multi-person scenes, each individual is assigned an independent tracking ID that stays stable over time.
• The IoU deduplication mechanism removes duplicate skeletons caused by the tracker.

3. Real-time Camera Mode

• Human poses in the camera feed are displayed in real time; latency depends on CPU performance.
• The top of the frame shows the model name, FPS, number of detected people, and number of keypoint groups.

4. Edge Cases

• Keypoint accuracy drops for non-standard poses such as side views, back views, or partial occlusion.
• In dense multi-person scenes (e.g. crowds), some people may be missed.
• Non-human targets (animals, robots) do not trigger pose estimation.

5. Program Analysis

Source code paths:

• Launch (real-time): ~/microros_ws/src/microros_v2_yolo_pkg/launch/yolo_pose_realtime.launch.py
• Launch (image): ~/microros_ws/src/microros_v2_yolo_pkg/launch/yolo_pose_image.launch.py
• Launch (video): ~/microros_ws/src/microros_v2_yolo_pkg/launch/yolo_pose_video.launch.py
• Node source: ~/microros_ws/src/microros_v2_yolo_pkg/microros_v2_yolo_pkg/yolo_pose_realtime_node.py

1. Core Node Analysis

The real-time pose estimation node yolo_pose_realtime_node subscribes to the raw image topic, calls the YOLO pose estimation model (-pose.pt), and publishes the image annotated with keypoints and skeleton connections along with JSON status.

Constructor (__init__): Declares the model, raw image input, output topics, ByteTrack, QoS, and display parameters. The node selects model.track() or model.predict() via use_track.

​x
# source file: ~/microros_ws/src/microros_v2_yolo_pkg/microros_v2_yolo_pkg/yolo_pose_realtime_node.py
​
    def __init__(self):
        super().__init__("yolo_pose_realtime_node")
​
        self.declare_parameter("model_dir", str(DEFAULT_MODEL_DIR))
        self.declare_parameter("model_name", "yolo11n-pose.pt")
        self.declare_parameter("model_path", "")
        self.declare_parameter("image_topic", "/camera/image_raw")
        self.declare_parameter("annotated_topic", "/yolo_pose/annotated_image")
        self.declare_parameter("status_topic", "/yolo_pose/status")
        self.declare_parameter("output_dir", str(DEFAULT_OUTPUT_DIR))
        self.declare_parameter("imgsz", 960)
        self.declare_parameter("conf", 0.25)
        self.declare_parameter("iou", 0.7)
        self.declare_parameter("max_det", 300)
        self.declare_parameter("device", "cpu")
        self.declare_parameter("classes", "")
        self.declare_parameter("tracker", "bytetrack.yaml")
        self.declare_parameter("show", True)
        self.declare_parameter("save_video", True)
        self.declare_parameter("output_fps", 20.0)
        self.declare_parameter("window_width", 1280)
        self.declare_parameter("window_height", 720)
        self.declare_parameter("qos_reliability", "best_effort")
        self.declare_parameter("qos_depth", 1)
        self.declare_parameter("process_every_n_frames", 1)
        self.declare_parameter("line_width", 2)
        self.declare_parameter("publish_status", True)
        self.declare_parameter("use_track", True)

Image callback (image_callback): Converts the raw Image to BGR via CvBridge, calls the tracking or prediction interface according to use_track, then uses Ultralytics result.plot() to draw keypoints, skeleton, and bounding boxes, and publishes the annotated image and status.

xxxxxxxxxx
# source file: ~/microros_ws/src/microros_v2_yolo_pkg/microros_v2_yolo_pkg/yolo_pose_realtime_node.py
​
    def image_callback(self, msg: Image) -> None:
        self.frame_index += 1
        if self.frame_index % self.process_every_n_frames != 0 or self.processing:
            return
        self.processing = True
        try:
            frame = self.bridge.imgmsg_to_cv2(msg, desired_encoding="bgr8")
            kwargs = dict(
                source=frame, imgsz=self.imgsz, conf=self.conf, iou=self.iou,
                max_det=self.max_det, device=self.device, classes=self.classes,
                line_width=self.line_width, verbose=False,
            )
            results = (
                self.model.track(persist=True, tracker=self.tracker, **kwargs)
                if self.use_track else self.model.predict(**kwargs)
            )
            annotated = results[0].plot(line_width=self.line_width)
            out_msg = self.bridge.cv2_to_imgmsg(annotated, encoding="bgr8")
            out_msg.header = msg.header
            self.annotated_pub.publish(out_msg)
        finally:
            self.processing = False

2. Key Parameters

Parameter definition files:

• Launch: ~/microros_ws/src/microros_v2_yolo_pkg/launch/yolo_pose_realtime.launch.py
• Node: ~/microros_ws/src/microros_v2_yolo_pkg/microros_v2_yolo_pkg/yolo_pose_realtime_node.py

3. Node Description

6. Common Problems