Image Classification

1. Experiment Objective

Use YOLO for image classification — determine which class a whole image belongs to and output the Top-K classification results with confidence. It supports three input sources: images, videos, and a real-time camera feed.

2. Experiment Principle

1. Glossary

2. Technical Architecture

YOLO image classification starts inference nodes through ROS2 launch files, supporting three input sources:

• Image/Video mode: The node reads local files for offline inference; it does not require micro_ros_agent or the state estimation layer.
• Real-time camera mode: The node subscribes to the /camera/image_raw topic to get real-time camera frames. It requires three processes to work together.

Real-time mode data flow:

ESP32 camera → micro_ros_agent → /camera/image_raw → YOLO classification node → /yolo_cls/annotated_image + /yolo_cls/status

The classification model runs inference with the small input size imgsz=224 (much smaller than the 960 used by detection/segmentation/pose), so classification is the fastest. The classification node draws text with a custom small font instead of using result.plot(), to avoid large text covering the frame.

YOLO inference depends on the ultralytics + PyTorch libraries installed in the current Python environment; the ROS2 and workspace environment must be sourced before starting.

3. Core Topic Quick Reference

ROS2 Internal Topics

3. Experiment Steps

1. Hardware Requirements

2. Start the YOLO Environment

xxxxxxxxxx
yolo_env

3. Image Mode (Single Image Inference)

xxxxxxxxxx
ros2 launch microros_v2_yolo_pkg yolo_cls_image.launch.py

4. Video Mode (Frame-by-Frame Inference on a Video File)

xxxxxxxxxx
ros2 launch microros_v2_yolo_pkg yolo_cls_video.launch.py

5. Real-time Camera Mode

Launch the proxy + camera + joint model (Terminal 1)

xxxxxxxxxx
ros2 launch microros_v2_bringup car_base.launch.py

Terminal 2 — Launch YOLO real-time classification:

xxxxxxxxxx
ros2 launch microros_v2_yolo_pkg yolo_cls_realtime.launch.py

6. Operating Instructions

• Image mode: The program reads a single image, runs inference on the whole image, and outputs the Top-3 classification results (default display_topk=3). The results are overlaid in the top-left corner of the frame as small green text showing class names and confidence. Press q or ESC to close the window and exit.
• Video mode: Frame-by-frame inference; each frame outputs the classification result for that frame. When the frame content changes, the classification result updates accordingly. Press q or ESC to stop playback and exit.
• Real-time camera mode: Real-time inference on the camera feed; the classification result updates instantly as the frame content changes. Image classification only outputs a whole-image label and does not output bounding boxes or location information — this is the core difference from object detection. Press q or ESC to exit.

7. How to Stop

Press Ctrl+C in each terminal, or press q or ESC to exit.

4. Experimental Phenomena

1. Image Mode

• The top-left corner of the frame shows the model name, FPS, and Top-3 classification results (class name: confidence).
• For example: yolo11n-cls, FPS:30.0, tabby cat: 0.85, tiger cat: 0.10, Egyptian cat: 0.03.
• The classification results are shown as small green text that is clear without covering the main content of the frame.

2. Video Mode

• Each video frame displays the classification result in real time; the Top-3 labels update as the content changes.
• On a static frame, the classification result stays stable.

3. Real-time Camera Mode

• When objects move in the camera feed, the classification labels quickly switch to the most matching class.
• Since the inference input is only 224px, classification is extremely fast (typically 20-30 FPS or more).

4. Edge Cases

• Blank/blurry frames may output random classification results (the model tries to match the closest known class).
• Objects not in the ImageNet 1000 classes (such as robots or ESP32 dev boards) may be misclassified into similar classes.

5. Program Analysis

Source code paths:

• Launch (real-time): ~/microros_ws/src/microros_v2_yolo_pkg/launch/yolo_cls_realtime.launch.py
• Launch (image): ~/microros_ws/src/microros_v2_yolo_pkg/launch/yolo_cls_image.launch.py
• Launch (video): ~/microros_ws/src/microros_v2_yolo_pkg/launch/yolo_cls_video.launch.py
• Node source: ~/microros_ws/src/microros_v2_yolo_pkg/microros_v2_yolo_pkg/yolo_cls_realtime_node.py

1. Core Node Analysis

The real-time classification node yolo_cls_realtime_node differs significantly from the detection/segmentation/pose nodes: the classification task does not need to draw bounding boxes or masks; instead, it overlays the Top-K classification results on the frame as small text.

Constructor (__init__): Parameters unique to the classification node include topk (number of results kept internally), display_topk (number shown on the frame), and text display control parameters (text_scale, text_thickness, text_x, text_y, text_line_gap).

​x
# source file: ~/microros_ws/src/microros_v2_yolo_pkg/microros_v2_yolo_pkg/yolo_cls_realtime_node.py
​
    def __init__(self):
        super().__init__("yolo_cls_realtime_node")
​
        self.declare_parameter("model_dir", str(DEFAULT_MODEL_DIR))
        self.declare_parameter("model_name", "yolo11n-cls.pt")
        self.declare_parameter("model_path", "")
        self.declare_parameter("image_topic", "/camera/image_raw")
        self.declare_parameter("annotated_topic", "/yolo_cls/annotated_image")
        self.declare_parameter("status_topic", "/yolo_cls/status")
        self.declare_parameter("output_dir", str(DEFAULT_OUTPUT_DIR))
        self.declare_parameter("imgsz", 224)
        self.declare_parameter("device", "cpu")
        self.declare_parameter("topk", 5)
        self.declare_parameter("display_topk", 3)
        self.declare_parameter("show", True)
        self.declare_parameter("save_video", True)
        self.declare_parameter("output_fps", 20.0)
        self.declare_parameter("window_width", 1280)
        self.declare_parameter("window_height", 720)
        self.declare_parameter("qos_reliability", "best_effort")
        self.declare_parameter("qos_depth", 1)
        self.declare_parameter("process_every_n_frames", 1)
        self.declare_parameter("publish_status", True)
        self.declare_parameter("text_scale", 0.45)
        self.declare_parameter("text_thickness", 1)
        self.declare_parameter("text_x", 10)
        self.declare_parameter("text_y", 22)
        self.declare_parameter("text_line_gap", 18)
        self.declare_parameter("show_fps_text", True)
        self.declare_parameter("show_model_text", True)
        self.declare_parameter("draw_text_shadow", True)

Classification result drawing (_draw_small_text): The classification node does not use result.plot(); instead, it manually draws small text based on the _extract_topk() results. It supports a shadow effect for better readability and can control the display of FPS and model name independently.

xxxxxxxxxx
# source file: ~/microros_ws/src/microros_v2_yolo_pkg/microros_v2_yolo_pkg/yolo_cls_realtime_node.py
​
    def _extract_topk(self, result):
        if result.probs is None:
            return []
        labels = []
        for idx in result.probs.top5[:self.topk]:
            labels.append((str(result.names[int(idx)]), float(result.probs.data[idx])))
        return labels
​
    def _draw_small_text(self, frame, labels, fps):
        annotated = frame.copy()
        lines = []
        if self.show_model_text:
            lines.append(self.model_name)
        if self.show_fps_text:
            lines.append(f"FPS:{fps:.1f}")
        lines.extend(f"{label}: {score:.2f}" for label, score in labels[:self.display_topk])
        x = self.text_x
        y = self.text_y
​
        for i, text in enumerate(lines):
            yy = y + i * self.text_line_gap
​
            if self.draw_text_shadow:
                cv2.putText(
                    annotated, text, (x + 1, yy + 1),
                    cv2.FONT_HERSHEY_SIMPLEX, self.text_scale,
                    (0, 0, 0), max(1, self.text_thickness + 1), cv2.LINE_AA,
                )
​
            cv2.putText(
                annotated, text, (x, yy),
                cv2.FONT_HERSHEY_SIMPLEX, self.text_scale,
                (0, 255, 0), self.text_thickness, cv2.LINE_AA,
            )
​
        return annotated
​
    def image_callback(self, msg: Image) -> None:
        frame = self.bridge.imgmsg_to_cv2(msg, desired_encoding="bgr8")
        results = self.model.predict(
            source=frame, imgsz=self.imgsz, device=self.device, verbose=False
        )
        fps = self._update_fps()
        labels = self._extract_topk(results[0]) if results else self.last_labels
        annotated = self._draw_small_text(frame, labels, fps)

2. Key Parameters

Parameter definition files:

• Launch: ~/microros_ws/src/microros_v2_yolo_pkg/launch/yolo_cls_realtime.launch.py
• Node: ~/microros_ws/src/microros_v2_yolo_pkg/microros_v2_yolo_pkg/yolo_cls_realtime_node.py

3. Node Description

6. Common Problems