Oriented Bounding Boxes Object Detection

1. Experiment Objective

Use YOLO OBB (Oriented Bounding Box) for oriented object detection — it outputs bounding boxes with a rotation angle, making it suitable for detecting tilted objects (such as vehicles, text, and signs in aerial imagery). It supports three input sources: images, videos, and a real-time camera feed.

2. Experiment Principle

1. Glossary

2. Technical Architecture

YOLO OBB detection starts inference nodes through ROS2 launch files, supporting three input sources:

• Image/Video mode: The node reads local files for offline inference; it does not require micro_ros_agent or the state estimation layer.
• Real-time camera mode: The node subscribes to the /camera/image_raw topic to get real-time camera frames. It requires three processes to work together: micro_ros_agent → state estimation layer → YOLO OBB node.

Real-time mode data flow:

ESP32 camera → micro_ros_agent → /camera/image_raw → YOLO OBB node → /yolo_obb/annotated_image + /yolo_obb/status

Regular object detection outputs horizontal rectangular boxes (HBB) aligned with the image axes; OBB outputs oriented rectangular boxes with a rotation angle. For tilted objects (such as a rotated book, a tilted sign, or vehicles in aerial imagery), OBB boxes are more accurate — the box area is smaller and includes less background.

YOLO inference depends on the ultralytics + PyTorch libraries installed in the current Python environment; the ROS2 and workspace environment must be sourced before starting.

3. Core Topic Quick Reference

ROS2 Internal Topics

3. Experiment Steps

1. Hardware Requirements

2. Start the YOLO Environment

xxxxxxxxxx
yolo_env

3. Image Mode (Single Image Inference)

xxxxxxxxxx
ros2 launch microros_v2_yolo_pkg yolo_obb_image.launch.py

4. Video Mode (Frame-by-Frame Inference on a Video File)

xxxxxxxxxx
ros2 launch microros_v2_yolo_pkg yolo_obb_video.launch.py

5. Real-time Camera Mode

Launch the proxy + camera + joint model (Terminal 1)

xxxxxxxxxx
ros2 launch microros_v2_bringup car_base.launch.py

Terminal 2 — Launch YOLO real-time OBB detection:

xxxxxxxxxx
ros2 launch microros_v2_yolo_pkg yolo_obb_realtime.launch.py

6. Operating Instructions

• Image mode: The program reads a single image and annotates detected tilted objects with rotated rectangular boxes. Each box shows the class name and confidence, and the box angle matches the actual orientation of the object. Press q or ESC to close the window and exit.
• Video mode: Frame-by-frame inference; each frame outputs OBB rotated-box detection results. The rotated box angle adjusts as the object orientation changes, making the box more accurate than a horizontal one. Press q or ESC to stop playback and exit.
• Real-time camera mode: Real-time inference on the live camera feed; the top-left corner of the frame shows the FPS and the number of OBB detections. The OBB model is pretrained on an aerial dataset (DOTA), so its detection of everyday objects in horizontal views is limited. Press q or ESC to exit.

7. How to Stop

Press Ctrl+C in each terminal, or press q or ESC to exit.

4. Experimental Phenomena

1. Image Mode

• Tilted objects are annotated with rotated rectangular boxes whose edges are parallel to the actual orientation of the objects.
• For tilted objects that a horizontal box cannot fit well (such as a book placed at 45°), the OBB box fits perfectly.
• Rotated boxes come with class names and confidence scores.

2. Video Mode

• The rotated box angle of objects in the video dynamically adjusts with the object orientation.
• As objects rotate, the OBB boxes follow in real time and always hug the object outline.

3. Real-time Camera Mode

• Tilted objects in the camera feed are annotated with rotated boxes.
• The top-left corner of the frame shows the model name, FPS, and the number of OBB boxes.
• For common vertical/horizontal objects seen in a horizontal view, the OBB box performs similarly to an HBB.

4. Edge Cases

• The pretrained model is based on the aerial dataset (DOTA) and recognizes everyday objects on desks/floors less well than general detection models.
• The OBB model has a limited set of training classes (about 15 DOTA classes: airplanes, ships, vehicles, etc.); unrelated classes will not be detected.

5. Program Analysis

Source code paths:

• Launch (real-time): ~/microros_ws/src/microros_v2_yolo_pkg/launch/yolo_obb_realtime.launch.py
• Launch (image): ~/microros_ws/src/microros_v2_yolo_pkg/launch/yolo_obb_image.launch.py
• Launch (video): ~/microros_ws/src/microros_v2_yolo_pkg/launch/yolo_obb_video.launch.py
• Node source: ~/microros_ws/src/microros_v2_yolo_pkg/microros_v2_yolo_pkg/yolo_obb_realtime_node.py

1. Core Node Analysis

The real-time OBB detection node yolo_obb_realtime_node uses an OBB model (-obb.pt) and outputs oriented bounding boxes with a rotation angle. The key difference is that results[0].obb contains rotated-box information (center point, width/height, angle), and it draws rotated rectangles on the frame instead of horizontal rectangles.

Constructor (__init__): Parameters unique to the OBB node include draw_fps_text (independent FPS text control) and print_classes (prints the class list at startup).

​x
# source file: ~/microros_ws/src/microros_v2_yolo_pkg/microros_v2_yolo_pkg/yolo_obb_realtime_node.py
​
    def __init__(self):
        super().__init__("yolo_obb_realtime_node")
​
        self.declare_parameter("model_dir", str(DEFAULT_MODEL_DIR))
        self.declare_parameter("model_name", "yolo11n-obb.pt")
        self.declare_parameter("model_path", "")
        self.declare_parameter("image_topic", "/camera/image_raw")
        self.declare_parameter("annotated_topic", "/yolo_obb/annotated_image")
        self.declare_parameter("status_topic", "/yolo_obb/status")
        self.declare_parameter("output_dir", str(DEFAULT_OUTPUT_DIR))
        self.declare_parameter("imgsz", 960)
        self.declare_parameter("conf", 0.25)
        self.declare_parameter("iou", 0.7)
        self.declare_parameter("max_det", 1)
        self.declare_parameter("device", "cpu")
        self.declare_parameter("classes", "")
        self.declare_parameter("show", True)
        self.declare_parameter("save_video", True)
        self.declare_parameter("output_fps", 10.0)
        self.declare_parameter("window_width", 960)
        self.declare_parameter("window_height", 720)
        self.declare_parameter("qos_reliability", "best_effort")
        self.declare_parameter("qos_depth", 1)
        self.declare_parameter("process_every_n_frames", 1)
        self.declare_parameter("line_width", 2)
        self.declare_parameter("publish_status", True)
        self.declare_parameter("draw_fps_text", True)
        self.declare_parameter("text_scale", 0.65)
        self.declare_parameter("text_thickness", 2)
        self.declare_parameter("print_classes", True)

Image callback (image_callback): Converts the raw Image to BGR, calls model.predict(), uses result.plot() to draw the OBB rotated boxes, then adds the model name, FPS, and OBB count via _draw_status_text(), and finally publishes the annotated image and status.

xxxxxxxxxx
# source file: ~/microros_ws/src/microros_v2_yolo_pkg/microros_v2_yolo_pkg/yolo_obb_realtime_node.py
​
    def _draw_status_text(self, frame, fps, num_obb):
        if not self.draw_fps_text:
            return frame
        text = f"{self.model_name} FPS:{fps:.1f} OBB:{num_obb}"
        cv2.putText(frame, text, (10, 28), cv2.FONT_HERSHEY_SIMPLEX,
                    self.text_scale, (0, 255, 0), self.text_thickness)
        return frame
​
    def image_callback(self, msg: Image) -> None:
        frame = self.bridge.imgmsg_to_cv2(msg, desired_encoding="bgr8")
        results = self.model.predict(
            source=frame, imgsz=self.imgsz, conf=self.conf, iou=self.iou,
            max_det=self.max_det, device=self.device, classes=self.classes,
            verbose=False,
        )
        result = results[0] if results else None
        annotated = result.plot() if result else frame
        num_obb = 0 if result is None or result.obb is None else len(result.obb)
        annotated = self._draw_status_text(annotated, self._update_fps(), num_obb)

2. Key Parameters

Parameter definition files:

• Launch: ~/microros_ws/src/microros_v2_yolo_pkg/launch/yolo_obb_realtime.launch.py
• Node: ~/microros_ws/src/microros_v2_yolo_pkg/microros_v2_yolo_pkg/yolo_obb_realtime_node.py

3. Node Description

6. Common Problems