Visual Line Tracking (PTZ)

1. Experiment Objective

The camera detects the line on the ground (HSV color segmentation), PID controls the chassis angular velocity to drive along the line, and the LiDAR stops the car when an obstacle is detected ahead. The PTZ tilts down (pitch=-90°) toward the ground.

2. Experiment Principle

1. Key Terms

2. Technical Architecture

Dual timers (image at 30 Hz + control at 20 Hz) + LiDAR callback + PTZ tilt-down:

/camera/image_raw (BEST_EFFORT)
  → follow_line_ptz_node
    ├── Image processing (30 Hz): image conversion → flip → ROI HSV binarization → morphology → largest contour → EMA filtering
    ├── Chassis control (20 Hz): pixel error → scaling → discrete PID → angular velocity limiting → /cmd_vel (BEST_EFFORT)
    └── PTZ: tilts down to -90° at startup → /cmd_ptz_angle
/scan (sensor_data) → obstacle avoidance callback: valid point count → stop + buzzer above threshold

Key PTZ vs. No-PTZ differences: The pan-tilt can tilt down -90° toward the ground, so there is no need to manually adjust the camera mounting angle. Line tracking performs better than the fixed-mount version.

3. Core Topics Quick Reference

Firmware Uplink

Firmware Downlink

3. Experiment Steps

1. Hardware Preparation

2. Start Agent + Camera + Joint Model (Terminal 1)

xxxxxxxxxx
ros2 launch microros_v2_bringup car_base.launch.py

3. Start Visual Line Tracking (Terminal 2)

xxxxxxxxxx
ros2 launch microros_v2_ptz_visual_control_pkg follow_line_ptz.launch.py

After startup, the pan-tilt tilts down to -90° and line tracking starts automatically (auto_start_tracking:=true).

4. Operation Instructions

5. How to Stop

Press Ctrl+C to stop Terminal 2 → Terminal 1 in order.

4. Experiment Observations

• Automatic line tracking after startup: The pan-tilt tilts down -90° toward the ground, driving along the line at 0.15 m/s
• Line centered: Drives straight within the 40 px deadband, with the PID outputting zero angular velocity
• Line shifted left: Turns right to correct (positive angular velocity); shifted right → turns left
• Obstacle ahead: Stops + buzzer within 0.35 m, resumes after the obstacle is removed
• Line lost > 1.5 s: Stops and exits automatically
• Joystick override: Stops + resets the PID, resumes after release

5. Program Analysis

Source code paths:

• Launch: ~/microros_ws/src/microros_v2_ptz_visual_control_pkg/launch/follow_line_ptz.launch.py
• Node: ~/microros_ws/src/microros_v2_ptz_visual_control_pkg/microros_v2_ptz_visual_control_pkg/follow_line_ptz_node.py

1. Core Node Analysis

follow_line_ptz_node integrates PTZ control on top of the No-PTZ version. Core features:

• Discrete PID controller (DiscretePidController): Independent P/I/D terms + integral limiting, with the pid_scale (1000) normalized gain (actual P=0.022, D=0.005)
• HSV hue wraparound: When Hmin > Hmax (e.g., red H[160,10]), merges two masks in segments
• EMA filtering: target_smooth_alpha (0.55) exponential moving average to smooth the detected center
• Mouse sampling: Box-select the ROI → 5th~95th percentile HSV → auto-update the threshold
• PTZ tilt-down: The pan-tilt tilts down to initial_pitch_deg (-90°) at startup, facing the camera toward the ground

Chassis control timer — PID line tracking + obstacle avoidance priority:

​x
# Source file: ~/microros_ws/src/microros_v2_ptz_visual_control_pkg/microros_v2_ptz_visual_control_pkg/follow_line_ptz_node.py
​
def publish_chassis_control_command(self) -> None:
    if not self.is_node_running:
        return
    # Joystick override takes priority
    if self.joy_is_active:
        self.publish_zero_twist()
        return
    # Obstacle avoidance takes priority over the tracking PID
    if self.enable_obstacle_stop and self.obstacle_is_active:
        self.publish_zero_twist()
        self.publish_beep(True)
        return
​
    if self.tracking_state != TRACKING_STATE_TRACKING:
        return
​
    # Pixel error → PID angular velocity
    line_center_x = self.filtered_line_center[0]
    pixel_error_x = line_center_x - self.image_width * 0.5
​
    twist_command = Twist()
    twist_command.linear.x = float(self.line_linear_speed)  # constant 0.15 m/s
​
    if abs(pixel_error_x) <= self.line_center_deadband_pixel:
        angular_command = 0.0  # drive straight within the deadband
    else:
        pid_input = pixel_error_x / self.line_error_divisor  # scaling
        pid_output = self.line_pid_controller.update(pid_input, 0.0)
        angular_command = -pid_output  # reverse_angular_direction
        angular_command = np.clip(angular_command, -self.max_angular_speed, self.max_angular_speed)
​
    twist_command.angular.z = angular_command
    self.cmd_vel_publisher.publish(twist_command)

2. Key Parameters

Parameter definition files:

• Launch: ~/microros_ws/src/microros_v2_ptz_visual_control_pkg/launch/follow_line_ptz.launch.py
• Node: ~/microros_ws/src/microros_v2_ptz_visual_control_pkg/microros_v2_ptz_visual_control_pkg/follow_line_ptz_node.py

3. Node Description

6. Frequently Asked Questions