Welcome to DOFBOT-Ultra repository

Color Block Positioning Experiment

Please ensure that the web control program backend process is not running in the background. To close it, refer to the [Get Started Quickly - 2. Access the Terminal] section in [Disable the Auto-Start Web Control Backend Process]. After closing, please restart DofbotUltra!!!

The color block positioning function detects the position of a color block by analyzing the HSV values of the camera image, outlines the block, and prints its position coordinates.

1. Startup and Operation

Open a browser (any browser on any host in the same local network) and enter the IP address:8888 in the address bar to access JupyterLab.

Code path:

xxxxxxxxxx
dofbot_color_follow/scripts/Color_position.ipynb

Click the Run All button on the JupyterLab toolbar, then scroll to the bottom.

The camera feed will be displayed. Click [color_position] and select the color to locate. For example, select [green], and the green block will be outlined.

Open the Log panel and set Log Level to Info to see the position coordinates of the color block printed.

To disable the function, click the [position_cancel] button.

To end the program, click [Exit] to avoid affecting resource usage by other programs.

Note: If color recognition is inaccurate, use /home/yahboom/dofbot_ultra_ws/src/dofbot_color_follow/scripts/HSV_calibration.ipynb to calibrate the colors.

2. Technical Overview

The core of color block positioning is target detection based on HSV color space:

1. Image Acquisition: Capture real-time images via the camera.

2. Color Space Conversion: Convert BGR images to HSV color space. HSV is more robust to lighting changes than BGR.

3. Color Threshold Filtering: Generate a binary mask using inRange based on preset HSV upper and lower limits, keeping only the target color region.

4. Morphological Processing: Apply erosion and dilation to remove noise.

5. Contour Detection: Find external contours in the mask and select the largest one.

6. Minimum Enclosing Circle: Use a minimum enclosing circle to frame the target. The center of the circle is the coordinate of the color block.

3. Core Code Analysis

• Main recognition function, gets the target center point (color_x, color_y) of the color block.

xxxxxxxxxx
def process(self, img, HSV_config):
        (color_lower, color_upper) = HSV_config
        self.img = img.copy()
        img1 = cv.GaussianBlur(self.img, (5, 5), 0)
        hsv = cv.cvtColor(img1, cv.COLOR_BGR2HSV)
        mask = cv.inRange(hsv, color_lower, color_upper)
        mask = cv.erode(mask, None, iterations=2)
        mask = cv.dilate(mask, None, iterations=2)
        mask = cv.GaussianBlur(mask, (5, 5), 0)
        cnts = cv.findContours(mask.copy(), cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)[-2]
        pos = None
        if len(cnts) > 0:
            cnt = max(cnts, key=cv.contourArea)
            (color_x, color_y), color_radius = cv.minEnclosingCircle(cnt)
            if color_radius > 10:
                cv.circle(self.img, (int(color_x), int(color_y)), int(color_radius), (255, 0, 255), 3)
                pos = (int(color_x), int(color_y))
        return self.img, pos

• Create widgets.

xxxxxxxxxx
button_layout = widgets.Layout(width='200px', height='100px', align_self='center')
output = widgets.Output()
color_position = widgets.Button(description='color_position', button_style='success', layout=button_layout)
choose_color = widgets.ToggleButtons(options=['red', 'green', 'blue', 'yellow'], button_style='success',
             tooltips=['Description of slow', 'Description of regular', 'Description of fast'])
position_cancel = widgets.Button(description='position_cancel', button_style='danger', layout=button_layout)
exit_button = widgets.Button(description='Exit', button_style='danger', layout=button_layout)
imgbox = widgets.Image(format='jpg', height=480, width=640, layout=widgets.Layout(align_self='auto'))
img_box = widgets.VBox([imgbox, choose_color], layout=widgets.Layout(align_self='auto'))
Slider_box = widgets.VBox([color_position,position_cancel,exit_button],
                          layout=widgets.Layout(align_self='auto'))
controls_box = widgets.HBox([img_box, Slider_box], layout=widgets.Layout(align_self='auto'))

• Main process.

def camera():
    global model
    capture = create_camera()
    while capture.isOpened():
        try:
            _, img = capture.read()
            fps.update_fps()
            if model == 'color_position':
                img, pos = position.process(img, color_hsv[choose_color.value])
                cv.putText(img, choose_color.value, (int(img.shape[0] / 2), 50), cv.FONT_HERSHEY_SIMPLEX, 2, color[random.randint(0, 254)], 2)
                if pos is not None:
                    print("x={}, y={}".format(pos[0], pos[1]))
            if model == 'Exit':
                cv.destroyAllWindows()
                capture.release()
                break
            fps.show_fps(img)
            imgbox.value = cv.imencode('.jpg', img)[1].tobytes()
        except:capture.release()