Welcome to DOFBOT-Ultra repository

Color Recognition and Grab Block

Please ensure that the web control program backend process is not running in the background. To close it, refer to the [Get Started Quickly - 2. Access the Terminal] section in [Disable the Auto-Start Web Control Backend Process]. After closing, please restart DofbotUltra!!!

The color recognition and grab block function mainly detects the HSV value of the color block in the image frame, distinguishes the color of the current block based on the HSV value, and then the arm rotates to the corresponding color position on the map to grab the block and place it at the center cross position.

For all functions involving grasping of square blocks, please use 30x30x30 mm wooden blocks. Non-standard sized objects may cause servo No. 6 stall and damage.

Note: Before starting the program, please follow the [Assembly Tutorial] -> [Install the Map] tutorial to correctly install the map before operation.

1. Startup and Operation

Open a browser (any browser on any host in the same local network) and enter the IP address:8888 in the address bar to access JupyterLab.

Code path:

xxxxxxxxxx
/dofbot_color_grab/scripts/Color_Grab.ipynb

Click the Run All button on the JupyterLab toolbar, then scroll to the bottom.

There will be a green frame in the center of the camera view. Place the color block to be grabbed into the frame, wait for recognition to complete, the buzzer will beep once, then place the block into the corresponding color area. The arm will automatically go to that color area to grab the block and place it on the center cross.

Before starting the next recognition and grab, please remove the block from the cross.

To end the program, click the [Exit] button to avoid affecting resource usage by other programs.

2. Technical Overview

The core workflow of this function:

1. Color Recognition: There is a frame area in the center of the camera view. When a block is placed in the frame, the HSV color space analysis identifies the block's color (red/green/blue/yellow)

2. Color Recording: After successful recognition, the buzzer beeps once, and the system records the map position corresponding to that color

3. Position Mapping: The 4 areas on the map correspond to 4 colors respectively. The system determines the grab position based on the recognized color

4. Arm Grasp: After placing the block in the corresponding color area, the arm rotates to that area and performs the grab action

5. Place and Return: Place the block at the center cross position, then return to the initial pose

3. Core Code Analysis

• Import header files

xxxxxxxxxx
import cv2 as cv
import threading
import ipywidgets as widgets
from IPython.display import display
from color_grab import Color_Grab
from dofbot_utils.fps import FPS
from dofbot_utils.robot_controller import Robot_Controller

• Create instances and initialize parameters

​x
grab = Color_Grab()
robot = Robot_Controller()
robot.move_look_front()

fps = FPS()
model = 'Start'

• Create widgets

xxxxxxxxxx
button_layout  = widgets.Layout(width='200px', height='70px', align_self='center')
output = widgets.Output()
exit_button = widgets.Button(description='Exit', button_style='danger', layout=button_layout)
imgbox = widgets.Image(format='jpg', height=480, width=640, layout=widgets.Layout(align_self='center'))
controls_box = widgets.VBox([imgbox, exit_button], layout=widgets.Layout(align_self='center'))

• Mode switching

xxxxxxxxxx
def exit_button_Callback(value):
    global model
    model = 'Exit'
exit_button.on_click(exit_button_Callback)

• Main program

xxxxxxxxxx
def camera():
    from dofbot_utils.camera import create_camera
    capture = create_camera()
    while capture.isOpened():
        try:
            if model == 'Exit':
                capture.release()
                break
            _, img = capture.read()
            fps.update_fps()
            img = grab.start_grab(img)
            fps.show_fps(img)
            imgbox.value = cv.imencode('.jpg', img)[1].tobytes()
        except KeyboardInterrupt:capture.release()
        except Exception as e:
            print(e)

• Startup

display(controls_box,output)
threading.Thread(target=camera, ).start()