Catch Game

If using a depth camera (DABAI_DCW2), make sure to execute ros2 run dofbot_ultra_driver calibrate_6point for a six-point calibration before running this example. Otherwise, the arm cannot accurately locate the target.

For all functions involving grasping of square blocks, please use 30x30x30 mm wooden blocks. Non-standard sized objects may cause servo No. 6 stall and damage.

Please ensure that the web control program backend process is not running in the background. To close it, refer to the [Get Started Quickly - 2. Access the Terminal] section in [Disable the Auto-Start Web Control Backend Process]. After closing, please restart DofbotUltra!!!

The Catch Game function uses HSV color recognition to identify the color of the block. After placing the block on the cross, the arm will automatically recognize the color, grab the block, and place it in the corresponding color area.

Note: Before starting the program, please follow the [Assembly Tutorial] -> [Install the Map] tutorial to correctly install the map before operation.

1. Startup and Operation

Open a browser (any browser on any host in the same local network) and enter the IP address:8888 in the address bar to access JupyterLab.

Code path:

xxxxxxxxxx
dofbot_color_grab/scripts/Put_and_Grab.ipynb

Click the Run All button on the JupyterLab toolbar, then scroll to the bottom.

After the program runs, the arm will identify the color, then go to the center cross in the recognition area to grab the block and place it in the corresponding color position.

For example: Place a yellow block on the cross with the colored side facing up. When the camera identifies yellow, the arm automatically goes to the center cross to grab the block, places it in the yellow area, and returns to the initial pose.

Effect after grabbing and placing

Before performing the next recognition operation, please remove the block to avoid conflicts during placement.

HSV Calibration Note: Color calibration tool path (JupyterLab): dofbot_color_identify/scripts/HSV_calibration.ipynb

2. Technical Overview

The core workflow of this function:

1. Color Detection: The camera captures real-time images and filters through HSV color space thresholds to detect the block color in the cross area

2. Color Matching: Match the detected color with the preset four colors (red/green/blue/yellow)

3. Grab Execution: After determining the color, the arm moves to the cross position to grab

4. Place and Return: Place the block in the corresponding color area on the map, then return to the initial pose

HSV parameters can be adjusted through the HSV_config.txt file, supporting custom color thresholds.

3. Core Code Analysis

• Import header files

xxxxxxxxxx
import cv2 as cv
import threading
from time import sleep
from dofbot_utils.dofbot_config import *
import ipywidgets as widgets
from IPython.display import display
from put_grab import Put_Grab
from dofbot_utils.fps import FPS
from dofbot_utils.robot_controller import Robot_Controller

• Create instances and initialize parameters

xxxxxxxxxx
grab = Put_Grab()
model = 'General'
color_hsv  = {"red"   : ((0, 43, 46), (10, 255, 255)),
              "green" : ((35, 43, 46), (77, 255, 255)),
              "blue"  : ((100, 43, 46), (124, 255, 255)),
              "yellow": ((26, 43, 46), (34, 255, 255))}
HSV_path="/home/yahboom/dofbot_ultra_ws/src/dofbot_color_identify/scripts/HSV_config.txt"
try: read_HSV(HSV_path,color_hsv)
except Exception: print("Read HSV_config Error!!!")

• Create widgets

xxxxxxxxxx
button_layout  = widgets.Layout(width='200px', height='70px', align_self='center')
output = widgets.Output()
exit_button = widgets.Button(description='Exit', button_style='danger', layout=button_layout)
imgbox = widgets.Image(format='jpg', height=480, width=640, layout=widgets.Layout(align_self='center'))
controls_box = widgets.VBox([imgbox, exit_button], layout=widgets.Layout(align_self='center'))

• Mode switching

xxxxxxxxxx
def exit_button_Callback(value):
    global model
    model = 'Exit'
exit_button.on_click(exit_button_Callback)

• Main program

xxxxxxxxxx
def camera():
    from dofbot_utils.camera import create_camera
    capture = create_camera()
    while capture.isOpened():
        try:
            _, img = capture.read()
            fps.update_fps()
            img = grab.process(img, color_hsv)
            if model == 'Exit':
                capture.release()
                break
            fps.show_fps(img)
            imgbox.value = cv.imencode('.jpg', img)[1].tobytes()
        except KeyboardInterrupt:capture.release()

• Startup

display(controls_box,output)
threading.Thread(target=camera, ).start()