Welcome to DOFBOT-Ultra repository

Color Recognition And Grasping With Voice Broadcast

If using a depth camera (DABAI_DCW2), before running this example, be sure to execute ros2 run dofbot_ultra_driver calibrate_6point for a six-point calibration, otherwise the robot arm cannot accurately locate the target.

Please ensure that the web control program backend process is not running in the background. To close it, refer to the [Get Started Quickly/ 3. Enter Terminal] section in [Close the web control program backend process started at boot]. After closing, please restart DofbotUltra!!!

This case requires placing building blocks within the camera's recognizable range, using 30x30x30mm building blocks, otherwise the arm cannot accurately grasp them.

1. Operation Instructions

1.1. Startup

Before running the function, you need to shut down the web control program backend process. For the shutdown method, please refer to the [4. Preparation - 1. Manage Web control services].

Raspberry Pi version (in Docker environment), enter the following command:

python3 /home/yahboom/dofbot_ultra/2d/dofbot_voice/scripts/color_grab_broadcast.py

RDK X5 version (native environment), enter the following command:

xxxxxxxxxx
python3 /home/sunrise/dofbot_ultra_ws/src/2d/dofbot_voice/scripts/color_grab_broadcast.py

1.2. Operation Description

After the program runs successfully, hold a color block and place the green square in the image on the color block, as shown in the figure below:

The program will recognize the block color, the voice module will broadcast the recognized block color, and after the buzzer beeps once, it will grab a block of the same color and place it in the center area. Finally, the voice module will broadcast "Placement complete".

2. Code Logic

This program adds robotic arm grasping actions on top of the color recognition and broadcasting function. After the program recognizes the block color, it broadcasts the color name via voice, then controls the robotic arm to move to the position of the corresponding colored block to grasp it, and finally places it in the center area.

The program uses Robot_Controller to get the predefined pose coordinates for each color area, implementing automated operations through the complete flow of "recognize → broadcast → buzzer → grasp → place → broadcast completion". The multi-threaded architecture ensures that color recognition and robotic arm control do not block each other.

3. Source Code

Raspberry Pi: /home/yahboom/dofbot_ultra/2d/dofbot_voice/scripts/color_grab_broadcast.py

RDK X5: /home/sunrise/dofbot_ultra_ws/src/2d/dofbot_voice/scripts/color_grab_broadcast.py

​x
frame, color_name = get_color(frame)
if len(color_name) == 1:
    if color_name['name'] == 'yellow':
        start_move_arm(1)
    elif color_name['name'] == 'red':
        start_move_arm(2)
    elif color_name['name'] == 'green':
        start_move_arm(3)
    elif color_name['name'] == 'blue':
        start_move_arm(4)

def number_action(index):
    if index == 1:
        arm_move(p_top, 1000)
        arm_move(p_Yellow, 1500)
    elif index == 2:
        arm_move(p_top, 1000)
        arm_move(p_Red, 1500)
    elif index == 3:
        arm_move(p_top, 1000)
        arm_move(p_Green, 1500)
    elif index == 4:
        arm_move(p_top, 1000)
        arm_move(p_Blue, 1500)
    arm_clamp_block(1)
    arm_move(p_top, 1500)

def ctrl_arm_move(index):
    arm_clamp_block(0)
    if index == 1:
        mySpeech.void_write(66)
        time.sleep(0.1)
        Arm.Arm_Buzzer_On(1)
        time.sleep(.5)
        number_action(index)
        put_down_block()
        mySpeech.void_write(65)

• get_color() - Detect the color of blocks within the camera's field of view, returning the color name

• start_move_arm(index) - Voice-triggered arm motion, internally calling the action function

• number_action(index) - Move to the corresponding color block position and grasp

• ctrl_arm_move(index) - Full control flow: broadcast -> buzzer -> action -> place -> broadcast completion