Welcome to DOFBOT-Ultra repository

Color Recognition With Voice Broadcast

If using a depth camera (DABAI_DCW2), before running this example, be sure to execute ros2 run dofbot_ultra_driver calibrate_6point for a six-point calibration, otherwise the robot arm cannot accurately locate the target.

Please ensure that the web control program backend process is not running in the background. To close it, refer to the [Get Started Quickly - 2. Access the Terminal] section in [Disable the Auto-Start Web Control Backend Process]. After closing, please restart DofbotUltra!!!

1. Operation Instructions

1.1. Startup

Before running the function, you need to shut down the web control program backend process. For the shutdown method, please refer to the [4. Preparation - 1. Manage Web control services].

Raspberry Pi version (in Docker environment), enter the following command:

python3 /home/yahboom/dofbot_ultra/2d/dofbot_voice/scripts/voice_color_identify.py

RDK X5 version (native environment), enter the following command:

xxxxxxxxxx
python3 /home/sunrise/dofbot_ultra_ws/src/2d/dofbot_voice/scripts/voice_color_identify.py

1.2. Operation Description

After the program runs successfully, the program will call the camera to capture the current image. Hold a color block and place the green square in the image on the color block, as shown in the figure below:

Then the terminal will print "get color!", then say "Hello, yahboom" to the voice module. The voice module will reply "here" to indicate successful wake-up. Then say "What color is it?" to the voice module. The voice module will broadcast the current color of the block. If the current recognized block is blue, the voice module will broadcast "This is blue". Note: If colors cannot be recognized, you may need to modify the HSV values in the program or adjust the ambient lighting.

2. Code Logic

This case integrates color recognition and voice broadcast, following a "voice trigger -> color recognition -> voice broadcast" workflow.

Colored block HSV positioning: Detect the area within the green frame (280-360, 180-260), extract the H (Hue) channel average value, and determine the color by the H value range (red/yellow/green/blue). The HSV color space is used instead of RGB for better lighting adaptability.

The program adopts a multi-threaded architecture: one thread continuously captures and processes camera images, while the main thread polls for voice commands. Upon receiving the voice query "What color is it?", the system broadcasts the current recognition result via voice audio output.

3. Source Code

Raspberry Pi: /home/yahboom/dofbot_ultra/2d/dofbot_voice/scripts/voice_color_identify.py

RDK X5: /home/sunrise/dofbot_ultra_ws/src/2d/dofbot_voice/scripts/voice_color_identify.py

​x
def get_color(img):
    H = []
    color_name = {}
    img = cv2.resize(img, (640, 480))
    HSV = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
    cv2.rectangle(img, (280, 180), (360, 260), (0, 255, 0), 2)
    for i in range(280, 360):
        for j in range(180, 260):
            H.append(HSV[j, i][0])
    H_min = min(H)
    H_max = max(H)
    if H_min >= 0 and H_max <= 10 or H_min >= 156 and H_max <= 180:
        color_name['name'] = 'red'
    elif H_min >= 21 and H_max <= 28:
        color_name['name'] = 'yellow'
    elif H_min >= 35 and H_max <= 78:
        color_name['name'] = 'green'
    elif H_min >= 100 and H_max <= 124:
        color_name['name'] = 'blue'
    return img, color_name
​
result = mySpeech.speech_read()
if result == 60:
    if color_name['name'] == 'yellow':
        mySpeech.void_write(64)
    elif color_name['name'] == 'red':
        mySpeech.void_write(61)
    elif color_name['name'] == 'green':
        mySpeech.void_write(63)
    elif color_name['name'] == 'blue':
        mySpeech.void_write(62)

• get_color() - Extract the H channel average from the specified area and determine the color based on H value range
• cvtColor(img, COLOR_BGR2HSV) - Convert BGR image to HSV color space
• ID 60 corresponds to "identify color" command, IDs 61-64 correspond to red/blue/green/yellow voice broadcasts