Welcome to DOFBOT-Ultra repository

Voice Control Stacking

If using a depth camera (DABAI_DCW2), before running this example, be sure to execute ros2 run dofbot_ultra_driver calibrate_6point for a six-point calibration, otherwise the robot arm cannot accurately locate the target.

Please ensure that the web control program backend process is not running in the background. To close it, refer to the [Get Started Quickly - 2. Access the Terminal] section in [Disable the Auto-Start Web Control Backend Process]. After closing, please restart DofbotUltra!!!

This case requires placing building blocks within the camera's recognizable range, using 30x30x30mm building blocks, otherwise the arm cannot accurately grasp them.

1. Operation Instructions

1.1. Startup

Before running the function, you need to shut down the web control program backend process. For the shutdown method, please refer to the [4. Preparation - 1. Manage Web control services].

Raspberry Pi version (in Docker environment), enter the following command:

python3 /home/yahboom/dofbot_ultra/2d/dofbot_voice/scripts/voice_control_stack.py

RDK X5 version (native environment), enter the following command:

xxxxxxxxxx
python3 /home/sunrise/dofbot_ultra_ws/src/2d/dofbot_voice/scripts/voice_control_stack.py

1.2. Operation Description

After the program runs successfully, place blocks according to the following arrangement: yellow blocks in the yellow area, red blocks in the red area, green blocks in the green area, blue blocks in the blue area, as shown in the figure below:

1.2.1 Stacking

Say "Hello, yahboom" to the voice module. The voice module will reply "here" to indicate successful wake-up. Then say "Stack the yellow block/Stack the red block/Stack the green block/Stack the blue block" to the voice module. The voice module will reply and broadcast "ok", and the robotic arm will go to the corresponding color block area, grab the block, and place it in the designated area. The voice module will reply and broadcast "ok". Each time you use voice commands to grab the target color block and place it, you will eventually complete the stacking.

1.2.2 Push Down

Say "Hello, yahboom" to the voice module. The voice module will reply "here" to indicate successful wake-up. Then say "Push down" to the voice module. The voice module will reply and broadcast "ok", then the robotic arm will execute the action to push down the stacked blocks.

2. Code Logic

This program uses voice commands to control a robotic arm to perform layer-by-layer stacking and push-down operations on colored blocks.

Unlike automatic sorting, the stacking function requires picking up blocks of different colors one by one in a user-specified order and stacking them layer by layer. The program maintains a grabbing status flag for each color (e.g., yellow_grabbed) to ensure that each color is grabbed only once. The number of layers placed automatically increases with each grab (layer 1 on the first grab, layer 2 on the second grab, and so on).

The "push-down" function resets all grabbing statuses, allowing a new round of stacking to begin.

3. Source Code

Raspberry Pi: /home/yahboom/dofbot_ultra/2d/dofbot_voice/scripts/voice_control_stack.py

RDK X5: /home/sunrise/dofbot_ultra_ws/src/2d/dofbot_voice/scripts/voice_control_stack.py

​x
while True:
    res = mySpeech.speech_read()
    time.sleep(0.01)
    if res == 77:
        if yellow_grabbed == 0:
            mySpeech.void_write(77)
            Arm.Arm_Buzzer_On(1)
            start_move_arm(1)
            yellow_grabbed = 1
    elif res == 78:
        if red_grabbed == 0:
            mySpeech.void_write(77)
            Arm.Arm_Buzzer_On(1)
            start_move_arm(2)
            red_grabbed = 1
    elif res == 82:
        mySpeech.void_write(82)
        Arm.Arm_Buzzer_On(1)
        start_move_arm(5)
        yellow_grabbed = 0
        red_grabbed = 0
        green_grabbed = 0
        blue_grabbed = 0

def number_action(index):
    if index == 1:
        arm_move(p_top, 1000)
        arm_move(p_Yellow, 1000)
        arm_clamp_block(1)
        Arm.Arm_serial_servo_write(2, 90, 1000)
        arm_move(p_top, 1000)

def put_down_block(layer):
    if layer == 1:
        arm_move(p_layer_1, 1000)
        arm_clamp_block(0)
        arm_move_6(look_at, 1000)
    mySpeech.void_write(81)

• Voice IDs: 77 = stack yellow, 78 = stack red, 79 = stack green, 80 = stack blue, 82 = reset

• *_grabbed flags track whether each color block has been stacked

• number_action(index) - Move arm to the corresponding color block position and grasp

• put_down_block(layer) - Place the block at the specified layer position (1-4 layers)