Welcome to DOFBOT-Ultra repository

Voice Action Control

If using a depth camera (DABAI_DCW2), before running this example, be sure to execute ros2 run dofbot_ultra_driver calibrate_6point for a six-point calibration, otherwise the robot arm cannot accurately locate the target.

Please ensure that the web control program backend process is not running in the background. To close it, refer to the [Get Started Quickly - 2. Access the Terminal] section in [Disable the Auto-Start Web Control Backend Process]. After closing, please restart DofbotUltra!!!

This case requires placing building blocks within the camera's recognizable range, using 30x30x30mm building blocks, otherwise the arm cannot accurately grasp them.

1. Operation Instructions

1.1. Startup

Before running the function, you need to shut down the web control program backend process. For the shutdown method, please refer to the [4. Preparation - 1. Manage Web control services].

Raspberry Pi version (in Docker environment), enter the following command:

python3 /home/yahboom/dofbot_ultra/2d/dofbot_voice/scripts/action_voice_ctrl.py

RDK X5 version (native environment), enter the following command:

xxxxxxxxxx
python3 /home/sunrise/dofbot_ultra_ws/src/2d/dofbot_voice/scripts/action_voice_ctrl.py

1.2. Operation Description

1.2.1 Robotic Arm Dancing

After the program runs, the voice module will broadcast "here". Then say "Hello, yahboom" to the voice module. The voice module will reply "here" to indicate successful wake-up. Then say "Dancing" to the voice module. The robotic arm will rotate its joints, moving to set positions each time to complete the "Dance" action.

1.2.2 Block Stacking

Pick up the blocks from the different sides in the order of yellow, red, green, and blue, and stack them into the central cross-shaped area. The arrangement of the blocks is shown in the diagram below:

After the program runs, the voice module will broadcast "here". Then say "Hello, yahboom" to the voice module. The voice module will reply "here" to indicate successful wake-up. Then say "Stack the block" to the voice module. The robotic arm will sequentially grab colored blocks for stacking. After executing the code, the robotic arm will stack the building blocks, with the final effect shown in the figure below:

1.2.3 Block Grabbing

After the program runs, the voice module will broadcast "here". Then say "Hello, yahboom" to the voice module. The voice module will reply "here" to indicate successful wake-up. Then say "Clip the block" to the voice module. Place blocks in the center cross area (the order of placing colored blocks is yellow, red, green, blue). The robotic arm will move blocks from the center cross area to different colored block areas around. After grabbing and placing the current block, you can place the next block in the center cross area.

1.2.4 Transport

Stack four different colored building blocks from bottom to top in the order of blue, green, red, and yellow on the center cross block. Then say "Hello, yahboom" to the voice module. The voice module will reply "here" to indicate successful wake-up. Then say "Carry the block" to the voice module. The robotic arm will execute in the following order: grab the fourth layer block and place it in the yellow area, grab the third layer block and place it in the red area, grab the second layer block and place it in the green area, grab the bottom layer block and place it in the blue area. The arrangement of building blocks is shown in the figure below:

After executing the code, the robotic arm will transport the building blocks to corresponding positions, with the final effect shown in the figure below:

2. Code Logic

This case uses the encapsulated Robot_Controller class for multi-action group control. Different colored blocks correspond to different position parameters, and different action modes call corresponding action functions.

By pre-defining position variables (such as P_YELLOW, P_RED, P_CENTER, etc.), combined with action functions (heap_up, dance, clamp_clock, move_block), the control of multiple complex actions is achieved.

3. Source Code

Raspberry Pi: /home/yahboom/dofbot_ultra/2d/dofbot_voice/scripts/action_voice_ctrl.py

RDK X5: /home/sunrise/dofbot_ultra_ws/src/2d/dofbot_voice/scripts/action_voice_ctrl.py

​x
from dofbot_utils.robot_controller import Robot_Controller
robot = Robot_Controller()

p_Yellow = robot.P_YELLOW
p_Red = robot.P_RED
p_Green = robot.P_GREEN
p_Blue = robot.P_BLUE

def arm_clamp_block(enable):
    if enable == 0:
        Arm.Arm_serial_servo_write(6, 60, 400)
    else:
        Arm.Arm_serial_servo_write(6, 135, 400)
    time.sleep(.5)

def arm_move(p, s_time=500):
    for i in range(5):
        id = i + 1
        if id == 5:
            time.sleep(.1)
            Arm.Arm_serial_servo_write(id, p[i], int(s_time*1.2))
        elif id == 1:
            Arm.Arm_serial_servo_write(id, p[i], int(3*s_time/4))
        else:
            Arm.Arm_serial_servo_write(id, p[i], int(s_time))
        time.sleep(.01)
    time.sleep(s_time/1000)

• Robot_Controller - Robot control class, provides pre-defined position constants and action group functions

• arm_clamp_block(enable) - Gripper control, 0 = open (60 degrees), 1 = close (135 degrees)

• arm_move(p, s_time) - Move servos 1-5 to specified positions, servo 1 moves faster (3/4 time), servo 5 moves slower (1.2x time), non-blocking

• Action group functions (e.g., heap_up()) encapsulate complete motion sequences like "grasp -> lift -> move -> place -> return"