Welcome to DOFBOT-Ultra repository

Garbage Sorting With Voice Broadcast

Please ensure that the web control program backend process is not running in the background. To close it, refer to the [Get Started Quickly - 2. Access the Terminal] section in [Disable the Auto-Start Web Control Backend Process]. After closing, please restart DofbotUltra!!!

This case requires placing building blocks within the camera's recognizable range, using 30x30x30mm building blocks, otherwise the arm cannot accurately grasp them.

1. Operation Instructions

1.1. Startup

Before running the function, you need to shut down the web control program backend process. For the shutdown method, please refer to the [4. Preparation - 1. Manage Web control services].

Raspberry Pi version (within Docker), after starting the ROS2 core node, run the following commands:

Step 1 - Start the ROS2 core node:

ros2 run dofbot_ultra_info kinematics_dofbotultra

Step 2 - Start the garbage sorting broadcast program:

xxxxxxxxxx
python3 /home/yahboom/dofbot_ultra/2d/dofbot_voice/scripts/garbage_sorting_broadcast.py

RDK X5 version (native environment), enter the following command:

xxxxxxxxxx
python3 /home/sunrise/dofbot_ultra_ws/src/2d/dofbot_voice/scripts/garbage_sorting_broadcast.py

1.2. Operation Description

lace the garbage label code block in the center of the image, then say "Hello, yahboom" to the voice module. The voice module will reply "here" to indicate successful wake-up. Then say "What garbage is this?" to the voice module. After the program recognizes it, the voice module will reply and broadcast what this garbage is and its type, and the robotic arm will grab this block and place it in the designated position according to its garbage type. Taking the figure below as an example, according to the recognition result, it will reply "This is a peach pit, which belongs to dry garbage". Then the robotic arm will lower its gripper to grab it and place it in the "dry garbage" area marked on the map.

2. Code Logic

This case combines garbage recognition and sorting, using YOLOv11 for garbage detection and pre-defined position points for sorting distribution. The voice module handles interaction throughout the process, implementing a complete "recognition -> broadcast -> sorting" closed loop.

There are 4 garbage categories, corresponding to 4 placement areas:

• Recyclable - Placement area A (P_RECYCLABLE_WASTE)
• Other Waste - Placement area B (P_OTHER_WASTE)
• Kitchen Waste - Placement area C (P_KITCHEN_WASTE)
• Hazardous Waste - Placement area D (P_HAZARDOUS_WASTE)

Workflow: Recognize -> determine category -> voice broadcast -> arm grasping -> rotate base -> place in corresponding area -> return to initial position. The Robot_Controller class encapsulates position data for all garbage sorting actions.

3. Source Code

Raspberry Pi: /home/yahboom/dofbot_ultra/2d/dofbot_voice/scripts/garbage_sorting_broadcast.py

RDK X5: /home/sunrise/dofbot_ultra_ws/src/2d/dofbot_voice/scripts/garbage_sorting_broadcast.py

Recognition module:

Raspberry Pi: /home/yahboom/dofbot_ultra_ws/src/dofbot_garbage_yolov11/speech_garbage_identify.py

RDK X5: /home/sunrise/dofbot_ultra_ws/src/2d/dofbot_garbage_yolov11/speech_garbage_identify.py

​x
from speech_garbage_identify import speech_garbage_identify
​
self.frame, msg = self.garbage_identify.garbage_run(self.frame)
​
# Voice triggers identification, start sorting
threading.Thread(target=self.single_garbage_grap, args=(self.garbage_class,)).start()
​
def single_garbage_grap(self, name):
    self.arm.Arm_Buzzer_On(1)
    sleep(0.5)
    if name == "04":
        joints_down = self.robot.P_HAZARDOUS_WASTE
        self.move(joints_down)
        self.status = 'waiting'
    if name == "01":
        joints_down = self.robot.P_RECYCLABLE_WASTE
        self.move(joints_down)
        self.status = 'waiting'
    if name == "03":
        joints_down = self.robot.P_KITCHEN_WASTE
        self.move(joints_down)
        self.status = 'waiting'
    if name == "02":
        joints_down = self.robot.P_OTHER_WASTE
        self.move(joints_down)
        self.status = 'waiting'
​
def move(self, joints_down):
    joints_uu = self.robot.P_TOP
    self.arm.Arm_serial_servo_write6_array(joints_uu, 1000)
    sleep(1)
    self.arm.Arm_serial_servo_write(6, self.release_joint, 500)
    sleep(0.5)
    joints_center = self.robot.P_CENTER
    joints_center[5] = self.release_joint
    self.arm.Arm_serial_servo_write6_array(joints_center, 1000)
    sleep(1.5)
    self.arm.Arm_serial_servo_write(6, self.grap_joint, 500)
    sleep(0.5)
    self.arm.Arm_serial_servo_write6_array(joints_uu, 1000)
    sleep(1)
    self.arm.Arm_serial_servo_write(1, joints_down[0], 1000)
    sleep(1)
    self.arm.Arm_serial_servo_write6_array(joints_down, 1000)
    sleep(1.5)
    self.arm.Arm_serial_servo_write(6, self.release_joint, 500)
    sleep(0.5)
    self.arm.Arm_serial_servo_write(2, 90, 1000)
    sleep(1)
    self.arm.Arm_serial_servo_write6_array(self.robot.P_LOOK_MAP, 1000)
    sleep(1.5)

• single_garbage_grap() - Select the corresponding placement area based on garbage category, call move() for the action
• move() - Full motion sequence: top position -> open gripper -> center -> grasp -> top -> rotate -> place -> release -> look at map
• Robot_Controller pre-defined constants (P_HAZARDOUS_WASTE, etc.) are 6-element joint angle arrays