Welcome to DOFBOT SE repository

Garbage Sorting

Since the garbage model training and simple garbage recognition have been completed previously, now we proceed with garbage recognition and robotic arm sorting.

ros2 run dofbot_info dofbot_server  # Start server node

Single Garbage Sorting Gameplay

Path address:

/home/yahboom/dofbot_ws/src/dofbot_garbage_yolov11/单个垃圾分拣玩法single_Garbage_sorting_play.ipynb

• Import header files

xxxxxxxxxx
#!/usr/bin/env python
# coding: utf-8
import Arm_Lib
import cv2 as cv
import threading
from time import sleep
import ipywidgets as widgets
from IPython.display import display
from single_garbage_identify import single_garbage_identify

• Create instance, initialize parameters

xxxxxxxxxx
# Create target acquisition instance
single_garbage = single_garbage_identify()
# Initialize mode
model = "General"

• Initialize robotic arm position

xxxxxxxxxx
import Arm_Lib
arm = Arm_Lib.Arm_Device()
joints_0 = [90, 135, 0, 0, 90, 30]
arm.Arm_serial_servo_write6_array(joints_0, 1000)

• Create controls

xxxxxxxxxx
button_layout = widgets.Layout(width='320px', height='60px', align_self='center')
output = widgets.Output()
# Exit
exit_button = widgets.Button(description='Exit', button_style="danger", layout=button_layout)
imgbox = widgets.Image(format='jpg', height=480, width=640, layout=widgets.Layout(align_self='center'))
controls_box = widgets.VBox([imgbox, exit_button], layout=widgets.Layout(align_self='center'))

• Exit program button callback

xxxxxxxxxx
def   exit_button_Callback(value):
    global model
    model = 'Exit'
    with output: print(model)
exit_button.on_click(exit_button_Callback)

• Main program

xxxxxxxxxx
def camera():
    # Open camera
    capture = cv.VideoCapture(0)
    # Loop while camera is opened normally
    while capture.isOpened():
        try:
            # Read each frame of the camera
            _, img = capture.read()
            # Uniform image size
            img = cv.resize(img, (640, 480))
            img =   single_garbage.single_garbage_run(img)
            if model == 'Exit':
                cv.destroyAllWindows()
                capture.release()
                break
            imgbox.value = cv.imencode('.jpg', img)[1].tobytes()
        except KeyboardInterrupt:capture.release()

• Start

xxxxxxxxxx
display(controls_box,output)
threading.Thread(target=camera, ).start()

When starting the above code block, it takes about 2-3 seconds to wait for the model to load. Place the block with garbage attached facing the robotic arm at the center of the cross on the frame. After the robotic arm accurately recognizes 10 times continuously, it automatically sorts.

Random Garbage Sorting Gameplay

Note: Before playing garbage sorting, start a command line window as the inverse solution server, running continuously. If this node has already been started, there is no need to repeat the startup!

xxxxxxxxxx
ros2 run dofbot_info dofbot_server # Start server node

Its operation process and sorting principle are similar to color sorting.

• First calibrate the frame. For specific calibration methods, see the [Visual Positioning] chapter introduction.
• After calibration is complete, click the [target_detection] button to load the model for recognition. This process takes some time, please wait patiently.
• When a block is detected, click the [grap] button to perform sorting and grabbing.
• When the game ends, click the [Exit] button to exit the program.

For detailed code, see /home/yahboom/dofbot_ws/src/dofbot_garbage_yolov11/Garbage_Sorting.ipynb