Welcome to DOFBOT-Ultra repository

Text Large Model + Robotic Arm Control

Please ensure that the web control program backend process is not running in the background. To close it, refer to the [Get Started Quickly - 2. Access the Terminal] section in [Disable the Auto-Start Web Control Backend Process]. After closing, please restart DofbotUltra!!!

Before running the feature, you need to close the web control program backend process.

1. Feature Description

After the program runs, enter a sequence of action instructions through the terminal. The large model will plan the action functions corresponding to the instructions and execute all instructions sequentially.

2. Startup

For the Raspberry Pi version, you need to enter the Docker container and run:

xxxxxxxxxx
ros2 launch largemodel largemodel_control.launch.py text_chat_mode:=True

Then open a second terminal and enter the following command:

xxxxxxxxxx
ros2 run text_chat text_chat

Then enter the instructions for controlling the robotic arm in the text_chat terminal. The following types of control instructions are currently supported:

• Single servo control: Set servo x to y degrees, x ranges from 1-6, y ranges from 0-180. For example, set servo 1 to 120 degrees.

• Single robotic arm action: Arm up/down/left/right, grip pose, track pose. For example, set the arm to grip pose, arm stretch upward.

• Robotic arm motion group: Arm dance/clap/nod/shake head. For example, please nod your head.

• Change arm end position: Arm adjust up/down/left/right/forward/backward x cm. For example, adjust the arm up 3 cm.

Combine robotic arm control instructions freely, and the large model will plan the action group corresponding to each instruction. Refer to the following example and enter it in the text_chat terminal:

x
First, adjust the robotic arm upward by 3 centimeters, wait 3 seconds, then extend the robotic arm downward, stay for 3 seconds, then have the robotic arm shake its head, pause for 3 seconds, and finally set servo 1 to 150 degrees.

3. Core Code Analysis

Let's mainly look at the adjust_joint function for setting servo angles. Other functions were analyzed in the first lesson [AI Large Model - Text Version\1. Semantic Understand Instruction Following] and will not be repeated here.

The source path of the adjust_joint function is:

Raspberry Pi:/home/yahboom/dofbot_ultra_ws/src/llm/largemodel/largemodel/action_service.py

RDK:/home/sunrise/dofbot_ultra_ws/src/llm/largemodel/largemodel/action_service.py

#Two parameters: joint_id is the servo number (1-6), angle is the angle value (0-180)
def adjust_joint(self,joint_id,angle):
    #Directly communicate with the top driver board, send the servo number and angle value
    Arm.Arm_serial_servo_write(int(joint_id), int(angle), 1000)
    time.sleep(1.0)
    if not self.combination_mode and not self.interrupt_flag:
        self.action_status_pub("adjust_joint_done")