Welcome to DOFBOT Pro repository

Semantic Understand Instruction Following（Voice Version）

Before running the function, you need to close the App and large programs. For the closing method, refer to [4. Preparation] - [1. Manage APP control services].

1. Function Description

After the program runs, wake up the voice module and say a sequence of action commands. The large model will plan the action functions corresponding to the action commands and execute all commands in sequence.

2. Startup

Users with Jetson-Nano mainboard version need to enter the docker container first and then input the following command. Users with Orin mainboard can directly open the terminal and input the following command:

ros2 launch largemodel largemodel_control.launch.py

After waking up the module, give voice commands. You can refer to the following example:

xxxxxxxxxx
Move the robotic arm up by 3 cm, then move the robotic arm forward by 3 cm, then turn on the red light, wait for 3 seconds, then turn on the green light, and finally have the robotic arm dance to finish.

The large model will plan a series of action command functions. Corresponding to the above action commands are: "arm_move('up',3)", "arm_move('forward',3)", "light_on('red')", 'wait(3)', "light_on('green')", 'arm_dance()'. The robotic arm end will first move forward by 3 cm, then turn on the drive board's LED red light, wait three seconds, then turn on the drive board's LED green light, and finally the robotic arm will execute dance movements.

3. Core Code Analysis

You can refer to the content in 3. Core Code Analysis from tutorial [17. AI Model - Text Version] - [1. Semantic Understanding and Instruction Following]. The voice version and text version have the same action functions, only the task command input method is different.