Welcome to DOFBOT SE repository

Semantic Understand Instruction Following

Function Description

After the program runs, after waking up the voice module, speak a series of action commands. The large model will plan the corresponding action functions for the action commands and execute all instructions in sequence.

Startup

Open the terminal and enter the following command:

ros2 launch largemodel largemodel_control.launch.py

After waking up the module, give a voice command, reference the following example:

xxxxxxxxxx
Move the robotic arm up 3 centimeters, then move the robotic arm forward 3 centimeters, then turn on the red light, wait 3 seconds, then turn on the green light, and finally have the robotic arm perform a dance.

The large model will plan a series of action command functions, corresponding to the above action commands respectively: "arm_move('up',3)", "arm_move('forward',3)", "light_on('red')", 'wait(3)', "light_on('green')", 'arm_dance()'. The robotic arm end effector will first move forward 3 centimeters, then light up the red LED on the driver board, after waiting three seconds, light up the green LED on the driver board, and finally the robotic arm will execute the dance action.

Core Code Analysis

Please refer to the Core Code Analysis content in the tutorial 【AI Model-Text Version】-【Semantic Understand Instruction Following】. The voice version and text version have the same action functions; only the input method for task commands is different.