Welcome to DOFBOT SE repository

AI Encyclopedia-You asked I Pointed

1. Function Description

After the program starts, ask the large model which word on the desktop can represent the object on the desktop. After thinking, the large model will reply with the English word that can represent the object, and the program will control the robotic arm to point to that word.

2. Startup

After entering the Docker container, enter the following commands in the Docker terminal.

ros2 launch largemodel largemodel_control.launch.py text_chat_mode:=True

Then open a second terminal and input the following command:

xxxxxxxxxx
ros2 run text_chat text_chat

Then input "Which English word represents the object in front of you?" in the text_chat terminal and press Enter; if it's the voice version, wake up the voice module and directly say to the voice module "Which English word represents the object in front of you?", then wait for the large model to think and reply, as shown in the figure below:

Then the robotic arm will point to the word Apple:

3. Task Planning

1. Call seewhat() to observe the environment;

2. Reply with what objects are in front of the robot;

3. Call seewhat() to check if there are English words representing the objects;

4. Call seewhat() to obtain the positions of the English words for the objects;

5. Call the point_to(x1, y1, x2, y2) function to point to the English words.