Welcome to DOFBOT SE repository

AI Encyclopedia-Nutritional Analysis

1. Function Description

After the program starts, ask the large model which fruit on the desktop has the highest vitamin content. After thinking, the large model will reply with the answer to the question, and the program will control the robotic arm to point to that fruit.

2. Startup

After entering the Docker container, enter the following commands in the Docker terminal.

ros2 launch largemodel largemodel_control.launch.py text_chat_mode:=True

Then open a second terminal and input the following command:

xxxxxxxxxx
ros2 run text_chat text_chat

Then input "Which of the fruits on the table has the highest vitamin content?" in the text_chat terminal and press Enter; if it's the voice version, wake up the voice module and directly say to the voice module "Which of the fruits on the table has the highest vitamin content?", then wait for the large model to think and reply, as shown in the figure below:

Then the robotic arm will point to the banana:

3. Task Planning

1. Call seewhat() to observe the environment;

2. Reply with which fruit on the table has the highest vitamin content;

3. Call the seewhat() function to check the position of the fruit with the highest vitamin content;

4. Call the point_to(x1, y1, x2, y2) function to point to the fruit with the highest vitamin content.