Welcome to DOFBOT-Pi repository

AI Encyclopedia-English Spelling

Before running the function, you need to close the App and large programs. For the closing method, refer to [4. Preparation] - [1. Manage APP control services].

1. Function Description

After the program starts, ask the large model what the word on the desktop might mean. After thinking, the large model will reply with the answer to the question and will check the spelling of the word, and control the robotic arm to point to the missing word.

2. Startup

After entering the Docker container, enter the following commands in the Docker terminal.

ros2 launch largemodel largemodel_control.launch.py text_chat_mode:=True

Then open a second terminal and input the following command,

xxxxxxxxxx
ros2 run text_chat text_chat

Then input "Guess what the word on the desktop might mean?" in the text_chat terminal and press Enter; if it's the voice version, wake up the voice module and directly say to the voice module "Guess what the word on the desktop might mean?", then wait for the large model to think and reply, as shown in the figure below,

Then the robotic arm will point to the letter c,

3. Task Planning

1. Call seewhat() to observe the environment and check the word(s) on the desktop;
2. Reply with what the word(s) on the desktop might be;
3. Call seewhat() to observe the environment and check whether the word(s) on the desktop are spelled completely;

• If incomplete, execute in accordance with the following steps, and output one function at a time:

(1) Call seewhat() to observe the environment and obtain the positions of the letters that can complete the spelling of the word(s);

(2) Call the point_to(x1, y1, x2, y2) function to point to the letters that can complete the spelling of the word(s) (where x1, y1, x2, y2 are the relevant coordinate parameters of the letters used to complete the spelling of the word(s)).

• If complete, call the finishtask() function.