Welcome to MicroROS V2 Robot Car repository

AI Large Model Preparation

1. Select the matching car model

Open a terminal and enter:

xxxxxxxxxx
gedit .bashrc

Find this line:

Fill in your current car model in the car_type field:

​x
ptz: with pan-tilt servo + camera

no_ptz: no servo + with fixed camera

basic_car: no servo + no camera

After modifying, click save and exit, then run the following command to apply the changes:

xxxxxxxxxx
source ~/.bashrc

2. Radar mapping

Refer to the content in the product tutorial 【Lidar Course】-【9. slam_toolx Mapping】 to start the car for mapping and save the map. This map will be used for navigation in AI large model applications later.

3. Color calibration

Start the color calibration program to calibrate and save the HSV values of the four colors red, green, blue and yellow in the parameter file. In the AI large model color line-following case later, llm_agent_node will load this parameter file via FOLLOW_LINE.hsv_file. When you input natural language commands such as "start following the red line" or "start following the yellow line" through the terminal text client, Dify will convert the commands into actions such as follow_line(red) and follow_line(yellow).

Source code path:

xxxxxxxxxx
~/microros_ws/src/multi_brains/multi_brains/utils/colorSelect.py

Parameter save file path:

xxxxxxxxxx
~/microros_ws/src/multi_brains/config/color_hsv.txt

Later, if the large model needs color-related data, it will read the HSV values of colors from this path.

Start the camera:

xxxxxxxxxx
# Start proxy + camera
ros2 launch microros_v2_bringup car_base.launch.py

Start the color calibration program:

xxxxxxxxxx
ros2 run multi_brains color_selector

After successful startup, the currently calibrating color will be displayed in the upper-left corner:

Use the mouse to select a region (this region can only contain one color):

After releasing the mouse, the binarized image will be displayed immediately:

After confirming that the color is properly calibrated, press the Space key, and the red HSV value will be written to the first line of the parameter file:

~/microros_ws/src/multi_brains/config/color_hsv.txt

Then move on to calibrating the next color. Continue calibrating the remaining colors in the same way, and the HSV values will be written to the second, third, and fourth lines of the parameter file:

When the last color calibration is completed, "all colors saved!" will be displayed in the upper-left corner:

Finally, you can press the Q/q key to exit the program.

4. Configure large model related API keys

All large model features require configuring the corresponding large model API keys. This chapter contains a lot of content and will not be elaborated here. Refer to the tutorial 【AI Large Model Development】-【2. Configure API-KEY】.

5. Start the text interaction client

Prerequisite: configure the large model related API keys

Currently, the multi_brains package uses terminal text interaction instead of speech recognition and voice wake-up. The core node is responsible for connecting to Dify, parsing actions, and controlling the robot, while the terminal client is only responsible for inputting text and displaying the robot's replies.

Start the agent core node:

xxxxxxxxxx
ros2 launch multi_brains llm_agent_control.launch.py

Open another terminal and start the text interaction client:

xxxxxxxxxx
ros2 run multi_brains terminal_client

After startup, you can directly input natural language commands, for example:

Follow the red line
Look at what's ahead
Move forward 2 meters then turn left 90 degrees