AI Model with Road Network Planning and Autonomous Driving

1. Course Content

• This course is a comprehensive advanced case study that builds on [13.Road Network Planning with Traffic Sign Recognition] to combine AI large models for more intelligent navigation. It can also combine multiple scenario cases to let AI take over autonomous driving navigation and complete complex cases in the track map.

[!CAUTION]

PI5 mainboard does not support this case!!

2. Preparation Work

2.1 Start kilted_agent Agent

• Use the following command to start the microROS agent and connect to the chassis

[!WARNING]

If you use the start_agent command to start the microROS agent, you need to close the previous agent first. Note that this is different from the previous microROS agent startup command

start_kilted_agent

• Start ros2kilted container (if already started, no need to repeat)

xxxxxxxxxx
bringup_ros2kilted

• Start Dify, all functions related to AI large models need to start Dify (if already started, no need to repeat)

xxxxxxxxxx
bringup_dify

[!WARNING]

RDK mainboard requires additional execution

Open ~/.bashrc, uncomment this line, and change it to the communication method as shown in the figure (after learning the road network planning function, please restore this comment to avoid affecting previous cases)

xxxxxxxxxx
vim ~/.bashrc

xxxxxxxxxx
:wq #Save and exit
source ~/.bashrc #Update environment

After environment update, the terminal display changes from green to red

2.2 Prerequisite Knowledge

For a better experience, please first learn and master the following content in the product tutorial:

• AI Large Model: [3.AI Model Basics], [4. AI Model Development], [5.AI Model-Voice Version]
• Road Network Planning Navigation: [7.Road Network Planning and Navigation]
• Autonomous Driving: [01.Environment Setup & Precautions], [07.Vehicle Centerline Calibration], [08.Lane Detection], [09.Lane-Keeping Autonomous Driving], [10.Traffic Sign Recognition], [14.Road Network Planning with Traffic Sign Recognition]

3. Run Case Program

[!IMPORTANT]

• Need to build the track map's grid map: map, pose map: pose_map, and road network description file: roadnet_file in advance according to [08.Autonomous Driving expand(Need purchase track map)]—[13.Road Network Planning with Track Map]

3.1 Start Road Network Navigation

• Open a terminal on the host machine to start the vehicle's radar fusion node and odometer fusion

xxxxxxxxxx
ros2 launch m3_bringup car_base.launch.py

• Open ros2kilted container terminal

xxxxxxxxxx
exec_ros2kilted

• (RDK mainboard) Visualization is started on the accompanying virtual machine Rosmaster-RoadNet-Ubuntu24.04,

xxxxxxxxxx
rviz2 -d /home/yahboom/code_ws/src/road_net_route/rviz/nav2_view.rviz 

• Start road network planning navigation node in container terminal

xxxxxxxxxx
ros2 launch road_net_route roadnet_nav2.launch.py 

[!NOTE]

Parameter Description

map Grid map file name to be loaded, default value is map.yaml

pose_map Pose map file to be loaded. When navigation global positioning uses slam_toolbox pure positioning, the pose map must be passed in, default value is map

roadnet_file Road network description file, default value is map.geojson

3.2 Start Autonomous Driving

• Run in host machine terminal

xxxxxxxxxx
ros2 launch m3_bringup auto_drive.launch.py conbine_nav:=True unkeeplane:=True start_camera:=False

[!NOTE]

Parameter Description

• conbine_nav Whether to combine road network navigation for autonomous driving, default is False. If True, the motion control of autonomous driving is provided by road network planning navigation, and traffic light response will directly replace the interruption and recovery acting on road network planning navigation
• unkeeplane Whether to close lane line detection model and thread, default is False. If road network navigation is started for autonomous driving, lane line detection and lane keeping functions will be automatically closed
• start_camera Whether to start camera node. If combined with AI large model, the camera node is started by the multi_brains launch file, not by this launch file

3.3 Start multi_brains Agent

Open a terminal on the vehicle and enter the command to start, or you can use the shortcut command multi_brains:

xxxxxxxxxx
ros2 launch m3_bringup llm_agent_control.launch.py enable_route_nav:=True

3.4 Demonstration Case

• Navigate to the construction site while obeying traffic rules

After waking up the vehicle with "Hello, yahboom", when the vehicle responds and the recording prompt sound "dang—" appears, give the command to the vehicle

• The vehicle will then follow the road network planned route to the target point "Construction Site"

• In AI large model combined with road network autonomous driving, you can also let AI not respond to traffic signals. You can add conditions to the command: ignore traffic signals, at which time AI will temporarily close the response to traffic signs.