Intention estimation

1. Course Content

Basic: Master the ability to customize unique user intent understanding functionality using the RAG knowledge base.

Advanced: Master the debugging methods for intent understanding effects on the DIfy platform.

[!IMPORTANT]

• Intent understanding is designed to increase the synergy between the robot and the human, allowing the robot to understand the user more uniquely. This function should not be used to perform "strange" and "illogical" tasks.

2. Start the Agent

Note: If it's already started, you don't need to start it again.

Enter the command on the vehicle terminal:

xxxxxxxxxx
start_agent

The terminal will print the following information, indicating a successful connection:

2.1 Configure the Intent Mapping File

• This file is used to store your vague intents and the corresponding task operations you want the robot to perform.
• Open the example file in this tutorial folder. Following the reference format, you can add multiple customized intents. Here's a simple example:

2.2 Configuring the Knowledge Base

• Next, we need to upload our edited intent mapping file to the RAG knowledge base in Dify.

[!TIP]

• For detailed instructions on operating the RAG knowledge base, please refer to the tutorial in [04.AI Model Development]—[06.Deploy the RAG knowledge base].

• Dify comes with a default Intent mapping reference knowledge base to demonstrate how to use intent understanding features.

• You can modify the Intent mapping.xlsx template, or you can delete it and add your own file.

[!TIP] Note:

• To ensure optimal intent understanding performance, it is recommended to set the Intent mapping intent understanding knowledge base to high-quality mode, as intent understanding often requires retrieving relevant fragments between similar semantic cues.

3. Running Examples

3.1 Starting the Program

• On the vehicle's infotainment system, open the terminal and enter the command to start the AI agent function, or use the shortcut multi_brains:

xxxxxxxxxx
ros2 launch m3_bringup llm_agent_control.launch.py

• Start the navigation function on the vehicle's infotainment system:

(ORIN & PI5 mainboard)

xxxxxxxxxx
ros2 launch m3_bringup nav2.launch.py

(RDK mainboard) Run step-by-step, visualize the startup on the accompanying virtual machine Rosmaster-M3-Ubuntu22.04:

xxxxxxxxxx
#Virtual machine side
rviz2 -d /home/yahboom/yahboomM3_ws/user_ws/src/m3_bringup/rviz/nav2.rviz
#Host side
ros2 launch m3_bringup nav2.launch.py gui:=False

Afterwards, follow the navigation startup process to initialize positioning. This will open the rviz2 visualization interface. Click on 2D Pose Estimate in the top toolbar to enter selection mode. Roughly mark the robot's location and orientation on the map. After initializing positioning, the preparation is complete.

3.2 Test Cases

The test cases here are for reference only. Users can create their own dialogue commands.

• I'm currently in the master bedroom, and I'm feeling a bit thirsty.

The steps for planning the decision-level model are as follows:

Following the steps for planning the large decision-level model, execute the following sequentially:

4. Effect Debugging

4.1 Visual Workflow

• Open the corresponding language version of the multi_brains application, then click preview. You can input test content to view the data flow.

• You can also open the workflow to view the time taken for each step and the input and output content of each process.

• If the task routing module here is inaccurate in certain contexts and is not categorized into Category 3: Personal Intent, you can add restrictions on personal intent or supplement the context description here.

• Additionally, the recall effect of the intent understanding statement to be tested in the knowledge base of the intent mapping can also be tested separately.