8. Intent Inference + Multimodal Visual Understanding + SLAM Navigation + Visual Functionality

Introduction

In this tutorial, we will learn how to issue commands via chat with Muto in the Dify interface.

Operation Steps

Preparatory Steps

This lesson requires that the Dify terminal be launched beforehand, and all configuration items have been configured as in the previous section.

This lesson requires MutoRS to be powered on. Following the steps in the previous chapters, start the large model terminal in Docker.

In this tutorial, we will use a plain text version of the large model, starting it with the --no-voice parameter. Using the --no-voice parameter eliminates the risk of being interrupted by external voice prompts during use!

cd muto-llm-2.0/ && ./rebuild_and_launch.sh --no-voice

Getting Started

The large text model requires you to enter Muto's IP address here, which is the address used to access the Dify page.

In this section, based on SLAM navigation, we will add pre-defined Intent Mapping to allow Muto to execute behaviors and actions that match your intentions. In this tutorial, we will learn how to add/modify prompts for large models in Dify.

The intent mapping file we use in this section is as follows:

Intent Mapping

• Instruction 1: "I'm currently in place1, and I'm a little thirsty."
• Intent Analysis 1: Find out which area on the map has bottled water and go to that area to find it.
• Example: 1. Navigate to area A, 2. Check if bottled water is available, 3. If bottled water is found, measure the distance between you and the user, 4. Navigate to the user's current location (place1) and announce the result.
• Instruction 2: "I want to sleep."
• Intent Analysis 2: 1. Navigate to the master bedroom and check if the environment is suitable for sleeping. 2. Tell the user a bedtime story to soothe their emotions.

Item Storage Locations

Master Bedroom

• Toilet paper, pen, phone

Kitchen

• Bottled water, Coke, Sprite, bread

Zoo

• Monkeys, tigers, peacocks

You can add intent mapping content later based on the actual scenario.

Open the Dify interface, find the decision layer node, and click the ENV button in the upper right corner.

Note: This step is already configured in the factory image. You can directly edit the Examples environment variable to modify the intended meaning.

Click the "Add Variable" button.

Add a variable named Examples, of type String. Enter the content from the Markdown file into the value.

After filling it in, click the Save button in the lower right corner.

Next, return to the System code block in the "Decision Layer Prompts" node.

Add the following content at the end: "# Decision Layer Reference"

Then type the forward slash "/" on your keyboard. Select the Example variables we just saved.

The result is shown in the image below.

After adding, click the "Publish" button in the upper right corner, then click "Publish Update".

After publishing, we can try out the intent we just added.

We will focus on observing the output results of the decision-making layer.

We can see the output results of the decision-making layer. In the result, the large model has already planned according to the prompts we added.

Here we mainly teach you how to add your own custom prompts, so we haven't demonstrated the complete effect.

If we want to achieve the full functionality, in addition to modifying the prompts, we also need to mark the actual locations of places like "kitchen" and "master bedroom" in advance, and start our navigation system in the background.