Multimodal Large Model + Intent Prediction + Robotic Arm Gripping

Please ensure that the web control program backend process is not running in the background. To close it, refer to the [Get Started Quickly - 2. Access the Terminal] section in [Disable the Auto-Start Web Control Backend Process]. After closing, please restart DofbotUltra!!!

The web control program backend process must be closed before starting this function.

Function Description

After the program starts, input action commands like "I'm a little hungry / I'm a little thirsty" via the terminal. The large model will search for food or water based on whether you are hungry or thirsty, then grip the food or water. The width of the object should not exceed 4 cm. This section primarily uses the intent prediction scenario to introduce the use of the RAG knowledge base.

1.1 Access Dify through a browser, select [Knowledge Base] from the top toolbar --> [decision_layer_training_samples_cn_1211_update.x...]

Click on this xlsx file in the knowledge base.

You can see the pre-set scenarios.

Intent prediction has the following two action commands:

• I'm hungry

• I'm a little thirsty

We can first test it. In the Dify page, find [Studio] --> [Decision Layer] in the top toolbar.

In the [Debug and Preview] area on the right, enter "I'm a little hungry" and click send.

You can see here that the decision layer referenced our knowledge base file while thinking.