Multimodal Large Model + Intent Prediction + Robotic Arm Gripping

Feature Description

After the program runs, enter action instructions like "I'm a bit hungry / I'm a bit thirsty" through the terminal. The large model will look for food or water based on hunger/thirst, then grip the food or water. The width of the object should not exceed 4 cm. This section mainly introduces the use of the RAG knowledge base through the intent prediction scenario.

1.1 Access Dify through a browser, select [Knowledge Base] --> [training_samples_for_decision_1211.x...] from the top toolbar

Click on this xlsx file in the knowledge base

You can see the preset scenarios we have configured.

Intent prediction has the following two action instructions:

• I'm hungry
• I'm a bit thirsty

We can first test it. In the Dify page, find [Studio] --> [decision_layer] from the top toolbar.

In the [Debug & Preview] area on the right, enter "I'm a bit hungry" and click send.

You can see that the decision layer referenced our knowledge base file while thinking.