yahboom
HomeLearn
Solution
Blog
Online Store
About Us
EN/中文
DOFBOT-Ultra📖
1. Get Started Quickly
  • 0.Unboxing Video
  • 1. Get Started Quickly
  • 2. Access the Terminal
  • 3. Calibration offset(Depth Camera)
  • 4. Calibration offset(USB Camera)
2. Assembly Course
  • 1. Installation Steps
  • 2. Calibrate the Robotic Arm
  • 3. Install the Map
3. Remote Control Course
  • 1. Mobile Phone Control
  • 2. USB Handle Control
  • 3. PC Web Page Programming
4. Preparation
  • 1. Manage Web control services
  • 2. Manage JupyterLab services
  • 3. Manage OLED services
5. 2D Visual Tracking Course
  • 1. Color block positioning
  • 2. Color block tracking
  • 3. Face positioning
  • 4. Face tracking
  • 5. Label code positioning
  • 6. Label code tracking
6. 2D Visual Grasping Course
  • 1. Identify grab color block
  • 2. Catch game
  • 3. Color sorting
  • 4. Color stacking
  • 5. Label code sorting
  • 6. Label code stacking
  • 7. Gesture grasping color blocks
  • 8. Gesture stacking color blocks
  • 9. Single garbage sorting
  • 10. Garbage sorting
  • 11. Tracking game
7. 2D Mediapipe Course
  • 1. Hand detection
  • 2. Posture detection
  • 3. Overall detection
  • 4. Facial Detection
  • 5. Face Detection
  • 6. Face effects
  • 7. 3D Object Recognition
  • 8. Paintbrush
  • 9. Finger control
  • 10. Gesture Recognition
  • 11. Gesture control robotic arm action group
  • 12. Arm posture control robotic arm
  • 13. Palm Targeting
  • 14. Robotic arm tracks the palm
  • 15. Fingertip trajectory recognition
  • 16. Fingertip trajectory control robotic arm
  • 17. Gesture Grasp and Release Objects
  • 18. Finger Control Robot Arm
8. AI Large Model Voice Module (Must Read)
  • 1. Introduction to AI Large Model Voice Module
  • 2. Flash the firmware (Must read)
  • 3. Modify the wake-up word and command word
9. 2D Voice Control Course
  • 1. Simple Voice Control
  • 2. Smart Voice Control
  • 3. Voice Action Control
  • 4. Memory Learning Mode
  • 5. Color Recognition with Voice Broadcast
  • 6. Color Recognition and Grasping with Voice Broadcast
  • 7. Color Recognition and Sorting with Voice Broadcast
  • 8. Voice Recognition Target Tracking
  • 9. Voice Control Stacking
  • 10. Voice Control Sorting
  • 11. Garbage Recognition with Voice Broadcast
  • 12. Garbage Sorting with Voice Broadcast
10. 3D Depth Camera Course
  • 1. Dabai_DCW2 camera
  • 2. Deep pseudo-color image
  • 3. Depth camera ranging
  • 4. Color block volume measurement
11. 3D Robotic Arm Control and Inverse Kinematics
  • 1. Robotic arm ROS control
  • 2. Inverse solution
12. 3D Sorting and Grasping Course
  • 1. AprilTag ID sorting
  • 2. AprilTag abnormal height sorting
  • 3. MediaPipe Gesture-AprilTag ID sorting
  • 4. MediaPipe Gesture-AprilTag height sorting
  • 5. MediaPipe Gesture-AprilTag distance sorting
  • 6. Block color sorting
  • 7. Block abnormal height sorting
  • 8. Block different shape sorting
  • 9. Block shape abnormal height sorting
  • 10. Yolov26 garbage sorting
13. 3D Tracking and Grasping Course
  • 1. AprilTag Tracking and Grasping
  • 2. Color Block Tracking and Grasping
  • 3. KCF Object Tracking and Grasping
14. 3D Voice Control for Grasping Course (Sorting & Tracking)
  • 1. Voice Control AprilTag ID Sorting
  • 2. Voice Control Remove Height Abnormal AprilTag
  • 3. Voice Control Tracking and Grasping AprilTag
  • 4. Voice Control Color Block Sorting
  • 5. Voice Control Remove Height Abnormal Color Block
  • 6. Voice Control Tracking and Grasping Color Block
  • 7. Voice Control Yolov26 Garbage Recognition Sorting
  • 8. Voice Control Object Tracking and Grasping
15. AI Model Basics
  • 1. AI Large Model Types and Principles
  • 2. RAG Retrieval Enhancement and Model Training Samples
  • 3. Embodied Intelligent Robot System Architecture
  • 4. Embodied Intelligent Functions Core Source Code
16. AI Model Development
  • 0. DofbotUltra Large Model Development Quick Start Guide
  • 1. Register a Model Service Provider Account
  • 2. Configure API-KEY
  • 3. Introduction to Dify
  • 4. Basic Dify Features
  • 5. AI Large Model Development - chatbot
  • 6. Deploy the RAG knowledge base
  • 7. RAG knowledge base + chatbot
  • 8. AI agent workflow
17. AI Model – Text Version
  • 1. Semantic understand instruction following
  • 2. Multimodal visual understand
  • 3. Text Large Model + robotic arm control
  • 4. Multimodal Large Model + Video Understanding
  • 5. Multimodal Large Model + Robotic Arm Gripping
  • 6. Multimodal Large Model + Robotic Arm Tracking
  • 7. Multimodal Large Model + Robotic Arm Sorting
  • 8. Multimodal Large Model + Robotic Arm Handling
  • 9. Multimodal Large Model + Color Blocks Back In Place
  • 10. Multimodal Large Model + Intent Prediction + Robotic Arm Gripping
  • 11. Text Large Model + YOLOv26 + Intelligent Garbage Classification
18. AI Model – Voice Version
  • 1. Semantic understand instruction following
  • 2. Multimodal visual understand
  • 3. Text Large Model + robotic arm control
  • 4. Multimodal Large Model + Video Understanding
  • 5. Multimodal Large Model + Robotic Arm Gripping
  • 6. Multimodal Large Model + Robotic Arm Tracking
  • 7. Multimodal Large Model + Robotic Arm Sorting
  • 8. Multimodal Large Model + Robotic Arm Handling
  • 9. Multimodal Large Model + Color Blocks Back In Place
  • 10. Multimodal Large Model + Intent Prediction + Robotic Arm Gripping
  • 11. Text Large Model + YOLOv26 + Intelligent Garbage Classification
19. AI Model – Fun Applications
  • 1. AI Butler-Intent Inference
  • 2. AI Encyclopedia-Arithmetic Master
  • 3. AI Encyclopedia-You asked I Pointed
  • 4. AI Encyclopedia-Nutritional Analysis
  • 5. AI Encyclopedia-Problem Solving Expert
  • 6. AI Encyclopedia-English Spelling
  • 7. AI Butler-Garbage Sorting
  • 8. AI Top Chef
20. MoveIt Course
  • 1. MoveIt Configuration
  • 2. MoveIt Usage
  • 3. MoveIt Drive Real Robot
  • 4. MoveIt Random Movement
  • 5. MoveIt Forward Kinematics Design
  • 6. MoveIt Inverse Kinematics Design
  • 7. MoveIt Cartesian Path
  • 8. MoveIt Scene Design
  • 9. MoveIt Trajectory Planning
  • 10. MoveIt Collision Detection
21. YOLOv11 Development Course
PI 5
  • 1. YOLO Environment Setup
  • 2. CLI Usage
  • 3. Object Detection
  • 4. Instance Segmentation
  • 5. Pose Estimation
  • 6. Image Classification
  • 7. Oriented Bounding Boxes Object Detection
  • 8. Model Conversion
  • 9. Dataset Annotation
  • 10. Model Training and Conversion
  • 11. Model Prediction
RDK X5
  • 1. YOLO Environment Setup
  • 2. CLI Usage
  • 3. Object Detection
  • 4. Instance Segmentation
  • 5. Pose Estimation
  • 6. Image Classification
  • 7. Oriented Bounding Boxes Object Detection
  • 8. Model Prediction
  • 9. Model Conversion
22. YOLOv26 Development Course
PI 5
  • 6. Image Classification
  • 1. YOLO Environment Setup
  • 2. CLI Usage
  • 3. Object Detection
  • 4. Instance Segmentation
  • 5. Pose Estimation
  • 6. Image Classification
  • 7. Oriented Bounding Boxes Object Detection
  • 8. Model Conversion
  • 9. Dataset Annotation
  • 10. Model Training and Conversion
  • 11. Model Prediction
RDK X5
  • 1. YOLO Environment Setup
  • 2. CLI Usage
  • 3. Object Detection
  • 4. Instance Segmentation
  • 5. Pose Estimation
  • 7. Oriented Bounding Boxes Object Detection
  • 8. Model Conversion
  • 9. Dataset Annotation
  • 10. Model Training and Conversion
  • 11. Model Prediction
23.Linux Operating System
  • 1.Introduction to Linux system
  • 2.Ubuntu file system
  • 3.Ubuntu common commands
  • 4.Ubuntu common editors
  • 5.Ubuntu software operation commands
  • 6.Virtual machine installation
  • 7.SSH remote control
  • 8.VNC remote control
  • 9.Transfer files remotely
  • 10.Driver library and communication
  • 11.Capacity expansion and resource allocation
  • 12.Bind device ID
  • 13.Capacity expansion and resource allocation
  • 14.Update system software sources
  • 15.Set root user password
  • 16.sudo free password
  • 17.Connect to WiFi network
  • 18.View system version
  • 19.Customized service management
  • 20.Back up system image
24. Basic Control Course
  • 1. Control RGB Light
  • 2. Control Buzzer
  • 3. Control Single Servo
  • 4. Read Servo Angle
  • 5. Control All Servos
  • 6. DOFBOT Up Down Left Right Movement
  • 7. DOFBOT Dancing
  • 8. DOFBOT Memory Actions
  • 9. DOFBOT Grab Blocks
  • 10. Porter
  • 11. Stack Blocks
  • 12. Two DOFBOT Synchronized Actions
25.RDKx5 Board Basic Course
  • 1.Introduction to RDK X5 board
  • 2.Write system file
  • 3.Hardware interface description
  • 4.Install case and other accessories
  • 5.Use 7-inch display
  • 6.SD card expansion
  • 7.Network configuration
  • 8.Remote login
  • 9.Update system
  • 10.srpi-config user guide
  • 11.config. txt configuration file
  • 12.Self-startup setting
  • 13.RDK exclusive command usage
  • 14.Linux command usage
  • 15.GPIO readwrite operation
  • 16.Image classification algorithm
  • 17.Use USB camera
  • 18.Use MIPI camera
  • 19.Binocular camera depth algorithm
26.PI5 Board Basic Course
  • 1.Introduction to Raspberry PI 5
  • 2.Raspberry PI system installation and backup
  • 3.Powering the Raspberry PI 5
  • 4.Startup of Raspberry PI 5
  • 5.Installation and introduction of shell and active radiator
  • 6.GPIO and 40-pin pins
  • 7.Tips for using Raspberry PI 5
  • 8.Raspberry PI PCIe connector
  • 9.MIPI CSI DSI connector
  • 10.UART connector
  • 11.Real-time clock
  • 12.Power over Ethernet (PoE) connector
  • 13.Update and upgrade operating system
  • 14.Introduction to raspi-config tool
  • 15.config.txt file description
  • 16.Network Configuration
  • 17.remote access
  • 18.Transfer files remotely
  • 19.Set display resolution and rotation
  • 20.Set screen to sleep
  • 21.Get real-time temperature of Raspberry Pi
  • 22.Play audio and video
  • 23.Using USB camera
  • 24.Using MIPI camera
27. Visual Basics Course
  • 1. Introduction to Open Source CV
  • 2. Image Reading and Display
  • 3. Image Writing
  • 4. Image Quality
  • 5. Pixel Operation
  • 6. Image Zoom
  • 7. Picture Cut
  • 8. Picture Pan
  • 9. Picture Mirroring
  • 10. Affine Transformation
  • 11. Picture Rotation
  • 12. Perspective Transformation
  • 13. Grayscale Processing
  • 14. Binary Image
  • 15. Edge Detection
  • 16. Line Segment Drawing
  • 17. Rectangular Circle Drawing
  • 18. Text and Picture Drawing
28. ROS2 Basic Course
  • 1.Introduction to ROS2
  • 2.ROS2 install Humble
  • 3.ROS2 development environment
  • 4.ROS2 workspace
  • 5.ROS2 function package
  • 6.ROS2 node
  • 7.ROS2 topic communication
  • 8.ROS2 service communication
  • 9.ROS2 action communication
  • 10.ROS2 custom interface message
  • 11.ROS2 parameter service case
  • 12.ROS2 meta-function package
  • 13.ROS2 distributed communication
  • 14.ROS2 DDS
  • 15.ROS2 time related API
  • 16.ROS2 common command tools
  • 17.ROS2 rviz2 use
  • 18.ROS2 rqt toolbox
  • 19.ROS2 Launch startup file configuration
  • 20.ROS2 recording and playback tool
  • 21.ROS2 URDF model
  • 22.ROS2 Gazebo simulation platform
  • 23.ROS2 TF2 coordinate transformation
29. Jupyter Lab Usage Course
  • 1. Read the document before operation
  • 2. Jupyter lab environment construction tutorial
  • 3. Basic tutorial of Jupyter lab
  • 4. Jupyter widgets configuration tutorial
  • 5. Getting Started Tutorial for JupyterWidgets Controls
Download
  • System_File
  • VM_File
  • Instruction Manual
  • Code
  • Software Tools
  • Communication_Protocols
  • Hardware_Info
Download ZIP

Welcome to DOFBOT-Ultra repository

2. Configure API-KEY

AI Large Model - Dify

Please ensure that the web control program backend process is not running in the background. To close it, refer to the [Get Started Quickly - 2. Access the Terminal] section in [Disable the Auto-Start Web Control Backend Process]. After closing, please restart DofbotUltra!!!

Before starting this feature, you need to close the web control program backend process.

1. Introduction to Dify

Dify is an open-source LLM application development platform that integrates BaaS and LLMOps concepts. It supports low-code/no-code rapid construction of production-grade generative AI applications, is compatible with hundreds of models, and has built-in enterprise-grade RAG and visual workflow. It supports private deployment and full-stack LLMOps management and control.

Dify (Define + Modify) targets developers and enterprises, using an integrated "Backend as a Service + BaaS + LLMOps" solution to lower the barrier from prototype to production for generative AI applications while ensuring data security and compliance.

One of Dify's product forms, the "AI Application Factory", enables "low-code interface for quickly creating chatbots and content generators". We configure the decision layer and execution layer to plan task steps and select execution action functions, achieving dual-model inference. Additionally, for more complex functions, we add knowledge bases so that the decision layer can reference the knowledge base when planning actions, ensuring correct steps.

2. Entering the Robotic Arm Dify Configuration Interface

The Dify backend runs as a Docker service on the DofbotUltra motherboard's host system (not inside a docker container). It must be started before using the robotic arm.

2.1 Start the Dify Backend

Enter the following command directly in any directory on the host system to start Dify (no need to cd to any path, and no need to add sh):

bringup_dify

Press Enter and wait for the containers to start. As shown in the figure below, when all services display green, Dify has started successfully.

image-20251219102631582

2.2 Verify Dify Backend Accessibility

Before using the large model functionality, please ensure the Dify backend is accessible. You can confirm it using either of the following methods:

  • Browser method: On a computer in the same LAN as DofbotUltra, enter the DofbotUltra IP address in the browser address bar (e.g., http://192.168.x.x). If the Dify login page opens normally, the backend is ready.

  • Command line method: On the host system of DofbotUltra (not inside a docker container), execute:

    xxxxxxxxxx
    docker ps

    In the output list, you should see several Dify-related containers such as langgenius/dify-* in the Up state. If not, Dify has not started or failed to start. Please re-execute bringup_dify.

2.3 Log in to Dify

Enter the robotic arm's IP address in the browser to access the Dify configuration interface.

The first startup will be very slow. Please wait patiently for 5-10 minutes.

image-20251219104054317

The first login requires an email and password.

Email: yahboom@163.com

Password: yahboom123

After clicking login, you can enter the configuration interface.

image-20260519193003379

3. Studio

When using the chat assistant in the studio, let's take a look at the domestic decision layer - [decision_layer] as an example.

image-20260519193139274

Item 1 contains the prompt. In the prompt, we give the large model an identity, tell it what it can do, output restrictions, basic requirements, special situations, etc.

Item 2 is the model we selected. Item 3 is the input box for sending chat content.

4. Configure Model Provider API Key

  • This section configures the model provider API Key in the Dify page Settings -> Model Provider.

  • This type of Key is generally provided by the model service provider, commonly in the format sk-.... Its purpose is to allow the Dify platform to call Tongyi Qianwen, OpenRouter, and other model services internally.

  • It is not the same as the decision_AI_api_key and execution_AI_api_key in the large_model_interface.yaml file later. Do not mix them.

As shown in the figure below, click the Y in the upper right corner to enter account settings.

image-20251219153106627

After entering, click [Model Provider].

image-20251219153235541

Click [Configure] on the right. In the Dify we provided, only Tongyi Qianwen has been added. To add other model providers, refer to the later tutorial.

image-20251219153350969

Click [Add API Key].

Then fill in the Tongyi Qianwen API-Key registered in the previous chapter. It is a key starting with sk-. For the [Use International Endpoint] option below, domestic users select No, and international users select Yes. After entering, click [Save].

image-20251219153732753

5. Import Other Provider Models (Provided for Users Needing Other Large Models)

The advantage of the Dify platform is its ability to seamlessly integrate with most global AI models: through Dify's model provider plugins, you can quickly and seamlessly connect to hundreds of global AI models, eliminating the hassle of deployment and setup. Here we use OpenRouter as an example.

Find the OpenRouter model and click Install.

image-20251226144933884

After installation is complete, refresh Dify, fill in the key, and save.

image-20251226145945244

Switch the OpenRouter model in the decision layer/execution layer (if not recharged, check the free model usage in OpenRouter). Finally, publish it. The execution layer must use a visual model with the VL name.

image-20251226150251712

6. International Users Configure Local Function Package

6.1 Check Dify Decision Layer/Execution Layer Application API Key

The decision_AI_api_key and execution_AI_api_key in large_model_interface.yaml are the API Keys used by the robotic arm program to access the Dify decision layer/execution layer applications, not IDs.

The current repository example enables the international version configuration by default. If you previously switched to the domestic version, you need to re-comment out the domestic version lines and enable the international version lines.

Raspberry Pi (inside Docker container) path: /home/yahboom/dofbot_ultra_ws/src/llm/largemodel/config

RDK (native environment) path: /home/sunrise/dofbot_ultra_ws/src/llm/largemodel/config

Open the file large_model_interface.yaml.

Switch decision_AI_api_key and execution_AI_api_key:

  • International users: Keep the international version lines enabled.

  • If you previously switched to the domestic version: Re-comment out the domestic version lines and enable the international version lines.

image-20251225151128312

6.2 Voice Parameter Configuration

File path:

Raspberry Pi: /home/yahboom/dofbot_ultra/llm/largemodel/config

RDK: /home/sunrise/dofbot_ultra_ws/src/llm/largemodel/config

Open the file: yahboom.yaml

Change the parameters for whether to use online speech recognition and speech synthesis to false.

Comment out the domestic version parameter options, and uncomment the international version parameters.

image-20251225151602404

6.3 Compile the Function Package

Raspberry Pi: Enter the following command to compile the function package to apply the changes.

xxxxxxxxxx
cd /home/yahboom/dofbot_ultra_ws
colcon build --packages-select largemodel
source install/setup.bash

RDK: Enter the following command to compile the function package to apply the changes.

xxxxxxxxxx
cd /home/sunrise/dofbot_ultra_ws
colcon build --packages-select largemodel
source install/setup.bash

 

Register a Model Service Provider Account
Introduction to Dify

COMPANY

  • About us
  • Team Style
  • Blog

LEARNING & EXPLORATION

  • Tutorials
  • Edge AI hardware
  • Educational robots
  • Embedded intelligent hardware

WHERE TO BUY

  • Yahboom Online Store
  • Amazon
  • Alibaba
  • AliExpress
  • Mercado

TECHNOLOGY SUPPORT

  • Email: support@yahboom.com

BE A DISTRIBUTOR

  • Email: sales@yahboom.com
QR Code
QR Code

Copyright © 2017-2026 Yahboom. All Rights Reserved.

EN/中文