yahboom
HomeTutorialsProduct
Solution
Blog
Online Store
About Us
EN/中文
Store
ROSMASTER M1 PRO📖
0.Robot Configuration and Operation Guidelines
  • 0.Unboxing and Review
  • 1.Assembly Videos
  • 2.User Manual
  • 3.Enter Docker (For Raspberry Pi)
  • 4.FAQ
  • 5.RobotManager User Guide
1.Chassis Control Course
  • 1.Robot Information Publishing
  • 2.Robot Handle Control
  • 3.Robot Keyboard Control
  • 4.Linear Velocity Calibration
  • 5.Angular Velocity Calibration
  • 6.Robot URDF Model
  • 7.Robot EKF State Estimation
2.Lidar Course
  • 1.Lidar Introduction And Use
  • 2.Lidar Obstacle Avoidance
  • 3.Lidar Following
  • 4.Lidar Guard
  • 5.Lidar Patrol
  • 6.GMapping-SLAM Mapping
  • 7.Cartographer-SLAM Mapping
  • 8.Slam-Toolbox Mapping
  • 9.Lifelong Mode Mapping
  • 10.Navigation2 Single Point Navigation
  • 11.Navigation2 Multi Point Navigation
  • 12.Cartographer Relocalization Navigation
  • 13.Slam-Toolbox Localization Navigation
  • 14.Synchronous Mapping And Navigation
  • 15.Aruco Visual Relocalization Navigation
  • 16.Mobile App Mapping And Navigation
3.USB Camera-Depth Camera Course
  • 1.USB Camera Introduction
  • 2.USB Camera Intrinsic Calibration
  • 3.Nuwa-HP60C Camera Introduction (Depth Camera Only)
  • 4.Depth Pseudo Color Image (Depth Camera Only)
  • 5.Depth Camera Distance Measurement (Depth Camera Only)
  • 6.Block Volume Calculation (Depth Camera Only)
  • 7.Edge Detection (Depth Camera Only)
  • 8.RTAB-Map SLAM Mapping (Depth Camera Only)
  • 9.RTAB-Map Navigation (Depth Camera Only)
4.Robotic Arm and 3D Space Gripping Course
  • 1.Robotic Arm Basic Usage
  • 2.Robotic Arm Kinematics
  • 3.Robotic Arm Chassis Coordination Control
  • 4.AprilTag ID Sorting
  • 5.AprilTag Height Anomaly Sorting
  • 6.AprilTag Tracking and Grasping
  • 7.Color Block Sorting
  • 8.Color Block Height Anomaly Sorting
  • 9.Color Block Tracking and Grasping
  • 10.Wood Block Shape Sorting
  • 11.KCF Tracking and Grasping Objects
  • 12.Mediapipe Gesture ID Sorting AprilTag
  • 13.Mediapipe Gesture Height Sorting AprilTag
  • 14.Desktop Tracking and Grasping AprilTag
  • 15.3D Spatial Tracking AprilTag
  • 16.Transformer Model Tracking Algorithm
  • 17.Line Following Obstacle Clearing
5.Road network planning and navigation
  • 1.Introduction To Road Network Planning
  • 2.Road Network Annotation
  • 3.Planning And Navigation Between Waypoints
  • 4.Road Network Navigation In Any Pose
  • 5.Path Re-planning
6.OpenClaw Basic Course
  • 1.Introduction to OpenClaw
  • 2.OpenClaw Core Concepts
  • 3.OpenClaw API-KEY Configuration
  • 4.OpenClaw Switch Models
  • 5.OpenClaw System Prompt Words
  • 6.OpenClaw Common CLI Commands
7.OpenClaw Interaction Method
  • 1.OpenClaw Webchat Interaction
  • 2.OpenClaw Tui Interaction
  • 3.OpenClaw Voice Dialogue Interaction
  • 4.OpenClaw WhatsApp Interaction
8.OpenClaw Development
  • 1.OpenClaw Connecting to Robot MCP Service
  • 2.Robot CLI Command Tool
  • 3.robot-control Plugin Tool
  • 4.reply-bridge Plugin Tool
  • 5.OpenClaw Multimodal Vision
  • 6.OpenClaw Chassis Control
  • 7.OpenClaw Robotic Arm Control
  • 8.OpenClaw Skills Development
  • 9.OpenClaw Scheduled Tasks
  • 10.OpenClaw Multi-Agent Intelligent Agent
  • 11.OpenClaw Active Interaction
  • 12.OpenClaw Vector Memory
9.OpenClaw Embodied intelligent application
  • 1.OpenClaw Target Tracking
  • 2.OpenClaw Arm Tracking And Grasping
  • 3.OpenClaw Arm Fixed-point Placement
  • 4.OpenClaw Waste Sorting
  • 5.OpenClaw SLAM Mapping And Navigation
  • 6.OpenClaw Navigation Transport
  • 7.OpenClaw AprilTag Grasping
  • 8.OpenClaw Color Line Patrol
  • 9.OpenClaw Visual Distance Measurement
10.OpenClaw scenario sandbox
  • 1.Map Description and Props Placement
  • 2.Building the Sandbox Map
  • 3.Visual Relocalization Marker
  • 4.Map Mapping Marker
  • 5.Sandbox Scenario Comprehensive Case
11.AI Large Model Basics
  • 1.AI Large Model Types and Principles
  • 2.RAG Retrieval Enhancement and Model Training Samples
  • 3.Embodied Intelligent Robot System Architecture
  • 4.Embodied Intelligent Functions Core Source Code
12.AI large Model Development
  • 1.Register a Model Service Provider Account
  • 2.Configuring API-KEY
  • 3.Introduction to Dify
  • 4.Basic Dify Features
  • 5.AI Large Model Development - Chatbot
  • 6.Deploy the RAG Knowledge Base
  • 7.RAG Knowledge Base + Chatbot
  • 8.AI Agent Workflow
  • 9.Custom Wake-up Response
13.AI Model - Voice Version
  • 1.Semantic Understanding and Command Following
  • 2.Multimodal Visual Understanding
  • 3.Multimodal Visual Understanding and Robotic Arm Gripping
  • 4.Multimodal Visual Understanding and SLAM Navigation
  • 5.Robotic arm gripping+Multimodal visual understand+SLAM navigation
  • 6.Intention Understanding
14.AI Model - Text Version
  • 1.Semantic Understanding and Command Following
  • 2.Multimodal Visual Understanding
  • 3.Multimodal Visual Understanding and Robotic Arm Gripping
  • 4.Multimodal Visual Understanding and SLAM Navigation
  • 5.Robotic arm gripping+Multimodal visual understand+SLAM navigation
  • 6.Intention estimation
15.Mediapipe Visual Course
  • 1.Hand detection
  • 2.Posture detection
  • 3.Overall detection
  • 4.Facial Landmark Detection
  • 5.Face detection
  • 6.Face special effects
  • 7.3D object recognition
  • 8.Brush
  • 9.Finger control
  • 10.Palm target positioning
  • 11.Fingertip trajectory recognition
  • 12.Fingertip gesture control robotic arm
  • 13.Gesture grabbing and releasing objects
  • 14.Finger control robotic arm
  • 15.Medipipe gesture control robotic arm action group
16.MoveIt2 Simulation Course
  • 1.MovelT2 configuration
  • 2.MovelT2 simulation-reality linkage
  • 3.Random movement
  • 4.Forward kinematics design
  • 5.Inverse kinematics design
  • 6.Cartesian path
  • 7.Trajectory Planning
  • 8.Collision Detection
  • 9.Scene Design
  • 10.3D perception obstacle‑avoidance
17.Multi-vehicle Course
  • 1.Multi-vehicle Control
  • 2.Multi-vehicle Navigation
  • 3.Multi-vehicle Formation
18.YOLO26 Development
  • 1.CLI Use
  • 2.Object Detection
  • 3.Instance Segmentation
  • 4.Pose Estimation
  • 5.Image Classification
  • 6.Oriented Object Detection
  • 7.YOLOv26 + DeepSORT Tracking
  • 8.Monocular Depth Estimation
19.Control Board STM32 Driver Course
  • 0.Notes (Must Read)
  • 1.About Expansion Board
  • 2.STM32 Development Environment
  • 3.Button Control
  • 4.FreeRTOS Application
  • 5.Serial Communication
  • 6.SBUS Model Aircraft Remote Control
  • 7.CAN Bus Communication
  • 8.RGB Colorful Light Bar
  • 9.Timer Interrupt to Control PWM Servo
  • 10.Control Serial Port Servos
  • 11.Nine-axis Attitude Sensor to Obtain Data
  • 12.Control Motor
  • 13.Timer Captures the Encoder Data
  • 14.1 Kinematic Analysis of Mecanum Wheel
  • 14.2 Kinematic Analysis of Ackerman
  • 14.3 Kinematic Analysis of 4WD
  • 15.PID Control Robot Movement
20.Docker Course
  • 1.Docker Introduction and Installation
  • 2.Docker Basic Commands
  • 3.Docker Submission and Multi-Terminal Access
  • 4. Docker Startup Script
  • 5.Docker-Vscode Environment Construction
21.Image Processing Basics Course
  • 1. Introduction to Open Source CV
  • 2. Image reading and display
  • 3. Image writing
  • 4. Image quality
  • 5. Pixel Operation
  • 6. Image zoom
  • 7. Image cutting
  • 8. Image translation
  • 9. Image Mirroring
  • 10. Affine Transformation
  • 11. Image Rotation
  • 12. Perspective Transformation
  • 13. Grayscale processing
  • 14. Image Binarization
  • 15. Edge Detection
  • 16. Line drawing
  • 17. Rectangle and circle drawing
  • 18. Drawing text and pictures
22.Linux System Course
  • 1.Introduction to Linux system
  • 2.Ubuntu file system
  • 3.Ubuntu common commands
  • 4.Ubuntu common editors
  • 5.Ubuntu software operation commands
  • 6.Virtual machine installation
  • 7.SSH remote control
  • 8.VNC remote control
  • 9.Transfer files remotely
  • 10.Driver library and communication
  • 11.Static IP and hotspot mode
  • 12.Bind device ID
  • 13.Capacity expansion and resource allocation
  • 14.Update system software sources
  • 15.Set root user password
  • 16.sudo free password
  • 17.Connect to WiFi network
  • 18.View system version
  • 19.Customized service management
  • 20.Back up system image
23.ROS2 Basic Course
  • 1.Introduction to ROS2
  • 2.ROS2 install Humble
  • 3.ROS2 development environment
  • 4.ROS2 workspace
  • 5.ROS2 function package
  • 6.ROS2 node
  • 7.ROS2 topic communication
  • 8.ROS2 service communication
  • 9.ROS2 action communication
  • 10.ROS2 custom interface message
  • 11.ROS2 parameter service case
  • 12.ROS2 meta-function package
  • 13.ROS2 distributed communication
  • 14.ROS2 DDS
  • 15.ROS2 time related API
  • 16.ROS2 common command tools
  • 17.ROS2 rviz2 use
  • 18.ROS2 rqt toolbox
  • 19.ROS2 Launch startup file configuration
  • 20.ROS2 recording and playback tool
  • 21.ROS2 URDF model
  • 22.ROS2 Gazebo simulation platform
  • 23.ROS2 TF2 coordinate transformation
24.Main Control Course
Jetson Orin Nano_NX
  • 1.Jetson Orin board introduction
  • 2.Restore the factory image system
  • 3.Backup SSD system
  • 4.SSD expansion
  • 5.Network configuration
  • 6.SSH remote login
  • 7.VNC remote control
  • 8.Remote file transfer
  • 9.Jtop tool
  • 10.Exchange space expansion
Raspberry Pi
  • 1. Introduction to Raspberry PI 5
  • 2. Raspberry PI system installation and backup
  • 3. Powering the Raspberry PI 5
  • 4. Startup of Raspberry PI 5
  • 5. Update and upgrade operating system
  • 6. Introduction to raspi-config tool
  • 7. config.txt file description
  • 8. Network Configuration
  • 9. remote access
  • 10. Transfer files remotely
  • 11. Set display resolution and rotation
  • 12. Set screen to sleep
  • 13. Play audio and video
  • 14. Using USB camera
  • 15. Using MIPI camera
  • 16. Get real-time temperature of Raspberry Pi
RDKX5
  • 1.Introduction to RDK X5 board
  • 2.Write system file
  • 3.Hardware interface description
  • 4.Install case and other accessories
  • 5.Network configuration
  • 6.Remote login
  • 7.Update system
  • 8.srpi-config user guide
  • 9.config. txt configuration file
  • 10.Self-startup setting
  • 11.RDK exclusive command usage
  • 12.Linux command usage
  • 13.GPIO readwrite operation
  • 14.Image classification algorithm
  • 15.Use USB camera
  • 16.Use MIPI camera
  • 17.Binocular camera depth algorithm
  • 18. Controlling RGB Cooling Fans
Download
  • System File
  • Code
  • 3D Model File
  • Software Tools
  • Hardware_Info
  • Deb function package
  • Dify AI Agent
Download ZIP

Welcome to ROSMASTER M1 PRO repository

3.Multimodal Visual Understanding and Robotic Arm Gripping

Multimodal Visual Understanding + Robotic Arm Gripping

Multimodal Visual Understanding + Robotic Arm Gripping1. Course Content2. Preparation3. Running the Cases4. Source Code Analysis

1. Course Content

Run the example program to perform a comprehensive task using the robot's visual understanding combined with the robotic arm gripping function.

Note

  • The only difference between the text version and the voice version is the command input method; the text version does not include the speech recognition and speech synthesis playback steps.

2. Preparation

ros2 launch yahboomcar_bringup dify_llm_agents.launch.py  use_camera:=true

image-20260912182502896

  • Text interaction terminal

xxxxxxxxxx
dify_tui

image-20260914110555466

  • Use the /clear command to clear the terminal conversation history, and ctrl+q to exit the interaction terminal

3. Running the Cases

  • Find the red block in front of you and grasp it; the wooden block used is 30x30x30mm.

  • Please help me remove the AprilTag in front of you whose height is greater than 5 cm; the wooden block used is 30x30x60mm.

 

4. Source Code Analysis

Refer to the analysis of the corresponding chapter of AI Model - Voice Version; the source code is the same, only the interaction method differs.

Multimodal Visual Understanding
Multimodal Visual Understanding and SLAM Navigation

COMPANY

  • About us
  • Team Style
  • Blog

LEARNING & EXPLORATION

  • Tutorials
  • Edge AI hardware
  • Educational robots
  • Embedded intelligent hardware

WHERE TO BUY

  • Yahboom Online Store
  • Amazon
  • Alibaba
  • AliExpress
  • Mercado

TECHNOLOGY SUPPORT

  • Email: support@yahboom.com

BE A DISTRIBUTOR

  • Email: sales@yahboom.com
QR Code

Copyright © 2017-2026 Yahboom. All Rights Reserved.

EN/中文