Welcome to DOFBOT-Ultra repository

Voice Recognition Target Tracking

If using a depth camera (DABAI_DCW2), before running this example, be sure to execute ros2 run dofbot_ultra_driver calibrate_6point for a six-point calibration, otherwise the robot arm cannot accurately locate the target.

Please ensure that the web control program backend process is not running in the background. To close it, refer to the [Get Started Quickly - 2. Access the Terminal] section in [Disable the Auto-Start Web Control Backend Process]. After closing, please restart DofbotUltra!!!

1. Operation Instructions

1.1. Startup

Before running the function, you need to shut down the web control program backend process. For the shutdown method, please refer to the [4. Preparation - 1. Manage Web control services].

Raspberry Pi version (in Docker environment), enter the following command:

v4l2-ctl -d /dev/video0 --set-ctrl=exposure_time_absolute=50
python3 /home/yahboom/dofbot_ultra/2d/dofbot_voice/scripts/voice_target_follow.py

RDK X5 version (native environment), enter the following command:

xxxxxxxxxx
v4l2-ctl -d /dev/video0 --set-ctrl=exposure_time_absolute=50
python3 /home/sunrise/dofbot_ultra_ws/src/2d/dofbot_voice/scripts/voice_target_follow.py

1.2. Operation Description

After the program runs, the voice module will broadcast "here", then say "Hello, yahboom" to the voice module. The voice module will reply "here" to indicate successful wake-up. Then say "Face following/Yellow following/Red following/Green following/Blue following" to the voice module. The voice module will reply and broadcast "ok", then the robotic arm will start tracking the target. If you want to stop tracking, say "Stop following" to the voice module. The robotic arm will stop tracking the target and the voice module will reply and broadcast "ok". Note: If colors cannot be recognized or the target color cannot be tracked correctly, you may need to modify the HSV values in the program or adjust the ambient lighting.

2. Code Logic

This case integrates voice control with color tracking and face tracking modes.

• Color Tracking: Uses the color_follow module, detects target position through HSV color space, and uses PID to control servos 1 and 2 for real-time tracking
• Face Tracking: Uses the Face_Follow module, detects face position through Haar cascade classifier, and uses PID for tracking

The model variable switches between modes (General standby / color_follow color tracking / follow2 face tracking), and corresponding tracking functions are called based on voice commands.

3. Source Code

Raspberry Pi: /home/yahboom/dofbot_ultra/2d/dofbot_voice/scripts/voice_target_follow.py

RDK X5: /home/sunrise/dofbot_ultra_ws/src/2d/dofbot_voice/scripts/voice_target_follow.py

Tracking modules:

• Color tracking: Raspberry Pi: /home/yahboom/dofbot_ultra/2d/dofbot_voice/scripts/color_follow.py RDK X5: /home/sunrise/dofbot_ultra_ws/src/2d/dofbot_voice/scripts/color_follow.py
• Face tracking: Raspberry Pi: /home/yahboom/dofbot_ultra/2d/dofbot_voice/scripts/face_follow.py RDK X5: /home/sunrise/dofbot_ultra_ws/src/2d/dofbot_voice/scripts/face_follow.py

​x
from color_follow import color_follow
from face_follow import Face_Follow
​
follow = color_follow()
follow2 = Face_Follow()
model = 'General'
​
result = mySpeech.speech_read()
if result == 73:
    choose_color = 'red'
    model = 'color_follow'
    mySpeech.void_write(71)
elif result == 71:
    model = 'follow2'
    mySpeech.void_write(71)
elif result == 76:
    model = 'General'
    mySpeech.void_write(76)
​
if model == 'color_follow':
    img = follow.follow_function(img, color_hsv[choose_color])
if model == 'follow2':
    img, pos = follow2.follow_function(img)

• color_follow.follow_function(img, hsv) - Color tracking, follows target based on HSV range, PID controls servos
• Face_Follow.follow_function(img) - Face tracking, follows face, PID controls servos
• Voice IDs: 73 = track red, 72 = track green, 74 = track blue, 75 = track yellow, 71 = start tracking, 76 = stop