Welcome to DOFBOT-Ultra repository

Voice Control Remove Height Abnormal AprilTag

Please ensure that the web control program backend process is not running in the background. To close it, refer to the [Get Started Quickly - 2. Access the Terminal] section in [Disable the Auto-Start Web Control Backend Process]. After closing, please restart DofbotUltra!!!

Before starting this function, you need to close the web control program backend process.

1. Function Description

Issue a command via voice for the robotic arm to remove AprilTags with abnormal height. (This section is only for the depth camera version.)

All functions involving picking up cubic blocks must use 30x30x30 mm wooden blocks. Non-standard sized objects may cause servo 6 stall and damage.

2. Startup and Operation

2.1. Startup

Raspberry Pi version:

​x
# ==================== Basic Service Layer ====================
# 1) Start camera and robotic arm inverse kinematics service
ros2 launch dofbot_ultra_info camera_arm_kin.launch.py

# 2) Start robotic arm driver
ros2 run dofbot_ultra_driver arm_driver


# ==================== Voice Input Layer ====================
# 3) Start speech recognition
ros2 launch dofbot_yahboom_speech speech.launch.py


# ==================== Core Control Layer ====================
# 4) Start voice router
ros2 run dofbot_ultra_voice_ctrl voice_router


# ==================== Visual Perception Layer ====================
# 5) Start height anomaly detection (modified apriltag_list.py)
#    Function: receives apriltag_trigger → detects tags with height > 0.05m → publishes PosInfo
ros2 run dofbot_ultra_driver apriltag_list


# ==================== Action Execution Layer ====================
# 6) Start grasp executor (receives PosInfo, executes grasping)
ros2 run dofbot_ultra_driver apriltag_remove_higher


# ==================== Voice Feedback Layer ====================
# 7) Start voice broadcast
ros2 run dofbot_yahboom_speech voice_player

For RDK:

xxxxxxxxxx
ros2 launch dofbot_ultra_info camera_arm_kin.launch.py
ros2 run dofbot_ultra_driver arm_driver
ros2 launch dofbot_yahboom_speech speech.launch.py
ros2 run dofbot_ultra_driver apriltag_list
ros2 run dofbot_ultra_driver apriltag_remove_higher

# Depth camera exposure settings
ros2 service call /camera/set_color_exposure orbbec_camera_msgs/srv/SetInt32 '{data: 30}'
ros2 service call /camera/set_depth_auto_exposure std_srvs/srv/SetBool '{data: true}'

2.2. Operation Steps

After all programs are running, say "Hello, yahboom" to the speech recognition module, and the speaker will broadcast "here". Then say "Remove machine codes with height anomalies" to the speech module. The robotic arm will lower its gripper to grasp AprilTag blocks higher than 4cm, and the speech module will broadcast "OK". After grasping the AprilTag block, it will place it at the designated position. Finally, the robotic arm returns to its initial posture, and the speech broadcast module will announce "Placement complete". If there are no height-abnormal AprilTags on the current desktop, the speech broadcast module will announce "Stop".

3. Core Code Analysis

3.1. AprilTag Detection Node apriltag_list.py

This tutorial actually starts with:

xxxxxxxxxx
ros2 run dofbot_ultra_driver apriltag_list

The corresponding real source code for this section is apriltag_list.py in the dofbot_ultra_driver package. In the current implementation, voice commands are not sent directly to this node; instead, they are first converted into standard control topics by voice_router. This node is mainly responsible for detecting AprilTags after receiving a trigger and publishing the target position to the grasping node.

xxxxxxxxxx
# Receive the "one-shot trigger" signal from the voice router
self.trigger_sub = self.create_subscription(
    Bool, 'apriltag_trigger', self._trigger_callback, 10)

# Publish the detected target position to the subsequent grasping node
self.pos_info_pub = self.create_publisher(
    AprilTagInfo, "PosInfo", qos_profile=10)

# Reset after grasping is complete to avoid repeated publishing of targets
self.subscription = self.create_subscription(
    Bool, 'grasp_done', self.GraspStatusCallback, qos_profile=1)

The workflow can be summarized as:

1. voice_router receives "Remove machine codes with height anomalies" and publishes apriltag_trigger=True;

2. apriltag_list.py receives the trigger signal and starts detecting AprilTags;

3. After finding a target that meets the conditions, it publishes PosInfo and set_joint5;

4. Waits for apriltag_remove_higher.py to complete grasping and send back grasp_done.

3.2. Robotic Arm Removal Node apriltag_remove_higher.py

This tutorial actually starts with:

xxxxxxxxxx
ros2 run dofbot_ultra_driver apriltag_remove_higher

Therefore, the source code that actually performs the removal action is dofbot_ultra_driver/apriltag_remove_higher.py. This node no longer directly processes voice_result; instead, it subscribes to the target pose sent by the vision node and uses apriltag_remove_higher_enable to control whether motion is allowed.

xxxxxxxxxx
self.sub = self.create_subscription(
    AprilTagInfo, 'PosInfo', self.pos_callback, 1)
self.sub_joint5 = self.create_subscription(
    Int16, "set_joint5", self.get_joint5_callback, 1)
self.enable_sub = self.create_subscription(
    Bool, 'apriltag_remove_higher_enable', self._enable_callback, 10)
self.pub_playID = self.create_publisher(Int8, 'player_id', 10)

When pos_callback() receives the target, the node completes the following work:

1. Performs forward and inverse kinematics calculations based on the current real pose of the robotic arm;

2. Converts PosInfo into graspable world coordinates;

3. Controls the robotic arm to execute descending, grasping, lifting, and placing;

4. Publishes grasp_done=True to notify the detection node to reset;

5. Publishes player_id=81, and the voice broadcasts "Placement complete".