Welcome to DOFBOT Pro repository

Voice Control Remove Height Abnormal Color Block

Before running the function, you need to close the App and large programs. For the closing method, refer to [4. Preparation] - [1. Manage APP control services].

1. Function Description

After the program starts, voice commands are issued to instruct the robotic arm to remove height-abnormal color blocks. After receiving the command, the robotic arm will lower its gripper to grasp color blocks that exceed the height threshold.

The wooden blocks used in this lesson are: 30x30x30mm and 30x30x60mm.

2. Startup and Operation

2.1. Startup

Users with Jetson-Nano board version need to enter the docker container and input the following commands. Orin board users can directly open the terminal and input the following commands,

xxxxxxxxxx
#Start camera and inverse kinematics
ros2 launch dofbot_pro_info camera_arm_kin.launch.py
#Start speech recognition and broadcast
ros2 launch yahboom_speech speech.launch.py
#Start color block height calculation program
ros2 run dofbot_pro_voice_ctrl color_hight_list
#Start color block grasping program
ros2 run dofbot_pro_voice_ctrl remove_heigher_VC

2.2. Operation Steps

2.2.1. Calibrate Color Block

Place color blocks of the same color but different heights in the camera view, then select the top area of any color block with the mouse and release to complete the color block color calibration. At this time, the height of the color block will also be displayed in the image. As shown in the figure below,

2.2.2. Voice Control

After color calibration is completed, say "Hello, yahboom" to the speech recognition module, and the speaker will broadcast "here". Then say "Remove color blocks with abnormal height" to the speech module. The robotic arm will lower its gripper to grasp color blocks higher than 5cm, and the speech module will broadcast "OK". After grasping the height-abnormal color block, it will place it at the designated position. Finally, the robotic arm returns to its initial posture, and the speech broadcast module will announce "Placement complete".

3. Core Code Analysis

3.1. Color Block Height Calculation Node color_hight_list

Source code path: ~/dofbot_pro_voice_ctrl/dofbot_pro_voice_ctrl/Color/color_hight_list.py

Mainly explains how to subscribe to and process speech recognition result topics and publish voice broadcast topics.

xxxxxxxxxx
#Create subscriber for speech recognition result topic
self.sub_voice =
self.create_subscription(Int8,"voice_result",self.getVoiceResultCallBack,1)
#Create publisher for voice broadcast topic
self.pub_playID = self.create_publisher(Int8,"player_id", 1)
​
#Callback function, if the received speech recognition result data is 104, it means the control command is "Remove color blocks with abnormal height", then change self.start_flag value to True indicating that color block position information can be published and publish voice broadcast topic, the broadcast audio is "OK"
def getVoiceResultCallback(self,msg):   
    if msg.data == 104:
        self.start_flag = True
        play_id = Int8()
        play_id.data = 45
        self.pub_playID.publish(play_id)

3.2. Robotic Arm Color Block Grasping Node remove_heigher_VC

Source code path: ~/dofbot_pro_voice_ctrl/dofbot_pro_voice_ctrl/Color/remove_heigher_VC.py

​x
#Create publisher for voice broadcast topic
self.pub_playID = self.create_publisher(Int8,"player_id", 1)
​
#At the end of the move function, publish voice broadcast topic, broadcasting "Placement completed" audio
play_id = Int8()
play_id.data = 81
self.pub_playID.publish(play_id)