Welcome to DOFBOT Pro repository

APP Control

Note: The mobile app control service is set to start automatically at boot in the factory image. If you have closed the app control program, please enable the relevant programs according to the [Manage App Control Service] tutorial.

I. APP Connection to Robotic Arm Function

1. Please install the robotic arm correctly and ensure the wiring is correct according to the instruction manual. Then, open the power switch.
2. Wait for about 1 minute and hear the buzzer beep three times, indicating that the system initialization is complete. At this time, the robotic arm will rotate to the initial posture.
3. Check the OLED screen on the expansion board and read the IP address of the device in the [IPA] line.

4. Open the WiFi settings on your phone, make your phone connect to the hotspot signal of the robotic arm. DOFBOT PRO hotspot name: DOFBOT-Pro password:12345678

5. Open the [YahboomRobot] APP. For the first time, you need to select [ROS Robot] --> [DOFBOT Pro] device

6. Click [IP Connect], then enter the IP address of the device, keep the others unchanged, and click [Connect]

7. Click [Connect] again. If the connection is successful, it will automatically jump to the configuration interface.

8. 【DOFBOT Calibration】: The center position of the robotic arm is very important. The deviation of the center position will directly affect the accuracy of all subsequent functions. Before using the robotic arm, we need to calibrate it, according to the following steps. Press the [Middle] button to check whether the robotic arm is vertically upward, left and right, and the gripper is closed. If it is the above state, please click the [Skip]; If not, click [Calibrate].

9. After clicking [Calibrate], robotic arm will enter calibration state, carefully check whether the robotic arm is upright,whether the left and right return to the center is normal, and gripper should be manually pressed to the closed state. After the adjustment is completed, click [Ok]. When the indicator rings of all numbers turn from blue to green, it means that the setting is successful. Click [Next].

10. 【Color Calibration】：Please place the red block to the position of the identification frame in the image, and ensure that the identification frame can completely cover the color block. Wait for the image to display the stable HSV value of current color in real time, and the video to display OK. Click [Color Calibration] to complete the calibration of the color. Then, calibrate the green, blue, and yellow block in the same way. If a color does not need to be calibrated, you can click [Skip]. Finally, click [Finish].

2. APP Control

The APP main interface is divided into eight area, each of which corresponds to different functions.

2.1 Remote Control

Click [Remote control] icon on the main interface, the following interface will appear.

The left side shows the camera screen. The upper right side shows a robotic arm model with numbers1 to 6, representing the six servos of the robotic arm. When it is displayed in red, it means that the servo with the current ID number is selected. Then, you can drag the slider below or press the buttons on the left and right sides to adjust the position of this servo.

【Angle】: After each click, APP will read the angle of the current servo and update it to the slider above.

【Middle】: Robotic arm stands upright.

【Stop】: After clicking this button, robotic arm turns off the torque, all servos no longer receives control commands. We can adjust manually the angles of all servos. Click this button again, robotic arm will turn on the torque, all servos will maintain their current positions, and can receive control commands.

2.2 Action Group

Click [Action Group] icon on the main interface, and the following interface will appear.

The left side is custom action group. The right side is fixed action group. In the middle, you can preview the action group screen, run the action group, and stop the action group. Click [Run] to run the current action group, and click [Stop] to stop the robot from running the action group.

【Custom action group】: Allows the robotic arm to learn and execute action groups. Click [Study mode], a dialog box will pop up, and RGB light on the expansion board will show a breathing light state.

Click [Record X Action] button, the robotic arm will record the current posture as an action group, and the RGB light breathing light on the robot expansion board will change color, indicating that the action has been recorded. If you want to exit after recording multiple groups of actions,click [Completed], the breathing light on the expansion board will go out. Click [Run] button in the action group interface to run the action group just learned

If red breathing light appears, it means that the learning is wrong or the recorded action group is full (up to 20 groups of actions can be stored). Please click [Completed] button to exit

【Fixed action group】: Click different number buttons to know the function of the corresponding action group from the preview window. When you click [Run], the robotic arm will run the fixed action group.

2.3 Gesture interaction

Click the [Gesture interaction] icon on the main interface, the following interface will appear.

Gesture interaction includes two functions: gesture action and gesture action. After selecting the corresponding function, click [Switch], the function preview window will display the guidance operation. Click [Switch] again to turn off current function.

【Gesture action】: Can recognize gestures 1~5 and perform corresponding actions.

【Gesture stack】: Recognize gestures 1234, respectively grab the blue, green, red, and yellow building blocks and stack them in order. When gesture 5 is recognized, push all color blocks down and reset the recognition data.

2.4 Color Interaction

Click [Color interaction] icon on the main interface, the following interface will appear.

Color interaction includes three functions: Catch game, color capture, color interaction. After selecting the corresponding function, click [Switch] to start the function, click [Switch] button again to turn off current function.

【Catch game】: Place the block in the cross area on the map,robotic arm will automatically identify the color of the current block, then grab the block and put it in the area of the corresponding color.

【Catch game】: Place the block in front of the camera, robotic arm will honk after identifying the color of block. We need to put this block in the area of the corresponding color, robotic arm will grab the block from the corresponding color area and place it in the cross area on the map.

【Color interaction】：After selecting the color below, click [Switch], put the selected color block in front of the camera,robotic arm will imitate the movement of the snake. When it isled to the longest, open its gripper to clamp the block and put it down. When the block is infinitely close to it, robotic arm will shake.

Note: If you need to switch to different colors of blocks, you need to close the current function first and then open it again for it to take effect.

2.5 Tracking Game

Click [Tracking game] icon on the main interface, the following interface will appear.

Tracking game includes three functions: Color tracking, custom color tracking, face tracking. After selecting the corresponding function, click [Switch] to start the function, click [Switch] button again to turn off current function.

【Color Tracking】: Select a color, click [Switch], then place the block of the corresponding color in front of the camera. When you move the block, robotic arm will move with this block.

【Custom color tracking】: After clicking [Custom color tracking]button, a box will be displayed on the camera image, and the color to be tracked will be placed in the box. After accurately picking the color, click [Switch], when the system completely detects the outline of the color, robotic arm will move with this block.

【Face Tracking】: Click [Face tracking] button, click [Switch],robotic arm will detect whether there is a face in the current camera image. If there is a face, it will mark and track the face movement.

2.6 Garbage Sorting

Click [Garbage sorting] icon on the main interface, the following interface will appear.

Click [Switch], the system will automatically load the model. After the red prompt [Model-Loading...] on the screen disappears, place the block with the garbage picture in the camera image. This function can identify the type of garbage on the current block. The camera collects information in the picture and displays the type of garbage on the APP. After continuous identification for 10 times, the garbage is sorted according to the garbage type to the corresponding garbage dumping area on the map.

2.7 Advanced Settings(Bate)

Click [Advanced Settings(Beta)] icon on the main interface, the following interface will appear.

Click [up, down, left, right] buttons to move the robotic arm,ensure the box appears completely in the field of view. Move the slider of threshold to adjust the border detection threshold until the four sides of the box completely detect the current color block, as shown above. Click [Ok] to enter the function interface.

[Sorting Game] interface, as shown below.

【Color sorting】: Click [1], [2], [3], [4] button to change the color(black means unselected). Click [Color sorting], wait for the color to be correctly identified, then click [Switch] to sort and grab in the order of color selection.

【Garbage sorting】: Click [Garbage Sorting], wait for the system to load the model, and the red prompt [Model-Loading...] in the image disappears. Place the block with the garbage picture in the camera image, and the system will automatically identify the current garbage type. Click [Switch], robotic arm will start sorting the block into the corresponding categories.

2.8 Robot Arm Settings

Click the [Robotic arm setting] icon on the main interface, the following interface will appear.

The robotic arm settings include three functions: servo calibration, color calibration, restore default.

【Servo Calibration】：The center position of the robotic arm is very important. The deviation of the center position will directly affect the accuracy of all subsequent functions. Before using the robotic arm, we need to calibrate it, according to the following steps.

Press the [Middle] button to check whether the robotic arm is vertically upward, left and right, and the gripper is closed. If it is the above state, please click the [Skip]; If not, click [Calibrate].

After clicking [Calibrate], robotic arm will enter calibration state, carefully check whether the robotic arm is upright,whether the left and right return to the center is normal, and gripper should be manually pressed to the closed state. After the adjustment is completed, click [Ok]. When the indicator rings of all numbers turn from blue to green, it means that the setting is successful. Click [Next].

【Color Calibration】: Color calibration has two functions. The first is [Basic color calibration], which is the same as the guided configuration function. The second is [Advanced color calibration].

【Basic color calibration】: Please place the red block to the position of the identification frame in the image, and ensure that the identification frame can completely cover the color block. Wait for the image to display the stable HSV value of current color in real time, and the video to display OK. Click [Color Calibration] to complete the calibration of the color. Then, calibrate the green, blue, and yellow block in the same way. If a color does not need to be calibrated, you can click [Skip]. Finally, click [Finish].

【Advanced color calibration】: Place four color blocks on the camera image at the same time, select the color to be calibrated. Then, click [Display switch] button to view the black and white image, and adjust it through the HSV slider until no interfering color is detected. Click [Color calibration] to complete the calibration of current color block, and calibrate other color block in the same way.

Verification result: Change the position of the color block at will, and the color block can be accurately identified in the color image, click [Back]. For detailed introduction, please refer to the course [Visual basic course]-->[Color calibration].

【Restore default】: Clear the configuration information of the APP. You will need to reconfigure the arm according to the instructions before it can be used again.