Get Started Quickly

Power On

Assemble the robotic arm according to the installation tutorial. Toggle the power switch on the motherboard to ON.

For smooth video transmission and control, it is strongly recommended to use a wired network connection.

When powered on for the first time, if a network cable is plugged in and wired network is available, the Raspberry Pi will automatically select LAN mode. If no network cable is plugged in, the robotic arm will start a hotspot named DofbotUltra.

The password is 12345678.

Note: The phone may prompt that the WIFI has no internet access. This is normal.

After powering on, wait about 1-2 minutes. You will hear a "ding" sound from the robotic arm, indicating it is ready to be connected and tested via phone / computer / controller.

Phone / PC Control

Before starting, check the IP address displayed on the OLED screen and record it.

By default, the DofbotUltra hotspot IP address is 10.42.0.1.

Then enter <IP>:6500 in the browser address bar on your phone or computer to access the control interface.

If opening the page on a phone, landscape mode is recommended. The webpage will prompt auto-rotation; if it does not switch automatically, simply rotate your phone.

Here we use a computer browser --> F12 Developer Tools --> Mobile Device Mode --> Landscape as an example:

The initial interface is as follows:

Note: The following content describes the web control functions. For more detailed and accurate information, please refer to the later chapter [3. Remote Control Tutorial] --> [1. Phone Control].

Initial Settings

When using for the first time, it is recommended to complete the basic setup in the following order. This will make testing various functions smoother and more stable.

1. Language Options

After entering the Settings page, you can switch between Chinese and English at the top.

2. Camera Type

If you are using a regular USB camera, select USB Camera. If you are using a depth camera, select Depth Camera.

When switching camera types, the left-side preview will briefly reconnect. This is normal. Wait for the preview to recover before proceeding.

Under normal circumstances, the left-side video will reappear and refresh in real time after switching.

Note: The only difference between depth camera and USB camera in web control is the distortion correction coefficient. When using the grasping function in web control, the same approach as a regular USB camera is used.

3. Camera Exposure

If your ambient lighting is relatively stable, it is recommended to use manual exposure. This will make color recognition, tracking, and sorting more stable.

Operation steps:

1. Go to Settings -> Vision & Calibration -> Camera Exposure and switch to manual exposure.
2. Drag the slider to adjust brightness.
3. Click Apply Exposure Value.

You can observe the left-side preview and adjust until the image shows "clear color blocks, not overexposed, not too dark."

4. Complete Calibration

If you plan to use functions like grasping and color sorting later, it is recommended to complete the following two steps:

1. Pixel-World Calibration (6 points)
2. Arm Offset Adjustment (only needed when there is a fixed error after pixel-world calibration. Generally, this does not need to be changed.)

Settings Page

Arm Settings

The 3 most commonly used functions are:

Servo Center (Check)

Normal effect: The arm slowly returns to the default center pose without obvious jitter or hitting the limit.

Servo Center Calibration (Offset)

If you find that a servo's center position is off, for example, the arm is still slightly tilted after centering, use this function to calibrate.

Operation sequence:

1. Click Start Calibration.
2. Manually adjust the arm to the standard pose.
3. Click Save All Servo Centers.
4. After completion, click End Calibration.

To restore factory settings, click Restore Default Center.

Normal effect: The next time you click "Servo Center," the arm pose will be more accurate.

Vision & Calibration

This section is recommended to be used in the following order.

Camera Type

Used to switch the currently connected camera.

Usage has been described earlier and will not be repeated here.

After switching cameras, it is recommended to check whether the preview is normal and recalibrate if necessary.

Camera Exposure

This mainly affects the brightness of the preview. When the ambient lighting is stable, manual exposure is recommended.

Normal effect: The left-side preview brightness is stable, and color blocks do not fluctuate in brightness.

Camera Calibration (Pixel-World Calibration)

If you need to perform grasping or sorting, it is recommended to complete this step. The calibration requires:

30x30x30 wood block x1 (with Apriltag code, any color)

3D calibration board x1

Operation steps:

1. Prepare the 3D calibration board.
2. Enter Pixel-World Calibration (6 points).
3. Follow the on-screen instructions to place the 30x30x30 block with the Apriltag face up at the corresponding positions.
4. Click Capture at each position.
5. After all 6 points are completed, the system will automatically calculate the calibration result.

After completion, you can use the Grasp Test below to verify the result.

Normal effect:

1. The calibration status gradually advances.
2. After collecting 6 points, a success message is displayed.
3. The arm can accurately target the marked positions during the grasp test.

Grasp Test

This function is used to quickly verify the calibration result.

Operation steps:

1. Complete the 6-point calibration on the calibration page.
2. Click Start.
3. Place the AprilTag in the view.
4. Click Grasp.

Normal effect: The arm performs a grasping action based on the currently recognized tag position to verify whether the positional mapping is accurate.

If the deviation is significant, continue adjusting the offset below.

Arm Offset Adjustment

This function is used to correct situations where "the target position looks correct, but the grasp is always slightly off."

Operation steps:

1. Enter Arm Offset Adjustment.
2. Fine-tune parameters one by one.
3. After achieving a more accurate grasp, click Save.

Normal effect: The gripper will more easily land on the correct position for the same target point.

Precision Color HSV Calibration

If color recognition is unstable, for example, red is recognized as another color, or the edges of color blocks are unclear, HSV calibration is needed.

Operation steps:

1. Enter Precision Color HSV Calibration.
2. Select the color to calibrate.
3. Adjust the sliders.
4. Observe the left-side binarization effect.
5. Make the target color appear as white as possible, and the background and other colors appear as black as possible.
6. Click Confirm Save.

Normal effect: After saving, color tracking, color grasping, and color sorting become more stable.

Hardware Control

The most direct function here is RGB Lighting.

Click different color buttons, and the RGB light on the arm will switch to the corresponding color. Click the power icon on the far right to turn off the light.

This function is mainly used for demonstration and quick confirmation of whether the motherboard control is normal.

Forward/Inverse Kinematics Test

This section is more suitable for quick validation. It is not recommended to spend much time on it during the first use.

Forward Kinematics (FK)

Click Read Current Pose, and the system will read the current servo angles and calculate the end-effector coordinates.

Inverse Kinematics (IK)

After entering the target X / Y / Z and Pitch:

1. Click Calculate to see if the system can compute a set of usable angles.
2. After confirming, click Execute Motion.

Remote Control Page

Directly control the angle values of Servo 1 to 6.

Basic Control

Center

Click to return the arm to the default pose.

Release

Click to release the servos so you can manually move the arm.

Restore

Click to re-engage the servos.

If the arm was in a released state, it will return to a controllable state.

Servo Control (ID 1-6)

Each slider below corresponds to a servo.

Usage:

1. Drag the slider of a servo.
2. Observe the real-time movement of that joint.
3. If needed, click Read Current Angle to check the actual angle.

Motion Page

This page is divided into Fixed Motion Groups and Custom Motions.

Fixed Motion Groups

The page provides Motion 1 ~ Motion 6.

Click directly to run.

Custom Motions

Custom motions allow you to record a series of poses and replay them.

Recommended usage sequence:

1. Click Start Learning.
2. Manually pose the arm.
3. Click Record Current Motion.
4. Continue to the next pose and click Record Current Motion again.
5. After recording all, click Finish Learning.
6. Click Single Run to verify the motion replay.

If you made a mistake, click Clear Motions.
If replay is in progress and you want to interrupt, click Stop Replay.

Gesture Page

Gesture Recognition

When enabled, the arm will perform corresponding actions based on recognized gestures.

Usage recommendations:

1. Keep your hand in the center of the frame.
2. Keep the background simple.
3. Ensure adequate lighting.

Gesture Stacking

When enabled, the arm performs stacking actions based on recognition results.

Color Page

This page mainly contains interactive features related to color recognition.

You Place, I Grasp

When enabled, place the target color block within the arm's field of view, and the arm will attempt to recognize and grasp it.

Color Grasp

This function is similar to "You Place, I Grasp" but more directly performs grasping based on color.

If color recognition is not stable enough, go back to the settings page and perform an HSV calibration.

Snake Attraction

When enabled, use the color selection buttons below.

Recommended sequence:

1. First turn off the switch.
2. Select the target color.
3. Then turn on the switch.

Tracking Page

Face Tracking

Turn on the switch, place your face in the frame, and the arm will follow your face position.

Color Tracking

There are 4 buttons provided:

1. Track Red
2. Track Green
3. Track Blue
4. Track Yellow

After selection, place the corresponding color block in front of the camera.

To stop, click Stop Tracking.

PID Parameter Adjustment

If you feel the tracking is too slow, too shaky, or not stable enough when returning to center, expand PID Parameter Adjustment.

Recommended approach:

1. Keep default parameters first.
2. Confirm that tracking works normally.
3. Make small adjustments.
4. Change only one parameter at a time and observe the effect.

Sorting Page

This page contains two very commonly used features.

YOLO Waste Sorting (Recognition)

Turn on the YOLO Waste Sorting switch, and the system will load the model first.

It may take a moment to load, and the preview may become laggy. This is normal.

Usage:

1. Turn on the switch.
2. Wait for the page to indicate that the model has loaded.
3. Place a waste item in the frame.
4. The arm identifies what type of waste it is.

Note: This is only for waste recognition, with no subsequent sorting action.

Color Sorting

Ensure calibration is completed before use.

Recommended operation sequence:

1. First set the colors for Slot 1 ~ Slot 4.
2. Click Start Detection.
3. Confirm that the color blocks in the frame are correctly identified.
4. Click Start Grasping.

If you want to restart:

1. Reset List
2. Stop

Normal effect: The arm grabs different color blocks to the corresponding positions according to your slot settings.