Welcome to DOFBOT-Ultra repository

Calibration Offset (Non-Depth Camera Version)

This tutorial is for standard USB 2D camera scenarios. Unlike the depth camera version, the non-depth camera calibration enters the Pixel-World Calibration script, which requires 6 fixed-point samples to generate the grasping mapping relationship.

Scope of Application

1. Applicable camera: Standard USB camera (not the DABAI_DCW2 depth camera).

   Check whether CAMERA_TYPE in ~/.bashrc is set to any value other than DABAI_DCW2.

2. Calibration goal: Establish the mapping relationship between image pixel points and the robotic arm working plane coordinates (X/Y).

3. Calibration method: Use the AprilTag to sample at 6 guided positions on the calibration board. The program automatically calculates the homography matrix.

4. Calibration result: Saved to pixel_world_homography.npy for subsequent recognition and grasping processes.

2. Preparation Before Calibration

1. Ensure the robotic arm and USB camera are properly connected and the camera image can be read normally.
2. Place the DOFBOT calibration board (30 cm x 21 cm) and the AprilTag (3 cm x 3 cm x 3 cm block) on the table.
3. Ensure uniform ambient lighting, avoiding strong reflections and shadows that block the Tag.
4. Fix the robotic arm body (place it in the designated area on the calibration board) during calibration. Only move the AprilTag position during sampling.
5. Confirm that the current environment variable is not in depth camera mode: CAMERA_TYPE should not be DABAI_DCW2.

Recommended check command:

echo $CAMERA_TYPE

If the output is DABAI_DCW2, switch to standard USB mode and reload the environment.

3. Start the Calibration Wizard

Run the following command in the terminal:

xxxxxxxxxx
ros2 run dofbot_ultra_driver calibration_wizard

The main menu will display the camera type and function items. In non-depth camera mode, option 1 is:

1. Pixel-World Calibration (Board-Guided)
2. Mechanical Bias & Height Comp Tuning
3. Run Grasp Test Q. Quit

First select 1 to enter the guided 6-point calibration.

4. Calibration Interface Key Instructions

Tip: When calibrating positions other than the first point, cover the fixed AprilTag on the calibration board to avoid interference.

Process summary: 1. Place the AprilTag at the specified position --> 2. Press SPACEBAR to sample --> 3. Repeat steps 1 and 2 six times for six points --> 4. Press the "C" key to complete calibration --> 5. Press Q to exit.

After entering the calibration window, the key logic is as follows:

1. SPACEBAR: Sample the current point.
2. S: Skip the current point (generally not used unless the point is truly uncollectable).
3. C: Calculate and save the calibration result (at least 4 valid points are required; it is recommended to follow the guide and collect all six points).
4. ESC: Exit the calibration process.

Notes:

1. Each time the SPACEBAR is pressed, the program does not take a single-frame sample immediately. Instead, it performs a brief multi-frame sampling and averages the result to reduce jitter error.
2. It is recommended to collect all 6 points, as accuracy is typically higher than with only 4 points.

Sample point illustration:

5. 6-Point Guided Sampling Steps

The program will prompt 6 points in a fixed order. Follow the prompts to move the AprilTag center and align it with the corresponding edges.

Point 1: Mark Center

1. Align the AprilTag center with the calibration board mark (X=18.1 cm, Y=0).
2. Once stable, press the SPACEBAR to sample.

Point 2: Far Center

1. Move the Tag to the far edge center (the side away from the arm).
2. Press the SPACEBAR to sample.

Point 3: Far + Left Side of Frame

1. Move the Tag to the intersection of the far edge and the left side of the frame (close to both edges).
2. Press the SPACEBAR to sample.

Point 4: Far + Right Side of Frame

1. Move the Tag to the intersection of the far edge and the right side of the frame (close to both edges).
2. Press the SPACEBAR to sample.

Point 5: Mark + Left Side of Frame

1. Return to the mark line height and move close to the left edge of the frame.
2. Press the SPACEBAR to sample.

Point 6: Mark + Right Side of Frame

1. Return to the mark line height and move close to the right edge of the frame.
2. Press the SPACEBAR to sample.

Tips:

1. After each successful sample, the interface will display the completed status.
2. If the Tag is not recognized in the current frame, pressing the SPACEBAR will show a failure prompt. Adjust and retry.

6. Calculation, Saving, and Online Verification

1. After sampling, press C. The program will calculate the pixel-to-world coordinate transformation matrix.
2. The terminal will output the error for each point, the average error, and the maximum error.
3. After saving, enter online verification mode. Move the Tag to observe the real-time calculated world coordinates.
4. Press ESC or Q to exit verification and return to the process end screen.

7. Result File and Path Description

1. Calibration result file name: pixel_world_homography.npy.
2. The save directory is the driver configuration path (the program automatically selects a valid directory).

Raspberry Pi configuration directory:

xxxxxxxxxx
/home/yahboom/dofbot_ultra/core/dofbot_ultra_driver/config

RDK X5 configuration directory:

xxxxxxxxxx
/home/sunrise/dofbot_ultra_ws/src/core/dofbot_ultra_driver/config

8. Common Issues and Solutions

1. Black screen or camera not opening after entering calibration

   • Check if the camera is occupied by another process.
   • Re-run the calibration wizard and ensure standard USB mode is used.

2. AprilTag not recognized

   • Adjust lighting and focus.
   • Ensure the Tag is clear, complete, and unobstructed.

3. Large error and obvious grasping offset

   • Re-run the full 6-point sampling.
   • Ensure each point is aligned with the indicated edge position.
   • Avoid moving the arm or camera during sampling.

4. Fixed deviation remains after calibration

   • Select option 2 (Mechanical Bias & Height Comp Tuning) from the main menu to fine-tune the mechanical error.

9. Grasp Test

After calibration is complete, you can immediately run a grasp test to verify the calibration result.

From the menu, select option 3 to enter the grasp test mode.

After the block is recognized, press the SPACEBAR to grasp. The target position is the center-lower part of the block. Test grasping with the block at several different positions:

1. If the grasp is accurate, calibration is successful.
2. If there is a fixed deviation (always offset forward or backward), use option 2 (Mechanical Bias) from the menu to fine-tune the mechanical error.
3. If the deviation is not fixed, recalibrate (minor deviations can be ignored).