Welcome to DOFBOT-Ultra repository

Label Code Positioning Experiment

Please ensure that the web control program backend process is not running in the background. To close it, refer to the [Get Started Quickly - 2. Access the Terminal] section in [Disable the Auto-Start Web Control Backend Process]. After closing, please restart DofbotUltra!!!

Identify AprilTag label codes through the camera, locate their position, and display their ID number and center coordinates.

1. Startup and Operation

Open a browser (any browser on any host in the same local network) and enter the IP address:8888 in the address bar to access JupyterLab.

Code path:

xxxxxxxxxx
dofbot_apriltag/scripts/Apriltag_Position.ipynb

Click the Run All button on the JupyterLab toolbar, then scroll to the bottom.

After the program starts, place the label code block in the camera view. The label code will be correctly located and its ID number displayed. Move the block, and the recognition follows.

Open the Log panel and set Log Level to Info to see the label code ID and position coordinates printed.

To end the program, click [Exit] to avoid affecting resource usage by other programs.

2. Technical Overview

AprilTag is a coded marker commonly used in machine vision. It offers high recognition rates and reliability for various tasks including augmented reality, robotics, and camera calibration. This implementation uses the TAG36H11 format. Factory-supplied tags are pre-attached to the blocks.

AprilTag positioning workflow:

1. Image Acquisition: Capture real-time images via the camera.

2. Tag Detection: Use the AprilTag detection library to scan for tags in the frame and decode the tag ID.

3. Corner Extraction: Get the coordinates of the four corners of the tag.

4. Center Calculation: Calculate the center position based on the corner coordinates.

5. Visualization: Draw the tag border and ID number on the image, and output position information.

3. Core Code Analysis

• Import header files.

xxxxxxxxxx
import cv2 as cv
import threading
import random
from time import sleep
import ipywidgets as widgets
from IPython.display import display
from apriltag_identify import ApriltagIdentify
from dofbot_utils.fps import FPS
from dofbot_utils.robot_controller import Robot_Controller
from dofbot_utils.camera import create_camera

• Create instances and initialize parameters.

xxxxxxxxxx
apriltag_Identify = ApriltagIdentify()
model = 'General'
robot = Robot_Controller()
robot.move_look_map()
fps = FPS()

• Main process.

xxxxxxxxxx
def camera():
    global HSV_learning,model
    capture = create_camera()
    while capture.isOpened():
        try:
            _, img = capture.read()
            fps.update_fps()
            img, msg = apriltag_Identify.getApriltagPosition(img)
            if len(msg):
                print(msg)
            if model == 'Exit':
                capture.release()
                break
            fps.show_fps(img)
            imgbox.value = cv.imencode('.jpg', img)[1].tobytes()
        except Exception as e:
            print("program end")
            print(e)
            capture.release()

• Startup.

display(controls_box,output)
threading.Thread(target=camera, ).start()