Welcome to DOFBOT-Ultra repository

Face Positioning Experiment

Please ensure that the web control program backend process is not running in the background. To close it, refer to the [Get Started Quickly - 2. Access the Terminal] section in [Disable the Auto-Start Web Control Backend Process]. After closing, please restart DofbotUltra!!!

The face positioning experiment detects faces in the camera frame, draws a box around the face, and prints the center point coordinates.

1. Startup and Operation

Open a browser (any browser on any host in the same local network) and enter the IP address:8888 in the address bar to access JupyterLab.

Code path:

xxxxxxxxxx
dofbot_face_follow/scripts/Face_position.ipynb

Click the Run All button on the JupyterLab toolbar, then scroll to the bottom.

After the program starts, place an image of a face in the camera view. A green box will outline the face. Move the face image, and the box will follow.

Open the Log panel and set Log Level to Info to see the face position coordinates printed.

To end the program, click [Exit] to avoid affecting resource usage by other programs.

2. Technical Overview

The core principle of face positioning is to determine the position of a face in the camera frame using computer vision algorithms:

1. Image Acquisition: Capture real-time images via the camera.

2. Face Detection: Use OpenCV's cascade classifier (Haar Cascade) or a deep learning model to detect the face region.

3. Position Calculation: Determine the bounding box of the face and calculate its center point coordinates.

4. Visualization: Draw a face box on the image and output the center point coordinates (x, y) in the log.

3. Core Code Analysis

• Import header files.

xxxxxxxxxx
import cv2 as cv
import threading
from time import sleep
import ipywidgets as widgets
from IPython.display import display
from face_position import Face_Position
from dofbot_utils.robot_controller import Robot_Controller
from dofbot_utils.fps import FPS
from dofbot_utils.dofbot_config import *
from dofbot_utils.camera import create_camera

• Create instances and initialize parameters.

xxxxxxxxxx
robot = Robot_Controller()
robot.move_init_pose()
fps = FPS()
face = Face_Position()
model = 'General'

• Main process.

xxxxxxxxxx
def camera():
    global model
    capture = create_camera()
    while capture.isOpened():
        try:
            _, img = capture.read()
            fps.update_fps()
            img, pos = face.process(img)
            if pos is not None:
                print("x={}, y={}".format(pos[0], pos[1]))
            if model == 'Exit':
                capture.release()
                break
            fps.show_fps(img)
            imgbox.value = cv.imencode('.jpg', img)[1].tobytes()
        except KeyboardInterrupt:capture.release()

• Startup.

display(controls_box,output)
threading.Thread(target=camera, ).start()