Face Effects

1. Experiment Objective

Use the dlib 68-point facial landmark detection model to add AR effects such as eye filling, lip beautification and eyebrow highlighting to the face.

2. Experiment Principle

1. Key Terms

2. Technical Architecture

Terminal 1: start agent + camera + joint models
Terminal 2: start face effects

3. Experiment Steps

1. Hardware Preparation

2. Start the agent + camera + joint models (Terminal 1)

xxxxxxxxxx
ros2 launch microros_v2_bringup car_base.launch.py

3. Start face effects (Terminal 2)

xxxxxxxxxx
ros2 launch microros_v2_mediapipe_pkg face_landmarks.launch.py

4. Observed Results

• The face is annotated with 68 cyan landmarks
• The eye region is filled with semi-transparent blue, and the lip region is beautified with a pink overlay
• The eyebrow region is highlighted

5. Code Walkthrough

Source code paths:

• Launch: ~/microros_ws/src/microros_v2_mediapipe_pkg/launch/face_landmarks.launch.py
• Node: ~/microros_ws/src/microros_v2_mediapipe_pkg/microros_v2_mediapipe_pkg/FaceLandmarks.py

1. Core Node Analysis

​x
# Source file: ~/microros_ws/src/microros_v2_mediapipe_pkg/microros_v2_mediapipe_pkg/FaceLandmarks.py
class FaceLandmarks(Node):
    def __init__(self):
        super().__init__("face_landmarks_node")
        self.detector = dlib.get_frontal_face_detector()
        self.predictor = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")
​
    def image_callback(self, msg):
        frame = self.bridge.imgmsg_to_cv2(msg, 'bgr8')
        gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)
        faces = self.detector(gray)
        for face in faces:
            landmarks = self.predictor(gray, face)
            # Eye ROI (landmarks 36-47)
            eye_pts = np.array([[landmarks.part(n).x, landmarks.part(n).y]
                for n in range(36, 48)], np.int32)
            cv.fillPoly(overlay, [eye_pts], (255, 0, 0))
            # Lip ROI (landmarks 48-67)
            lip_pts = np.array([[landmarks.part(n).x, landmarks.part(n).y]
                for n in range(48, 68)], np.int32)
            cv.fillPoly(overlay, [lip_pts], (255, 0, 255))
        frame = cv.addWeighted(frame, 0.7, overlay, 0.3, 0)
        cv.imshow(self.window_name, frame)

Key logic:

• dlib.shape_predictor("shape_predictor_68_face_landmarks.dat"): loads the 68-point model file
• Eye landmark indices 36-47 (12 points, 6 per eye); lip indices 48-67 (20 points)
• cv.fillPoly draws filled polygons on the overlay layer, and cv.addWeighted blends them with the original image at 0.3 opacity
• dlib 68 points vs MediaPipe Face Mesh 468 points: dlib is lighter and starts faster; the 468-point mesh is finer but more computationally expensive

2. Key Parameters

Parameter definition files:

• Launch: ~/microros_ws/src/microros_v2_mediapipe_pkg/launch/face_landmarks.launch.py
• Node: ~/microros_ws/src/microros_v2_mediapipe_pkg/microros_v2_mediapipe_pkg/FaceLandmarks.py

3. Node Description

6. Frequently Asked Questions