Welcome to ROSMASTER A1 repository

Hand detection

1. Content Description

This course implements the function of acquiring color images and using the MediaPipe framework to detect hand joints. This section requires entering commands in the terminal. The appropriate terminal will be selected based on the motherboard type. This section uses a Raspberry Pi 5 as an example.

For Raspberry Pi and Jetson Nano boards, you need to open a terminal on the host computer and enter the command to enter the Docker container. Once inside the Docker container, enter the commands mentioned in this section in the terminal. For instructions on entering the Docker container from the host computer, refer to [01. Robot Configuration and Operation Guide] --[5.Enter Docker (For JETSON Nano and RPi 5)].

For Orin and RDK motherboards, simply open the terminal and enter the commands mentioned in this section.

2. Program Startup

For the Raspberry Pi 5 and jetson nano controller, you must first enter the Docker container. For the Orin and RDK controller, this is not necessary.

Entering a Docker container (see [Docker Course] --- [4. Docker Startup Script] for steps).

All the following commands must be executed within the same Docker container.(See [Docker Course] --- [3. Docker Commit and Multi-Terminal Entry] for steps.)

First, in the terminal, enter the following command to start the camera.

xxxxxxxxxx
#usb camera
ros2 launch usb_cam camera.launch.py
#nuwa camera
ros2 launch ascamera hp60c.launch.py

After successfully starting the camera, open another terminal and enter the following command to start the hand detection program.

xxxxxxxxxx
ros2 run yahboomcar_mediapipe 01_HandDetector

After running the program, the following image appears. The detected hand joints are displayed on the right side of the image.

3. Core Code Analysis

Program Code Path:

• Raspberry Pi 5 and Jetson Nano Board

  The program code is running in Docker. The path in Docker is /root/yahboomcar_ros2_ws/yahboomcar_ws/src/yahboomcar_mediapipe/yahboomcar_mediapipe/01_HandDetector.py

• Orin board

  The program code path is /home/jetson/yahboomcar_ros2_ws/yahboomcar_ws/src/yahboomcar_mediapipe/yahboomcar_mediapipe/01_HandDetector.py

• RDK X5 board

  The program code path is /home/sunrise/yahboomcar_ros2_ws/yahboomcar_ws/src/yahboomcar_mediapipe/yahboomcar_mediapipe/01_HandDetector.py

Import the required library files.

xxxxxxxxxx
import rclpy
from rclpy.node import Node
import mediapipe as mp
import cv2 as cv
import numpy as np
import time
import os
from cv_bridge import CvBridge
from sensor_msgs.msg import Image
from arm_msgs.msg import ArmJoints
import cv2

Initialize data and define publishers and subscribers,

xxxxxxxxxx
def __init__(self):
    super().__init__('hand_detector')
    self.mpHand = mp.solutions.hands
    self.mpDraw = mp.solutions.drawing_utils
    self.hands = self.mpHand.Hands(
        static_image_mode=False,
        max_num_hands=2,
        min_detection_confidence=0.5,
        min_tracking_confidence=0.5
    )
    self.bridge = CvBridge()
    self.publisher_ = self.create_publisher(PointArray, '/mediapipe/points', 10)
    self.lmDrawSpec = mp.solutions.drawing_utils.
 URDF Model
Posture detection