Welcome to MicroROS-Pi5 repository

Gesture control of basic movement of the car

Note: The ROS_DOMAIN_ID of the Raspberry Pi and the microROS control board need to be consistent. You can check [MicroROS Control Board Parameter Configuration] to set the microROS control board ROS_DOMAIN_ID. Check the tutorial [Connect MicroROS Agent] to determine whether the IDs are consistent.

1、Program function description

After the function is turned on, the camera captures images and recognizes gestures to control the movement of the car.

MediaPipe Hands infers the 3D coordinates of 21 hand-valued joints from a frame

2、Program code reference path

After entering the docker container, the location of the source code of this function is as follows

/root/yahboomcar_ws/src/yahboomcar_mediapipe/yahboomcar_mediapipe/

3、Program start

3.1、Start command

After entering the docker container, enter the following command in the terminal.

xxxxxxxxxx
ros2 run yahboomcar_mediapipe HandCtrl

Enable this function, then put your hand in front of the camera, and the shape of the finger will be drawn on the screen. After the program recognizes the gesture, it will send the speed to the chassis to control the movement of the car.

4、core code

4.1、HandCtrl.py

• Import key library

  x
  from media_library import * #This library contains functions such as detecting hands and obtaining gestures.
  

• Get finger data

  xxxxxxxxxx
  frame, lmList, _ = self.hand_detector.findHands(frame)
  fingers = self.hand_detector.fingersUp(lmList)
  sum(fingers)
  fingers[]
  

  It can be seen that the hand is detected first, the value of lmList is obtained, and then passed into the fingersUp function. The fingersUp function is used to detect which fingers are straight. The value of straight fingers is 1. The specific code here can also be seen in media_library, py function. There is a detailed explanation in it. In fact, it is judged by judging the xy value of the finger joint. Time to straighten. The sum(fingers) function is used to calculate the number of straightened fingers. fingers[] can be used to enumerate fingers. For example, the index finger is represented by fingers[1].

• Release speed to chassis

  x
  self.media_ros.pub_vel(x,y,z)   #This function is also in media_library,py
  

4.2、flow chart