Gesture following

1. Experimental purpose

Drive the car to follow gestures

2. Experimental path source code

Enter the car system, end the car program, enter "ip (ip is the car's ip): 8888" in the browser, enter the password "yahboom" Then log in Enter the path of Rider-pi_class/6.AI Visual Interaction Course/1. Color tracking and run color.ipynb.

3. Experimental phenomenon

After running the source code, the car can squat and move left and right following the gestures. Clenching a fist means stopping the operation, and other gestures will follow the movement. This case will be a bit stuck because the running memory of the car is only 2GB. The car cannot be in a stopped state (that is, not a standing balance state), otherwise it cannot move.

Follow gesture:

Stop following gesture:

4. Analysis of main source code parameters

​x
def Gesture_follow():
    try:
        while True:
            global bot
            ret, frame = cap.read()
            img_height, img_width, _ = frame.shape
            hand_detector.findHands(frame, draw=False) 
            if len(hand_detector.lmList) != 0:
                # 转向控制部分
                # Turning control section
                # MediaPipe中, 手部最中心的指关节的编号为9
                # In MediaPipe, the index of the central finger joint is 9
                x,y = hand_detector.findPoint(9)
                cv2.circle(frame,(int(x),int(y)),2,(0,255,255),6)
​
                value_x = x - 160
                value_y = y - 120
                rider_x=value_x
                if value_x > 55:
                    value_x = 55
                elif value_x < -55:
                    value_x = -55
                if value_y > 75:
                    value_y = 75
                elif value_y < -75:
                    value_y = -75
​
                if dog_type=='L' or dog_type=='M':
                    g_car.attitude(['y','p'],[-value_x/15, value_y/15])
                elif dog_type=='R':
                    #print(y,75+int((190-y)/160*40))
                    g_car.rider_height(75+int((190-y)/160*40))
                    if rider_x==9999:
                        g_car.rider_turn(0)
                    else:
                        if rider_x > 35:
                            g_car.rider_turn(-25)
                        elif rider_x < -35:
                            g_car.rider_turn(25)
                        else:
                            g_car.rider_turn(0)
​
    
                # 前进控制部分
                # Forward control section
                finger_number = hand_detector.get_gesture()
                finger_str=f"Number:{finger_number}"
    
                if(finger_number == "Zero"):
                    g_car.rider_turn(0)
​
            else:
                x = 0
                y = 0
                rider_x=9999
                g_car.rider_turn(0)
​
            try:
                #图片显示在lcd屏上
                #The picture is displayed on the LCD screen
                image_widget.value = bgr8_to_jpeg(frame)
                b,g,r = cv2.split(frame)
                frame = cv2.merge((r,g,b))
                frame = cv2.flip(frame, 1)
                imgok = Image.fromarray(frame)
                mydisplay.ShowImage(imgok)
​
               
            except:
                continue
    finally:
        cap.release()

From the source code analysis, we can get that the car will follow the gestures according to the camera. If the gesture is upward, the car will stand up; if the gesture is downward, the car will squat; if the gesture is left or right, the car will also move left or right.