Welcome to DOFBOT-Pi repository

1.Gesture Recognition

Gesture Recognition

1. Gesture Recognition Instructions

This example uses gesture recognition based on Baidu Smart Cloud platform services. This service provides 50,000 free uses per day for learning purposes only. Do not use for commercial purposes. If you need long-term use, please purchase the relevant services. Our company is not responsible for any issues.
API Application Instructions. Official API address reference: https://cloud.baidu.com/doc/BODY/s/ajwvxyyn0. After opening the official API address in a browser, log in with your Baidu account in the upper right corner. If you don't have a Baidu account, please register first.

After logging in, click 'Products' - 'Artificial Intelligence' - 'Face and Body Recognition' - 'Body Analysis' on the menu bar.

Click 'Gesture Recognition' in the product list.

Click Use Now

Next, you will automatically enter the Body Analysis backend management. Click Create Application.

After filling in the relevant information, click Create Now. After creation is complete, click the application list to see the information of the newly created application. At this time, copy and save the information under AppID, API Key and Secret Key.

Note: You need to claim the free recognition quota here. When claiming, you can select all functions

Supported gestures and example images for gesture recognition:

API Functions


from aip import AipBodyAnalysis
""" Your APPID AK SK """
APP_ID = 'Your App ID'
API_KEY = 'Your Api Key'
SECRET_KEY = 'Your Secret Key'
client = AipBodyAnalysis(APP_ID, API_KEY, SECRET_KEY)
""" Read image """
def get_file_content(filePath):
with open(filePath, 'rb') as fp:
return fp.read()
image = get_file_content('example.jpg')
""" Call gesture recognition """
Res = client.gesture(image);

Gesture Recognition Return Data Parameter Details:

Gesture Recognition Return Example:


xxxxxxxxxx
{
"log_id": 4466502370458351471,
"result_num": 2,
"result": [{
"probability": 0.9844077229499817,
"top": 20,
"height": 156,
"classname": "Face",
"width": 116,
"left": 173
},
{
"probability": 0.4679304957389832,
"top": 157,
"height": 106,
"classname": "Heart_2",
"width": 177,
"left": 183
}
}

2. Main Code

Start Docker

After entering the main board desktop, open the terminal and run the following command to start the corresponding Dofbot container:


xxxxxxxxxx
./Docker_Ros.sh

Then enter the following command in the docker terminal to start jupyter-lab,


xxxxxxxxxx
cd
jupyter-lab --allow-root

Then enter the main board's IP address plus :9999 in the browser


xxxxxxxxxx
IP:9999         // Example: 192.168.1.11:9999

Code path: /root/Dofbot/6.AI_Visuall/4.Gesture recognition.ipynb

Import Baidu API, importantly need to change the secret key to the one you applied for.


x
import cv2 
import time
import demjson
import pygame 
from aip import AipBodyAnalysis
from aip import AipSpeech
from PIL import Image, ImageDraw, ImageFont
import numpy
import ipywidgets.widgets as widgets
# Refer to official documentation for specific gestures https://ai.baidu.com/ai-doc/BODY/4k3cpywrv
#Please refer to the official information for specific gestures https://ai.baidu.com/ai-doc/BODY/4k3cpywrv
hand={'One':'Number 1','Two':'Number 2','Three':'Number 3','Four':'Number 4',
      'Five':'Number 5', 'Six':'Number 6','Seven':'Number 7',
      'Eight':'Number 8','Nine':'Number 9','Fist':'Fist','Ok':'OK',
      'Prayer':'Prayer','Congratulation':'Bow','Honour':'Farewell',
      'Heart_single':'Heart','Thumb_up':'Thumbs up','Thumb_down':'Thumbs down',
      'ILY':'I love you','Palm_up':'Palm up','Heart_1':'Double heart 1',
      'Heart_2':'Double heart 2','Heart_3':'Double heart 3','Rock':'Rock',
      'Insult':'Middle finger','Face':'Face'}
# The key below needs to be replaced with your own 
#The key below needs to be replaced with one's own
""" Body Analysis APPID AK SK """
APP_ID = '31069241'
API_KEY = 'pxVueLwdAGX4dafYeLsLdZa1'
SECRET_KEY = 'VsDmfGRlWGqzGhcWowoCT5km4TG4Gylq'
client = AipBodyAnalysis(APP_ID, API_KEY, SECRET_KEY)
g_camera = cv2.VideoCapture(0)
g_camera.set(3, 640)
g_camera.set(4, 480)
g_camera.set(5, 30)  # Set frame rate
g_camera.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter.fourcc('M', 'J', 'P', 'G'))
g_camera.set(cv2.CAP_PROP_BRIGHTNESS, 40) # Set brightness -64 - 64  0.0
g_camera.set(cv2.CAP_PROP_CONTRAST, 50) # Set contrast -64 - 64  2.0
g_camera.set(cv2.CAP_PROP_EXPOSURE, 156) # Set exposure value 1.0 - 5000  156.0
ret, frame = g_camera.read()

Camera display component


xxxxxxxxxx
image_widget = widgets.Image(format='jpeg', width=600, height=500)  # Set camera display component
display(image_widget)   
image_widget.value = bgr8_to_jpeg(frame)

Main display and result display program.


xxxxxxxxxx
try:
    while True:
        """1. Take photo """
        ret, frame = g_camera.read()
        #image = get_file_content('./image.jpg')
        """ 2. Call gesture recognition """
        raw = str(client.gesture(image_widget.value))
        text = demjson.decode(raw)
        try:
            res = text['result'][0]['classname']
        except:
    #         print('识别结果：什么也没识别到哦~' )
    #         img = cv2ImgAddText(frame, "未识别", 250, 30, (0, 0 , 255), 30)
            img = frame
        else:
    #         print('识别结果：' + hand[res])
    #         img = cv2ImgAddText(frame, hand[res], 250, 30, (0, 255 , 0), 30)
            if res == 'Prayer': # 1 祈祷
                print('识别结果：' + hand[res])
                img = cv2ImgAddText(frame, hand[res], 250, 30, (0, 255 , 0), 30)
              
            elif res == 'Thumb_up':# 2 点赞
                print('识别结果：' + hand[res])
                img = cv2ImgAddText(frame, hand[res], 250, 30, (0, 255 , 0), 30)
               
            elif res == 'Ok': # 3 OK
                print('识别结果：' + hand[res])
                img = cv2ImgAddText(frame, hand[res], 250, 30, (0, 255 , 0), 30)
              
            elif res == 'Heart_single': # 4 单手比心
                print('识别结果：' + hand[res])
                img = cv2ImgAddText(frame, hand[res], 250, 30, (0, 255 , 0), 30)
               
            elif res == 'Five': # 5 数字5
                print('识别结果：' + hand[res])
                img = cv2ImgAddText(frame, hand[res], 250, 30, (0, 255 , 0), 30)
               
            elif res == "Eight": # 数字8
                print('识别结果：' + hand[res])
                img = cv2ImgAddText(frame, hand[res], 250, 30, (0, 255 , 0), 30)
                
            elif res == "Rock": # rock
                print('识别结果：' + hand[res])
                img = cv2ImgAddText(frame, hand[res], 250, 30, (0, 255 , 0), 30)
                
            elif res == "Congratulation": # 作揖
                print('识别结果：' + hand[res])
                img = cv2ImgAddText(frame, hand[res], 250, 30, (0, 255 , 0), 30)
              
            elif res == "Seven": # 数字7
                print('识别结果：' + hand[res])
                img = cv2ImgAddText(frame, hand[res], 250, 30, (0, 255 , 0), 30)
               
            elif res == "Thumb_down": # 拇指向下
                print('识别结果：' + hand[res])
                img = cv2ImgAddText(frame, hand[res], 250, 30, (0, 255 , 0), 30)                
            else:
                img = frame
        image_widget.value = bgr8_to_jpeg(img)
except KeyboardInterrupt:
    print(" Program closed! ")
    pass

After running the code block, you can see the recognition screen. Note: If recognition fails, apply for your own secret key from Baidu.

USB camera display

Color threshold adjustment&Color calibration