k230 gesture recognition

K230 and Raspberry Pi communication

1. Experimental Prerequisites

This tutorial uses the Raspberry Pi 5 development board, and the corresponding routine path is [14.export\Raspberrypi-K230\12_k230_gesture.py].

K230 needs to run the [14.export\CanmvIDE-K230\12.gesture.py] program to start the experiment. It is recommended to download it as an offline program.

Things you need:

Windows computer Raspberry Pi 5 development board microUSB cable K230 visual module (including TF card with image burned in) type-C cable connection cable

2. Experimental wiring

3. Main code explanation

​x
import serial
​
com="/dev/ttyUSB0"
ser = serial.Serial(com, 115200)
​
FUNC_ID = 12
​
def parse_data(data):
    if data[0] == ord('$') and data[len(data)-1] == ord('#'):
        data_list = data[1:len(data)-1].decode('utf-8').split(',')
        data_len = int(data_list[0])
        data_id = int(data_list[1])
        if data_len == len(data) and data_id == FUNC_ID:
            # print(data_list)
            gesture = data_list[2]
            return gesture
        elif (data_len != len(data)):
            print("data len error:", data_len, len(data))
        elif(data_id != FUNC_ID):
            print("func id error:", data_id, FUNC_ID)
    else:
        print("pto error", data)
    return ""
​
while True:
    if ser.in_waiting:
        data = ser.readline()
        # print("rx:", data)
        gesture = parse_data(data.rstrip(b'\n'))
        print("gesture:", gesture)

The above program is for parsing K230 data. Only when it complies with specific protocols can the corresponding data be parsed.

in

• gesture: msg is the gesture information, 'UP', 'DOWN', 'LEFT', 'RIGHT', 'MIDDLE'

4. Experimental Phenomenon

1. After connecting the cables, the k230 visual module runs offline. After K230 is connected to Canmv IDE, open the corresponding program, click [Save open script to CanMV board (as main.py)] on the toolbar, and then restart K230.

2. Transfer the program file to the system, open the terminal and enter the corresponding directory, then run the following command to start the program.

xxxxxxxxxx
python3 12_k230_gesture.py

3. When the K230 camera image recognizes a palm and the palm makes a corresponding gesture, the terminal will parse and print out the information transmitted by the K230.

   in

• gesture: gesture information, 'UP', 'DOWN', 'LEFT', 'RIGHT', 'MIDDLE'

  As shown in the figure below