k230 gaze direction

k230 and PICO2 communication

1. Experimental Prerequisites

This tutorial uses the PICO2 development board, and the corresponding routine path is [14.export\PICO-K230\07_pico_k230_eye_gaze.py].

K230 needs to run the [14.export\CanmvIDE-K230\07.eye_gaze.py] program to start the experiment. It is recommended to download it as an offline program.

Things you need:

Windows computer PICO2 development board microUSB cable K230 visual module (including TF card with image burned in) type-C cable connection cable

2. Experimental wiring

3. Main code explanation

​x
from machine import UART, Pin
​
FUNC_ID = 7
​
​
uart1 = UART(1, baudrate=115200, tx=Pin(8), rx=Pin(9), bits=8, parity=None, stop=0)
​
print("hello yahboom")
​
​
def parse_data(data):
    if data[0] == ord('$') and data[len(data)-1] == ord('#'):
        data_list = data[1:len(data)-1].decode('utf-8').split(',')
        data_len = int(data_list[0])
        data_id = int(data_list[1])
        if data_len == len(data) and data_id == FUNC_ID:
            # print(data_list)
            x0 = int(data_list[2])
            y0 = int(data_list[3])
            x1 = int(data_list[4])
            y1 = int(data_list[5])
            return x0, y0, x1, y1
        elif (data_len != len(data)):
            print("data len error:", data_len, len(data))
        elif(data_id != FUNC_ID):
            print("func id error:", data_id, FUNC_ID)
    return -1, -1, -1, -1
​
last_data = bytearray()
while True:
    if uart1.any() > 0:
        cur_data = uart1.readline()
        # print("rx:", cur_data)
        if ord('\n') in cur_data:
            # data = bytearray(last_data + cur_data.decode('utf-8'), 'utf-8')
            data = last_data + cur_data
            last_data = bytearray()
            x0, y0, x1, y1 = parse_data(data.rstrip(b'\n'))
            print("eye:x0:%d, y0:%d, x1:%d, y1:%d" % (x0, y0, x1, y1))
        else:
            last_data = last_data + cur_data

The above program is for parsing K230 data. Only when it complies with specific protocols can the corresponding data be parsed.

in

• x0: is the horizontal coordinate of the eye starting point
• y0: is the vertical coordinate of the eye starting point
• x1: is the horizontal coordinate of the gaze direction
• y1: is the vertical coordinate of the gaze direction

4. Experimental Phenomenon

1. After connecting the cables, the k230 visual module runs offline. After K230 is connected to Canmv IDE, open the corresponding program, click [Save open script to CanMV board (as main.py)] on the toolbar, and then restart K230.

2. Open the Thonny editor, connect the PICO2 mainboard, open the program file and run it.

   Note: The PICO2 mainboard needs to have the microPython firmware downloaded in advance.

3. When the K230 camera image recognizes a face, the terminal will parse and print out the information transmitted by the K230.

in

• x0: is the horizontal coordinate of the eye starting point

• y0: is the vertical coordinate of the eye starting point

• x1: is the horizontal coordinate of the gaze direction

• y1: is the vertical coordinate of the gaze direction

  As shown in the figure below