K210 Scenario-Based Autonomous Driving

This case requires the use of an eight-channel intelligent line-following module (infrared line-following module)

Serial Communication

USART: Universal Synchronous Asynchronous Receiver/Transmitter. It can automatically generate data frame timing based on a byte of data in the data register and send it out from the TX pin. It can also automatically receive data frame timing from the RX pin, concatenate them into a byte of data, and store it in the data register. This experiment uses UART2.

When using serial communication, both communicating parties must be "on the same channel." "On the same channel" refers to using the same communication protocol. USART (Ultra-Mobile Serial Port) Conventions: During communication, data must be transmitted in "frames." A single frame of data in a USART port includes: Start Bit + Data Bits + Parity Bit + Stop Bit. Where: 1) Start Bit: Fixed as a low-level signal for one cycle. 2) Data Bits: Can be agreed upon by both communicating parties, ranging from 5 to 9 bits. 3) Parity Bit: The USART port uses parity checking, which can be agreed upon by both communicating parties. 4) Stop Bit: Optional high-level signal for 0.5 to 2 cycles. Simultaneously, to synchronize the transmission and reception speeds of both parties, the number of data frames transmitted per second needs to be agreed upon, called the baud rate. Typical baud rates include 9600, 115200, 57600, etc.

K210 Vision Module

The Kendrick K210 is a system-on-a-chip (SoC) integrating machine vision and machine hearing capabilities. It utilizes TSMC's ultra-low-power 28nm advanced process, features a dual-core 64-bit processor with a total computing power of up to 1 TOPS, and incorporates various hardware acceleration units (KPU, FPU, FFT, etc.), while also boasting excellent power efficiency, stability, and reliability.

Functional Distribution as follows:

Hardware Connection

K210 Protocol

After the K210 identifies a road sign, it sends a data frame via serial port in the format "$09" + "signature content ID," + "#". For example: "$091, #".

When the microcontroller receives this data frame, it checks each bit. If the content is "$09", it continues receiving. When it receives "#", it indicates that a data packet has been completed. The microcontroller parses the information transmitted by the K210 and compares the signature content ID with a specific command. If it matches the command, the vehicle will perform the corresponding operation.

Control Flowchart

Partial Code Analysis

bsp_k210.c

xxxxxxxxxx
void data_deal_k210(uint8_t data) {
    
    // Process data sent by K210 device, parse single-byte data according to specific format
    switch(data_step) {
        case 0:
            
            // Reset all states to ensure a clean start
            len = 0;
            memset(k210_data, 0, K210_DATA_MAX_LEN);
            if(data == '$') {
               
                // Detected start character '$', move to next state
                data_step++;
            }
            break;
        
        case 1:
            if(data == '0') {
             
                // Matched second feature character '0', move to next state
                data_step++;
            } else {
                
                // Reset state if not matched
                data_step = 0;
            }
            break;
        
        case 2:
            if(data == '9') {
                
                // Matched third feature character '9', move to next state
                data_step++;
            } else {
                
                // Reset state if not matched
                data_step = 0;
            }
            break;
        
        case 3:  
                 // Receiving x value (numeric digits)
            if(data == ',') {  
                               // End of x reception, enter end character verification phase
                if(len == 0 || len > K210_DATA_MAX_LEN) {
                   
                    // x is empty or exceeds max length, invalid data
                    data_step = 0;
                } else {
                   
                    // Valid data length, enter end character waiting state
                    data_step++; 
                                  // Move to state 4 waiting for '#'
                }
            } else if(data >= '0' && data <= '9') {  
                                                    // Only receive numeric digits
                if(len < K210_DATA_MAX_LEN) {  
                                               // Limit max length to prevent overflow
                    k210_data[len++] = data;
                } else {
                    
                    // Exceeds max length, treat as invalid data
                    data_step = 0;
                    len = 0;
                    memset(k210_data, 0, K210_DATA_MAX_LEN);
                }
            } else {
                
                // Received non-numeric and non-',' character, invalid data
                data_step = 0;
                len = 0;
                memset(k210_data, 0, K210_DATA_MAX_LEN);
            }
            break;
        
        case 4:  
                 // Verify end character '#'
            if(data == '#') {  
                               // Complete format match
                deal_msg(len);  
                                // Process received valid message
            }
            
            // Reset state regardless of whether '#' is received (to avoid residue)
            data_step = 0;
            len = 0;
            memset(k210_data, 0, K210_DATA_MAX_LEN);
            break;
    }
}
​
void deal_msg(uint8_t length) {
    
    // Process parsed message, convert digits and verify range
    data_id = 0;
    if(length == 1) {
       
        // Single-byte digit conversion
        data_id = k210_data[0] - '0';
    } else if(length == 2) {
       
        // Two-byte digit conversion (tens place + units place)
        uint8_t data_H = k210_data[0] - '0';
        uint8_t data_L = k210_data[1] - '0';
        data_id = data_H * 10 + data_L;
    }
    
    
    // Verify if data_id is within 1-11 range
    if(data_id >= 1 && data_id <= MAX_ID) {
        stop_flag = 0;  
                        // Valid data, normal processing
    } else {
        stop_flag = 1;  
                        // Invalid data, set stop flag
    }
}

main.py

...
​
# Model selection: 1-tinybit_AI_01.kmodel 2-tinybit_AI_02.kmodel 3-M0_AI_03.kmodel
kmdel = 3 
...init...
​
# Label name list (order cannot be modified, must match training)
label_num = ["red","green","school","walk","one","right","two","freeSpeed","left","limitSpeed","horn"]
​
​
# Configure parameters according to selected model
...
elif kmdel == 3 :
    labels = ["one","two","freeSpeed","green","horn","left","limitSpeed","red","right"]
    anchor = (1.78, 1.61, 2.03, 2.03, 2.22, 2.19, 2.50, 2.47, 3.19, 3.66)
    kpu.load_kmodel('/sd/M0_AI_03.kmodel')  # 加载模型 Load model
​
​
# Initialize YOLOv2 model parameters
kpu.init_yolo2(anchor, 
               anchor_num=(int)(len(anchor)/2), 
               img_w=320, 
               img_h=240, 
               net_w=320, 
               net_h=240, 
               layer_w=10, 
               layer_h=8, 
               threshold=0.4, 
               nms_value=0.3, 
               classes=len(labels))
​
msg_ = ""  # Recognition result message variable
​
while(True):
    gc.collect()  #  Clear memory
​
    clock.tick()  #  Update clock
    img = sensor.snapshot()  #  Capture a frame of image
​
   
    # Run KPU for object detection
    kpu.run_with_output(img)
    dect = kpu.regionlayer_yolo2()  #  Get detection results
​
    fps = clock.fps()  #  Get FPS
​
    
    # If targets are detected
    if len(dect) > 0:
        for l in dect :
            
            # Draw target rectangle on image
            img.draw_rectangle(l[0],l[1],l[2],l[3],color=(0,255,0))
           
            # Display label and confidence
            info = "%s %.3f" % (labels[l[4]], l[5])
            img.draw_string(l[0],l[1],info,color=(255, 255, 0),scale=1.5)
            print(info)
            del info  #  Free variable memory
​
          ）
            # Convert label to corresponding ID (starting from 1)
            msg_ = labels[l[4]]
            for i in range(len(label_num)):
                if msg_ == label_num[i]:
                    idd = str(i+1)  # ID从1开始 ID starts from 1
                    print(idd)
                    break
​
    
    # If targets are detected, send data to lower computer
    if len(dect) > 0:
       send_data ="$"+"09"+ idd+','+"#"  # Data format: $09+ID+,#
       time.sleep_ms(5)  #  Short delay
       serial.send(send_data)  #  Send data
   # else :
    #    serial.send("#")  # Send empty command when no result (commented备用)
​
    
    # Display FPS on LCD
    img.draw_string(0, 0, "%2.1ffps" %(fps),color=(0,60,255),scale=2.0)
    lcd.display(img)  #  Display image

Main Functions

data_deal_k210

deal_msg

set_dataid

get_dataid

Reverse_parking_no2

Reverse_parking_no1

Road_sign_right

Road_sign_left

Road_sign_speedlimit

LineWalking_PWM

LineCheck

Experimental Phenomena

The road signs placed on the map in this tutorial are shown in the image below. After successfully downloading the program, place the car on the map, turn on the switch, and wait a while until the K210 screen lights up. The car will then begin line-following.

Upon detecting a speed limit sign, the car will move at a speed not exceeding the speed limit sign.

Upon detecting an end-of-speed-limit sign, the vehicle resumes normal lane-following movement.

Upon detecting a left/right turn, the vehicle prioritizes the left/right turn.

Upon detecting a horn, the vehicle stops, sounds the horn twice, and then resumes normal lane-following movement.

Upon recognizing traffic lights, the vehicle stops at red lights and proceeds at green lights.

Regarding parking spaces 1 and 2, if the vehicle recognizes space 1, it will enter space 1; if it recognizes space 2, it will reverse into space 2.