License Plate Recognition

Routine Experiment Effect

In this section, we will learn how to use K230 to realize the license plate recognition function. License plate recognition is based on the license plate detection we talked about in the previous section. When we detect that there is a license plate on the screen, we regard the area of the license plate as the ROI area that needs to be recognized.

Let’s use the picture from the previous section as an example:

The current routine has added serial port output

For the protocol format, please refer to [Routine Communication Protocol.xlsx] in the document.

Code Explanation

Code structure

1. Initialization phase:

   • Loading detection and recognition models
   • Initialize the character dictionary
   • Setting up the AI2D Processor
   • Configuring Threshold Parameters

2. Main processing flow:

   • Detection stage: Detect the license plate position and crop
   • Recognition stage: preprocessing, model reasoning, character recognition
   • Display stage: drawing detection boxes and recognition results

3. Other treatments:

   • Resource Management
   • Exit Check
   • Results

Code Analysis

For the complete code, please refer to the file [Source Code/09.Scene/07.licence_det_rec.py]

xxxxxxxxxx
class LicenceRec:
    """
    车牌检测和识别的整合类，包含检测和识别两个功能模块
    Integrated class for license plate detection and recognition, including detection and recognition modules
    """
    def __init__(self, licence_det_kmodel, licence_rec_kmodel, det_input_size, rec_input_size, 
                 confidence_threshold=0.25, nms_threshold=0.3, rgb888p_size=[1920,1080], 
                 display_size=[1920,1080], debug_mode=0):
        """
        初始化函数
        Initialization function
​
        参数说明 Parameters:
        licence_det_kmodel: 车牌检测模型路径 Path to license plate detection model
        licence_rec_kmodel: 车牌识别模型路径 Path to license plate recognition model
        det_input_size: 检测模型的输入尺寸 Input size for detection model
        rec_input_size: 识别模型的输入尺寸 Input size for recognition model
        confidence_threshold: 置信度阈值 Confidence threshold for detection
        nms_threshold: 非极大值抑制阈值 Non-maximum suppression threshold
        rgb888p_size: 输入图像尺寸 Input image size
        display_size: 显示尺寸 Display size
        debug_mode: 调试模式 Debug mode
        """
        # 初始化成员变量 Initialize member variables 
        self.licence_det_kmodel = licence_det_kmodel
        self.licence_rec_kmodel = licence_rec_kmodel
        self.det_input_size = det_input_size
        self.rec_input_size = rec_input_size
        self.confidence_threshold = confidence_threshold
        self.nms_threshold = nms_threshold
        
        # 确保图像宽度是16的倍数 Ensure image width is multiple of 16
        self.rgb888p_size = [ALIGN_UP(rgb888p_size[0],16), rgb888p_size[1]]
        self.display_size = [ALIGN_UP(display_size[0],16), display_size[1]]
        self.debug_mode = debug_mode
​
        # 初始化检测和识别模型 Initialize detection and recognition models
        self.licence_det = LicenceDetectionApp(
            self.licence_det_kmodel,
            model_input_size=self.det_input_size,
            confidence_threshold=self.confidence_threshold,
            nms_threshold=self.nms_threshold,
            rgb888p_size=self.rgb888p_size,
            display_size=self.display_size,
            debug_mode=0)
        self.licence_rec = LicenceRecognitionApp(
            self.licence_rec_kmodel,
            model_input_size=self.rec_input_size,
            rgb888p_size=self.rgb888p_size)
            
        # 配置检测模型的预处理 Configure preprocessing for detection model
        self.licence_det.config_preprocess()
​
    def run(self, input_np):
        """
        执行车牌检测和识别的主要流程
        Main pipeline for license plate detection and recognition
        
        参数 Parameters:
        input_np: 输入图像(numpy array格式) Input image in numpy array format
        
        返回 Returns:
        det_boxes: 检测到的车牌位置 Detected license plate positions
        rec_res: 识别的车牌字符 Recognized license plate characters
        """
        # 执行车牌检测 Perform license plate detection
        det_boxes = self.licence_det.run(input_np)
        
        # 对检测到的区域进行预处理 Preprocess detected regions
        imgs_array_boxes = aidemo.ocr_rec_preprocess(
            input_np,
            [self.rgb888p_size[1], self.rgb888p_size[0]],
            det_boxes)
        imgs_array = imgs_array_boxes[0]
        boxes = imgs_array_boxes[1]
        
        # 对每个检测到的车牌进行识别 Recognize each detected license plate
 License plate detection
Target Tracking