Welcome to RPi AI Camera repository
2. Object Detection
2. Object Detection
2. Object Detection
This section mainly addresses the problem of how to use the dnn module in OpenCV to import a trained object detection network. However, there are requirements for the version of opencv.
Currently, there are three main methods for object detection using deep learning:
Faster R-CNNs
You Only Look Once (YOLO)
Single Shot Detectors (SSDs)
Faster R-CNNs is the most commonly heard neural network based on deep learning. However, this method is technically difficult to understand (especially for deep learning novices), difficult to implement, and difficult to train.
In addition, even if the "Faster" method is used to implement R-CNNs (here R stands for Region Proposal), the algorithm is still relatively slow, about 7FPS.
If we pursue speed, we can turn to YOLO, because it is very fast, reaching 40-90 FPS on TianXGPU, and the fastest version may reach 155 FPS. But the problem with YOLO is that its accuracy needs to be improved.
SSDs was originally developed by Google and can be said to be a balance between the above two. Compared with Faster R-CNNs, its algorithm is more direct. Compared with YOLO, it is more accurate.
2.1, Model structure
The main work of MobileNet is to replace the past standard convolutions with depthwise sparable convolutions to solve the problems of computational efficiency and parameter quantity of convolutional networks. The MobileNets model is based on depthwise sparable convolutions, which can decompose standard convolutions into a depth convolution and a point convolution (1 × 1 convolution kernel). Deep convolution applies each convolution kernel to each channel, while 1 × 1 convolution is used to combine the output of channel convolution.
Batch Normalization (BN) is added to the basic components of MobileNet, that is, at each SGD (stochastic gradient descent), the standardization process is performed so that the mean of the result (each dimension of the output signal) is 0 and the variance is 1. Generally, when you encounter slow convergence or gradient explosion during neural network training, you can try to solve the problem. In addition, in general use, you can also add BN to speed up training and improve model accuracy.
In addition, the model also uses the ReLU activation function, so the basic structure of the depthwise separable convolution is shown in the figure below:
The MobileNets network is composed of many depthwise separable convolutions shown in the figure above. The specific network structure is shown in the figure below:
2.2, Code Analysis
List of Recognizable Objects
xxxxxxxxxx[person, bicycle, car, motorcycle, airplane, bus, train,truck, boat, traffic light, fire hydrant, street sign,stop sign, parking meter, bench, bird, cat, dog, horse,sheep, cow, elephant, bear, zebra, giraffe, hat, backpack,umbrella, shoe, eye glasses, handbag, tie, suitcase,frisbee, skis, snowboard, sports ball, kite, baseball bat,baseball glove, skateboard, surfboard, tennis racket,bottle, plate, wine glass, cup, fork, knife, spoon, bowl,banana, apple, sandwich, orange, broccoli, carrot, hot dog,pizza, donut, cake, chair, couch, potted plant, bed, mirror,dining table, window, desk, toilet, door, tv, laptop, mouse,remote, keyboard, cell phone, microwave, oven, toaster,sink, refrigerator, blender, book, clock, vase, scissors,teddy bear, hair drier, toothbrush]Load category [object_detection_coco.txt], import model [frozen_inference_graph.pb], specify deep learning framework [TensorFlow]
xxxxxxxxxx# Load COCO class nameswith open('object_detection_coco.txt', 'r') as f: class_names = f.read().split('\n')# Display different colors for different targetsCOLORS = np.random.uniform(0, 255, size=(len(class_names), 3))# Load DNN image modelmodel = cv.dnn.readNet(model='frozen_inference_graph.pb', config='ssd_mobilenet_v2_coco.txt', framework='TensorFlow')Import the image, extract the height and width, calculate the 300x300 pixel blob, and pass this blob into the neural network
xxxxxxxxxxdef Target_Detection(image):image_height, image_width, _ = image.shape# Create a blob from the imageblob = cv.dnn.blobFromImage(image=image, size=(300, 300), mean=(104, 117, 123), swapRB=True)model.setInput(blob)output = model.forward()# Iterate over each detectionfor detection in output[0, 0, :, :]:# Extract the confidence of the detectionconfidence = detection[2]# Draw bounding box only if detection confidence is above a certain threshold, skip otherwiseif confidence > .4:# Get class IDclass_id = detection[1]# Map class id to classclass_name = class_names[int(class_id) - 1]color = COLORS[int(class_id)]# Get bounding box coordinatesbox_x = detection[3] * image_widthbox_y = detection[4] * image_height# Get bounding box width and heightbox_width = detection[5] * image_widthbox_height = detection[6] * image_height# Draw a rectangle around each detected objectcv.rectangle(image, (int(box_x), int(box_y)), (int(box_width), int(box_height)), color, thickness=2)# Write class name text on detected objectscv.putText(image, class_name, (int(box_x), int(box_y - 5)), cv.FONT_HERSHEY_SIMPLEX, 1, color, 2)return image2.3, Startup
Note: The AI camera in this case does not have a computing power bonus, and is called as a normal camera!