imx219 camera (RDK X5)

The basis of this tutorial is to operate on the image downloaded from the official website of Digua Robot

1. Identify the camera

The development board provides 2 MIPI CSI interfaces (CAM1) to enable the access of MIPI cameras. The camera module is connected to the development board through a flat cable.

Note: It is strictly forbidden to plug and unplug the camera when the development board is not powered off, otherwise it is very easy to burn the camera module.

Wiring diagram:

After the installation is complete, the user can use the i2cdetect command to confirm whether the module's I2C address can be detected normally. If it cannot be detected, the user needs to check whether the FPC cable connection is normal.

When the I2C address is successfully detected, the log print is as follows.

sudo i2cdetect -y -r 6

2. Preview camera video

This sample code opens the mipi camera to obtain video data and displays it in real time through HDMI

Environment preparation:

• Power on RDK X5, the LED indicator light turns red, and wait for the desktop to be displayed. Users can connect to the device through the vnc method introduced in [remote login];
• Connect the MIPI camera supported by RDK X5. The connection of the MIPI camera can refer to the first point.
• Connect the development board and the display through the HDMI cable

Operation method:

xxxxxxxxxx
cd /app/cdev_demo/vio2display
sudo make
sudo ./vio2display -w 1920 -h 1080

Expected results:

The make command completes the compilation of the program, and the ./vio2display command executes the program. After the program runs correctly, the display connected to the HDMI of the X3 Pi will display the real-time image captured by the MIPI camera

Effect picture:

3. Fcos target detection algorithm based on mipi camera

In this example, we want to achieve:

• Load the fcos image target detection algorithm model on RDK X5 (target detection of 80 categories based on COCO dataset training)
• Render the detection results to the picture
• Output the image data to the display through HDMI, and learn how to use the Display function of the Hobot_vio module of the Sweet Potato Robot

Environment preparation:

• Connect the camera module to the MIPI CSI interface of the development board through the FPC cable, and connect the MIPI camera
• Connect the development board and the display through the HDMI cable

Operation method:

xxxxxxxxxx
cd /app/pydev_demo/03_mipi_camera_sample
sudo python3 ./mipi_camera.py 

Expected results:

After the program is executed, the command line will output the version information of the DNN module, the sensor initialization information, and the result output of the visual detection algorithm (target type, confidence), as shown below:

4. Fcos target detection algorithm based on MIPI camera (web display)

Objectives of this example:

• Load the fcos image target detection algorithm model on RDK X5 (target detection based on 80 categories trained with COCO dataset)
• Read video images from MIPI camera (imx219) and learn how to use the Camera function of hobot_vio module of Digua robot
• Display the detection results on the web

Start web_service service:

Before using web display, make sure the board and personal PC are in the same network segment and the PC can ping the IP of the board.

Execute the following command to start the web service

xxxxxxxxxx
cd /app/pydev_demo/05_web_display_camera_sample
sudo sh ./start_nginx.sh
sudo python3 ./mipi_camera_web.py 

Notes:

If the following error is reported when running start_nginx.sh, it means that TCP port 80 is already occupied.

1. Repeated execution of start_nginx.sh causes this. This situation does not need to be handled
2. There is already a service running httpd on the device. You need to find and end the process occupying port 80. Execute the command lsof -i:80 to obtain the process information and its PID of the occupied port, and use kill -9 PID to end the process

View the display effect on the web

Open the browser on your x3 pie and write your own IP address in the address bar to view the model detection results in real time.

Effect picture: