Collect data

Note: This course is only applicable to the R2L car and Yahboom's autopilot map.

If you use other models or other maps, you need to develop and debug the code by yourself, and the code provided in this chapter cannot be used directly.

1. Install the cvui library

Open the system terminal and enter the following command to install the cvui library.

sudo pip3 install cvui

2. Code analysis

Code path:Rosmaster/auto_drive/data_collection.py

1. Initialize the cvui library. The cvui library is based on the image operation library of opencv and can display multiple windows.

2. Create a new data table named road_following_A/B, which is used to store data and picture information. A/B are two independent data sets, as long as A is used in practical applications.

3. Initialize the camera, where 0 in cv2.VideoCapture(0) is the device number of the camera /dev/video0. Generally, it is 0 when only one camera is connected,if multiple cameras are connected, obtain the camera number by searching ls /dev/video*. Since the image size required by the model dataset is 224*224, it is necessary to modify the image size to 224*224.

4. Initialize the data table and print the number of currently existing data pictures.

5. Create a while loop to read the camera image and display it on the cvui window. Then listen for mouse click events.

6. When it is detected that the left mouse button is clicked on the window, the onMouse function event is triggered. If it is judged to be in the camera area, record the X Y coordinates of the marked point and save the image.

7. Save the current camera picture to the local and data table, and print out the current X Y value and the number of data pictures at the same time.

3. Start collecting data

Note: Since opencv needs to open the camera screen, it needs to run on the actual desktop. It can be run on the remote VNC desktop, but it cannot be run by remote SSH login. Please close other programs that open the camera before running the command, otherwise conflicts and errors will be reported.

Run the following commands

xxxxxxxxxx
cd ~/Rosmaster/auto_drive
python3 data_collection.py

If you use the Jetson Xavier NX master car, due to internal multi-version issues, you need to use python3.7 to run the program.

xxxxxxxxxx
cd ~/Rosmaster/auto_drive
python3.7 data_collection.py

At this point, the number of currently existing image datasets will be printed out, and one window and two screens will be displayed. The picture on the left is the real-time display picture of the camera, and the picture on the right is the image picture captured after the mouse is clicked, and a green circle with XY coordinates is added. Every time you click on the left camera screen, the count_value will automatically increase by 1 and save the picture and coordinate information.

Note: When collecting data sets, you need to put the car on the map, and click on the optimal path point on the camera screen of the car to push the car slowly along the track, and record every small step, and record the entire map. The data set can be recorded multiple times to improve the recognition accuracy. You can also open another terminal to run the handle control program, and use the handle to control the movement of the car.

xxxxxxxxxx
cd ~/Rosmaster/auto_drive/
python3 joystick_R2L.py

Finally, press the keyboard q key to exit the data collection function.

4. View data pictures

Picture generation path:

xxxxxxxxxx
Rosmaster/auto_drive/road_following_A/apex

Note: If you find that there is an incorrectly entered picture and you need to cancel it, please find the corresponding picture in this folder and delete it. Then re-run the data_collection.py program.