Welcome to Raspberry Pi 5 repository

4. AR vision

1 Overview

Augmented Reality, referred to as "AR", is a technology that cleverly integrates virtual information with the real world. It widely uses multimedia, three-dimensional modeling, real-time tracking and registration, intelligent interaction, sensing and other technologies. It simulates computer-generated text, images, three-dimensional models, music, videos and other virtual information and then applies it to the real world. The two types of information complement each other, thereby achieving "enhancement" of the real world.

The AR system has three outstanding characteristics: ① information integration between the real world and the virtual world; ② real-time interactivity; ③ adding positioning virtual objects in the three-dimensional scale space.

Augmented reality technology includes new technologies and methods such as multimedia, three-dimensional modeling, real-time video display and control, multi-sensor fusion, real-time tracking and registration, and scene fusion.

2. How to use

When using AR cases, you must have the internal parameters of the camera, otherwise it will not work. The internal parameter files are in the same directory as the code, and different cameras correspond to different internal parameters. This course takes a USB driver-free camera as an example. Internal parameter calibration can be quickly calibrated with a checkerboard** (this step has been completed in the docker image)**

There are a total of 12 effects in the cases in this section.

["Triangle", "Rectangle", "Parallelogram","WindMill","TableTennisTable", "Ball","Arrow", "Knife", "Desk","Bench", "Stickman", "ParallelBars"]

3. Start command

Code reference path

xxxxxxxxxx
/root/yahboomcar_ros2_ws/yahboomcar_ws/src/yahboomcar_visual/yahboomcar_visual/simple_AR.py

After entering the docker container, enter in the docker terminal

xxxxxxxxxx
ros2 run yahboomcar_visual simple_AR

[q] key to exit, [f] key to switch different effects.

Note: Due to the average performance of some motherboards, the GUI display is very laggy, causing the button function to fail. Sometimes the button can be used to switch, and sometimes there is no response after pressing it. If you need to switch to different effects, you can refer to the following posts on different topics to achieve the switching effect.

3.1.1, ROS deployment

This course also deploys ROS, which mainly has the following two functions:

• Subscribe to topic data and switch between different effects

• Post images

View the ros topic through the following command, enter in docker terminal

xxxxxxxxxx
ros2 topic list

• /Graphics_topic: The topic name of the effect, the effect that needs to be identified when subscribing.

• /simpleAR/camera: The topic name of the image, publish the image.

To modify the effect, you can modify it through the following command. For example, I first changed it to Desk (the effect name has been given above, you can modify it yourself)

docker terminal input

xxxxxxxxxx
ros2 topic pub /Graphics_topic std_msgs/msg/String "data: Desk"

To view the published image, you can use rqt_image_view to view it and enter it in the docker terminal.

xxxxxxxxxx
ros2 run rqt_image_view rqt_image_view

Select the topic /simpleAR/camera in the upper left corner to view the image.