Welcome to Transbot SE repository

7. AR QR code

7.1. Overview

wiki: http://wiki.ros.org/ar_track_alvar/

Source code: https://github.com/ros-perception/ar_track_alvar.git

Function package location: ~/transbot_ws/src/transbot_visual

ARTag (AR tag, AR means "augmented reality") is a benchmark marking system, which can be understood as a reference for other objects. It looks similar to a QR code, but its encoding system and QR code are still very different. The difference is mostly used in camera calibration, robot positioning, augmented reality (AR) and other applications. One of the most important functions is to identify the pose relationship between the object and the camera. ARTag can be attached to the object, or ARTag label can be attached to the plane to calibrate the camera. After the camera recognizes the ARTag, it can calculate the position and attitude of the tag in the camera coordinates.

ar_track_alvar has 4 main functions:

• Generate AR tags with different sizes, resolutions and data/ID encodings.
• Recognize and track poses of individual AR tags, optionally integrating kinect depth data (when kinect is available) for better pose estimation.
• Recognize and track poses in "bundles" consisting of multiple tags. This allows for more stable pose estimation, robustness to occlusions, and tracking of multilateral objects.
• Automatically calculate spatial relationships between tags in bundles using camera images so users don't have to manually measure and enter tag locations in an XML file to use the bundle feature.

Alvar is newer and more advanced than ARToolkit, which has been the basis for several other ROS AR tag packages. Alvar features adaptive thresholding to handle various lighting conditions, optical flow-based tracking for more stable pose estimation, and an improved tag recognition method that does not slow down significantly as the number of tags increases.

7.2. Create ARTag

7.2.1. Install software package

The factory image of the car has already been installed, no need to reinstall it.

sudo apt install ros-melodic-ar-track-alvar

ar_track_alvar is an open source marker library that provides pr2+kinect examples. The first use case of this package is to identify and track gestures from (possibly) multiple AR tags, each considered individually.

7.2.2. Create AR QR code

• Continuously generate multiple tags on one image

  jetson/Raspberry Pi 4B

xxxxxxxxxx
rosrun ar_track_alvar createMarker

Raspberry Pi 5

Before running, please confirm that the large program has been permanently closed

Enter docker

Note: If there is a terminal that automatically starts docker, or there is a docker terminal that has been opened, you can directly enter the docker terminal to run the command, and there is no need to manually start docker

Start docker manually

xxxxxxxxxx
./run_docker.sh

xxxxxxxxxx
rosrun ar_track_alvar createMarker

You can enter [ID] and location information here, and enter [-1] to end. One or more can be generated, and the layout can be designed by yourself.

• Generate a single number

Command + parameters directly generate digital images; for example

jetson/Raspberry Pi 4B

xxxxxxxxxx
rosrun ar_track_alvar createMarker 11
rosrun ar_track_alvar createMarker -s 5 33

Raspberry Pi 5

Before running, please confirm that the large program has been permanently closed

Enter docker

Note: If there is a terminal that automatically starts docker, or there is a docker terminal that has been opened, you can directly enter the docker terminal to run the command, and there is no need to manually start docker

Start docker manually

xxxxxxxxxx
./run_docker.sh

xxxxxxxxxx
rosrun ar_track_alvar createMarker 11
rosrun ar_track_alvar createMarker -s 5 33

11: The number is the QR code of 11. -s: Specify image size. 5: 5x5 picture. 33: The number is the QR code of 33.

7.3. ARTag identification

Note: When starting the camera, you need to load the camera calibration file, otherwise it will not be recognized.

7.3.1, ar_track_alvar node

• Subscribed topic

• Published Topics

• Provided tf Transforms

Single QR code: camera coordinate system → AR tag coordinate system

Multiple QR codes: Provides transformation from the camera coordinate system to each AR tag coordinate system (named ar_marker_x), where x is the ID number of the marker.

7.3.2. Launch file analysis

xxxxxxxxxx
<launch>
     <arg name="camDevice" default="USBCam" doc="camDevice type [USBCam]"/>
     <arg name="open_rviz" default="true"/>
     <arg name="marker_size" default="5.0"/>
     <arg name="max_new_marker_error" default="0.08"/>
     <arg name="max_track_error" default="0.2"/>
     <arg name="cam_image_topic" default="/usb_cam/image_raw" if="$(eval arg('camDevice') == 'USBCam')"/>
     <arg name="cam_info_topic" default="/usb_cam/camera_info" if="$(eval arg('camDevice') == 'USBCam')"/>
     <arg name="output_frame" default="/usb_cam" if="$(eval arg('camDevice') == 'USBCam')"/>
     <include if="$(eval arg('camDevice') == 'USBCam')" file="$(find usb_cam)/launch/usb_cam-test.launch"/>
     <node if="$(eval arg('camDevice') == 'USBCam')" pkg="tf" type="static_transform_publisher" name="world_to_cam"
           args="0 0 0.5 0 1.57 0 world usb_cam 10"/>
     <node name="ar_track_alvar" pkg="ar_track_alvar" type="individualMarkersNoKinect" respawn="false" output="screen">
         <param name="marker_size" type="double" value="$(arg marker_size)"/>
         <param name="max_new_marker_error" type="double" value="$(arg max_new_marker_error)"/>
         <param name="max_track_error" type="double" value="$(arg max_track_error)"/>
         <param name="output_frame" type="string" value="$(arg output_frame)"/>
         <remap from="camera_image" to="$(arg cam_image_topic)"/>
         <remap from="camera_info" to="$(arg cam_info_topic)"/>
     </node>
     <group if="$(arg open_rviz)">
         <node pkg="rviz" type="rviz" name="rviz" args="-d $(find transbot_visual)/rviz/ar_track.rviz"/>
     </group>
</launch>

Node parameters:

• marker_size (double): Width (in centimeters) of one side of the black square marker border.
• max_new_marker_error (double): Threshold for determining when a new marker can be detected under uncertain conditions.
• max_track_error (double): A threshold that determines how many tracking errors can be observed before the marker disappears.
• camera_image (string): Provides the image topic name used to detect AR tags. This can be monochrome or color, but should be an uncorrected image since correction is done in this package.
• camera_info (string): Subject name that provides camera calibration parameters for correcting images.
• output_frame (string): Publish the coordinate position of the AR tag in the camera coordinate system.

7.3.3. Start identification instance

• jetson/Raspberry Pi 4B

  xxxxxxxxxx
    roslaunch transbot_visual ar_track.launch camDevice:=USBCam
  

  Raspberry Pi 5

  Before running, please confirm that the large program has been permanently closed

  Enter docker

  Note: If there is a terminal that automatically starts docker, or there is a docker terminal that has been opened, you can directly enter the docker terminal to run the command, and there is no need to manually start docker

  Start docker manually

  xxxxxxxxxx
  ./run_docker.sh
  

  

  xxxxxxxxxx
    roslaunch transbot_visual ar_track.launch camDevice:=USBCam
  

  • camDevice: USBCam

  

  In rviz, you need to set the corresponding camera topic name.

  • Image_Topic: The high frame rate camera is [/usb_cam/image_raw].
  • Marker: The display component of rviz. Different squares display the location of the AR QR code.
  • TF: The display component of rviz, used to display the coordinate system of AR QR codes.
  • Camera: The display component of rviz, which displays the camera screen.
  • world: world coordinate system.
  • usb_cam/camera_link: camera coordinate system.

  7.3.4. View node graph

  jetson/Raspberry Pi 4B/Raspberry Pi 4B

  xxxxxxxxxx
  rqt_graph
  

  Raspberry Pi 5

  Enter the same docker from multiple terminals

  Keep the program of the previous docker terminal running and open a new terminal

  Enter the following command

  xxxxxxxxxx
  docker ps
  

  Enter the same docker and use the following 18870bc3dc00 to modify the ID displayed on the actual terminal.

  xxxxxxxxxx
  docker exec -it 18870bc3dc00 /bin/bash
  

  

  xxxxxxxxxx
  rqt_graph
  

  

   

  7.3.5. View tf tree

  jetson/Raspberry Pi 4B/Raspberry Pi 4B

  xxxxxxxxxx
  rosrun rqt_tf_tree rqt_tf_tree
  

  Raspberry Pi 5

  Enter the same docker from multiple terminals

  Keep the program of the previous docker terminal running and open a new terminal

  Enter the following command

  xxxxxxxxxx
  docker ps
  

  Enter the same docker and use the following 18870bc3dc00 to modify the ID displayed on the actual terminal.

  xxxxxxxxxx
  docker exec -it 18870bc3dc00 /bin/bash
   AR vision
  ROS+Opencv foundation