Welcome to DOFBOT-JetsonNANO repository

2. Human Pose Estimation

Enter the Docker container, and then execute the tutorial commands within Docker to launch the program.

Path to the provided source code package:

dofbot_pro_vision/dofbot_pro_vision

2.1. Overview

Human Pose Estimation involves estimating a person's posture by correctly connecting the human keypoints detected within an image. These keypoints typically correspond to joints on the human body that possess a certain degree of freedom—such as the neck, shoulders, elbows, wrists, waist, knees, and ankles—as illustrated in the figure below.

2.2. Principles

Upon receiving an input image, a convolutional network extracts features to produce a set of feature maps. These maps are then split into two branches, where separate CNN networks are employed to extract Part Confidence Maps and Part Affinity Fields, respectively. Once this information is obtained, we utilize Bipartite Matching (from graph theory) to determine Part Associations—specifically, to connect the keypoints belonging to the same individual. Thanks to the inherent vectorial nature of PAFs, the resulting bipartite matches are highly accurate, ultimately allowing us to assemble a complete skeletal structure for each person. Finally, based on the PAFs, we perform Multi-Person Parsing. This involves transforming the multi-person parsing problem into a graph-based problem, which is then solved using the Hungarian Algorithm. (The Hungarian Algorithm is the most common algorithm for bipartite graph matching. Its core principle lies in finding "augmenting paths"—a technique used to determine the maximum matching within a bipartite graph.)

2.3. Startup

xxxxxxxxxx
# Start the camera (enter this command within the Docker container terminal):
ros2 launch dofbot_info camera_arm_kin.launch.py
# Start the recognition program (enter this command within the Docker container terminal):
ros2 run dofbot_pro_vision detect_pose

Input Image

Output Image

You can observe the actual footage captured by the camera.