Welcome to Jetson Orin NANO Developer Kit repository
09.Person action and pose classification
TAO Human Action and Pose Classification
TAO Human Action and Pose Classification1. Case Goal and Processing Flow2. Pre-Run Checks3. Starting Action Recognition4. USB Camera Real-Time Detection4.1 Direct Run4.2 Common Parameters5. CSI Camera Real-Time Detection5.1 Direct Run5.2 Common Parameters6. Common Issues
Applicable environment: Jetson Orin Nano, JetPack 7.2.1, DeepStream 9.1
After setting the power mode to MAXN, running the case may pop up an over-current warning; just ignore it.
1. Case Goal and Processing Flow
This case first detects and tracks persons, then estimates keypoints, and finally recognizes actions with a time series. A single frame cannot verify the complete action classification functionality.
video/USB/CSI → PeopleNet → NvDCF track ID → BodyPose3DNet 34 points → time-series window → PoseClassificationNet (ST-GCN) → action classEach target needs to accumulate about 300 frames of keypoints; at 30 FPS, the action class may only become stable after about 10 seconds.
2. Pre-Run Checks
Video mode requires two input files:
xxxxxxxxxxcd ~/DeepStream/tao-toolkit-shell-demols -lh video_in/bodypose.mp4 video_in/sample_walk.movls -ld video_outCamera mode must run in a Jetson desktop/VNC session with the camera connected. Targets newly entering the field of view need to wait for the time-series window to accumulate.
3. Starting Action Recognition
Offline video:
xxxxxxxxxx./run_video.sh pose
Uses video_in/bodypose.mp4 and video_in/sample_walk.mov, outputting video_out/out_pose.mp4. The person boxes, skeletons, and action labels come from detection, pose estimation, and time-series classification respectively.
After waiting for both input sources to reach EOS, check:
xxxxxxxxxxgst-discoverer-1.0 video_out/out_pose.mp4Replacing the test video: the two sources of video mode are specified by the configuration. Before replacing, check each input path and format, prepare material of sufficient length containing continuous actions, and keep the original files. If you change the frame rate or the clip is too short, the time-series window may not be able to produce a stable class.
4. USB Camera Real-Time Detection
4.1 Direct Run
The course USB entry reuses the real-time camera source and can run Pose:
xxxxxxxxxxls -l /dev/video*cd ~/DeepStream/tao-toolkit-shell-demo./run_usb.sh poseEnsure there are people continuously in the field of view and allow time for keypoint accumulation. USB real-time mode is for preview; do not assume out_pose.mp4 will be generated. Press Ctrl+C once and wait for the source EOS and device release.

4.2 Common Parameters
Before running, you can check the formats, resolutions, and frame rates supported by the camera:
xxxxxxxxxxv4l2-ctl -d /dev/video0 --list-formats-ext
Note: the USB pipeline of Object Embedding Vector Generation is fixed to YUY2, so among the parameters below, choose the YUY2 parameters
x# View all parameters./run_usb.sh pose --help
# Use another video device./run_usb.sh pose --device /dev/video1
# Choose from the configurations found above that the camera supports; here we use 1280×720, 9FPS./run_usb.sh pose --width 1280 --height 720 --fps 95. CSI Camera Real-Time Detection
5.1 Direct Run
The CSI entry uses Argus; after connecting a supported module and confirming the driver/service, you can try:
xxxxxxxxxxcd ~/DeepStream/tao-toolkit-shell-demo./run_csi.sh poseCSI supports live action preview and still requires continuous keypoint accumulation. The video mode's output file cannot be directly used as the acceptance basis for camera mode.

5.2 Common Parameters
Before running, you can check the formats, resolutions, and frame rates supported by the camera; this must be run in a graphical interface:
xxxxxxxxxxgst-launch-1.0 nvarguscamerasrc
xxxxxxxxxx# View all parameters./run_csi.sh pose --help
# Use the second CSI camera./run_csi.sh pose --sensor-id 1
# Adjust input size and frame rate./run_csi.sh pose --width 1920 --height 1080 --fps 306. Common Issues
| Symptom | Check and Handling |
|---|---|
| No action labels right after startup | Wait for about 300 frames of time-series data; a new target having no label for a short time is expected. |
| Detection boxes but no skeleton | Check BodyPose3DNet, person crops, and engine loading. |
| Skeleton but unstable labels | Confirm continuous input, frame rate, and accumulation duration; avoid judging from a single frame. |
| No USB image | Check the node, format, occupancy, permissions, and graphical session. |
| No CSI image | Check the compatible module connection, sensor driver, Argus service, and camera occupancy. |