Welcome to Jetson Orin NANO Developer Kit repository
10.Transformer-based person detection
TAO PeopleNet Transformer Person Detection
TAO PeopleNet Transformer Person Detection1. Case Goal and Processing Flow2. Pre-Run Checks3. Starting Detection4. USB Camera Real-Time Detection4.1 Direct Run4.2 Common Parameters5. CSI Camera Real-Time Detection5.1 Direct Run5.2 Common Parameters6. Common Issues
Applicable environment: Jetson Orin Nano 8GB, JetPack 7.2.1, DeepStream 9.1. The current video entry can run and preview, and does not generate an MP4
After setting the power mode to MAXN, running the case may pop up an over-current warning; just ignore it.
1. Case Goal and Processing Flow
PeopleNet Transformer detects classes such as person, face, and bag. It uses different model assets and configurations from the CNN version of PeopleNet; engines/parsers must not be mixed.
video/USB/CSI → Transformer TensorRT inference → detection boxes/classes → desktop preview2. Pre-Run Checks
xxxxxxxxxxcd ~/DeepStream/tao-toolkit-shell-demols -lh video_in/sample_720p.mp4Please operate in a Jetson desktop/VNC graphical session. Confirm the input exists; the Transformer compute load is high, and the offline frame rate may be lower than the material's frame rate.
3. Starting Detection
xxxxxxxxxx./run_video.sh transformer
The entry uses video_in/sample_720p.mp4, and the window displays person-related detection results. The current configuration passes -d, which only previews and does not generate an MP4. The display sink and the official default file sink are mutually exclusive; removing -d changes the run mode and output location and is not part of this tutorial's operations.
Replacing the test video: put new material at the video_in/sample_720p.mp4 path used by the configuration, or check the configuration and then modify the input URI. First use gst-discoverer-1.0 to confirm the device can decode it, keep the original material; the current entry has no output file.
4. USB Camera Real-Time Detection
4.1 Direct Run
USB mode supports live preview. The dedicated camera entry feeds frames to the application through a local RTSP relay; do not treat the video file output path as camera recording results.
After confirming the USB camera device node exists, run:
xxxxxxxxxxls -l /dev/video*cd ~/DeepStream/tao-toolkit-shell-demo./run_usb.sh transformer
4.2 Common Parameters
Before running, you can check the formats, resolutions, and frame rates supported by the camera:
xxxxxxxxxxv4l2-ctl -d /dev/video0 --list-formats-ext
Note: the USB pipeline of Object Embedding Vector Generation is fixed to MJPEG, so among the parameters below, choose the MJPEG parameters
x# View all parameters./run_usb.sh transformer --help
# Use another video device./run_usb.sh transformer --device /dev/video1
# Choose from the configurations found above that the camera supports; here we use 1280×720, 60FPS./run_usb.sh transformer --width 1280 --height 720 --fps 605. CSI Camera Real-Time Detection
5.1 Direct Run
CSI mode supports live preview; the entry captures via Argus and sends frames into the dedicated application through a local RTSP relay.
xxxxxxxxxxcd ~/DeepStream/tao-toolkit-shell-demo./run_csi.sh transformer
USB/CSI stream continuously and will not end naturally. To stop, press Ctrl+C once in the launching terminal, wait for the application and camera relay to exit, then start other camera tasks.
5.2 Common Parameters
Before running, you can check the formats, resolutions, and frame rates supported by the camera; this must be run in a graphical interface:
xxxxxxxxxxgst-launch-1.0 nvarguscamerasrc
xxxxxxxxxx# View all parameters./run_csi.sh transformer --help
# Use the second CSI camera./run_csi.sh transformer --sensor-id 1
# Adjust input size and frame rate./run_csi.sh transformer --width 1920 --height 1080 --fps 306. Common Issues
| Symptom | Check and Handling |
|---|---|
| No display window | Check the Jetson graphical session, input file, and display authorization. |
| Abnormal classes/results | Verify the Transformer-specific engine, parser, and labels; do not use CNN PeopleNet assets. |
| Low processing frame rate | The Transformer load is heavy; distinguish offline processing throughput from the input video's nominal frame rate. |
| MP4 not found | The current entry only previews and has no file sink. |