yahboom
HomeTutorialsProduct
Solution
Blog
Online Store
About Us
EN/中文
Store
Jetson Orin NANO Developer Kit📖
01.Get started
  • 01. Learning route
  • 02. Get started quickly
  • 03. System Update Notes
  • 04. Disabling automatic updates and troubleshooting issues
  • 05. Jetson Orin Nano official kit screwdriver instructions
02.Jetson basic course
  • 01.Jetson board introduction
  • 02.Write Jetson original system
  • 03.Installing Jetson environment
  • 04.Backup SSD system
  • 05.Restore initial image system
  • 06.Upgrade to SUPER version (Optional)
  • 07.Write SUPER boot (Must read)
  • 08.SSD expansion
  • 09.Jetpack old system upgraded to Yahboom SUPER factory system
  • 10.Write the system onto your own SSD
03.Basic setting course
  • 01.Basic knowledge
  • 02.Network configuration
  • 03.SSH remote login
  • 04.VNC remote control
  • 05.RDP remote control
  • 06.Remote file transfer
  • 07.Jtop tool
  • 08.Exchange space expansion
  • 09.Use VSCode
  • 10.Use VMware
  • 11.Modify screen resolution
04.GPIO control course
  • 01.GPIO description
  • 02.GPIO library installation
  • 03.GPIO reading
  • 04.GPIO output
  • 05.Serial communication
  • 06.I2C communication
05.Vision basic course
  • 01.CSI camera preview
  • 02.USB camera preview
  • 03.Use Jupyter Lab
  • 04.Use JetCam
06.OpenCV basic course
  • 01.Image reading
  • 02.Image saving
  • 03.Image modification
  • 04.Image scaling
  • 05.Image cropping
  • 06.Image translation
  • 07.Image mirroring
  • 08.Image grayscale conversion
  • 09.Image binarization
  • 10.Image edge detection
  • 11.Add line segments
  • 12.Add rectangles
  • 13.Add circular shape
  • 14.Add ellipses
  • 15.Add polygons
  • 16.Add Text
  • 17.Preview USB camera image
  • 18.Preview CSI camera image
07.Vision advanced course
01.Environment construction
  • 01.TensorFlow
  • 02.PyTorch
  • 03.Torchvision
02.DeepStream development
  • 01.DeepStream environment
  • 02.License plate detection and recognition
  • 03.Parallel multi-model inference
  • 04.Object embedding vector generation
  • 05.Pose recognition
  • 06.NV optical flow anomaly detection
  • 07.Single-view 3D tracking
  • 08.Multi-view 3D tracking
03.NVIDIA TAO toolkit application
  • 01.Traffic scene object detection
  • 02.Person object detection
  • 03.Vehicle brand classification
  • 04.Vehicle model classification
  • 05.License plate detection and recognition
  • 06.Person semantic segmentation
  • 07.Retail product detection and recognition
  • 08.Person feature extraction and matching (ReID)
  • 09.Person action and pose classification
  • 10.Transformer-based person detection
  • 11.3D human skeleton detection
  • 12.LiDAR 3D point cloud detection
04.YOLO11 development
  • 01.YOLO environment construction
  • 02.CLI use
  • 03.Object detection
  • 04.Instance segmentation
  • 05.Pose estimation
  • 06.Image classification
  • 07.Oriented object detection
  • 08.Model conversion
  • 09.Dataset annotation
  • 10.Model training conversion
  • 11.Model prediction
05.Mediapipe development
  • 01.Mediapipe environment
  • 02.Hand detection
  • 03.Face detection
  • 04.Face effects
  • 05.3D object recognition
  • 06.Virtual brush
  • 07.Gesture recognition
  • 08.Finger control
  • 09.Posture estimation
  • 10.Face mask
  • 11.Overall detection
  • 12.Green scene removal(Human)
06.YOLO26 development
  • 01.YOLO environment construction
  • 02.CLI use
  • 03.Object detection
  • 04.Instance segmentation
  • 05.Pose estimation
  • 06.Image classification
  • 07.Oriented object detection
  • 08.Model conversion
  • 09.Dataset annotation
  • 10.Model training conversion
  • 11.Model prediction
07.QR code recognition
  • QR code recognition
08.Docker basic course
  • 01.Docker introduction
  • 02.Docker installation
  • 03.Docker command
  • 04.Docker usage
  • 05.Docker interaction
09.ROS2-Jazzy
01.ROS2 basic course
  • 1.Introduction to ROS2
  • 2.ROS2 install Jazzy
  • 3.ROS2 development environment
  • 4.ROS2 workspace
  • 5.ROS2 function package
  • 6.ROS2 node
  • 7.ROS2 topic communication
  • 8.ROS2 service communication
  • 9.ROS2 action communication
  • 10.ROS2 custom interface message
  • 11.ROS2 parameter service case
  • 12.ROS2 meta-function package
  • 13.ROS2 distributed communication
  • 14.ROS2 DDS
  • 15.ROS2 time related API
  • 16.ROS2 common command tools
  • 17.ROS2 rviz2 use
  • 18.ROS2 rqt toolbox
  • 19.ROS2 Launch startup file configuration
  • 20.ROS2 recording and playback tool
  • 21.ROS2 URDF model
  • 22.ROS2 Gazebo simulation platform
  • 23.ROS2 TF2 coordinate transformation
02.ROS2 vision course
  • 01.Camera preview (USB)
  • 02.Camera preview (CSI)
  • 03.Camera calibration (USB)
  • 04.Camera calibration (CSI)
  • 05.AR vision (USB)
  • 06.AR vision (CSI)
10.Large model basic course
01.Offline large model
  • 01.AI large model environment deployment
  • 02.Install large model dialogue platform
  • 03.Meta AI - Llama model
  • 04.Alibaba Cloud - Qwen model
  • 05.Microsoft - Phi-4-mini model
  • 06.DeepSeek - DeepSeek model
  • 07.Alibaba Cloud - Qwen-VL visual multimodal large model
  • 08.Google - Gemma visual multimodal large model
  • 9.Llava visual multimodal large model
  • 10.MiniCPM-V visual multimodal large model
  • 11.High-quality vision-language grounding (LocateAnything)
02.Online large model
  • 01.Tongyi Qianwen large model
  • 02.OpenRouter large model API aggregation platform
11.Large model application practice
01.Voice interaction application
  • 1.Voice interaction hardware connection (Must read)
  • 2.Offline speech to text (ASR)
  • 3.Offline text to speech (TTS)
  • 4.AI large model voice interaction
  • 5.AI large model offline voice assistant
02.Multimodal large model application (Online_Offline large model + Text_Voice interaction)
  • 01.Multimodal visual understand application
  • 02.Multimodal text image application
  • 03.Multimodal video analysis application
  • 04.Multi module visual position application
  • 05.Multimodal table scanning application
  • 06.Multimodal autonomous agent application
12.NVIDIA Isaac ROS course
  • 01.Build Isaac ROS environment
  • 02.Deep segmentation
  • 03.DNN stereoscopic depth
  • 04.Free space segmentation
  • 05.Image distortion processing
  • 06.Image segmentation
  • 07.3D scene reconstruction and mapping
  • 08.Object detection
  • 09.3D attitude estimation
  • 10.Visual SLAM
13.OpenClaw deployment and basic usage
01.OpenClaw deployment and introduction
  • 01.Quick start with the pre-configured environment (Recommended)
  • 02.OpenClaw Deployment
  • 03.OpenClaw WhatsApp Plugin Application
  • 04.OpenClaw WebChat Interaction
  • 05.OpenClaw TUI Interaction
  • 06.OpenClaw Tools Introduction
  • 07.OpenClawGate Gateway User Manual
  • 08.OpenClaw Features Overview
  • 09.OpenClawHub Introduction (Skill Installation)
02.OpenClaw basic application
  • 01.OpenClaw Application - File Management
  • 02.OpenClaw Application - Camera
  • 03.OpenClaw Application - Browser Control
  • 04.OpenClaw Application - Script Execution
  • 05.OpenClaw Application - AI Programming (Peripheral & GPIO Control)
  • 06.OpenClaw Application - Dedicated AI Assistant
  • 07.OpenClaw Web Search Capability Configuration
03.NVIDIA official Agent Skills application
  • 01.AI-assisted development with VS Code or Cursor
  • 02.Jetson device skills
  • 03.Jetson BSP custom agent skills
14.OpenClaw advanced development
01.Preparation Before Use
  • 01.Peripheral Hardware Configuration
  • 02.OpenClaw API-KEY Configuration
  • 03.OpenClaw Model Switching
  • 04.OpenClaw Prompts
  • 05.AI Voice Interaction Configuration
  • 06.Three Dialogue Configuration Tests
02.OpenClaw AI Programming Peripheral Control
  • 1. Servo Control
  • 2. RGB Strip
  • 3. OLED Display
  • 4. Temperature and Humidity Sensor
  • 5. Camera Application
  • 6. AI Script Peripheral Control
03.OpenClaw Peripheral Integration Applications
  • 1. Plant Care Assistant
  • 2. AI Calorie Estimation
  • 3. AI Rock Paper Scissors Game
  • 4. AI Pet
  • 5. AI Weather Station
  • 6. Temperature Humidity Dashboard
  • 7. Scheduled Tasks
Download
  • System_File
  • VMFile_BootFile
  • 3D Model File
  • DataSheet
  • Software_Tools
  • Code
  • Manual
Download ZIP

Welcome to Jetson Orin NANO Developer Kit repository

03.Parallel multi-model inference

DeepStream Parallel Multi-Model Inference

DeepStream Parallel Multi-Model Inference1. Factory Environment and First Deployment Reference1.1 Obtaining the Official Source Code1.2 Preparing Triton, Models, and TensorRT Engines1.3 Compiling the Official Parallel Application2. Example Goal3. Launching via Scripts: Choosing the Input Source4. Local Video Verification5. Real-Time Verification with a USB Camera5.1 Direct Run5.2 Common Parameters6. Real-Time Verification with a CSI Camera6.1 Direct Run6.2 Common Parameters7. Traffic Tracking and Line-Crossing Counting8. Output, Stop, and Checks9. Common Symptoms

The factory system already has the DeepStream, Triton runtime, models, TensorRT engines, course launch entry points, and compatibility configurations required by this example ready. For normal study, simply run from Section 3 by choosing an input source; the deployment commands in this section are only for reference when the environment is missing, needs restoration, or is being redeployed.

After the power mode is set to MAXN, running the example may pop up an over-current warning; just ignore it

1. Factory Environment and First Deployment Reference

The factory system already contains:

  • The video, USB, and CSI Python entry points under ~/DeepStream/course-examples/.

  • The Triton runtime, launch scripts, and the single-camera four-pane program under ~/DeepStream/parallel-inference/.

  • The models and TensorRT engines required by YOLOv4, TrafficCamNet, and BodyPose2D.

  • The compatibility launch configuration for embedded Triton.

Therefore, do not repeat the commands in this section; only when the environment is missing, needs restoration, or is being deployed on your own, follow the order below.

1.1 Obtaining the Official Source Code

cd ~
git clone https://github.com/NVIDIA/DeepStream.git
cd ~/DeepStream
git checkout v9.1.0
git submodule update --init --recursive

Confirm the version:

xxxxxxxxxx
git describe --tags --exact-match

1.2 Preparing Triton, Models, and TensorRT Engines

​x
# This script is located in the DeepStream repository root, not in the parallel multi-model example directory.
cd ~/DeepStream/scripts
sudo ./triton_backend_setup.sh

cd ~/DeepStream/src/apps/reference_apps/deepstream_parallel_inference_app/tritonserver
sudo ./build_engine.sh

The first build downloads the models and generates engines for the Jetson, which takes a long time. If it reports that trtexec cannot be found, first check:

xxxxxxxxxx
command -v trtexec

This course environment uses a toolchain matched to CUDA 13.2 and TensorRT 10.16; do not directly upgrade the CUDA or TensorRT metapackages just to install trtexec.

1.3 Compiling the Official Parallel Application

xxxxxxxxxx
cd ~/DeepStream/src/apps/reference_apps/deepstream_parallel_inference_app/tritonclient/sample
make -C apps/deepstream-parallel-infer CUDA_VER=13.2

2. Example Goal

This example derives multiple processing branches from the same video frame or the same camera frame, and is used to learn how nvinfer, nvinferserver (Triton), the Tracker, nvdsanalytics, and human pose post-processing work together.

The final window is a four-pane layout:

PositionDisplayed Content
Top leftYOLOv4 general object detection: persons, vehicles, bicycles, backpacks, etc.
Top rightYOLO boxes + NVTracker + TrafficCamNet + line-crossing counting
Bottom leftYOLO boxes + BodyPose2D: human keypoints and skeleton
Bottom rightOriginal frame for comparison

For a USB camera, the frame is captured only once; DeepStream copies the same frame internally to the four display branches and does not open the camera four times.

Official reference source code: https://github.com/NVIDIA/DeepStream/tree/main/src/apps/reference_apps/deepstream_parallel_inference_app

3. Launching via Scripts: Choosing the Input Source

The current unified launch script is located at ~/DeepStream/deepstream-shell-demo/:

xxxxxxxxxx
cd ~/DeepStream/deepstream-shell-demo
Input SourceCommandApplicable Scenario
Local video./run_video.sh parallelFirst verification of the four-pane layout and each model's effect, with MP4 recording
USB camera./run_usb.sh parallelReal-time input from a UVC/V4L2 USB camera, display only
CSI camera./run_csi.sh parallelReal-time input from a MIPI CSI camera, display only

All three entry points display the four-pane layout in real time; only the local video entry records an MP4, while the USB/CSI entries do not write video files.

4. Local Video Verification

For the first run, it is recommended to run the local video first to confirm that the models, the four-pane layout, and the display environment are correct:

xxxxxxxxxx
cd ~/DeepStream/deepstream-shell-demo
./run_video.sh parallel

5. Real-Time Verification with a USB Camera

5.1 Direct Run

Note: the bundled factory USB camera has been adapted by default to the MJPEG 1280×720@60 mode

xxxxxxxxxx
cd ~/DeepStream/deepstream-shell-demo
./run_usb.sh parallel

5.2 Common Parameters

Before running, you can check the device nodes and the formats, resolutions, and frame rates supported by the camera:

xxxxxxxxxx
v4l2-ctl --list-devices
v4l2-ctl -d /dev/video0 --list-formats-ext

Then pass parameters according to the actual device.

xxxxxxxxxx
# View all parameters
./run_usb.sh parallel --help

# Use another video device
./run_usb.sh parallel --device /dev/video1

# Choose among the supported configurations listed above; here 1920×1080, 30 FPS is used; without the suffix, this is the YUY2 stream
./run_usb.sh parallel --width 1920 --height 1080 --fps 6

# Add the mjpeg parameter to use MJPG mode
./run_usb.sh parallel --width 1920 --height 1080 --fps 30 --mjpeg

6. Real-Time Verification with a CSI Camera

6.1 Direct Run

xxxxxxxxxx
cd ~/DeepStream/deepstream-shell-demo
./run_csi.sh parallel

6.2 Common Parameters

Before running, you can check the formats, resolutions, and frame rates supported by the camera; this needs to be run in a graphical interface:

xxxxxxxxxx
gst-launch-1.0 nvarguscamerasrc

xxxxxxxxxx
# View all parameters
./run_csi.sh parallel --help

# Use the second CSI camera
./run_csi.sh parallel --sensor-id 1

# Adjust the input size and frame rate
./run_csi.sh parallel --width 1920 --height 1080 --fps 30

7. Traffic Tracking and Line-Crossing Counting

The processing chain at the top right is:

xxxxxxxxxx
YOLO detection boxes → NVTracker → line-crossing counting → TrafficCamNet
  • YOLO finds vehicles, persons, and other targets and generates detection boxes.

  • NVTracker assigns a stable ID to the same target across consecutive frames, avoiding repeated per-frame counting of the same vehicle.

  • The line-crossing counting accumulates once when a tracked target crosses the manually configured counting line.

  • TrafficCamNet is a traffic scene detection model, used for the parallel traffic model demonstration.

The green line, Entry, and Exit are manually defined business rules and count displays, not obtained by the model automatically recognizing the road. Before real deployment, the counting line must be re-calibrated according to the road, lanes, or entrances and exits in the actual camera view; the default line in the tutorial is only for demonstration.

8. Output, Stop, and Checks

The real-time window is displayed on the Jetson local desktop or the VNC desktop. The video output location of the local video version is fixed at:

xxxxxxxxxx
~/DeepStream/src/apps/reference_apps/deepstream_parallel_inference_app/tritonclient/sample/out.mp4

Press Ctrl+C once in the terminal where the run command was executed to stop normally. The program sends EOS and finishes writing the MP4 index; do not use kill -9, otherwise the video may become unplayable.

After stopping, you can play or inspect:

xxxxxxxxxx
totem ~/DeepStream/src/apps/reference_apps/deepstream_parallel_inference_app/tritonclient/sample/out.mp4

gst-discoverer-1.0 \
  ~/DeepStream/src/apps/reference_apps/deepstream_parallel_inference_app/tritonclient/sample/out.mp4

9. Common Symptoms

SymptomHandling
The video version runs, but the USB version does not startFirst run v4l2-ctl --list-formats-ext, and verify the device number, resolution, and frame rate; MJPEG input requires adding --mjpeg.
CSI cannot be openedConfirm the sensor-id, the camera wiring, and whether the basic CSI example works normally.
A fixed counting line or Exit text appears at the top rightThis is the configuration overlay of nvdsanalytics; it only has business meaning after re-calibration according to the actual road.
The MP4 is 0 bytes or unplayableDo not force-kill the process; run again and press Ctrl+C once, and wait for the program to exit completely.
License plate detection and recognition
Object embedding vector generation

COMPANY

  • About us
  • Team Style
  • Blog

LEARNING & EXPLORATION

  • Tutorials
  • Edge AI hardware
  • Educational robots
  • Embedded intelligent hardware

WHERE TO BUY

  • Yahboom Online Store
  • Amazon
  • Alibaba
  • AliExpress
  • Mercado

TECHNOLOGY SUPPORT

  • Email: support@yahboom.com

BE A DISTRIBUTOR

  • Email: sales@yahboom.com
QR Code

Copyright © 2017-2026 Yahboom. All Rights Reserved.

EN/中文