yahboom
HomeTutorialsProduct
Solution
Blog
Online Store
About Us
EN/中文
Store
Jetson Orin NANO Developer Kit📖
01.Get started
  • 01. Learning route
  • 02. Get started quickly
  • 03. System Update Notes
  • 04. Disabling automatic updates and troubleshooting issues
  • 05. Jetson Orin Nano official kit screwdriver instructions
02.Jetson basic course
  • 01.Jetson board introduction
  • 02.Write Jetson original system
  • 03.Installing Jetson environment
  • 04.Backup SSD system
  • 05.Restore initial image system
  • 06.Upgrade to SUPER version (Optional)
  • 07.Write SUPER boot (Must read)
  • 08.SSD expansion
  • 09.Jetpack old system upgraded to Yahboom SUPER factory system
  • 10.Write the system onto your own SSD
03.Basic setting course
  • 01.Basic knowledge
  • 02.Network configuration
  • 03.SSH remote login
  • 04.VNC remote control
  • 05.RDP remote control
  • 06.Remote file transfer
  • 07.Jtop tool
  • 08.Exchange space expansion
  • 09.Use VSCode
  • 10.Use VMware
  • 11.Modify screen resolution
04.GPIO control course
  • 01.GPIO description
  • 02.GPIO library installation
  • 03.GPIO reading
  • 04.GPIO output
  • 05.Serial communication
  • 06.I2C communication
05.Vision basic course
  • 01.CSI camera preview
  • 02.USB camera preview
  • 03.Use Jupyter Lab
  • 04.Use JetCam
06.OpenCV basic course
  • 01.Image reading
  • 02.Image saving
  • 03.Image modification
  • 04.Image scaling
  • 05.Image cropping
  • 06.Image translation
  • 07.Image mirroring
  • 08.Image grayscale conversion
  • 09.Image binarization
  • 10.Image edge detection
  • 11.Add line segments
  • 12.Add rectangles
  • 13.Add circular shape
  • 14.Add ellipses
  • 15.Add polygons
  • 16.Add Text
  • 17.Preview USB camera image
  • 18.Preview CSI camera image
07.Vision advanced course
01.Environment construction
  • 01.TensorFlow
  • 02.PyTorch
  • 03.Torchvision
02.DeepStream development
  • 01.DeepStream environment
  • 02.License plate detection and recognition
  • 03.Parallel multi-model inference
  • 04.Object embedding vector generation
  • 05.Pose recognition
  • 06.NV optical flow anomaly detection
  • 07.Single-view 3D tracking
  • 08.Multi-view 3D tracking
03.NVIDIA TAO toolkit application
  • 01.Traffic scene object detection
  • 02.Person object detection
  • 03.Vehicle brand classification
  • 04.Vehicle model classification
  • 05.License plate detection and recognition
  • 06.Person semantic segmentation
  • 07.Retail product detection and recognition
  • 08.Person feature extraction and matching (ReID)
  • 09.Person action and pose classification
  • 10.Transformer-based person detection
  • 11.3D human skeleton detection
  • 12.LiDAR 3D point cloud detection
04.YOLO11 development
  • 01.YOLO environment construction
  • 02.CLI use
  • 03.Object detection
  • 04.Instance segmentation
  • 05.Pose estimation
  • 06.Image classification
  • 07.Oriented object detection
  • 08.Model conversion
  • 09.Dataset annotation
  • 10.Model training conversion
  • 11.Model prediction
05.Mediapipe development
  • 01.Mediapipe environment
  • 02.Hand detection
  • 03.Face detection
  • 04.Face effects
  • 05.3D object recognition
  • 06.Virtual brush
  • 07.Gesture recognition
  • 08.Finger control
  • 09.Posture estimation
  • 10.Face mask
  • 11.Overall detection
  • 12.Green scene removal(Human)
06.YOLO26 development
  • 01.YOLO environment construction
  • 02.CLI use
  • 03.Object detection
  • 04.Instance segmentation
  • 05.Pose estimation
  • 06.Image classification
  • 07.Oriented object detection
  • 08.Model conversion
  • 09.Dataset annotation
  • 10.Model training conversion
  • 11.Model prediction
07.QR code recognition
  • QR code recognition
08.Docker basic course
  • 01.Docker introduction
  • 02.Docker installation
  • 03.Docker command
  • 04.Docker usage
  • 05.Docker interaction
09.ROS2-Jazzy
01.ROS2 basic course
  • 1.Introduction to ROS2
  • 2.ROS2 install Jazzy
  • 3.ROS2 development environment
  • 4.ROS2 workspace
  • 5.ROS2 function package
  • 6.ROS2 node
  • 7.ROS2 topic communication
  • 8.ROS2 service communication
  • 9.ROS2 action communication
  • 10.ROS2 custom interface message
  • 11.ROS2 parameter service case
  • 12.ROS2 meta-function package
  • 13.ROS2 distributed communication
  • 14.ROS2 DDS
  • 15.ROS2 time related API
  • 16.ROS2 common command tools
  • 17.ROS2 rviz2 use
  • 18.ROS2 rqt toolbox
  • 19.ROS2 Launch startup file configuration
  • 20.ROS2 recording and playback tool
  • 21.ROS2 URDF model
  • 22.ROS2 Gazebo simulation platform
  • 23.ROS2 TF2 coordinate transformation
02.ROS2 vision course
  • 01.Camera preview (USB)
  • 02.Camera preview (CSI)
  • 03.Camera calibration (USB)
  • 04.Camera calibration (CSI)
  • 05.AR vision (USB)
  • 06.AR vision (CSI)
10.Large model basic course
01.Offline large model
  • 01.AI large model environment deployment
  • 02.Install large model dialogue platform
  • 03.Meta AI - Llama model
  • 04.Alibaba Cloud - Qwen model
  • 05.Microsoft - Phi-4-mini model
  • 06.DeepSeek - DeepSeek model
  • 07.Alibaba Cloud - Qwen-VL visual multimodal large model
  • 08.Google - Gemma visual multimodal large model
  • 9.Llava visual multimodal large model
  • 10.MiniCPM-V visual multimodal large model
  • 11.High-quality vision-language grounding (LocateAnything)
02.Online large model
  • 01.Tongyi Qianwen large model
  • 02.OpenRouter large model API aggregation platform
11.Large model application practice
01.Voice interaction application
  • 1.Voice interaction hardware connection (Must read)
  • 2.Offline speech to text (ASR)
  • 3.Offline text to speech (TTS)
  • 4.AI large model voice interaction
  • 5.AI large model offline voice assistant
02.Multimodal large model application (Online_Offline large model + Text_Voice interaction)
  • 01.Multimodal visual understand application
  • 02.Multimodal text image application
  • 03.Multimodal video analysis application
  • 04.Multi module visual position application
  • 05.Multimodal table scanning application
  • 06.Multimodal autonomous agent application
12.NVIDIA Isaac ROS course
  • 01.Build Isaac ROS environment
  • 02.Deep segmentation
  • 03.DNN stereoscopic depth
  • 04.Free space segmentation
  • 05.Image distortion processing
  • 06.Image segmentation
  • 07.3D scene reconstruction and mapping
  • 08.Object detection
  • 09.3D attitude estimation
  • 10.Visual SLAM
13.OpenClaw deployment and basic usage
01.OpenClaw deployment and introduction
  • 01.Quick start with the pre-configured environment (Recommended)
  • 02.OpenClaw Deployment
  • 03.OpenClaw WhatsApp Plugin Application
  • 04.OpenClaw WebChat Interaction
  • 05.OpenClaw TUI Interaction
  • 06.OpenClaw Tools Introduction
  • 07.OpenClawGate Gateway User Manual
  • 08.OpenClaw Features Overview
  • 09.OpenClawHub Introduction (Skill Installation)
02.OpenClaw basic application
  • 01.OpenClaw Application - File Management
  • 02.OpenClaw Application - Camera
  • 03.OpenClaw Application - Browser Control
  • 04.OpenClaw Application - Script Execution
  • 05.OpenClaw Application - AI Programming (Peripheral & GPIO Control)
  • 06.OpenClaw Application - Dedicated AI Assistant
  • 07.OpenClaw Web Search Capability Configuration
03.NVIDIA official Agent Skills application
  • 01.AI-assisted development with VS Code or Cursor
  • 02.Jetson device skills
  • 03.Jetson BSP custom agent skills
14.OpenClaw advanced development
01.Preparation Before Use
  • 01.Peripheral Hardware Configuration
  • 02.OpenClaw API-KEY Configuration
  • 03.OpenClaw Model Switching
  • 04.OpenClaw Prompts
  • 05.AI Voice Interaction Configuration
  • 06.Three Dialogue Configuration Tests
02.OpenClaw AI Programming Peripheral Control
  • 1. Servo Control
  • 2. RGB Strip
  • 3. OLED Display
  • 4. Temperature and Humidity Sensor
  • 5. Camera Application
  • 6. AI Script Peripheral Control
03.OpenClaw Peripheral Integration Applications
  • 1. Plant Care Assistant
  • 2. AI Calorie Estimation
  • 3. AI Rock Paper Scissors Game
  • 4. AI Pet
  • 5. AI Weather Station
  • 6. Temperature Humidity Dashboard
  • 7. Scheduled Tasks
Download
  • System_File
  • VMFile_BootFile
  • 3D Model File
  • DataSheet
  • Software_Tools
  • Code
  • Manual
Download ZIP

Welcome to Jetson Orin NANO Developer Kit repository

08.Multi-view 3D tracking

DeepStream Multi-View 3D Tracking (MV3DT)

DeepStream Multi-View 3D Tracking (MV3DT)1. Example Goal and Chain2. Input Data and Calibration3. Dependencies and Resource Preparation4. Running the Official 4-Stream Sample5. Checking Tracking and Kafka Metadata6. 12-Stream Resource Limitations7. Common Issues

Applicable environment: JetPack 7.2.1, Ubuntu 24.04, DeepStream 9.1, CUDA 13.2, TensorRT 10.16, Jetson Orin Nano 8GB. The official 4-stream offline sample has been accepted on the course device; although the 12-stream sample has an entry point, it fails on the Orin Nano 8GB due to insufficient GPU/video buffer memory, and is not provided as a usable configuration. Real USB/CSI multi-camera input is currently not connected

After the power mode is set to MAXN, running the example may pop up an over-current warning; just ignore it

1. Example Goal and Chain

MV3DT (Multi-View 3D Tracking) associates the observations of multiple calibrated cameras into shared world coordinates, maintains global target IDs across viewpoints, and publishes structured 3D tracking metadata via Kafka.

4 synchronized video streams → per-stream person detection/pose estimation → camera coordinate projection
             → cross-camera target association and global 3D tracking
             → Kafka mv3dt topic → BEV visualization

MV3DT four video streams and BEV results

The DeepStreamTest5App on the right displays the four camera views; the Bird-Eye View on the left projects cross-camera trajectories onto the field map. The desktop also displays performance information. The over-current/thermal-limit notifications in the screenshot are not algorithm outputs; during operation, use adequate power supply and maintain cooling.

2. Input Data and Calibration

The official 4-stream data is located in the dataset directory of the DeepStream multi-view reference application, containing synchronized videos, per-camera configurations, world-to-pixel projection matrices, and BEV map resources. Custom data must satisfy:

  • All camera videos are time-synchronized, with alignable frame rates/timestamps.

  • The resolutions are identical and correspond to the calibration parameters.

  • Every camera must have its own projectionMatrix_3x4_w2p; the matrix of another camera must not be reused.

  • The camera fields of view need sufficient overlap or associable areas; the field map and coordinate transformation should cover the actual area.

MV3DT still requires calibration. It does not automatically know the spatial relationships between cameras just because multiple cameras are used.

3. Dependencies and Resource Preparation

Official reference application: https://github.com/NVIDIA/DeepStream/tree/main/src/apps/reference_apps/deepstream-tracker-3d-multi-view

The first deployment usually requires: Git LFS data, the official multi-view video/camera YAMLs, the PeopleNet detection model plus BodyPose3DNet, Kafka, MQTT/Mosquitto, Python visualization dependencies, and TensorRT engines. The course target machine already has the 4-stream configuration, the Kafka mv3dt topic, and the run dependencies ready; the prerequisites do not need to be run again.

Unified Jetson entry point:

xxxxxxxxxx
~/DeepStream/deepstream-shell-demo/run_video.sh

4. Running the Official 4-Stream Sample

Run from a terminal in the Jetson graphical desktop or a VNC session:

xxxxxxxxxx
cd ~/DeepStream/deepstream-shell-demo
export DISPLAY=:0
xhost +si:localuser:root
./run_video.sh mv3dt --4

The current default detection model uses PeopleNet2.6.3, with a batch-4 engine adapted to the Orin Nano 8GB. If an engine still needs to be built on the first run, wait for completion and watch the terminal log; do not start other large inference tasks at the same time. The run requires a desktop display and a local Kafka service; the launcher checks/prepares Kafka and the topic, and sets up the local X11 authorization required by the container.

5. Checking Tracking and Kafka Metadata

Four OSD streams and the BEV window are expected to appear on the desktop, and the terminal shows 4 active sources. After pressing Ctrl+C once, the main program should exit and close the BEV child-process window.

Open another terminal to read the cross-camera messages:

xxxxxxxxxx
cd ~/DeepStream/src/apps/reference_apps/deepstream-tracker-3d-multi-view
source ./mv3dt_venv/bin/activate
python utils/kafka_client.py --broker localhost:9092 --topic mv3dt

The messages should contain fields such as the camera identifier, target ID, footLocation3D, and bbox3d. The ID association across consecutive frames/different cameras is used to determine whether global tracking is working; Kafka having a topic does not mean the inference succeeded — you still need to confirm that new messages are received.

6. 12-Stream Resource Limitations

Although the batch-12 engine of PeopleNet2.6.3 can be built successfully and the launcher keeps the following experimental entry, actual 12-stream runs showed GPU out of memory, unavailable video output buffers, and tracker submission failures:

xxxxxxxxxx
./run_video.sh mv3dt --12

7. Common Issues

SymptomHandling
BEV window or four-pane layout does not appearLaunch from the Jetson graphical desktop/VNC, and check DISPLAY, X11 authorization, and the run log.
No messages in KafkaCheck whether Kafka is running, whether the mv3dt topic exists, and whether the main pipeline is processing video.
IDs do not match across camerasCheck the time synchronization, each stream's projectionMatrix_3x4_w2p, the coordinate systems, and the field-of-view overlap.
12-stream OOMA known hardware limitation; go back to --4; reduce the number of streams or switch to a platform with more memory.
Residual BEV window after Ctrl+CThe updated official script should close the child processes via exit cleanup; if it still remains, first confirm the actually running script version.
Single-view 3D tracking
Traffic scene object detection

COMPANY

  • About us
  • Team Style
  • Blog

LEARNING & EXPLORATION

  • Tutorials
  • Edge AI hardware
  • Educational robots
  • Embedded intelligent hardware

WHERE TO BUY

  • Yahboom Online Store
  • Amazon
  • Alibaba
  • AliExpress
  • Mercado

TECHNOLOGY SUPPORT

  • Email: support@yahboom.com

BE A DISTRIBUTOR

  • Email: sales@yahboom.com
QR Code

Copyright © 2017-2026 Yahboom. All Rights Reserved.

EN/中文