yahboom
HomeTutorialsProduct
Solution
Blog
Online Store
About Us
EN/中文
Store
Jetson Orin NANO Developer Kit📖
01.Get started
  • 01. Learning route
  • 02. Get started quickly
  • 03. System Update Notes
  • 04. Disabling automatic updates and troubleshooting issues
  • 05. Jetson Orin Nano official kit screwdriver instructions
02.Jetson basic course
  • 01.Jetson board introduction
  • 02.Write Jetson original system
  • 03.Installing Jetson environment
  • 04.Backup SSD system
  • 05.Restore initial image system
  • 06.Upgrade to SUPER version (Optional)
  • 07.Write SUPER boot (Must read)
  • 08.SSD expansion
  • 09.Jetpack old system upgraded to Yahboom SUPER factory system
  • 10.Write the system onto your own SSD
03.Basic setting course
  • 01.Basic knowledge
  • 02.Network configuration
  • 03.SSH remote login
  • 04.VNC remote control
  • 05.RDP remote control
  • 06.Remote file transfer
  • 07.Jtop tool
  • 08.Exchange space expansion
  • 09.Use VSCode
  • 10.Use VMware
  • 11.Modify screen resolution
04.GPIO control course
  • 01.GPIO description
  • 02.GPIO library installation
  • 03.GPIO reading
  • 04.GPIO output
  • 05.Serial communication
  • 06.I2C communication
05.Vision basic course
  • 01.CSI camera preview
  • 02.USB camera preview
  • 03.Use Jupyter Lab
  • 04.Use JetCam
06.OpenCV basic course
  • 01.Image reading
  • 02.Image saving
  • 03.Image modification
  • 04.Image scaling
  • 05.Image cropping
  • 06.Image translation
  • 07.Image mirroring
  • 08.Image grayscale conversion
  • 09.Image binarization
  • 10.Image edge detection
  • 11.Add line segments
  • 12.Add rectangles
  • 13.Add circular shape
  • 14.Add ellipses
  • 15.Add polygons
  • 16.Add Text
  • 17.Preview USB camera image
  • 18.Preview CSI camera image
07.Vision advanced course
01.Environment construction
  • 01.TensorFlow
  • 02.PyTorch
  • 03.Torchvision
02.DeepStream development
  • 01.DeepStream environment
  • 02.License plate detection and recognition
  • 03.Parallel multi-model inference
  • 04.Object embedding vector generation
  • 05.Pose recognition
  • 06.NV optical flow anomaly detection
  • 07.Single-view 3D tracking
  • 08.Multi-view 3D tracking
03.NVIDIA TAO toolkit application
  • 01.Traffic scene object detection
  • 02.Person object detection
  • 03.Vehicle brand classification
  • 04.Vehicle model classification
  • 05.License plate detection and recognition
  • 06.Person semantic segmentation
  • 07.Retail product detection and recognition
  • 08.Person feature extraction and matching (ReID)
  • 09.Person action and pose classification
  • 10.Transformer-based person detection
  • 11.3D human skeleton detection
  • 12.LiDAR 3D point cloud detection
04.YOLO11 development
  • 01.YOLO environment construction
  • 02.CLI use
  • 03.Object detection
  • 04.Instance segmentation
  • 05.Pose estimation
  • 06.Image classification
  • 07.Oriented object detection
  • 08.Model conversion
  • 09.Dataset annotation
  • 10.Model training conversion
  • 11.Model prediction
05.Mediapipe development
  • 01.Mediapipe environment
  • 02.Hand detection
  • 03.Face detection
  • 04.Face effects
  • 05.3D object recognition
  • 06.Virtual brush
  • 07.Gesture recognition
  • 08.Finger control
  • 09.Posture estimation
  • 10.Face mask
  • 11.Overall detection
  • 12.Green scene removal(Human)
06.YOLO26 development
  • 01.YOLO environment construction
  • 02.CLI use
  • 03.Object detection
  • 04.Instance segmentation
  • 05.Pose estimation
  • 06.Image classification
  • 07.Oriented object detection
  • 08.Model conversion
  • 09.Dataset annotation
  • 10.Model training conversion
  • 11.Model prediction
07.QR code recognition
  • QR code recognition
08.Docker basic course
  • 01.Docker introduction
  • 02.Docker installation
  • 03.Docker command
  • 04.Docker usage
  • 05.Docker interaction
09.ROS2-Jazzy
01.ROS2 basic course
  • 1.Introduction to ROS2
  • 2.ROS2 install Jazzy
  • 3.ROS2 development environment
  • 4.ROS2 workspace
  • 5.ROS2 function package
  • 6.ROS2 node
  • 7.ROS2 topic communication
  • 8.ROS2 service communication
  • 9.ROS2 action communication
  • 10.ROS2 custom interface message
  • 11.ROS2 parameter service case
  • 12.ROS2 meta-function package
  • 13.ROS2 distributed communication
  • 14.ROS2 DDS
  • 15.ROS2 time related API
  • 16.ROS2 common command tools
  • 17.ROS2 rviz2 use
  • 18.ROS2 rqt toolbox
  • 19.ROS2 Launch startup file configuration
  • 20.ROS2 recording and playback tool
  • 21.ROS2 URDF model
  • 22.ROS2 Gazebo simulation platform
  • 23.ROS2 TF2 coordinate transformation
02.ROS2 vision course
  • 01.Camera preview (USB)
  • 02.Camera preview (CSI)
  • 03.Camera calibration (USB)
  • 04.Camera calibration (CSI)
  • 05.AR vision (USB)
  • 06.AR vision (CSI)
10.Large model basic course
01.Offline large model
  • 01.AI large model environment deployment
  • 02.Install large model dialogue platform
  • 03.Meta AI - Llama model
  • 04.Alibaba Cloud - Qwen model
  • 05.Microsoft - Phi-4-mini model
  • 06.DeepSeek - DeepSeek model
  • 07.Alibaba Cloud - Qwen-VL visual multimodal large model
  • 08.Google - Gemma visual multimodal large model
  • 9.Llava visual multimodal large model
  • 10.MiniCPM-V visual multimodal large model
  • 11.High-quality vision-language grounding (LocateAnything)
02.Online large model
  • 01.Tongyi Qianwen large model
  • 02.OpenRouter large model API aggregation platform
11.Large model application practice
01.Voice interaction application
  • 1.Voice interaction hardware connection (Must read)
  • 2.Offline speech to text (ASR)
  • 3.Offline text to speech (TTS)
  • 4.AI large model voice interaction
  • 5.AI large model offline voice assistant
02.Multimodal large model application (Online_Offline large model + Text_Voice interaction)
  • 01.Multimodal visual understand application
  • 02.Multimodal text image application
  • 03.Multimodal video analysis application
  • 04.Multi module visual position application
  • 05.Multimodal table scanning application
  • 06.Multimodal autonomous agent application
12.NVIDIA Isaac ROS course
  • 01.Build Isaac ROS environment
  • 02.Deep segmentation
  • 03.DNN stereoscopic depth
  • 04.Free space segmentation
  • 05.Image distortion processing
  • 06.Image segmentation
  • 07.3D scene reconstruction and mapping
  • 08.Object detection
  • 09.3D attitude estimation
  • 10.Visual SLAM
13.OpenClaw deployment and basic usage
01.OpenClaw deployment and introduction
  • 01.Quick start with the pre-configured environment (Recommended)
  • 02.OpenClaw Deployment
  • 03.OpenClaw WhatsApp Plugin Application
  • 04.OpenClaw WebChat Interaction
  • 05.OpenClaw TUI Interaction
  • 06.OpenClaw Tools Introduction
  • 07.OpenClawGate Gateway User Manual
  • 08.OpenClaw Features Overview
  • 09.OpenClawHub Introduction (Skill Installation)
02.OpenClaw basic application
  • 01.OpenClaw Application - File Management
  • 02.OpenClaw Application - Camera
  • 03.OpenClaw Application - Browser Control
  • 04.OpenClaw Application - Script Execution
  • 05.OpenClaw Application - AI Programming (Peripheral & GPIO Control)
  • 06.OpenClaw Application - Dedicated AI Assistant
  • 07.OpenClaw Web Search Capability Configuration
03.NVIDIA official Agent Skills application
  • 01.AI-assisted development with VS Code or Cursor
  • 02.Jetson device skills
  • 03.Jetson BSP custom agent skills
14.OpenClaw advanced development
01.Preparation Before Use
  • 01.Peripheral Hardware Configuration
  • 02.OpenClaw API-KEY Configuration
  • 03.OpenClaw Model Switching
  • 04.OpenClaw Prompts
  • 05.AI Voice Interaction Configuration
  • 06.Three Dialogue Configuration Tests
02.OpenClaw AI Programming Peripheral Control
  • 1. Servo Control
  • 2. RGB Strip
  • 3. OLED Display
  • 4. Temperature and Humidity Sensor
  • 5. Camera Application
  • 6. AI Script Peripheral Control
03.OpenClaw Peripheral Integration Applications
  • 1. Plant Care Assistant
  • 2. AI Calorie Estimation
  • 3. AI Rock Paper Scissors Game
  • 4. AI Pet
  • 5. AI Weather Station
  • 6. Temperature Humidity Dashboard
  • 7. Scheduled Tasks
Download
  • System_File
  • VMFile_BootFile
  • 3D Model File
  • DataSheet
  • Software_Tools
  • Code
  • Manual
Download ZIP

Welcome to Jetson Orin NANO Developer Kit repository

04.Object embedding vector generation

Object Embedding Vector Generation

Object Embedding Vector Generation1. NVIDIA Official Application2. Example Goals and Processing Chains2.1 Person Feature Vector2.2 Retail Item Feature Vector3. Course Script Entry Points and Output4. Local Video Recognition4.1 Running4.2 Changing the Local Video4.3 Why the Retail Video Preview Plays Slowly Frame by Frame, but out.mp4 Is Smooth5. Real-Time Recognition with a USB Camera5.1 Direct Run5.2 Common Parameters5.3 Recognition Strategy and the Cause of Stuttering6. Real-Time Recognition with a CSI Camera6.1 Direct Run6.2 Common Parameters7. Stopping, Video Output, and Troubleshooting

Applicable environment: JetPack 7.2.1, Ubuntu 24.04, DeepStream 9.1, Jetson Orin Nano Super. The factory environment already has the MDX application, models, TensorRT engines, and course entry points ready. Daily study starts from Section 4; Section 1 only needs to be consulted for redeployment when the models, engines, or program are damaged

After the power mode is set to MAXN, running the example may pop up an over-current warning; just ignore it

1. NVIDIA Official Application

This example uses the NVIDIA DeepStream official deepstream-mdx-perception-app, whose source code is located at:

~/DeepStream/src/apps/tao_apps/apps/tao_others/deepstream-mdx-perception-app

Official documentation: https://github.com/NVIDIA/DeepStream/tree/main/src/apps/tao_apps/apps/tao_others/deepstream-mdx-perception-app

The highlight of MDX is not just detection boxes, but attaching an embedding (feature vector) to each target. A feature vector is a string of numbers that can be used for subsequent matching of same-class targets, similar-target retrieval, cross-frame association, or database analysis.

The current course environment has completed application compilation, model download, and engine building. Do not casually delete the .engine files under models/; the engines are tied to the current Jetson, CUDA, and TensorRT environment, and after deletion the first run will rebuild them and take considerably longer.

2. Example Goals and Processing Chains

The course provides two MDX chains: person and retail item.

2.1 Person Feature Vector

xxxxxxxxxx
PeopleNet Transformer person detection
        ↓
NvDCF Tracker target tracking
        ↓
ReIdentificationNet
        ↓
Person embedding (256 dimensions)

The person mode is usually faster than the retail mode; about 20 FPS was measured with CSI. The reason is that the person detection model is lighter, and person targets in the frame are generally fewer than product targets on shelves.

2.2 Retail Item Feature Vector

xxxxxxxxxx
Retail DINO product detection
        ↓
NvDCF Tracker target tracking
        ↓
Retail Object Recognition
        ↓
Product category + embedding (448 dimensions)

The retail mode performs secondary recognition and extracts a feature vector for each detected product. The input of Retail DINO is 960×540, noticeably heavier than person detection, so real-time camera scenarios require a dedicated low-latency strategy.

3. Course Script Entry Points and Output

The current unified launch script is located at ~/DeepStream/deepstream-shell-demo/:

xxxxxxxxxx
cd ~/DeepStream/deepstream-shell-demo
Input SourcePerson ModeRetail Mode
Local video./run_video.sh mdx --people./run_video.sh mdx --retail
USB camera./run_usb.sh mdx --people./run_usb.sh mdx --retail
CSI camera./run_csi.sh mdx --people./run_csi.sh mdx --retail

All three entry points display a window on the Jetson local desktop or the VNC desktop. Embeddings are written as JSONL by default; each line corresponds to one target's detection information and full feature vector:

ModeDefault JSONL Output
Local video person~/DeepStream/src/apps/tao_apps/apps/tao_others/deepstream-mdx-perception-app/mdx_people_reid_embeddings.jsonl
Local video retail~/DeepStream/src/apps/tao_apps/apps/tao_others/deepstream-mdx-perception-app/mdx_retail_embeddings.jsonl
USB person / retailmdx_people_usb_embeddings.jsonl / mdx_retail_usb_embeddings.jsonl
CSI person / retailmdx_people_csi_embeddings.jsonl / mdx_retail_csi_embeddings.jsonl

You can use --output to specify your own JSONL file, for example:

xxxxxxxxxx
cd ~/DeepStream/deepstream-shell-demo
./run_video.sh mdx --retail --output ~/mdx-result.jsonl

4. Local Video Recognition

4.1 Running

​x
cd ~/DeepStream/deepstream-shell-demo

# Person feature vectors
./run_video.sh mdx --people

# Retail product feature vectors
./run_video.sh mdx --retail

The video entry displays the detection results in real time and generates an out.mp4 with detection boxes in the MDX application directory:

xxxxxxxxxx
~/DeepStream/src/apps/tao_apps/apps/tao_others/deepstream-mdx-perception-app/out.mp4

4.2 Changing the Local Video

For the person mode, modify:

xxxxxxxxxx
~/DeepStream/src/apps/tao_apps/configs/app/sources.csv

For the retail mode, modify:

xxxxxxxxxx
~/DeepStream/course-examples/common/sources_retail_object_visual.csv

Change the video URI in the CSV to an absolute path on the Jetson:

xxxxxxxxxx
1,3,file:///home/jetson/videos/test.mp4,1,0,0

MP4/H.264 files are recommended. A Jetson path must be filled in here; a Windows path cannot be used.

4.3 Why the Retail Video Preview Plays Slowly Frame by Frame, but out.mp4 Is Smooth

The speed of the full retail model chain is lower than the frame rate of the original video. For example, when the source video is 30 FPS while Retail DINO plus secondary recognition can only process more slowly, the file source can be back-pressured by the downstream pipeline: the program reduces the file reading and advancement speed, waiting for the previous frame to finish inference.

xxxxxxxxxx
Local video file → inference slows down → file reading slows accordingly → frames processed one by one

Therefore the preview window looks like a slow frame-by-frame playback, but it is still processing the complete video frames in order. The output MP4 keeps the original video timestamps, and the player plays it on the normal timeline, so out.mp4 looks continuous and smooth.

This is not "automatically reducing the number of pushed frames" or actively dropping video frames; rather, the file input can wait for inference to complete. For a local video, the cost is a longer total processing time; for a real-time camera, the same approach cannot be used, because the camera does not pause its output.

When stopping a local video early, press Ctrl+C once and wait for the terminal to show the EOS finishing message before exiting. The program lets the MP4 muxer finish writing the moov index; do not press Ctrl+C repeatedly, close the terminal directly, or use kill -9, otherwise out.mp4 may become unplayable.

5. Real-Time Recognition with a USB Camera

5.1 Direct Run

xxxxxxxxxx
cd ~/DeepStream/deepstream-shell-demo

./run_usb.sh mdx --people
# or
./run_usb.sh mdx --retail

The current USB defaults are:

xxxxxxxxxx
/dev/video0
640×480
30 FPS

5.2 Common Parameters

Before running, you can check the formats, resolutions, and frame rates supported by the camera:

xxxxxxxxxx
v4l2-ctl -d /dev/video0 --list-formats-ext

Note: the USB pipeline of object embedding vector generation is fixed to YUY2, so when choosing from the parameters below, select the YUY2 ones

xxxxxxxxxx
# View all parameters
./run_usb.sh mdx --help

# Use another video device
./run_usb.sh mdx --people --device /dev/video1

# Choose among the supported configurations listed above; here 1280×720, 9 FPS is used
./run_usb.sh mdx --people --width 1280 --height 720 --fps 9

5.3 Recognition Strategy and the Cause of Stuttering

The current setup no longer drops frames at the source, and the camera keeps the full 30 FPS input:

xxxxxxxxxx
drop-frame-interval=0

However, the heavy inference is executed only every 6 frames:

xxxxxxxxxx
# Retail DINO main detection
interval: 5

# Retail Recognition and embedding
interval: 5

The actual structure is as follows:

xxxxxxxxxx
Camera 30 FPS
    ↓
Every 6 frames: Retail DINO + Recognition + embedding
    ↓
Remaining frames: NvDCF Tracker continues the target boxes
    ↓
OSD real-time display

This does not remove MDX's embeddings: each execution of the secondary recognition still generates a 448-dimensional product feature vector; only the vector update frequency for the same product is about 5 FPS instead of recomputing every frame. The intermediate frames have their box positions maintained by the Tracker.

Note that this strategy can only mitigate the stuttering; it cannot guarantee that the preview is as smooth as the original camera. The reason is that the current preview is still inside the same DeepStream inference pipeline: camera frames must pass through the Tracker, the OSD, and the interval-executed DINO/Recognition before reaching the preview window. When there are many products, targets appear frequently, or the Tracker metadata is insufficient, the later processing may still block the earlier display.

Therefore, the actual effect of the current USB retail mode is: the input stays at 30 FPS, the heavy model runs about once every 6 frames, and the preview frame rate is much higher than per-frame inference, but brief pauses or delayed detection box updates may still occur. The purpose of interval: 5 is to reduce repeated inference, not to decouple the preview from inference; nor does it make Retail DINO reach the camera's 30 FPS.

If you need "the original frame always at 30 FPS while Retail embeddings are kept in the background", the camera feed must be split into an independent preview branch and an inference branch, with the latest detection results asynchronously overlaid onto the preview. The current course entry does not yet adopt this dual-branch structure.

gstnvtracker: Unable to acquire a user meta buffer may appear in the current configuration. Previously, increasing the Tracker's user-meta-pool-size caused a noticeable drop in retail performance, so the current setup prioritizes actual preview smoothness, and adjusting this parameter on your own is not recommended.

If the USB camera only outputs MJPEG and the current generic V4L2 configuration cannot capture the stream, a dedicated adaptation with jpegparse → nvv4l2decoder → nvvideoconvert → NVMM needs to be added; first confirm the camera format with the command below:

xxxxxxxxxx
v4l2-ctl -d /dev/video0 --list-formats-ext

6. Real-Time Recognition with a CSI Camera

6.1 Direct Run

xxxxxxxxxx
cd ~/DeepStream/deepstream-shell-demo

./run_csi.sh mdx --people
# or
./run_csi.sh mdx --retail

CSI uses the Jetson native Argus camera source:

xxxxxxxxxx
sensor-id=0
1280×720
30 FPS

The CSI camera of the current device is imx219. The person mode has been measured at about 20 FPS; the retail mode keeps the embeddings and is configured for low-latency display. Before running, confirm that no other program is occupying the CSI camera.

6.2 Common Parameters

Before running, you can check the formats, resolutions, and frame rates supported by the camera; this needs to be run in a graphical interface:

xxxxxxxxxx
gst-launch-1.0 nvarguscamerasrc

img

xxxxxxxxxx
# View all parameters
./run_csi.sh mdx --people --help

# Use the second CSI camera
./run_csi.sh mdx --people --cn --sensor-id 1

# Adjust the input size and frame rate
./run_csi.sh mdx --people --width 1920 --height 1080 --fps 30

7. Stopping, Video Output, and Troubleshooting

The real-time USB/CSI modes by default only preview and write JSONL, without generating out.mp4. The local video mode generates out.mp4.

When stopping a real-time camera or file task:

  1. Return to the terminal where the launch command was run.

  2. Press Ctrl+C only once.

  3. Wait for Received EOS. Exiting ... or App run successful before closing the terminal.

SymptomPriority Checks
No JSONL records in the person modeWhether persons actually appear in the frame; no embeddings are generated when there are no detection targets.
The retail preview still has noticeable latencyConfirm the USB-specific configuration is used and both PGIE/SGIE are interval: 5; do not change back to per-frame inference.
Product boxes update slowlyDetection currently runs once every 6 frames, which is a normal trade-off of the low-latency strategy; interval can be lowered to increase the update frequency, but the preview frame rate will drop.
USB shows no pictureRun v4l2-ctl -d /dev/video0 --list-formats-ext and check the device number, output format, and whether it is occupied.
CSI shows no pictureFirst confirm the basic CSI example works, then check sensor-id=0 and nvargus-daemon.
out.mp4 cannot be playedCheck whether the program was force-terminated before finishing its EOS wrap-up; run again and wait for a normal end.
Parallel multi-model inference
Pose recognition

COMPANY

  • About us
  • Team Style
  • Blog

LEARNING & EXPLORATION

  • Tutorials
  • Edge AI hardware
  • Educational robots
  • Embedded intelligent hardware

WHERE TO BUY

  • Yahboom Online Store
  • Amazon
  • Alibaba
  • AliExpress
  • Mercado

TECHNOLOGY SUPPORT

  • Email: support@yahboom.com

BE A DISTRIBUTOR

  • Email: sales@yahboom.com
QR Code

Copyright © 2017-2026 Yahboom. All Rights Reserved.

EN/中文