yahboom
HomeTutorialsProduct
Solution
Blog
Online Store
About Us
EN/中文
Store
Jetson Orin NANO Developer Kit📖
01.Get started
  • 01. Learning route
  • 02. Get started quickly
  • 03. System Update Notes
  • 04. Disabling automatic updates and troubleshooting issues
  • 05. Jetson Orin Nano official kit screwdriver instructions
02.Jetson basic course
  • 01.Jetson board introduction
  • 02.Write Jetson original system
  • 03.Installing Jetson environment
  • 04.Backup SSD system
  • 05.Restore initial image system
  • 06.Upgrade to SUPER version (Optional)
  • 07.Write SUPER boot (Must read)
  • 08.SSD expansion
  • 09.Jetpack old system upgraded to Yahboom SUPER factory system
  • 10.Write the system onto your own SSD
03.Basic setting course
  • 01.Basic knowledge
  • 02.Network configuration
  • 03.SSH remote login
  • 04.VNC remote control
  • 05.RDP remote control
  • 06.Remote file transfer
  • 07.Jtop tool
  • 08.Exchange space expansion
  • 09.Use VSCode
  • 10.Use VMware
  • 11.Modify screen resolution
04.GPIO control course
  • 01.GPIO description
  • 02.GPIO library installation
  • 03.GPIO reading
  • 04.GPIO output
  • 05.Serial communication
  • 06.I2C communication
05.Vision basic course
  • 01.CSI camera preview
  • 02.USB camera preview
  • 03.Use Jupyter Lab
  • 04.Use JetCam
06.OpenCV basic course
  • 01.Image reading
  • 02.Image saving
  • 03.Image modification
  • 04.Image scaling
  • 05.Image cropping
  • 06.Image translation
  • 07.Image mirroring
  • 08.Image grayscale conversion
  • 09.Image binarization
  • 10.Image edge detection
  • 11.Add line segments
  • 12.Add rectangles
  • 13.Add circular shape
  • 14.Add ellipses
  • 15.Add polygons
  • 16.Add Text
  • 17.Preview USB camera image
  • 18.Preview CSI camera image
07.Vision advanced course
01.Environment construction
  • 01.TensorFlow
  • 02.PyTorch
  • 03.Torchvision
02.DeepStream development
  • 01.DeepStream environment
  • 02.License plate detection and recognition
  • 03.Parallel multi-model inference
  • 04.Object embedding vector generation
  • 05.Pose recognition
  • 06.NV optical flow anomaly detection
  • 07.Single-view 3D tracking
  • 08.Multi-view 3D tracking
03.NVIDIA TAO toolkit application
  • 01.Traffic scene object detection
  • 02.Person object detection
  • 03.Vehicle brand classification
  • 04.Vehicle model classification
  • 05.License plate detection and recognition
  • 06.Person semantic segmentation
  • 07.Retail product detection and recognition
  • 08.Person feature extraction and matching (ReID)
  • 09.Person action and pose classification
  • 10.Transformer-based person detection
  • 11.3D human skeleton detection
  • 12.LiDAR 3D point cloud detection
04.YOLO11 development
  • 01.YOLO environment construction
  • 02.CLI use
  • 03.Object detection
  • 04.Instance segmentation
  • 05.Pose estimation
  • 06.Image classification
  • 07.Oriented object detection
  • 08.Model conversion
  • 09.Dataset annotation
  • 10.Model training conversion
  • 11.Model prediction
05.Mediapipe development
  • 01.Mediapipe environment
  • 02.Hand detection
  • 03.Face detection
  • 04.Face effects
  • 05.3D object recognition
  • 06.Virtual brush
  • 07.Gesture recognition
  • 08.Finger control
  • 09.Posture estimation
  • 10.Face mask
  • 11.Overall detection
  • 12.Green scene removal(Human)
06.YOLO26 development
  • 01.YOLO environment construction
  • 02.CLI use
  • 03.Object detection
  • 04.Instance segmentation
  • 05.Pose estimation
  • 06.Image classification
  • 07.Oriented object detection
  • 08.Model conversion
  • 09.Dataset annotation
  • 10.Model training conversion
  • 11.Model prediction
07.QR code recognition
  • QR code recognition
08.Docker basic course
  • 01.Docker introduction
  • 02.Docker installation
  • 03.Docker command
  • 04.Docker usage
  • 05.Docker interaction
09.ROS2-Jazzy
01.ROS2 basic course
  • 1.Introduction to ROS2
  • 2.ROS2 install Jazzy
  • 3.ROS2 development environment
  • 4.ROS2 workspace
  • 5.ROS2 function package
  • 6.ROS2 node
  • 7.ROS2 topic communication
  • 8.ROS2 service communication
  • 9.ROS2 action communication
  • 10.ROS2 custom interface message
  • 11.ROS2 parameter service case
  • 12.ROS2 meta-function package
  • 13.ROS2 distributed communication
  • 14.ROS2 DDS
  • 15.ROS2 time related API
  • 16.ROS2 common command tools
  • 17.ROS2 rviz2 use
  • 18.ROS2 rqt toolbox
  • 19.ROS2 Launch startup file configuration
  • 20.ROS2 recording and playback tool
  • 21.ROS2 URDF model
  • 22.ROS2 Gazebo simulation platform
  • 23.ROS2 TF2 coordinate transformation
02.ROS2 vision course
  • 01.Camera preview (USB)
  • 02.Camera preview (CSI)
  • 03.Camera calibration (USB)
  • 04.Camera calibration (CSI)
  • 05.AR vision (USB)
  • 06.AR vision (CSI)
10.Large model basic course
01.Offline large model
  • 01.AI large model environment deployment
  • 02.Install large model dialogue platform
  • 03.Meta AI - Llama model
  • 04.Alibaba Cloud - Qwen model
  • 05.Microsoft - Phi-4-mini model
  • 06.DeepSeek - DeepSeek model
  • 07.Alibaba Cloud - Qwen-VL visual multimodal large model
  • 08.Google - Gemma visual multimodal large model
  • 9.Llava visual multimodal large model
  • 10.MiniCPM-V visual multimodal large model
  • 11.High-quality vision-language grounding (LocateAnything)
02.Online large model
  • 01.Tongyi Qianwen large model
  • 02.OpenRouter large model API aggregation platform
11.Large model application practice
01.Voice interaction application
  • 1.Voice interaction hardware connection (Must read)
  • 2.Offline speech to text (ASR)
  • 3.Offline text to speech (TTS)
  • 4.AI large model voice interaction
  • 5.AI large model offline voice assistant
02.Multimodal large model application (Online_Offline large model + Text_Voice interaction)
  • 01.Multimodal visual understand application
  • 02.Multimodal text image application
  • 03.Multimodal video analysis application
  • 04.Multi module visual position application
  • 05.Multimodal table scanning application
  • 06.Multimodal autonomous agent application
12.NVIDIA Isaac ROS course
  • 01.Build Isaac ROS environment
  • 02.Deep segmentation
  • 03.DNN stereoscopic depth
  • 04.Free space segmentation
  • 05.Image distortion processing
  • 06.Image segmentation
  • 07.3D scene reconstruction and mapping
  • 08.Object detection
  • 09.3D attitude estimation
  • 10.Visual SLAM
13.OpenClaw deployment and basic usage
01.OpenClaw deployment and introduction
  • 01.Quick start with the pre-configured environment (Recommended)
  • 02.OpenClaw Deployment
  • 03.OpenClaw WhatsApp Plugin Application
  • 04.OpenClaw WebChat Interaction
  • 05.OpenClaw TUI Interaction
  • 06.OpenClaw Tools Introduction
  • 07.OpenClawGate Gateway User Manual
  • 08.OpenClaw Features Overview
  • 09.OpenClawHub Introduction (Skill Installation)
02.OpenClaw basic application
  • 01.OpenClaw Application - File Management
  • 02.OpenClaw Application - Camera
  • 03.OpenClaw Application - Browser Control
  • 04.OpenClaw Application - Script Execution
  • 05.OpenClaw Application - AI Programming (Peripheral & GPIO Control)
  • 06.OpenClaw Application - Dedicated AI Assistant
  • 07.OpenClaw Web Search Capability Configuration
03.NVIDIA official Agent Skills application
  • 01.AI-assisted development with VS Code or Cursor
  • 02.Jetson device skills
  • 03.Jetson BSP custom agent skills
14.OpenClaw advanced development
01.Preparation Before Use
  • 01.Peripheral Hardware Configuration
  • 02.OpenClaw API-KEY Configuration
  • 03.OpenClaw Model Switching
  • 04.OpenClaw Prompts
  • 05.AI Voice Interaction Configuration
  • 06.Three Dialogue Configuration Tests
02.OpenClaw AI Programming Peripheral Control
  • 1. Servo Control
  • 2. RGB Strip
  • 3. OLED Display
  • 4. Temperature and Humidity Sensor
  • 5. Camera Application
  • 6. AI Script Peripheral Control
03.OpenClaw Peripheral Integration Applications
  • 1. Plant Care Assistant
  • 2. AI Calorie Estimation
  • 3. AI Rock Paper Scissors Game
  • 4. AI Pet
  • 5. AI Weather Station
  • 6. Temperature Humidity Dashboard
  • 7. Scheduled Tasks
Download
  • System_File
  • VMFile_BootFile
  • 3D Model File
  • DataSheet
  • Software_Tools
  • Code
  • Manual
Download ZIP

Welcome to Jetson Orin NANO Developer Kit repository

11.High-quality vision-language grounding (LocateAnything)

LocateAnything High-Quality Visual-Language Grounding

LocateAnything High-Quality Visual-Language Grounding1. What Is LocateAnything2. Applicable Devices and Capability Scope3. Preparation Before Deployment3.1 Checking the System and Disk3.2 Installing Build and Download Tools4. Downloading the Model5. Compiling the Image Grounding Program5.1 Getting the Adapted Version of the Source Code5.2 Configuring and Compiling the CUDA Version5.3 Organizing the Runtime Files6. Creating the Single-Image Run Script7. Preparing the Input Image8. Running Image Grounding8.1 Using the Default Image and Target8.2 Specifying the Image and Recognition Target9. Video Usage and Performance Limits10. Common Problems11. References

Note: Due to performance limitations, the Jetson Orin Nano 4GB cannot run this case

1. What Is LocateAnything

LocateAnything is a visual-language grounding model. It takes an image and a text description, finds the targets in the image that match the description, and returns the bounding box coordinates of the targets.

The difference between it and ordinary image classification models is: a classification model usually answers "what is in the image", while LocateAnything also points out the location of the target in the image. For example, input a street scene and query person, and the model returns the position of the pedestrian; query red car, and the model tries to locate the red car in the picture.

This tutorial uses the GGUF quantized model of LocateAnything-3B, and runs it through the llama.cpp multimodal program adapted to this model format. After the model is loaded, image inference can be completed offline on the local Jetson; the first download and installation of the model files and the program still require a network connection.

2. Applicable Devices and Capability Scope

This tutorial uses Jetson Orin Nano 8GB, JetPack 7.2.1, CUDA 13.2 as the baseline. Other Jetson models or software versions need to re-check the CUDA, compiler, and memory compatibility; the same parameters cannot be guaranteed to work directly.

The current solution is suitable for a single image or low-frequency offline grounding of frames extracted from a video, and is not a real-time video detector. On Orin Nano 8GB, a 1280×720 image measured in practice takes approximately 18–19 seconds; the memory usage during inference is close to the device limit. Do not start other large models or high-load GPU tasks at the same time.

3. Preparation Before Deployment

If you are using the factory image, the locateanything environment has already been set up, and you can skip this subsection and go to 8.1 to run the script directly

3.1 Checking the System and Disk

Open a Jetson terminal and confirm that CUDA 13.2 is available:

nvcc --version
free -h
df -h /home

nvcc --version should show CUDA 13.2. The model files occupy approximately 2.8GB, and compiling the source code also requires additional disk space; it is recommended to reserve at least 10GB for /home. The first download of the model and the source code requires a network connection that can access Hugging Face and GitHub.

When you need to observe the real-time memory of the device, you can run tegrastats and press Ctrl+C to stop the monitoring:

xxxxxxxxxx
tegrastats --interval 1000

3.2 Installing Build and Download Tools

xxxxxxxxxx
sudo apt update
sudo apt install -y git cmake build-essential wget libssl-dev libcurl4-openssl-dev

Confirm that Python OpenCV is available; the subsequent script uses it to read and draw images:

xxxxxxxxxx
python3 -c 'import cv2; print(cv2.__version__)'

If it reports that cv2 cannot be found, you can install the system package:

xxxxxxxxxx
sudo apt install -y python3-opencv

4. Downloading the Model

The main model and the vision projector are matching files. The combination specified in this tutorial has been measured on Orin Nano 8GB; please download both files, and do not replace only one of them.

​x
mkdir -p ~/locateanything/LocateAnything-3B-4bit
cd ~/locateanything/LocateAnything-3B-4bit

wget -c -O LocateAnything-3B-Q4_K_M.gguf \
  "https://hf-mirror.com/liodon-ai/LocateAnything-3B-imatrix-GGUF/resolve/main/LocateAnything-3B-Q4_K_M.gguf"

wget -c -O mmproj-locateanything-3b-bf16.gguf \
  "https://hf-mirror.com/sabafallah/LocateAnything-3B-GGUF/resolve/main/mmproj-locateanything-3b-bf16.gguf"

-c means the download can continue after an interruption. Check the files after the download is complete:

xxxxxxxxxx
ls -lh ~/locateanything/LocateAnything-3B-4bit/*.gguf

You should see a main model of approximately 2.0GB and a vision projector of approximately 833MB. The actual file size may show slight differences due to unit conversion.

5. Compiling the Image Grounding Program

5.1 Getting the Adapted Version of the Source Code

The vision projector type of LocateAnything requires multimodal support from the corresponding version. Use the mtmd-grounders branch:

xxxxxxxxxx
cd ~/locateanything
git clone --depth 1 --branch mtmd-grounders \
  https://github.com/yuuko-eth/llama.cpp.git

5.2 Configuring and Compiling the CUDA Version

Orin Nano uses compute capability 87. Specify CUDA 13.2, which corresponds to JetPack 7.2.1, and build only the llama-mtmd-cli target required by this tutorial:

xxxxxxxxxx
export PATH=/usr/local/cuda-13.2/bin:$PATH
export CUDACXX=/usr/local/cuda-13.2/bin/nvcc

cd ~/locateanything/llama.cpp
cmake -S . -B build \
  -DGGML_CUDA=ON \
  -DBUILD_SHARED_LIBS=ON \
  -DCMAKE_CUDA_ARCHITECTURES=87 \
  -DCMAKE_BUILD_TYPE=Release \
  -DCUDAToolkit_ROOT=/usr/local/cuda-13.2

cmake --build build --target llama-mtmd-cli -j2

When the compilation succeeds, the end of the output shows:

xxxxxxxxxx
[100%] Built target llama-mtmd-cli

The CUDA compilation time varies with device load and system status. Do not run other large model tasks at the same time during compilation.

5.3 Organizing the Runtime Files

Place the program and its shared libraries in the LocateAnything home directory; the subsequent script loads them from this directory:

xxxxxxxxxx
cd ~/locateanything
mkdir -p lib
cp llama.cpp/build/bin/llama-mtmd-cli ./llama-mtmd-cli
cp -a llama.cpp/build/bin/lib*.so* ./lib/
chmod +x ./llama-mtmd-cli

Check whether the dynamic libraries are complete:

xxxxxxxxxx
LD_LIBRARY_PATH="$HOME/locateanything/lib:/usr/local/cuda-13.2/lib64" \
  ldd ~/locateanything/llama-mtmd-cli | grep "not found"

If there is no output at all, no missing dynamic libraries were found. If not found is displayed, check whether the shared library copy in the previous step has been completed, and confirm that the CUDA 13.2 path exists.

6. Creating the Single-Image Run Script

By default, the script reads ~/locateanything/input.jpg, queries person, and saves the result with red boxes as out.jpg. Copy the entire command below to create the script:

xxxxxxxxxx
cat > ~/locateanything/run_locateanything.sh <<'SCRIPT'
#!/usr/bin/env bash
set -u

BASE_DIR="$HOME/locateanything"
BIN="$BASE_DIR/llama-mtmd-cli"
MODEL="$BASE_DIR/LocateAnything-3B-4bit/LocateAnything-3B-Q4_K_M.gguf"
MMPROJ="$BASE_DIR/LocateAnything-3B-4bit/mmproj-locateanything-3b-bf16.gguf"
INPUT="${1:-$BASE_DIR/input.jpg}"
QUERY="${2:-person}"
OUTPUT="${3:-$BASE_DIR/out.jpg}"
RAW="$BASE_DIR/locate-result.txt"

if [[ ! -f "$INPUT" ]]; then
    echo "Input image does not exist: $INPUT" >&2
    exit 2
fi

export LD_LIBRARY_PATH="$BASE_DIR/lib:/usr/local/cuda-13.2/lib64:${LD_LIBRARY_PATH:-}"
echo "Recognizing: $INPUT"
echo "Target: $QUERY"

"$BIN" \
    -m "$MODEL" \
    --mmproj "$MMPROJ" \
    --image "$INPUT" \
    -p "$QUERY" \
    -ngl 10 \
    -n 128 \
    --temp 0 2>&1 | tee "$RAW"
STATUS=${PIPESTATUS[0]}
if [[ $STATUS -ne 0 ]]; then
    echo "Model inference failed, exit code: $STATUS" >&2
    exit "$STATUS"
fi

python3 - "$INPUT" "$OUTPUT" "$QUERY" "$RAW" <<'PY'
import re
import sys

import cv2

input_path, output_path, query, raw_path = sys.argv[1:]
image = cv2.imread(input_path)
if image is None:
    raise SystemExit(f"Cannot read image: {input_path}")

height, width = image.shape[:2]
with open(raw_path, "r", encoding="utf-8", errors="replace") as file:
    text = file.read()

boxes = [tuple(map(int, match)) for match in re.findall(
    r"<box>\s*<(-?\d+)>\s*<(-?\d+)>\s*<(-?\d+)>\s*<(-?\d+)>\s*</box>",
    text,
)]

for index, (x1, y1, x2, y2) in enumerate(boxes, 1):
    left = max(0, min(width - 1, round(x1 * width / 1000)))
    top = max(0, min(height - 1, round(y1 * height / 1000)))
    right = max(left + 1, min(width - 1, round(x2 * width / 1000)))
    bottom = max(top + 1, min(height - 1, round(y2 * height / 1000)))
    cv2.rectangle(image, (left, top), (right, bottom), (0, 0, 255), 3)
    cv2.putText(image, f"{query} {index}", (left, max(28, top - 8)),
                cv2.FONT_HERSHEY_SIMPLEX, 0.75, (0, 0, 255), 2, cv2.LINE_AA)

if not cv2.imwrite(output_path, image):
    raise SystemExit(f"Cannot write output image: {output_path}")
print(f"Output: {output_path}")
print(f"Parsed {len(boxes)} boxes")
PY
SCRIPT

chmod +x ~/locateanything/run_locateanything.sh

7. Preparing the Input Image

Copy the image to be recognized to the Jetson and save it with the default file name:

xxxxxxxxxx
cp /path/to/image.jpg ~/locateanything/input.jpg

You can also extract a static image from a video as the input. The following example reads the frame at the 5th second of the video and saves it as input.jpg:

xxxxxxxxxx
python3 - <<'PY'
import os
import cv2

video = os.path.expanduser("~/locateanything/sample_720p.mp4")
output = os.path.expanduser("~/locateanything/input.jpg")
capture = cv2.VideoCapture(video)
capture.set(cv2.CAP_PROP_POS_MSEC, 5000)
ok, frame = capture.read()
capture.release()
if not ok:
    raise SystemExit(f"Failed to read video frame: {video}")
if not cv2.imwrite(output, frame):
    raise SystemExit(f"Failed to save image: {output}")
print(f"Saved: {output}")
PY

If the video path is different, change the path in the command to the actual location. This step only extracts one image and does not process the entire video.

8. Running Image Grounding

8.1 Using the Default Image and Target

xxxxxxxxxx
cd ~/locateanything
./run_locateanything.sh

The default parameters are:

ItemDefault value
Input image~/locateanything/input.jpg
Query targetperson
Output image~/locateanything/out.jpg
Raw model result~/locateanything/locate-result.txt

After the recognition is complete, the terminal displays the number of detected boxes; open out.jpg to view the image and the red grounding boxes. Running it again overwrites out.jpg and locate-result.txt.

8.2 Specifying the Image and Recognition Target

Command format:

xxxxxxxxxx
./run_locateanything.sh [image path] [recognition target] [output image path]

For example, to recognize the car in the image:

xxxxxxxxxx
./run_locateanything.sh test.jpg car out.jpg

The recognition target can be an English word or phrase; enclose it in quotes when it contains spaces:

xxxxxxxxxx
./run_locateanything.sh test.jpg "white car" out.jpg

You can also specify other input and output paths:

xxxxxxxxxx
./run_locateanything.sh /home/jetson/Pictures/street.jpg person /home/jetson/Pictures/street-result.jpg

Currently, it is recommended to use a short English target description. The predicted boxes of the model may miss targets or have inaccurate positions, especially for small distant targets and adjacent targets; please check manually based on the output image.

9. Video Usage and Performance Limits

The current run script accepts a single image and does not take an entire video directly. When a video needs to be analyzed, images can be extracted at time points and then passed to the script one by one. Each image requires a separate model inference.

On Orin Nano 8GB, a single 1280×720 image measured in practice takes approximately 18–19 seconds. If a 40-second, 30 FPS video is processed frame by frame, 1200 inferences are required in total, which is estimated to take approximately 6–6.3 hours; the actual time varies with the picture and the system load. Therefore, this solution is not suitable for real-time 30 FPS video analysis.

The peak memory during inference is approximately 7.0GB/7.5GB. Please avoid running other large models or GPU-intensive tasks at the same time; if CUDA out of memory occurs, close other high-memory applications and retry. Do not use increasing the number of GPU layers as the default solution.

10. Common Problems

SymptomChecking method
Input image does not existCheck the image path; the default input should be ~/locateanything/input.jpg.
libllama-common.so.0: cannot open shared object fileConfirm that ~/locateanything/lib/ contains the shared libraries and start it through the run script; do not copy only llama-mtmd-cli and miss lib/.
CUDA error: out of memory or insufficient system memoryClose other large model/GPU tasks before running; the inference memory headroom of Orin Nano 8GB is limited.
No boxes in the output, or the boxes do not fit the targetCheck whether the query word is clear and whether the target is visible in the picture, and check the original image manually; LocateAnything is a prediction model, and the results are not guaranteed to be accurate every time.
Download interruptedRun the wget command with -c again in the original download directory to continue the download.
CUDA compiler not foundConfirm the JetPack/CUDA version, and set PATH and CUDACXX to the CUDA 13.2 path before compiling.

11. References

  • LocateAnything research introduction: NVIDIA LocateAnything

  • Model source: LocateAnything-3B Q4_K_M and LocateAnything-3B mmproj

  • Inference program source: yuuko-eth/llama.cpp, using the mtmd-grounders branch

MiniCPM-V visual multimodal large model
Tongyi Qianwen large model

COMPANY

  • About us
  • Team Style
  • Blog

LEARNING & EXPLORATION

  • Tutorials
  • Edge AI hardware
  • Educational robots
  • Embedded intelligent hardware

WHERE TO BUY

  • Yahboom Online Store
  • Amazon
  • Alibaba
  • AliExpress
  • Mercado

TECHNOLOGY SUPPORT

  • Email: support@yahboom.com

BE A DISTRIBUTOR

  • Email: sales@yahboom.com
QR Code

Copyright © 2017-2026 Yahboom. All Rights Reserved.

EN/中文