Welcome to Jetson Orin NANO Developer Kit repository
01.AI large model environment deployment
Ollama
Ollama1. Large Language Models (LLMs)2. Offline Models and 4GB Compatibility3. Installing OllamaScript installation4. Using Ollama5. Uninstalling OllamaReferences
Demo environment
Development board: Jetson Orin series mainboards
Ollama is an open source tool designed to simplify deploying and running large language models, so that users can run high-quality language models in a local environment.
1. Large Language Models (LLMs)
Large Language Models (LLMs) are a class of advanced text generation systems built on artificial intelligence technology. Their main characteristic is that they can learn and understand human language from large-scale training data and can generate natural and fluent text.
2. Offline Models and 4GB Compatibility
Before downloading a model, first select a version based on the board's memory. The table below summarizes the models covered in this chapter's offline model tutorials and adds the Qwen3-VL model used by the Chapter 11 visual understanding case. Compatibility is assessed for Jetson Orin Nano 4GB with this course's configuration; it is not a general conclusion for every quantization, context length, and runtime configuration of these models.
The table marks "Supported" or "Not supported" according to this course's compatibility results; even a supported model still needs memory reserved for the system and other runtime components. The model download size does not equal the runtime memory: inference also requires context cache, vision components, and system memory.
| Model family and tutorial | Model version | Main use | Orin Nano 4GB |
|---|---|---|---|
| Llama 3.2 | llama3.2:1b | Text chat | Supported |
| Llama 3.2 | llama3.2:3b | Text chat | Supported |
| Qwen3 | qwen3:0.6b | Text chat | Supported |
| Qwen3 | qwen3:1.7b | Text chat | Supported |
| Qwen3 | qwen3:4b | Text chat | Supported |
| Qwen3 | qwen3:8b | Text chat | Not supported |
| Phi-4-mini | phi4-mini:3.8b | Text chat, reasoning | Not supported |
| DeepSeek-R1 | deepseek-r1:1.5b | Text chat, reasoning | Supported |
| DeepSeek-R1 | deepseek-r1:7b | Text chat, reasoning | Not supported |
| DeepSeek-R1 | deepseek-r1:8b | Text chat, reasoning | Not supported |
| Qwen2.5-VL | qwen2.5vl:3b | Image understanding, text chat | Not supported |
| Qwen2.5-VL | qwen2.5vl:7b | Image understanding, text chat | Not supported |
| Qwen3-VL | qwen3-vl:2b | Image understanding, text chat | Not supported (this course's multimodal case) |
| Gemma 3 | gemma3:1b | Text chat | Supported |
| Gemma 3 | gemma3:4b | Image understanding, text chat | Not supported |
| LLaVA | llava:7b | Image understanding, text chat | Not supported |
| MiniCPM-V | minicpm-v:8b | Image understanding, text chat | Not supported |
| LocateAnything | LocateAnything-3B Q4_K_M + BF16 mmproj | Visual object grounding | Not supported |
Course scope: The range of small-parameter text models in the table above does not mean that complete multimodal applications can also run. The six cases in Chapter 11, "Multimodal large model application (online/offline large model + text/voice interaction)", do not support a 4GB configuration; use a Jetson Orin Nano 8GB. The fact that a model itself can run does not mean it supports the image input or tool calling required by the cases.
3. Installing Ollama
This tutorial demonstrates installing Ollama with a script on Jetson Orin series mainboards. It is already installed on the factory system, so you can skip this section.
Note: If pulling from Ollama fails, that is normal. Try several times, or pull through a proxy. How to set up a proxy is something you need to look up yourself.
Script installation
sudo apt install curl -ysudo curl -fsSL https://ollama.com/install.sh | sh

The whole installation process takes a long time; please be patient.
4. Using Ollama
Enter ollama in the terminal to see the help output:

| Command | Purpose |
|---|---|
| ollama serve | Start Ollama |
| ollama create | Create a model from a model file |
| ollama show | Show model information |
| ollama run | Run a model |
| ollama pull | Pull a model from the registry |
| ollama push | Push a model to the registry |
| ollama list | List models |
| ollama ps | List running models |
| ollama cp | Copy a model |
| ollama rm | Remove a model |
| ollama help | Get help on any command |
5. Uninstalling Ollama
Remove the service
xxxxxxxxxxsudo systemctl stop ollamasudo systemctl disable ollamasudo rm /etc/systemd/system/ollama.service
Remove the files
xxxxxxxxxxsudo rm $(which ollama)
Remove the models, service user, and group
xxxxxxxxxxsudo rm -r /usr/share/ollamasudo userdel ollamasudo groupdel ollama
References
Ollama
Official website: https://ollama.com/
GitHub: https://github.com/ollama/ollama