AI Large Model Image Usage Instructions

This tutorial mainly introduces the differences between the mainboard AI large model image and the car's factory image.

Important Notes

1. The TF card/SSD included with the purchased robot kit contains the factory default image for the robot, which is only suitable for running the AI large language model development chapters for the robot. To run the examples in this chapter, please flash the corresponding image for your main board:

2. Because Ollam requires a lot of memory to run large models, mainboards with limited memory can only use models with small parameters, or even cannot run them. Sometimes, when memory usage is at its limit, the performance will be poor. For a better experience of this feature, please refer to the online large model chapter.

3. The commands in the AI ​​large model section of the Raspberry Pi 5 are mostly run within Docker containers. You need to enter a Docker container before running any commands. This is not necessary for RDK X5 and Orin mainboards. Slowing down may occur when running within a Docker container; this is normal.

4. The ros2_docker.sh script on the Raspberry Pi 5 creates a container on its first run. If you don't manually delete this container, subsequent runs of the script will continuously enter it. If you encounter any issues or need to modify the Docker script, please delete the existing Docker container before rerunning the script.

System Information

• Raspberry Pi 5

Based on Raspberry Pi OS (64-bit): 2025-05-13

Username: pi

Password: yahboom

Hotspot Name: Pi_Hot

Hotspot Password: 12345678

Hotspot IP: 192.168.1.11

• RDK X5

Based on rdk_os_3.3.1-2025-08-08

Username: sunrise

Password: sunrise

• ORIN

Based on jetpack 6.2, ubuntu 22.04

Username: jetson

Password: yahboom

System Environment

• Raspberry Pi 5, RDK X5, ORIN
• Ollama
• Docker
• Open WebUI
• Offline large model storage path: /usr/share/ollama/.ollama/models

​x
Due to SD (TF) card limitations, we have not downloaded all models from the tutorial to the AI large model image system (except ORIN). Some high-parameter versions of models in the tutorials have been deleted. Users can delete and download the models themselves.
​

Parameter Description

• Raspberry Pi 5

Raspberry Pi 5B (16GB RAM): Runs models with 14B or fewer parameters

Raspberry Pi 5B (8GB RAM): Runs models with 8B or fewer parameters

Raspberry Pi 5B (4GB RAM): Runs models with 3B or fewer parameters

Raspberry Pi 5B (2GB RAM): Runs models with 1B or fewer parameters

• RDK X5

RDK X5 (8GB RAM): Runs models with 8B or fewer parameters

RDK X5 (4GB RAM): Runs models with 3B or fewer parameters

While the above conclusions are not entirely accurate, they can be used as a reference!

Ollam's official website states that the following specifications apply to running models: 7B parameter models require at least 8GB of available RAM; 13B parameter models require at least 16GB of available RAM; and 33B parameter models require at least 32GB of available RAM.