0. AI large model image description

This tutorial primarily explains the differences between the factory image for the Raspberry Pi 5 Large Model package and the factory images for other packages.

Important Preface

The Raspberry Pi 5 AI Large Model Development Tutorial requires the pre-configured large model image: 12.Raspberry Pi 5 Image Operating System\Raspberry Pi 5_Factory Image_Online + Offline Large Model\Pi5_AI_Pure.img

1. If you purchased the Raspberry Pi 5 AI Large Model package, the default image on the TF card you received is the AI ​​Large Model image, which is only suitable for running the AI ​​Large Model Development chapter. To run examples in other chapters, please burn the Pi-OS-ROS image in the "Factory Image_ROS+AI Vision" folder on the Raspberry Pi 5.

2. Because Ollama requires a large amount of memory to run large models, motherboards with limited memory may only be able to use models with smaller parameters or may not run at all. Sometimes, performance may be poor when running at the maximum memory usage. For a better experience with this feature, please refer to the Online Large Model section.

3. The commands in the Large AI Model section are primarily executed within Docker. Before running these commands, you must first enter the Docker container. You may experience some lag when running within a Docker container, which is normal.

4. The ros2_docker.sh script creates a container upon initial execution. If you do not manually delete this container, subsequent executions of the script will continue to enter this container. If you encounter any issues or need to modify the Docker script, please delete the existing Docker container and then re-run the script.

System Information

Based on Raspberry Pi OS (64-bit): 2025-05-13

Username: pi

User Password: yahboom

Hotspot Name: Pi_Hot

Hotspot Password: 12345678

Hotspot IP: 192.168.1.11

System Environment

• Ollama

• Docker

• Open WebUI

• Offline Large Model Storage Path: /usr/share/ollama/.ollama/models

xxxxxxxxxx
Due to SD (TF) card limitations, we have not downloaded all tutorial models to the Pi5_AI_Pure.img image. We have deleted some high-parameter versions of the models in the tutorial. Users can delete and download models on their own.

Parameter Description

Raspberry Pi 5B (16GB RAM): Runs models with 14GB or less parameters.

Raspberry Pi 5B (8GB RAM): Runs models with 14GB or less parameters. RAM): Runs models with 8GB parameters or less.

Raspberry Pi 5B (4GB RAM): Runs models with 3GB parameters or less.

Raspberry Pi 5B (2GB RAM): Runs models with 1GB parameters or less.

Raspberry Pi 5B (1G RAM): Cannot run offline large models. Please refer to the online large model routines.

While the above conclusions are not completely accurate, they serve as a reference!

Ollam's official website explains that models with 7GB parameters require at least 8GB of available RAM, models with 13GB parameters require at least 16GB of available RAM, and models with 33GB parameters require at least 32GB of available RAM.