AI Large Model Environment Usage Guide

This tutorial mainly introduces the differences between the factory image and SSD factory image for the Raspberry Pi 5 large model package.

Important Note Before Starting

The Raspberry Pi 5 AI Large Model Development Tutorial requires the pre-configured large model image: SystemFlie\RPi5(2_4_8_16GB)_System_Online+OfflineLLM_openclaw_SSD-TFCard_xx\PI5_AI_Pure_openclaw_xx.img

SSD users: The image is an integrated factory image. You can directly run all tutorial examples.

1. If you purchased the Raspberry Pi 5 AI Large Model package, the default image on the TF card you received is the AI Large Model image, which is only suitable for running the AI Large Model Development chapter. To run examples in other chapters, please burn the Pi-OS-ROS image: SystemFlie\RPi5(2-4-8-16G)_ROS+openclaw+AIVisual_SSD-TFCard_xx\Pi-os-ROS-openclaw_xx.zip.

2. Because Ollama requires a large amount of memory to run large models, motherboards with limited memory may only be able to use models with smaller parameters or may not run at all. Sometimes, performance may be poor when running at the maximum memory usage. For a better experience with this feature, please refer to the Online Large Model section.

3. The commands in the Large AI Model section are primarily executed within Docker. Before running these commands, you must first enter the Docker container. You may experience some lag when running within a Docker container, which is normal.

4. The ros2_docker.sh script creates a container upon initial execution. If you do not manually delete this container, subsequent executions of the script will continue to enter this container. If you encounter any issues or need to modify the Docker script, please delete the existing Docker container and then re-run the script.

System Information

Based on Raspberry Pi OS (64-bit): 2025-05-13

Username: pi

User password: yahboom

Hotspot name: Pi_Hot

Hotspot password: 12345678

Hotspot IP: 192.168.1.11

System Environment

• Ollama

• Docker

• Open WebUI

• Offline large model storage path: /usr/share/ollama/.ollama/models

xxxxxxxxxx
Due to SD (TF) card limitations, the factory image may not have all models pre-downloaded. If the system prompts that a model does not exist when running, you can download it according to the corresponding model tutorial or delete models you don't need.

Parameter Description

Raspberry Pi 5B (16G RAM): Run models with 14B parameters or below

Raspberry Pi 5B (8G RAM): Run models with 8B parameters or below

Raspberry Pi 5B (4G RAM): Run models with 3B parameters or below

Raspberry Pi 5B (2G RAM): Run models with 1B parameters or below

Raspberry Pi 5B (1G RAM): Cannot run offline large models, please refer to the online large model tutorial

Although the above conclusions are not completely accurate, they can be used as a reference!

Ollama official website model running description: Running 7B parameter model requires at least 8G available memory, running 13B parameter model requires at least 16G available memory, running 33B parameter model requires at least 32G available memory.