Model usage instructions

This tutorial mainly introduces some Raspberry Pi 5 system environments and models that can run on different RAM versions.

System image Description

The offline AI large model development tutorial for Raspberry Pi 5 can use the image already configured by Yahboom: Pi5-AI-Pure.img

System Information

user name：pi

password：yahboom

WiFi hotspot name：Pi_Hot

WiFi hotspot password：12345678

System environment

• Ollama

  • Llama3
  • Qwen2
  • Phi-3
  • Gemma
  • WizardLM2
  • Code Llama
  • DeepSeek Coder
  • LLaVA

• Docker

• Open WebUI

​x
Due to the limitations of the SD (TF) card, we did not install all the models in the tutorial onto the Pi5AI-Pure.img file. User can dlete and download models by yourself.

Parameter Description

Raspberry Pi 5B (8G RAM): Run 8B and below parameter models

Raspberry Pi 5B (4G RAM): Run 3B and below parameter models

The above conclusion may not be entirely accurate, but it can be used as a reference!

x
Ollam official website operation model description:
Run the 7B parameter model requires at least 8GB of available running memory.
Run the 13B parameter model with at least 16GB of available running memory.
Run the 33B parameter model with at least 32GB of available running memory.

Model Statistics

The following table shows common models based on the Ollama official website model library statistics.

Users can search for the desired models on the Ollama official website by themselves.

x
In theory, the larger the model parameters, the greater the memory required, and running the model can choose versions with smaller parameters to experience the functionality

Reference material

Ollama

Website：https://ollama.com/

GitHub：https://github.com/ollama/ollama