Ollama

If you are using the factory image we provided, manual installation and configuration are not required.

Demo Environment

Development Board: Raspberry Pi 5

SD (TF) Card: 64G (16G or above - the larger the capacity, the more models you can experience)

Raspberry Pi 5 (16G RAM): Run models with 14B parameters or below
Raspberry Pi 5 (8G RAM): Run models with 8B parameters or below
Raspberry Pi 5 (4G RAM): Run models with 3B parameters or below
Raspberry Pi 5 (2G RAM): Run models with 1B parameters or below
Raspberry Pi 5 (1G RAM): Cannot run offline large models, please refer to the online large model tutorial

Ollama is an open-source tool designed to simplify the deployment and operation of large language models, enabling users to use high-quality language models in a local environment.

1. Large Language Models (LLM)

Large Language Models (LLM) are a class of advanced text generation systems based on AI technology. Their main characteristic is the ability to learn and understand human language through large-scale training data, and generate natural and fluent text.

2. Ollama Installation

Note: Failure to pull ollama is normal. You can try multiple times or use a proxy to pull. How to use a proxy needs to be researched online.

Script Installation

xxxxxxxxxx
sudo apt install curl -y
sudo curl -fsSL https://ollama.com/install.sh | sh

The installation process takes a while, please be patient. When the Raspberry Pi terminal displays "Install complete", Ollama is successfully installed!

xxxxxxxxxx
The warning indicates that NVIDIA/AMD GPU was not detected. Ollama will run in CPU mode. We can directly ignore this warning.

3. Using Ollama

Enter ollama in the terminal to see the prompt:

4. Uninstall Ollama

• Remove service

xxxxxxxxxx
sudo systemctl stop ollama
sudo systemctl disable ollama
sudo rm /etc/systemd/system/ollama.service

• Remove files

xxxxxxxxxx
sudo rm $(which ollama)

• Remove models and service users and groups

xxxxxxxxxx
sudo rm -r /usr/share/ollama
sudo userdel ollama
sudo groupdel ollama

References

Ollama

Website: https://ollama.com/

GitHub: https://github.com/ollama/ollama