Ollama

Demonstration Environment

Development Board: RDK X5 mainboard

SD Card: 64G

Tutorial Scope

Ollama is an open-source tool designed to simplify the deployment and running process of large language models, allowing users to use high-quality language models in a local environment.

1. Large Language Models (LLM)

Large Language Models (LLMs) are a class of advanced text generation systems based on artificial intelligence technology. Their main feature is their ability to learn and understand human language through large-scale training data and to generate naturally fluent text.

2. Ollama Installation

Note: It is recommended to use the image with the environment configured by Yahboom. It takes a long time to configure the environment personally, and there may be exceptions during installation on different system versions.

The tutorial demonstrates how to install Ollama on the rdk x5 mainboard using a script. (The large language model image provided by Yahboom has the environment configured and does not need to be installed again.)

Script Installation

sudo apt install curl -y
sudo curl -fsSL https://ollama.com/install.sh | sh

The entire installation process takes a long time, please wait patiently!

Note: It is normal for ollama pull to fail; you can try multiple times, or pull after setting up a proxy. The method for setting up a proxy requires you to search online yourself.

It is complete when the installation is displayed:

3. Using Ollama

Enter ollama in the terminal to see the prompt:

4. Uninstalling Ollama

• Delete the service

xxxxxxxxxx
sudo systemctl stop ollama
sudo systemctl disable ollama
sudo rm /etc/systemd/system/ollama.service

• Delete the file

xxxxxxxxxx
sudo rm $(which ollama)

• Delete the model and service user and group

xxxxxxxxxx
sudo rm -r /usr/share/ollama
sudo userdel ollama
sudo groupdel ollama

References

Ollama

Official Website: https://ollama.com/

GitHub: https://github.com/ollama/ollama