Open WebUI

The large models available in Open WebUI are synchronized with Ollama. Users need to manually pull the large model from Ollama before switching to the corresponding model.

Demo Environment

Development Board: RDK S600 Development Board

Open WebUI is an open-source project designed to provide a simple and easy-to-use user interface (UI) for managing and monitoring open-source software and services.

xxxxxxxxxx
When using Open WebUI, there is a high probability of encountering unresponsive dialogues or timeouts. Try restarting Open WebUI or using the Ollama tool to run the model!

1. Environment Requirements

Installing Open WebUI on the host machine and Conda: Requires Node.js >= 20.10, Python = 3.11:

This tutorial demonstrates installing Open WebUI using Docker.

2. Docker Setup

2.1 Installing Docker from the Official Website

If Docker is not installed, you can use a script to install it with one click.

xxxxxxxxxx
Domestic users may not be able to install Docker through the official methods. We recommend using Alibaba Cloud to install Docker or directly using our image.

• Update local package list

xxxxxxxxxx
sudo apt update

• Upgrade installed packages

xxxxxxxxxx
sudo apt upgrade

• Download and run script

Download the get-docker.sh file and save it in the current directory.

xxxxxxxxxx
sudo apt install curl
curl -fsSL https://get.docker.com -o get-docker.sh

Run the get-docker.sh script file with sudo privileges.

xxxxxxxxxx
sudo sh get-docker.sh

2.2 Installing Docker on Alibaba Cloud

If you cannot install it yourself, please use the image we provide.

• Update local package list

xxxxxxxxxx
sudo apt update

• Install essential software

xxxxxxxxxx
sudo apt install apt-transport-https ca-certificates curl gnupg2 lsb-release software-properties-common

• Add GPG key for software repository

xxxxxxxxxx
curl -fsSL https://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

• Add Alibaba Cloud mirror software repository

xxxxxxxxxx
echo "deb [arch=arm64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://mirrors.aliyun.com/docker-ce/linux/ubuntu noble stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

• Install Docker

xxxxxxxxxx
sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io docker-compose-plugin

2.3 Add Access Permissions

Add access permissions for the current system user to the Docker daemon: Docker commands can be used without using the sudo command

xxxxxxxxxx
sudo usermod -aG docker $USER
newgrp docker

2.4 Compatibility Modification

After successful installation, starting Docker may fail:

This issue stems from a compatibility conflict between Linux network stack architecture updates and Docker's network implementation. This error typically occurs on newer Linux distributions that default to the nf_tables backend, which Docker may have compatibility issues with. Therefore, switching to legacy mode (using iptables-legacy) can resolve most such issues.

1. Switch to legacy iptables

xxxxxxxxxx
sudo update-alternatives --set iptables /usr/sbin/iptables-legacy
sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy

2. Check the current iptables mode

xxxxxxxxxx
sudo iptables --version
# It should display "legacy" instead of "nf_tables"

3. Clean up iptables rules

xxxxxxxxxx
sudo iptables -t nat -F
sudo iptables -t nat -X
sudo iptables -F
sudo iptables -X

4. Restart Docker

xxxxxxxxxx
sudo systemctl restart docker

5. Check if the Docker service is running successfully

xxxxxxxxxx
sudo systemctl status docker

3. Installing Open WebUI

For systems with Docker already installed, you can directly enter the following command in the terminal: The image shows the result after pulling.

xxxxxxxxxx
docker pull ghcr.io/open-webui/open-webui:main

4. Running Open WebUI

Enter the following command in the terminal to start the specified Docker container:

xxxxxxxxxx
docker run --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main

After successful startup, access it in your browser using the following URL:

xxxxxxxxxx
http://localhost:8080/

Access can be made via the motherboard IP:8080 on the same local area network:

xxxxxxxxxx
Assuming the motherboard IP:192.168.2.105, we can access it via 192.168.2.105:8080

This method is recommended because when accessing the site locally on the motherboard, the browser requires some memory. If you then try to run a 7B or 8B model, it will prompt an insufficient memory error and fail to run.

4.1 Administrator Account

First-time users need to register an account. This account is the administrator account. You can fill in the required information yourself!

xxxxxxxxxx
Since all content in our mirror has been set up and tested, users can log in directly using their registered account:
Username: yahboom
Email: yahboom@163.com
Password: yahboom

4.2 Registration and Login

4.3 User Interface

5 Model Dialogue

Using the Open WebUI for dialogue will be slower than using the Ollam tool directly, and may even result in timeouts and connection failures. This is related to the Jetson motherboard's memory and is unavoidable!

xxxxxxxxxx
Users interested can switch to other Linux environments to set up the Ollam and Open WebUI tools for interaction.

5.1 Switching Models

Clicking Select a model in the upper left corner allows you to select a specific model for interaction.

xxxxxxxxxx
Models retrieved using Ollam are automatically added to the Open WebUI model options. Refreshing the webpage will display the new model! 

5.2 Demo Model: minicpm-v:8b

xxxxxxxxxx
What do you see in this picture?

6. Frequently Asked Questions

6.1 Disabling Open WebUI

Disable the automatically starting Open WebUI.

• View running Docker containers

xxxxxxxxxx
docker ps

• Shut down running Docker containers

xxxxxxxxxx
docker stop [CONTAINER ID] # Example docker stop 5f42ee9cf784

• View stopped containers

xxxxxxxxxx
docker ps -a

• Clean up stopped containers

xxxxxxxxxx
docker rm [CONTAINER ID] # Example docker rm 5f42ee9cf784

Clean up all stopped containers:

xxxxxxxxxx
docker container prune

6.2 Common Errors

Unable to start Open WebUI

• docker: Error response from daemon: Conflict. The container name "/open-webui" is already in use by container "cfc05c84f8e38b290337e7178c76fd1c49076f94b11ed3d49d9448be72b7f20f". You have to remove (or rename) that container to reuse that name.

Solution: Close and restart Open WebUI!

Service Connection Timeout

• Open WebUI: Server Connection Error

Close and restart Open WebUI, then try asking the question again, or use the Ollam tool to run the model and ask the question!

6.3 Insufficient Memory Error

When using a model with a large data volume to generate queries, the following error occurs:

• 500: model requires more system memory (6.6 GiB) than is available (5.2 GiB)

1. It is recommended to use a computer browser accessing the motherboard via IP:8080. Accessing the model locally on the motherboard requires some memory, and running a 7B or 8B model under these conditions will result in an insufficient memory error and the model will fail to run.
2. Running Open WebUI also consumes some memory, which may prevent some larger models from running, such as Ollama_Instead of running it directly, consider pulling a larger model with a smaller dataset, or migrating it to Ollama to run the larger model.
3. Increase swap space content.

# Create a 4GB swap file
sudo fallocate -l 4G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile