Multimodal Text-to-Image Application

The Raspberry Pi 5 1GB version cannot run due to memory limitations.

Since Ollama doesn't support text-to-image, we need to use other tools to implement this functionality locally.

1. Concept Introduction

1.1 What is Text-to-Image?

Text-to-Image is an AI technology that automatically generates images based on text descriptions. Simply enter a text (e.g., "A Shiba Inu wearing sunglasses surfing on the beach"), and the AI ​​model will generate an image that matches the description based on semantic understanding. No painting or design skills are required.

Core Principles

1. Language Understanding: The AI ​​first parses your text and identifies keywords (e.g., "Shiba Inu," "sunglasses," "beach").

2. Image Generation: Based on training from massive image data, the AI ​​converts text into visual elements and combines them into a reasonable image.

3. Style Adaptation: You can specify a style (realistic, cartoon, watercolor, etc.), and the AI ​​will adjust the generated image accordingly.

1.2 What is FastSDCPU?

FastSDCPU is an open-source Stable Diffusion image processing project optimized for CPU devices. Through algorithmic and engineering optimizations, it enables rapid generation of high-quality images on standard computers without a GPU, significantly lowering the hardware barrier to entry for AI painting.

Core Features

1. Pure CPU Operation: No dedicated graphics card required, allowing it to run directly on laptops, desktops, or edge devices.

2. High-Speed ​​Inference: Incorporating technologies such as Latent Consistency Models (LCM), image generation takes only 4–8 steps, significantly reducing latency.

3. Lightweight Deployment: Supports model quantization (e.g., INT8) and OpenVINO acceleration, reducing resource usage and improving responsiveness.

Applicable Scenarios

• Localized AI Creation Tool

• Intranet Image Generation Service

• Teaching Demonstrations and Prototyping

• Lightweight Deployment in Resource-Constrained Environments

2. Project Deployment

2.1 Deployment Environment

Note: If using our pre-installed image, there's no need to deploy an environment and you can skip the deployment steps. Simply refer to <2.4 How to Start After Deployment> at the bottom to start the application. Then, return to <2.3 Using the Wenshengtu Function> to learn how to use it.

Open a terminal and execute the following code:

​x
#If there is no git on the motherboard, run
sudo apt update
sudo apt install git -y
sudo apt install python3.10-venv -y

#Add environment variables
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc

# Clone the project code
git clone https://github.com/rupeshs/fastsdcpu.git
cd fastsdcpu

# Create a virtual environment and install dependencies
python -m venv venv
source venv/bin/activate

#Install uv
curl -Ls https://astral.sh/uv/install.sh | sh(This step may not be successful if there is no proxy in China. If not, you can skip this step and execute the next command)

#If the curl command failed to install uv in the previous step, run these three instructions
wget https://mirrors.huaweicloud.com/astral/uv/0.8.4/uv-aarch64-unknown-linux-gnu -O ~/.local/bin/uv
chmod +x ~/.local/bin/uv
~/.local/bin/uv --version

chmod +x install.sh start-webui.sh
./install.sh --disable-gui

Installation successful, press any key to exit:

2.2 LAN Access

Before starting, you need to modify a file to support LAN access. Otherwise, the webUI can only be accessed locally:

xxxxxxxxxx
vim ~/fastsdcpu/src/frontend/webui/ui.py

After opening the ui.py file, scroll to the last line and find webui.launch(share=share). Change it to webui.launch(server_name="0.0.0.0",share=share).

Save the file.

Start:

xxxxxxxxxx
./start-webui.sh

Then, enter your motherboard's IP address: 7860 in your browser to access the webUI.

2.3 Using the Vinyl Image Function

Use ifconfig in the terminal to query your motherboard's IP address. For example, mine is 192.168.2.106.

Then open your browser and enter your motherboard's IP address: 7860 . For example, I entered 192.168.2.106:7860 . This will allow you to access the webUI.

Next, click LCM-LoRA. This model uses less memory. If you prefer, feel free to explore other models.

Next, click Models to view the LCM LoRA model settings. You can change the model you want, or just stick with the default, as I did.

Next, click Generation Settings and increase the Inference Steps setting to improve the quality of the generated image. I've set it to 5.

Next, return to the Text to Image dialog box and enter the content you want to generate. Click Generate to begin generating the image.

For first-time users, you'll need to download the model. You'll see the default model being downloaded in the terminal. Once it's finished downloading, the text-to-image function will begin.

Generated result:

This project has better support for English, and the generated images are more consistent with the text. We recommend using English descriptions when generating images.

2.4 How to start after successful deployment

xxxxxxxxxx
cd fastsdcpu #Enter the fastsdcpu directory
source venv/bin/activate #Enter the virtual environment
./start-webui.sh #Start the webui

After the webui is successfully started, enter your motherboard's IP address: 7860 in your browser to start the image generation function.