yahboom
HomeTutorialsProduct
Solution
Blog
Online Store
About Us
EN/中文
Store
Jetson Orin NANO Developer Kit📖
01.Get started
  • 01. Learning route
  • 02. Get started quickly
  • 03. System Update Notes
  • 04. Disabling automatic updates and troubleshooting issues
  • 05. Jetson Orin Nano official kit screwdriver instructions
02.Jetson basic course
  • 01.Jetson board introduction
  • 02.Write Jetson original system
  • 03.Installing Jetson environment
  • 04.Backup SSD system
  • 05.Restore initial image system
  • 06.Upgrade to SUPER version (Optional)
  • 07.Write SUPER boot (Must read)
  • 08.SSD expansion
  • 09.Jetpack old system upgraded to Yahboom SUPER factory system
  • 10.Write the system onto your own SSD
03.Basic setting course
  • 01.Basic knowledge
  • 02.Network configuration
  • 03.SSH remote login
  • 04.VNC remote control
  • 05.RDP remote control
  • 06.Remote file transfer
  • 07.Jtop tool
  • 08.Exchange space expansion
  • 09.Use VSCode
  • 10.Use VMware
  • 11.Modify screen resolution
04.GPIO control course
  • 01.GPIO description
  • 02.GPIO library installation
  • 03.GPIO reading
  • 04.GPIO output
  • 05.Serial communication
  • 06.I2C communication
05.Vision basic course
  • 01.CSI camera preview
  • 02.USB camera preview
  • 03.Use Jupyter Lab
  • 04.Use JetCam
06.OpenCV basic course
  • 01.Image reading
  • 02.Image saving
  • 03.Image modification
  • 04.Image scaling
  • 05.Image cropping
  • 06.Image translation
  • 07.Image mirroring
  • 08.Image grayscale conversion
  • 09.Image binarization
  • 10.Image edge detection
  • 11.Add line segments
  • 12.Add rectangles
  • 13.Add circular shape
  • 14.Add ellipses
  • 15.Add polygons
  • 16.Add Text
  • 17.Preview USB camera image
  • 18.Preview CSI camera image
07.Vision advanced course
01.Environment construction
  • 01.TensorFlow
  • 02.PyTorch
  • 03.Torchvision
02.DeepStream development
  • 01.DeepStream environment
  • 02.License plate detection and recognition
  • 03.Parallel multi-model inference
  • 04.Object embedding vector generation
  • 05.Pose recognition
  • 06.NV optical flow anomaly detection
  • 07.Single-view 3D tracking
  • 08.Multi-view 3D tracking
03.NVIDIA TAO toolkit application
  • 01.Traffic scene object detection
  • 02.Person object detection
  • 03.Vehicle brand classification
  • 04.Vehicle model classification
  • 05.License plate detection and recognition
  • 06.Person semantic segmentation
  • 07.Retail product detection and recognition
  • 08.Person feature extraction and matching (ReID)
  • 09.Person action and pose classification
  • 10.Transformer-based person detection
  • 11.3D human skeleton detection
  • 12.LiDAR 3D point cloud detection
04.YOLO11 development
  • 01.YOLO environment construction
  • 02.CLI use
  • 03.Object detection
  • 04.Instance segmentation
  • 05.Pose estimation
  • 06.Image classification
  • 07.Oriented object detection
  • 08.Model conversion
  • 09.Dataset annotation
  • 10.Model training conversion
  • 11.Model prediction
05.Mediapipe development
  • 01.Mediapipe environment
  • 02.Hand detection
  • 03.Face detection
  • 04.Face effects
  • 05.3D object recognition
  • 06.Virtual brush
  • 07.Gesture recognition
  • 08.Finger control
  • 09.Posture estimation
  • 10.Face mask
  • 11.Overall detection
  • 12.Green scene removal(Human)
06.YOLO26 development
  • 01.YOLO environment construction
  • 02.CLI use
  • 03.Object detection
  • 04.Instance segmentation
  • 05.Pose estimation
  • 06.Image classification
  • 07.Oriented object detection
  • 08.Model conversion
  • 09.Dataset annotation
  • 10.Model training conversion
  • 11.Model prediction
07.QR code recognition
  • QR code recognition
08.Docker basic course
  • 01.Docker introduction
  • 02.Docker installation
  • 03.Docker command
  • 04.Docker usage
  • 05.Docker interaction
09.ROS2-Jazzy
01.ROS2 basic course
  • 1.Introduction to ROS2
  • 2.ROS2 install Jazzy
  • 3.ROS2 development environment
  • 4.ROS2 workspace
  • 5.ROS2 function package
  • 6.ROS2 node
  • 7.ROS2 topic communication
  • 8.ROS2 service communication
  • 9.ROS2 action communication
  • 10.ROS2 custom interface message
  • 11.ROS2 parameter service case
  • 12.ROS2 meta-function package
  • 13.ROS2 distributed communication
  • 14.ROS2 DDS
  • 15.ROS2 time related API
  • 16.ROS2 common command tools
  • 17.ROS2 rviz2 use
  • 18.ROS2 rqt toolbox
  • 19.ROS2 Launch startup file configuration
  • 20.ROS2 recording and playback tool
  • 21.ROS2 URDF model
  • 22.ROS2 Gazebo simulation platform
  • 23.ROS2 TF2 coordinate transformation
02.ROS2 vision course
  • 01.Camera preview (USB)
  • 02.Camera preview (CSI)
  • 03.Camera calibration (USB)
  • 04.Camera calibration (CSI)
  • 05.AR vision (USB)
  • 06.AR vision (CSI)
10.Large model basic course
01.Offline large model
  • 01.AI large model environment deployment
  • 02.Install large model dialogue platform
  • 03.Meta AI - Llama model
  • 04.Alibaba Cloud - Qwen model
  • 05.Microsoft - Phi-4-mini model
  • 06.DeepSeek - DeepSeek model
  • 07.Alibaba Cloud - Qwen-VL visual multimodal large model
  • 08.Google - Gemma visual multimodal large model
  • 9.Llava visual multimodal large model
  • 10.MiniCPM-V visual multimodal large model
  • 11.High-quality vision-language grounding (LocateAnything)
02.Online large model
  • 01.Tongyi Qianwen large model
  • 02.OpenRouter large model API aggregation platform
11.Large model application practice
01.Voice interaction application
  • 1.Voice interaction hardware connection (Must read)
  • 2.Offline speech to text (ASR)
  • 3.Offline text to speech (TTS)
  • 4.AI large model voice interaction
  • 5.AI large model offline voice assistant
02.Multimodal large model application (Online_Offline large model + Text_Voice interaction)
  • 01.Multimodal visual understand application
  • 02.Multimodal text image application
  • 03.Multimodal video analysis application
  • 04.Multi module visual position application
  • 05.Multimodal table scanning application
  • 06.Multimodal autonomous agent application
12.NVIDIA Isaac ROS course
  • 01.Build Isaac ROS environment
  • 02.Deep segmentation
  • 03.DNN stereoscopic depth
  • 04.Free space segmentation
  • 05.Image distortion processing
  • 06.Image segmentation
  • 07.3D scene reconstruction and mapping
  • 08.Object detection
  • 09.3D attitude estimation
  • 10.Visual SLAM
13.OpenClaw deployment and basic usage
01.OpenClaw deployment and introduction
  • 01.Quick start with the pre-configured environment (Recommended)
  • 02.OpenClaw Deployment
  • 03.OpenClaw WhatsApp Plugin Application
  • 04.OpenClaw WebChat Interaction
  • 05.OpenClaw TUI Interaction
  • 06.OpenClaw Tools Introduction
  • 07.OpenClawGate Gateway User Manual
  • 08.OpenClaw Features Overview
  • 09.OpenClawHub Introduction (Skill Installation)
02.OpenClaw basic application
  • 01.OpenClaw Application - File Management
  • 02.OpenClaw Application - Camera
  • 03.OpenClaw Application - Browser Control
  • 04.OpenClaw Application - Script Execution
  • 05.OpenClaw Application - AI Programming (Peripheral & GPIO Control)
  • 06.OpenClaw Application - Dedicated AI Assistant
  • 07.OpenClaw Web Search Capability Configuration
03.NVIDIA official Agent Skills application
  • 01.AI-assisted development with VS Code or Cursor
  • 02.Jetson device skills
  • 03.Jetson BSP custom agent skills
14.OpenClaw advanced development
01.Preparation Before Use
  • 01.Peripheral Hardware Configuration
  • 02.OpenClaw API-KEY Configuration
  • 03.OpenClaw Model Switching
  • 04.OpenClaw Prompts
  • 05.AI Voice Interaction Configuration
  • 06.Three Dialogue Configuration Tests
02.OpenClaw AI Programming Peripheral Control
  • 1. Servo Control
  • 2. RGB Strip
  • 3. OLED Display
  • 4. Temperature and Humidity Sensor
  • 5. Camera Application
  • 6. AI Script Peripheral Control
03.OpenClaw Peripheral Integration Applications
  • 1. Plant Care Assistant
  • 2. AI Calorie Estimation
  • 3. AI Rock Paper Scissors Game
  • 4. AI Pet
  • 5. AI Weather Station
  • 6. Temperature Humidity Dashboard
  • 7. Scheduled Tasks
Download
  • System_File
  • VMFile_BootFile
  • 3D Model File
  • DataSheet
  • Software_Tools
  • Code
  • Manual
Download ZIP

Welcome to Jetson Orin NANO Developer Kit repository

02.Multimodal text image application

Multimodal Text-to-Image Application

Multimodal Text-to-Image Application1. Concept Introduction1.1 What Is Text-to-Image?Core Principles1.2 What Is FastSDCPU?Core FeaturesApplicable Scenarios2. Project Deployment2.1 Deployment Environment2.2 LAN Access2.3 Using the Text-to-Image Feature2.4 Startup Method After Successful Deployment3. Online Large Model Text-to-Image (Text/Voice Interaction)3.1 Configuring Tongyi Wanxiang3.2 Online Text Input3.3 Online Voice Input

Hardware requirement: a 4GB Jetson cannot run this case; use a Jetson with 8GB or more.

Since Ollama does not support the text-to-image feature, we need to use other tools to implement the local text-to-image feature.


1. Concept Introduction

1.1 What Is Text-to-Image?

Text-to-image is an artificial intelligence technology that automatically generates a corresponding image from a text description. You only need to enter a piece of text (for example "a Shiba Inu wearing sunglasses surfing on a beach"), and the AI model generates a picture that matches the description based on semantic understanding, without any drawing or design background.

Core Principles

  1. Language understanding: The AI first parses your text and identifies keywords (such as "Shiba Inu", "sunglasses", "beach").

  2. Image generation: Trained on massive image data, the AI converts the text into visual elements and combines them into a reasonable picture.

  3. Style adaptation: A style can be specified (realistic, cartoon, watercolor, and so on), and the AI adjusts the generated result.

1.2 What Is FastSDCPU?

FastSDCPU is an open-source Stable Diffusion text-to-image project optimized specifically for CPU devices. Through algorithmic and engineering optimization, it achieves fast generation of high-quality images on ordinary computers without a GPU, significantly lowering the hardware threshold for AI painting.

Core Features

  1. Pure CPU operation: No discrete graphics card is required; it can run directly on a laptop, desktop, or edge device.

  2. High-speed inference: Combined with technologies such as Latent Consistency Models (LCM), an image can be generated in only 4–8 steps, greatly shortening the waiting time.

  3. Lightweight deployment: Supports model quantization (such as INT8) and OpenVINO acceleration, reducing resource usage and improving response efficiency.

Applicable Scenarios

  • Localized AI creation tools

  • Image generation services on enterprise intranets

  • Teaching demonstrations and prototype development

  • Lightweight deployment in resource-constrained environments

2. Project Deployment

2.1 Deployment Environment

Note: If you are using our factory image, no environment deployment is required; skip the deployment steps directly. Refer to <2.4 Startup Method After Successful Deployment> at the bottom and start it directly.

Open a terminal and execute the following code:

​x
#If git is not installed on the mainboard, run this first
sudo apt update
sudo apt install git -y
sudo apt install python3.10-venv -y

#Add environment variables
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc

# Clone the project code
git clone https://github.com/rupeshs/fastsdcpu.git
cd fastsdcpu

# Create a virtual environment and install dependencies
python -m venv venv
source venv/bin/activate

#Install uv
curl -Ls https://astral.sh/uv/install.sh | sh(this step may fail in China without a proxy; if it does not succeed, you can skip this step and run the next command)

#If uv was not installed successfully with the curl command in the previous step, run these three commands
wget https://mirrors.huaweicloud.com/astral/uv/0.8.4/uv-aarch64-unknown-linux-gnu -O ~/.local/bin/uv
chmod +x ~/.local/bin/uv
~/.local/bin/uv --version

chmod +x install.sh start-webui.sh
./install.sh --disable-gui

The installation is successful; press any key to exit:

image-20250801214649785

2.2 LAN Access

Before starting, you need to modify a file to support LAN access; otherwise the webui can only be accessed locally:

xxxxxxxxxx
vim ~/fastsdcpu/src/frontend/webui/ui.py

After opening the ui.py file, scroll to the last line, find the code webui.launch(share=share), and change it to webui.launch(server_name="0.0.0.0",share=share)

Then save

Start:

xxxxxxxxxx
./start-webui.sh

image-20250801221311739

Then you can enter your mainboard IP:7860 in the browser to access this webui.

Local text-to-image runs the model on the Jetson itself, so generation speed depends on the mainboard's compute and is usually slower than online services; the model size and the number of generation steps also affect the waiting time.

2.3 Using the Text-to-Image Feature

In the terminal, use ifconfig to query the IP of our mainboard; for example, mine is 192.168.2.106.

Then open the browser and enter your mainboard ip:7860. For example, I would enter 192.168.2.106:7860, and then I can enter the webui.

image-20250804105134704

Next, click LCM-LoRA; this model uses relatively little memory. If you want to use other models, you can explore them yourself.

Then click Models, and you can see the model settings of LCM LoRA. You can change to the model you want, or you can choose the default as I did.

image-20250804105439656

Next, click Generation Settings and increase the Inference Steps inside; this can improve the quality of the generated images. Here I set it to 5.

image-20250804111046973

Then go back to our Text to Image, enter the content we want to generate in the dialog box, and then press Generate to start generating the image.

image-20250804105656393

If you are using it for the first time, the model needs to be downloaded. You can see in the terminal that the model selected by default just now is being downloaded; after it finishes downloading, the text-to-image feature will start executing.

image-20250804105710057

The generated result:

image-20250804113118977

In actual testing, this project supports English better, and the generated images match the text more closely. It is recommended to use English descriptions to generate images.

2.4 Startup Method After Successful Deployment

xxxxxxxxxx
cd fastsdcpu #Enter the fastsdcpu directory
source venv/bin/activate #Enter the virtual environment
./start-webui.sh #Start the webui

After the webui starts successfully, enter your mainboard IP:7860 in the browser, and you can start the text-to-image feature.


3. Online Large Model Text-to-Image (Text/Voice Interaction)

The local FastSDCPU path is operated through a web page and does not support voice control; online text-to-image calls Tongyi Wanxiang through largemodel and can use text or voice input. Online and local are two independent operation modes; choose as needed.

3.1 Configuring Tongyi Wanxiang

First apply for a Tongyi Qianwen API Key according to the online large model basic tutorial, and confirm that the account has available quota. Open the model interface configuration file:

xxxxxxxxxx
vim ~/yahboom_ws/src/largemodel/config/large_model_interface.yaml

Find the Tongyi Qianwen configuration item, and fill in the API Key and the online text-to-image model. Text-to-image uses the tongyi_media_model parameter, not the qianwen_model used for visual question answering:

xxxxxxxxxx
qianwen_api_key: "Replace with your API-Key"
tongyi_media_model: "flux-schnell"

After saving, open the main configuration file:

xxxxxxxxxx
vim ~/yahboom_ws/src/largemodel/config/yahboom.yaml

In the existing model_service.ros__parameters parameter block, select the Tongyi platform; if llm_platform already exists, modify its value and do not add it again:

xxxxxxxxxx
model_service:
  ros__parameters:
    llm_platform: 'tongyi'

After saving both configuration files, restart largemodel for the settings to take effect. Online calling requires that the Jetson can access the Tongyi service, that the API Key is valid, and that the account has available quota. Do not disclose or show real API Keys in screenshots.

3.2 Online Text Input

Start text mode:

xxxxxxxxxx
ros2 launch largemodel largemodel_control.launch.py text_chat_mode:=true

In another terminal that has already loaded the ROS 2 and workspace environments, run:

xxxxxxxxxx
ros2 run text_chat text_chat

Enter, for example, "Generate a pig soaring in the universe". Observe the main program logs and the generated image that pops up; the generated file is saved in ~/yahboom_ws/src/largemodel/resources_file/generated_images.

3.3 Online Voice Input

Voice input and announcement are controlled independently by the ASR/TTS parameters. Configure the credentials according to the online voice ASR/TTS tutorials, and set in yahboom.yaml:

xxxxxxxxxx
model_service:
  ros__parameters:
    llm_platform: 'tongyi'
    use_oline_asr: True
    useolinetts: True

Start voice mode:

xxxxxxxxxx
ros2 launch largemodel largemodel_control.launch.py

After waking the device, say "Generate a pig soaring in the universe". Confirm that the terminal recognizes the voice request, that the model call succeeds, that the image pops up, and that the speaker announces the result. If you only need an online LLM with offline ASR or TTS, adjust the corresponding switches separately and configure the services according to the respective tutorials.

The running speed of local FastSDCPU is affected by the computing power of the Jetson mainboard; the generation time of online Tongyi Wanxiang depends on the network and the response of the cloud service.

Multimodal visual understand application
Multimodal video analysis application

COMPANY

  • About us
  • Team Style
  • Blog

LEARNING & EXPLORATION

  • Tutorials
  • Edge AI hardware
  • Educational robots
  • Embedded intelligent hardware

WHERE TO BUY

  • Yahboom Online Store
  • Amazon
  • Alibaba
  • AliExpress
  • Mercado

TECHNOLOGY SUPPORT

  • Email: support@yahboom.com

BE A DISTRIBUTOR

  • Email: sales@yahboom.com
QR Code

Copyright © 2017-2026 Yahboom. All Rights Reserved.

EN/中文