yahboom
HomeTutorialsProduct
Solution
Blog
Online Store
About Us
EN/中文
Store
Jetson Orin NANO Developer Kit📖
01.Get started
  • 01. Learning route
  • 02. Get started quickly
  • 03. System Update Notes
  • 04. Disabling automatic updates and troubleshooting issues
  • 05. Jetson Orin Nano official kit screwdriver instructions
02.Jetson basic course
  • 01.Jetson board introduction
  • 02.Write Jetson original system
  • 03.Installing Jetson environment
  • 04.Backup SSD system
  • 05.Restore initial image system
  • 06.Upgrade to SUPER version (Optional)
  • 07.Write SUPER boot (Must read)
  • 08.SSD expansion
  • 09.Jetpack old system upgraded to Yahboom SUPER factory system
  • 10.Write the system onto your own SSD
03.Basic setting course
  • 01.Basic knowledge
  • 02.Network configuration
  • 03.SSH remote login
  • 04.VNC remote control
  • 05.RDP remote control
  • 06.Remote file transfer
  • 07.Jtop tool
  • 08.Exchange space expansion
  • 09.Use VSCode
  • 10.Use VMware
  • 11.Modify screen resolution
04.GPIO control course
  • 01.GPIO description
  • 02.GPIO library installation
  • 03.GPIO reading
  • 04.GPIO output
  • 05.Serial communication
  • 06.I2C communication
05.Vision basic course
  • 01.CSI camera preview
  • 02.USB camera preview
  • 03.Use Jupyter Lab
  • 04.Use JetCam
06.OpenCV basic course
  • 01.Image reading
  • 02.Image saving
  • 03.Image modification
  • 04.Image scaling
  • 05.Image cropping
  • 06.Image translation
  • 07.Image mirroring
  • 08.Image grayscale conversion
  • 09.Image binarization
  • 10.Image edge detection
  • 11.Add line segments
  • 12.Add rectangles
  • 13.Add circular shape
  • 14.Add ellipses
  • 15.Add polygons
  • 16.Add Text
  • 17.Preview USB camera image
  • 18.Preview CSI camera image
07.Vision advanced course
01.Environment construction
  • 01.TensorFlow
  • 02.PyTorch
  • 03.Torchvision
02.DeepStream development
  • 01.DeepStream environment
  • 02.License plate detection and recognition
  • 03.Parallel multi-model inference
  • 04.Object embedding vector generation
  • 05.Pose recognition
  • 06.NV optical flow anomaly detection
  • 07.Single-view 3D tracking
  • 08.Multi-view 3D tracking
03.NVIDIA TAO toolkit application
  • 01.Traffic scene object detection
  • 02.Person object detection
  • 03.Vehicle brand classification
  • 04.Vehicle model classification
  • 05.License plate detection and recognition
  • 06.Person semantic segmentation
  • 07.Retail product detection and recognition
  • 08.Person feature extraction and matching (ReID)
  • 09.Person action and pose classification
  • 10.Transformer-based person detection
  • 11.3D human skeleton detection
  • 12.LiDAR 3D point cloud detection
04.YOLO11 development
  • 01.YOLO environment construction
  • 02.CLI use
  • 03.Object detection
  • 04.Instance segmentation
  • 05.Pose estimation
  • 06.Image classification
  • 07.Oriented object detection
  • 08.Model conversion
  • 09.Dataset annotation
  • 10.Model training conversion
  • 11.Model prediction
05.Mediapipe development
  • 01.Mediapipe environment
  • 02.Hand detection
  • 03.Face detection
  • 04.Face effects
  • 05.3D object recognition
  • 06.Virtual brush
  • 07.Gesture recognition
  • 08.Finger control
  • 09.Posture estimation
  • 10.Face mask
  • 11.Overall detection
  • 12.Green scene removal(Human)
06.YOLO26 development
  • 01.YOLO environment construction
  • 02.CLI use
  • 03.Object detection
  • 04.Instance segmentation
  • 05.Pose estimation
  • 06.Image classification
  • 07.Oriented object detection
  • 08.Model conversion
  • 09.Dataset annotation
  • 10.Model training conversion
  • 11.Model prediction
07.QR code recognition
  • QR code recognition
08.Docker basic course
  • 01.Docker introduction
  • 02.Docker installation
  • 03.Docker command
  • 04.Docker usage
  • 05.Docker interaction
09.ROS2-Jazzy
01.ROS2 basic course
  • 1.Introduction to ROS2
  • 2.ROS2 install Jazzy
  • 3.ROS2 development environment
  • 4.ROS2 workspace
  • 5.ROS2 function package
  • 6.ROS2 node
  • 7.ROS2 topic communication
  • 8.ROS2 service communication
  • 9.ROS2 action communication
  • 10.ROS2 custom interface message
  • 11.ROS2 parameter service case
  • 12.ROS2 meta-function package
  • 13.ROS2 distributed communication
  • 14.ROS2 DDS
  • 15.ROS2 time related API
  • 16.ROS2 common command tools
  • 17.ROS2 rviz2 use
  • 18.ROS2 rqt toolbox
  • 19.ROS2 Launch startup file configuration
  • 20.ROS2 recording and playback tool
  • 21.ROS2 URDF model
  • 22.ROS2 Gazebo simulation platform
  • 23.ROS2 TF2 coordinate transformation
02.ROS2 vision course
  • 01.Camera preview (USB)
  • 02.Camera preview (CSI)
  • 03.Camera calibration (USB)
  • 04.Camera calibration (CSI)
  • 05.AR vision (USB)
  • 06.AR vision (CSI)
10.Large model basic course
01.Offline large model
  • 01.AI large model environment deployment
  • 02.Install large model dialogue platform
  • 03.Meta AI - Llama model
  • 04.Alibaba Cloud - Qwen model
  • 05.Microsoft - Phi-4-mini model
  • 06.DeepSeek - DeepSeek model
  • 07.Alibaba Cloud - Qwen-VL visual multimodal large model
  • 08.Google - Gemma visual multimodal large model
  • 9.Llava visual multimodal large model
  • 10.MiniCPM-V visual multimodal large model
  • 11.High-quality vision-language grounding (LocateAnything)
02.Online large model
  • 01.Tongyi Qianwen large model
  • 02.OpenRouter large model API aggregation platform
11.Large model application practice
01.Voice interaction application
  • 1.Voice interaction hardware connection (Must read)
  • 2.Offline speech to text (ASR)
  • 3.Offline text to speech (TTS)
  • 4.AI large model voice interaction
  • 5.AI large model offline voice assistant
02.Multimodal large model application (Online_Offline large model + Text_Voice interaction)
  • 01.Multimodal visual understand application
  • 02.Multimodal text image application
  • 03.Multimodal video analysis application
  • 04.Multi module visual position application
  • 05.Multimodal table scanning application
  • 06.Multimodal autonomous agent application
12.NVIDIA Isaac ROS course
  • 01.Build Isaac ROS environment
  • 02.Deep segmentation
  • 03.DNN stereoscopic depth
  • 04.Free space segmentation
  • 05.Image distortion processing
  • 06.Image segmentation
  • 07.3D scene reconstruction and mapping
  • 08.Object detection
  • 09.3D attitude estimation
  • 10.Visual SLAM
13.OpenClaw deployment and basic usage
01.OpenClaw deployment and introduction
  • 01.Quick start with the pre-configured environment (Recommended)
  • 02.OpenClaw Deployment
  • 03.OpenClaw WhatsApp Plugin Application
  • 04.OpenClaw WebChat Interaction
  • 05.OpenClaw TUI Interaction
  • 06.OpenClaw Tools Introduction
  • 07.OpenClawGate Gateway User Manual
  • 08.OpenClaw Features Overview
  • 09.OpenClawHub Introduction (Skill Installation)
02.OpenClaw basic application
  • 01.OpenClaw Application - File Management
  • 02.OpenClaw Application - Camera
  • 03.OpenClaw Application - Browser Control
  • 04.OpenClaw Application - Script Execution
  • 05.OpenClaw Application - AI Programming (Peripheral & GPIO Control)
  • 06.OpenClaw Application - Dedicated AI Assistant
  • 07.OpenClaw Web Search Capability Configuration
03.NVIDIA official Agent Skills application
  • 01.AI-assisted development with VS Code or Cursor
  • 02.Jetson device skills
  • 03.Jetson BSP custom agent skills
14.OpenClaw advanced development
01.Preparation Before Use
  • 01.Peripheral Hardware Configuration
  • 02.OpenClaw API-KEY Configuration
  • 03.OpenClaw Model Switching
  • 04.OpenClaw Prompts
  • 05.AI Voice Interaction Configuration
  • 06.Three Dialogue Configuration Tests
02.OpenClaw AI Programming Peripheral Control
  • 1. Servo Control
  • 2. RGB Strip
  • 3. OLED Display
  • 4. Temperature and Humidity Sensor
  • 5. Camera Application
  • 6. AI Script Peripheral Control
03.OpenClaw Peripheral Integration Applications
  • 1. Plant Care Assistant
  • 2. AI Calorie Estimation
  • 3. AI Rock Paper Scissors Game
  • 4. AI Pet
  • 5. AI Weather Station
  • 6. Temperature Humidity Dashboard
  • 7. Scheduled Tasks
Download
  • System_File
  • VMFile_BootFile
  • 3D Model File
  • DataSheet
  • Software_Tools
  • Code
  • Manual
Download ZIP

Welcome to Jetson Orin NANO Developer Kit repository

3.Offline text to speech (TTS)

2. Text-to-Speech (TTS, Online/Offline)

2. Text-to-Speech (TTS, Online/Offline)1. Concept Introduction1.1 What is "TTS"?1.2 Implementation Principles Overview1. Text Analysis2. Language Processing3. Speech Synthesis4. Sound Waveform Generation2. Code AnalysisKey Code1. TTS Initialization and Invocation (largemodel/largemodel/model_service.py)2. TTS Backend Implementation (largemodel/utils/large_model_interface.py)Code Analysis3. Practical Operations3.1 Configuring TTS3.1.1 Configuring Offline TTS3.1.2 Configuring Online TTS3.2 Starting and Testing the Feature4. Common Problems and Solutions4.1 Playback ProblemsProblem 1: The program runs normally with no errors, but no sound can be heard.


1. Concept Introduction

1.1 What is "TTS"?

TTS technology converts written text into speech output that humans can hear. It enables computers to "read aloud" text content and is widely used in accessible reading, intelligent assistants, navigation systems, educational software, and many other fields. Through TTS, users can hear a natural and fluent machine-generated human voice, which greatly improves the convenience and flexibility of information acquisition.

1.2 Implementation Principles Overview

The implementation of a TTS system mainly includes the following key steps and technologies:

1. Text Analysis

  • At this stage, the input text is first preprocessed, including but not limited to removing irrelevant characters, normalizing punctuation and letter case, word segmentation, and identifying special symbols such as numbers and converting them into the corresponding word forms.

  • At the same time, linguistic analysis of the text is also required, for example determining the pronunciation of each word (which usually requires a pronunciation dictionary), the stress position, the intonation pattern, and the sentence structure.

2. Language Processing

  • This step mainly focuses on how to correctly pronounce and adjust intonation according to the context. For example, the word "read" has different pronunciations in different tenses (past tense/past participle pronounced /red/, while in other cases pronounced /riːd/). Therefore, a powerful language model is needed to understand these subtle differences.

  • Prosody modeling is also involved here, that is, deciding which parts should be emphasized, whether the speaking rate should be fast or slow, and the emotional tone of the whole sentence.

3. Speech Synthesis

  • The text information processed in the previous two stages is fed into a speech synthesis engine, which is responsible for generating the actual sound waveform.

  • Traditional TTS systems use concatenative synthesis, that is, selecting suitable units from a database of pre-recorded speech segments and concatenating them to form a complete sentence. Although this method can produce high-quality speech, it is limited by the samples in the database.

  • Modern TTS systems rely more on parametric synthesis or neural network synthesis (such as WaveNet, Tacotron, and so on). These methods can predict speech features directly from text and generate continuous sound signals. Deep learning-based methods in particular can better capture the subtle variations in speech, thereby producing more natural and fluent voices.

4. Sound Waveform Generation

  • Finally, the generated sound waveform is further processed to ensure that its quality meets the expected standards, for example adjusting the volume and equalizing the frequency response.

  • After that, these audio data can be played through speakers or other audio playback devices for people to listen to.

With the development of artificial intelligence and machine learning technologies, especially the application of deep learning, TTS systems have not only improved significantly in accuracy but also made considerable progress in naturalness and emotional expression, making machine-generated speech increasingly close to real human voice.

2. Code Analysis

Key Code

1. TTS Initialization and Invocation (largemodel/largemodel/model_service.py)

​x
# From largemodel/largemodel/model_service.py
class LargeModelService(Node):
    def __init__(self):
        # ...
        self.system_sound_init()
        # ...

    def init_param_config(self):
        # ...
        self.declare_parameter('useolinetts', False)
        self.useolinetts = self.get_parameter('useolinetts').get_parameter_value().bool_value
        if self.useolinetts:
            self.tts_out_path = os.path.join(self.pkg_path, "resources_file", "tts_output.mp3")
        else:
            self.tts_out_path = os.path.join(self.pkg_path, "resources_file", "tts_output.wav")

    def system_sound_init(self):
        """Initialize TTS system"""
        model_type = "oline" if self.useolinetts else "local"
        self.model_client.tts_model_init(model_type, self.language)
        self.get_logger().info(f'TTS initialized with {model_type} model')

    def _safe_play_audio(self, text_to_speak: str):
        """
        Synthesizes and plays all non-empty messages only in non-text chat mode.
        """
        if not self.text_chat_mode and text_to_speak:
            try:
                self.model_client.voice_synthesis(text_to_speak, self.tts_out_path)
                self.play_audio_async(self.tts_out_path)
            except Exception as e:
                self.get_logger().error(f"Safe audio playback failed: {e}")

2. TTS Backend Implementation (largemodel/utils/large_model_interface.py)

xxxxxxxxxx
# From largemodel/utils/large_model_interface.py
class model_interface:
    # ...
    def tts_model_init(self,model_type='oline',language='zh'):
        if model_type=='oline':
            if self.tts_supplier=='baidu':
                self.token=self.fetch_token()
    
            self.model_type='oline'      
        elif model_type=='local':
            self.model_type='local'
            if language=='zh':
                tts_model=self.zh_tts_model
                tts_json=self.zh_tts_json
            elif language=='en':
                tts_model=self.en_tts_model
                tts_json=self.en_tts_json
            self.synthesizer = piper.PiperVoice.load(tts_model, config_path=tts_json, use_cuda=False)      

    def voice_synthesis(self,text,path):
        if self.model_type=='oline':
            if self.tts_supplier=='baidu':
                # ... (Baidu TTS implementation)
                pass
            elif self.tts_supplier=='aliyun':
                # ... (Aliyun TTS implementation)
                pass
        elif self.model_type=='local':
            with wave.open(path, 'wb') as wav_file:
                wav_file.setnchannels(1)
                wav_file.setsampwidth(2)
                wav_file.setframerate(self.synthesizer.config.sample_rate)
                self.synthesizer.synthesize(text, wav_file)

Code Analysis

The text-to-speech (TTS) feature is invoked by the LargeModelService node, with the specific implementation provided by the model_interface class. Its design switches between different backend services through parameter configuration.

  1. Initialization Flow (model_service.py):

    • When LargeModelService is initialized, the init_param_config function reads the Boolean value useolinetts from the ROS parameter server.

    • Based on the value of useolinetts, the system_sound_init function decides whether to pass the string 'local' or 'oline' to the self.model_client.tts_model_init method.

    • In large_model_interface.py, the tts_model_init method executes the corresponding initialization logic according to the received string parameter. If it is 'local', piper.PiperVoice.load is used to load the local model file.

  2. Synthesis and Playback Flow (model_service.py):

    • When voice announcement is needed, the _safe_play_audio function is called.

    • This function first calls the self.model_client.voice_synthesis method, passing in the text to be converted and the target audio path self.tts_out_path.

    • After the voice_synthesis method finishes executing and generates the audio file, _safe_play_audio then calls self.play_audio_async to play the file asynchronously.

  3. Backend Implementation Selection (large_model_interface.py):

    • The voice_synthesis method is the backend dispatch center of the TTS feature. Internally, it chooses the execution path by checking the value of the self.model_type attribute set during initialization.

    • If self.model_type is 'local', the code block uses Python's wave library to open a WAV file, sets its header parameters (channels, sample width, sample rate), and then calls the self.synthesizer.synthesize method to write the audio stream synthesized from the text directly into that file.

    • If self.model_type is 'oline', it enters the logic branches for different cloud service providers (such as Baidu, Aliyun).

    • This structure separates the invocation issued by the upper-layer node ("say this passage") from the specific synthesis technology of the lower layer (which library to use, which API to call).

3. Practical Operations

3.1 Configuring TTS

Online and offline TTS share the language, speaker, and audio device settings, and useolinetts selects the synthesis backend. First open the main configuration file:

xxxxxxxxxx
vim ~/yahboom_ws/src/largemodel/config/yahboom.yaml

Set the language and backend in the existing model_service.ros__parameters parameter block:

xxxxxxxxxx
model_service:
  ros__parameters:
    language: 'zh'              # 'zh': Chinese; 'en': English
    useolinetts: False          # False: offline TTS; True: online TTS
    llm_platform: 'ollama'      # Available platforms: 'ollama', 'tongyi', 'spark', 'qianfan'
    regional_setting: "China"

If the parameters already exist, simply modify the original values; do not add duplicate keys to the same YAML parameter block.

3.1.1 Configuring Offline TTS

Set useolinetts to False, and confirm in large_model_interface.yaml that the Piper model and configuration file paths for the selected language exist:

xxxxxxxxxx
vim ~/yahboom_ws/src/largemodel/config/large_model_interface.yaml::
xxxxxxxxxx
zh_tts_model: "/home/jetson/yahboom_ws/src/largemodel/MODELS/tts/zh/zh_CN-huayan-medium.onnx"
zh_tts_json: "/home/jetson/yahboom_ws/src/largemodel/MODELS/tts/zh/zh_CN-huayan-medium.onnx.json"
en_tts_model: "/home/jetson/yahboom_ws/src/largemodel/MODELS/tts/en/en_US-libritts-high.onnx"
en_tts_json: "/home/jetson/yahboom_ws/src/largemodel/MODELS/tts/en/en_US-libritts-high.onnx.json"

3.1.2 Configuring Online TTS

Online TTS requires the API credentials of the corresponding service provider. The source tutorial takes Baidu AI Cloud as an example; fill in your own credentials and voice parameters in large_model_interface.yaml:

xxxxxxxxxx
# large_model_interface.yaml

# Baidu AI Cloud text-to-speech configuration
baidu_API_KEY: 'your api key'        # Baidu platform API key
baidu_SECRET_KEY: 'your secret key'  # Baidu platform SECRET key
CUID: 'your device identifier '         # Device identifier
PER: 4  # Speaker: basic voice library (0=Duxiaomei, 1=Duxiaoyu, 3=Duxiaoyao, 4=Duyaya)
SPD: 5  # Speed (0-15), default 5
PIT: 5  # Pitch (0-15), default 5
VOL: 5  # Volume (0-9), default 5

Confirm that model_service.ros__parameters.useolinetts in yahboom.yaml is True, and set the online service provider to use according to the project's current tts_supplier. Online synthesis requires that the Jetson can access the service provider's API; do not disclose real keys.

3.2 Starting and Testing the Feature

  1. Start the TTS Node: Run the following command:

    xxxxxxxxxx
    ros2 launch largemodel tts_only.launch.py

    image-20250807165129463

  2. Send the Text to Be Synthesized: Open a new terminal and run the following command to publish a voice announcement message:

    xxxxxxxxxx
    ros2 topic pub --once /tts_text_input std_msgs/msg/String '{data: "Speech synthesis test succeeded"}'
  3. Test: If everything works properly, you should hear the robot say "Speech synthesis test succeeded" from your speaker using the synthesized voice.


4. Common Problems and Solutions

4.1 Playback Problems

Problem 1: The program runs normally with no errors, but no sound can be heard.

Solution:

  1. Check the audio output: Confirm that the system audio output device is selected correctly and that the volume is not muted. You can try playing an ordinary music file to test the hardware.

  2. In the Sound settings, keep Balance in the middle position

    image-20250807155119783

Offline speech to text (ASR)
AI large model voice interaction

COMPANY

  • About us
  • Team Style
  • Blog

LEARNING & EXPLORATION

  • Tutorials
  • Edge AI hardware
  • Educational robots
  • Embedded intelligent hardware

WHERE TO BUY

  • Yahboom Online Store
  • Amazon
  • Alibaba
  • AliExpress
  • Mercado

TECHNOLOGY SUPPORT

  • Email: support@yahboom.com

BE A DISTRIBUTOR

  • Email: sales@yahboom.com
QR Code

Copyright © 2017-2026 Yahboom. All Rights Reserved.

EN/中文