Welcome to Jetson Orin NANO Developer Kit repository
5.AI large model offline voice assistant
10. AI Large Model Voice Assistant (Online/Offline)
10. AI Large Model Voice Assistant (Online/Offline)1. Online/Offline Voice Configuration2. Creating a Boot Autostart Service (Systemd)2.1 Creating a Startup Script2.2 Creating the Systemd Service File2.3 Managing and Debugging the Service
1. Online/Offline Voice Configuration
Note: Due to performance limitations, the Jetson Orin Nano 4GB may not be able to run a fully offline combination smoothly; you can select an online LLM, ASR, or TTS in this chapter, or refer to the corresponding section in <Online Large Model (Voice Interaction)>.
Before setting up autostart, first confirm that the voice assistant works properly in the selected mode. ASR, LLM, and TTS can each select online or offline; only when the corresponding part is set to online is that service provider's network and API credentials required.
Locate the configuration file: In your project code, find and open the configuration file:
~/yahboom_ws/src/largemodel/config/yahboom.yamlModify the configuration parameters: Check the following parameters in the file and make sure their values match those shown below. If a parameter does not exist, add it.
xasr#Voice node parametersros__parametersVAD_MODE2 #vad sensitivitysample_rate16000 #asr recording audio sample rateframe_duration_ms30 #vad frame size in msuse_oline_asrFalse # False: offline ASR; True: online ASRmic_serial_port"/dev/ttyUSB0" #microphone serial port aliasmic_index0 #microphone indexlanguage'zh' #asr languageregional_setting "China" #international: international edition China: China editionmodel_service#model server node parametersros__parameterslanguage'zh' #large model interface languageuseolinettsFalse # False: offline TTS; True: online TTS# Large model configuration# llm_platform: 'ollama' # Available platforms: 'ollama', 'tongyi', 'spark', 'qianfan', 'openrouter'llm_platform'ollama' # 'ollama': offline LLM; 'tongyi': online Tongyiregional_setting "China"Fully offline:
use_oline_asr: False,useolinetts: False,llm_platform: 'ollama'.Fully online example:
use_oline_asr: True,useolinetts: True,llm_platform: 'tongyi'. Online ASR/TTS still requires configuring the respective service provider credentials; follow articles 1 and 2 of this chapter.ASR, TTS, and LLM can be selected in mixed combinations; for example, an online LLM with offline ASR/TTS.
Save the file and rebuild the project to apply the changes:
xxxxxxxxxxcd ~/yahboom_wscolcon buildsource install/setup.bash
The online Tongyi LLM requires qianwen_api_key and the model name to be filled in; online ASR requires online_asr_api_key to be filled in; online TTS requires the corresponding service provider credentials to be filled in. In offline mode, confirm that the Ollama model, SenseVoiceSmall, and Piper model paths exist. See articles 1–3 of this chapter for detailed fields and examples.
Starting in online mode additionally requires the network to be available; offline mode requires the local models to be installed.
Note: If you are using the factory image, the scripts and service-related configurations are already complete, and the following operations are not required
2. Creating a Boot Autostart Service (Systemd)
Now we will create a systemd service so that largemodel_control.launch.py runs automatically at system startup. In online mode, it is recommended to wait until the network is ready before starting; if the device connects to the network slowly, you can check interface connectivity and the service logs after the service starts.
2.1 Creating a Startup Script
In order for systemd to load the ROS 2 environment correctly, the best practice is to create a simple bash script to wrap our startup command.
Create the script file: In the directory (
~/yahboom_ws/src/largemodel/), create a file namedstart_largemodel.sh.xxxxxxxxxxvim ~/yahboom_ws/src/largemodel/start_largemodel.shWrite the script content: Copy and paste the following content into the script file.
xxxxxxxxxx# Source the ROS 2 Jazzy environmentsource /opt/ros/jazzy/setup.bash# Source the Yahboom workspace environmentsource /home/jetson/yahboom_ws/install/setup.bash# Start the largemodel control scriptros2 launch largemodel largemodel_control.launch.pyFirst run
whoamito confirm the Linux user name and runecho $HOMEto confirm the home directory; replace/home/jetsonin the script with the actual home directory of the device.Save and exit
Grant the script execute permission:
xxxxxxxxxxchmod +x ~/yahboom_ws/src/largemodel/start_largemodel.sh
2.2 Creating the Systemd Service File
This is the most essential step. We will tell the system that we have a new service to manage.
Create the service file: You need to use
sudoprivileges to create this file.xxxxxxxxxxsudo vim /etc/systemd/system/largemodel.serviceWrite the service configuration: Copy and paste the following content completely into the service file.
xxxxxxxxxx[Unit]Description=Robot ServiceAfter=network-online.target sound.target graphical.targetWants=network-online.target sound.target graphical.target[Service]Type=simpleUser=jetsonGroup=jetsonEnvironment=DISPLAY=:0Environment=XDG_RUNTIME_DIR=/run/user/1000Environment=PULSE_SERVER=unix:/run/user/1000/pulse/nativeSupplementaryGroups=audio videoExecStartPre=/bin/sleep 10ExecStart=/home/jetson/yahboom_ws/src/largemodel/start_largemodel.shRestart=on-failureStandardOutput=journalStandardError=journal[Install]WantedBy=multi-user.targetChange the user name, UID, and paths in
User,Group,XDG_RUNTIME_DIR,PULSE_SERVER, andExecStartto the actual values of the device (you can query them withwhoami,id -u, andecho $HOME).ExecStartmust point to the script created in the previous step that has execute permission. Online mode usesnetwork-online.target; the system also needs the wait-online service of the corresponding network manager enabled in order to wait until the network is truly ready.Save and exit.
2.3 Managing and Debugging the Service
Now that your service has been created, we need to have systemd load it and set it to start on boot.
Reload the
systemddaemon so that it reads our newly created service file:xxxxxxxxxxsudo systemctl daemon-reloadSet the service to start on boot:
xxxxxxxxxxsudo systemctl enable largemodel.serviceStart the service immediately:
xxxxxxxxxxsudo systemctl start largemodel.serviceCheck the service status: This is the most important command for verifying whether the service is running successfully.
xxxxxxxxxxsudo systemctl status largemodel.serviceIf you see
Active: active (running), the service process has started; if it isfailed, continue checking the service logs and verify the configuration, network, and audio devices.View the service logs (essential for debugging): If the service fails to start, you can view all real-time logs generated by the
ros2 launchcommand with the following command, which is crucial for locating the problem.xxxxxxxxxxjournalctl -u largemodel.service -f
After completing the steps above, the system will start the largemodel voice service according to the selected online or offline configuration.