Voice Interaction

Repository Address

• Voice stream processing code: https://github.com/D-Robotics/magicbox_audio_io
• Large language model code: https://github.com/D-Robotics/magicbox_qwen_llm

Function Description

1. This function includes an audio processing node and a voice large model node. The large model uses Qwen2.5-1.5B. ASR is based on SensorVoice. TTS and KWS are based on the Sherpa-onnx framework.
2. In addition to the default continuous dialogue mode, it also supports keyword wake-up mode, namely "one wake-up one dialogue". Users can wake up by saying "Hello, Digua" and then have a round of dialogue, or directly have a dialogue by saying "Hello, Digua + your question.". After detecting "Hello, Digua", the light will blink. It can be switched via continuous_wake_mode in audio_io.launch.py.
3. In addition to the default voice dialogue function, it also supports voice control. However, this function will add a lot of extra prompts, and the function loading time is longer. For configuration, refer to enable_function_call in qwen_llm.launch.py. After starting, you can use commands such as "Raise right hand" , "Raise right hand to the highest“ , "Stand up" , "Sit down" .
4. Due to the long initialization time of the voice processing node, this node starts automatically after boot. The button only controls the start and stop of the large language model node, but the two nodes will wait for each other to start before working.
5. Functional architecture diagram: