Children's Storytelling

1. Function Introduction

This example program can be used to tell stories to children aged 4-7. Based on the child's speech, it generates a related short story and accompanying illustrations.

2. Hardware Connection and Setup

This example program requires not only a power adapter but also a large language model module for language input and output, a DP monitor, and a mouse and keyboard for displaying the interface and operation, or logging into a remote desktop.

Open system settings, click Sound, select Speakers-USB Audio Device for Output Device, and Microphone-USB Audio Device for Input Device.

3. Code Analysis

Source Code Path:

/home/jetson/showing/gpt-oss-storyteller/storyteller.py

The default is to use Chinese voice. If you need English voice, change the language variable to "en". The program will automatically switch to the English voice model and prompts.

xxxxxxxxxx
language = "en"

If you need to adjust the image window size, modify the values of self.win_width and self.win_height.

If you need to modify the control buttons, control the start and stop buttons accordingly.

The background will start a task to detect language activation. When you say "Hello, Xiaoya" to the language module, the module will be activated and start recording.

If you need to adjust the recording duration, modify the value of the -d parameter; this value specifies the number of seconds after which the recording will automatically stop. The default setting is 10 seconds.

If you need to manually input specific text, uncomment the text variable and enter your custom text.

If you wish to change the opening prompt, search for OPENING_TEXT and modify the associated parameters. If you wish to modify the content used to generate the story and images, search for STORY_PROMPT and modify the associated parameters.

Note: For Chinese text, modify the variables located under the "zh" section; for English text, modify the variables located under the "else" section.

If you wish to change the background image, modify the file path specified in initial_image. Please ensure that the image file is placed in the same directory as the program files.

If you need to modify the serial port for the Large Language Model (LLM) module, the ComfyUI port, the Ollama server port, or the Piper model file, please modify the parameters listed below.

4. Running the Code

Open a new system terminal and launch the ComfyUI service:

xxxxxxxxxx
cd ~/ComfyUI
source venv/bin/activate
python main.py

Open another new system terminal, activate the virtual environment, and launch the main storytelling program:

xxxxxxxxxx
cd ~/showing/
source venvs/bin/activate
cd gpt-oss-storyteller
python storyteller.py

5. Demonstration Results

Once the program launches, a background image will appear.

At this point, you can wake up the Large Language Model (LLM) module by saying, "Hi, yahboom" or press the 's' key on your keyboard to begin recording. Then, simply voice your ideas. Wait for the recording to finish—or press the Spacebar to stop recording manually—and the speakers will play an audio prompt announcing that the story is about to begin. The system will then generate the story and convert it into speech for playback. As the narrative shifts between scenes, the background image will automatically update to display visuals relevant to the current setting.

If you wish to end the story, you can press the ESC key. Alternatively, you can interrupt the current story—and restart the recording and story generation process—by saying, "Hello, Xiaoya," to the LLM module.