Children's Storytelling

1. Function Introduction

This example program can be used to tell stories to children aged 4-7. Based on the child's speech, it generates a related short story and accompanying illustrations.

2. Hardware Connection and Setup

This example program requires not only a power adapter but also a large language model module for language input and output, a DP monitor, and a mouse and keyboard for displaying the interface and operation.

Open system settings, click Sound, select Speakers-USB Audio Device for Output Device, and Microphone-USB Audio Device for Input Device.

3. Code Analysis

Source code path:

/home/jetson/showing/gpt-oss-storyteller/storyteller.py

If you need to adjust the image window size, modify the values of self.win_width and self.win_height.

If you need to modify the control buttons, control the start and stop buttons accordingly.

To modify the recording duration, change the value of the -d parameter, which indicates how many seconds of recording will automatically stop. The default is 10 seconds.

To force text input, uncomment the text parameter and write the custom text.

To modify the start message, modify the parameter passed to generate_tts.

To modify the generated story and images, you can modify the story_prompt variable.

To modify the background image, change the path and name of initial_image. Ensure the file is placed in the same directory as the program files.

To modify the serial port of the large language model module, the ComfyUI port, the Ollam server port, and the piper model file, modify the following parameters.

4. Running the Code

Open a new system terminal and start the ComfyUI service.

xxxxxxxxxx
cd ~/ComfyUI
source venv/bin/activate
python main.py

Open another new system terminal, start the virtual environment, and start the main storytelling program.

xxxxxxxxxx
cd ~/showing/
source venvs/bin/activate
cd gpt-oss-storyteller
python storyteller.py

5. Experimental Results

After the program starts, a background image will pop up.

At this time, say "Hi, yahboom" to the large language model module to wake it up, or press the 's' key on the keyboard to start recording, then speak your thoughts aloud, wait for the recording to end, or press the space bar to end the recording. The speaker will play a prompt that the story is about to begin, then generate the story and convert it into speech for playback. When the story scene changes, the background image will automatically display an image related to the scene.

To end the story, press the ESC key. Alternatively, you can say "Hi,yahboom" to the large language model module to interrupt the story and restart recording and story generation.