Cooking Assistant

1. Function Introduction

This example can generate recipes for projects, including ingredient preparation, seasoning amounts, cooking methods, and the final result image.

2. Hardware Connection and Setup

This example requires not only a power adapter but also a large language model module for language input and output, a DP monitor, and a mouse and keyboard for displaying the interface and operations.

Open system settings, click Sound, select Speakers-USB Audio Device for Output Device, and Microphone-USB Audio Device for Input Device.

3. Code Analysis

Source Code Path:

/home/jetson/showing/cooking_assistant/cooking_assistant.py

To adjust the image window size, modify the values ​​of self.win_width and self.win_height.

To modify the recording duration, modify the value of the -d parameter, which indicates how many seconds of recording will automatically stop; the default is 10 seconds.

If you need to force text input, remove the comment symbol from text and write the custom text.

If you need to modify the start prompt, modify the parameter passed to generate_tts.

If you need to modify the content of the generated story and images, you can modify the story_prompt variable.

If you need to modify the serial port of the large language model module, the port of ComfyUI, the Ollam server port, or the piper model file, modify the parameters below.

4. Running the Code

Open a new system terminal and start the ComfyUI service.

xxxxxxxxxx
cd ~/ComfyUI
source venv/bin/activate
python main.py

Open a new system terminal, start the virtual environment, and launch the main storytelling program.

xxxxxxxxxx
cd ~/showing/
source venvs/bin/activate
cd cooking_assistant
python cooking_assistant.py

5. Experimental Results

After the program starts.

Say "Hi,yahboom" to the large language model module to wake it up, then say the dish you want to eat. Wait for the recording to finish, and the speaker will play a prompt to prepare the recipe. Then, the recipe will be generated, converted into speech, and played. Finally, a picture of the food will be displayed.

To end playback, you can say "Hi, yahboom" to the large language model module to interrupt and restart recording and recipe generation.