Cooking Assistant

1. Feature Overview

This example routine generates recipes for dishes, covering ingredients preparation, seasoning quantities, cooking instructions, and the generation of final dish images.

2. Hardware Connections and Setup

This example requires not only a connection to a power adapter but also a connection to a Large Language Model (LLM) module for voice input and output. Additionally, you must connect a DP monitor, mouse, and keyboard to interact with the user interface, or alternatively, access the system via a remote desktop connection.

Open System Settings, click on "Sound," and set the "Output Device" to "Speakers - USB Audio Device" and the "Input Device" to "Microphone - USB Audio Device."

3. Code Analysis

Source Code Path:

/home/jetson/showing/cooking_assistant/cooking_assistant.py

The program defaults to using Chinese voice prompts. If you require English voice prompts, please change the value of the language variable to "en"; the program will automatically switch to the English voice model and prompts.

xxxxxxxxxx
language = "en"

If you need to adjust the size of the image display window, modify the values ​​of self.win_width and self.win_height.

A background task is launched to monitor for the voice wake-up command. When you speak the phrase "Nihao XiaYa" (Hello, XiaYa) into the voice module, the module will wake up and begin recording audio.

If you need to adjust the recording duration, modify the value of the -d parameter; this value specifies the number of seconds after which the recording will automatically stop (the default duration is 10 seconds).

If you need to force-feed specific text input, uncomment the text variable and insert your custom text.

If you need to modify the opening prompt, search for OPENING_TEXT and adjust the relevant parameters. If you need to modify the content generated for the story and images, search for PROMPT_TEXT and adjust the relevant parameters.

Note: For Chinese, modify the variables located under the "zh" section; for English, modify the variables located under the "else" section.

If you need to modify the serial port for the Large Language Model (LLM) module, the ComfyUI port, the Ollama server port, or the Piper model file, adjust the parameters shown below.

4. Running the Code

Open a new system terminal and launch the ComfyUI service.

xxxxxxxxxx
cd ~/ComfyUI
source venv/bin/activate
python main.py

Open another new system terminal, activate the virtual environment, and launch the main storytelling program.

xxxxxxxxxx
cd ~/showing/
source venvs/bin/activate
cd cooking_assistant
python cooking_assistant.py

5. Experimental Results

Once the program has launched:

At this point, say "Hi,yahboom" to the Large Language Model (LLM) module to wake it up. Then, state the dish you would like to eat. After the recording concludes, the speaker will play a prompt indicating that the recipe is being prepared; subsequently, the recipe will be generated and read aloud via speech synthesis. Finally, an image of the finished dish will be displayed.

If you wish to stop the playback, you can interrupt the process by saying "Hi,yahboom" to the LLM module, which will allow you to restart the recording and recipe generation.