Image to Animation

1. Function Introduction

This example program can generate a 5-second animation from a single image.

2. Hardware Connection and Setup

This example program requires not only a power adapter but also a DP monitor, mouse, and keyboard for displaying the interface and operation.

3. Code Analysis

Source code path:

/home/jetson/showing/animation/animation.py

If you need to replace the image, please replace the image in the path ~/ComfyUI/input/image.jpg.

If you need to modify the prompt message, you can modify the PROMPT variable.

xxxxxxxxxx
TEXT_PROMPT = '''The background is a cyberpunk-style Tang Dynasty Never Sleeping City, with flying eaves and holographic lanterns. The camera slowly zooms in and circles around. The image shows a robotic dog, and a person is slowly waving a yellow bone in front of it. The robotic dog follows the bone's sway, and the movements are natural and seamless, with a smooth transition between real and virtual elements. \nConstraint:\nDo not change the robotic dog's appearance.'''

If you need to change the output file name, modify the OUTPUT_NAME variable. The final output will be in the ~/ComfyUI/output/video/ directory.

xxxxxxxxxx
OUTPUT_NAME = "myvideo"

If you need to change the ComfyUI port, modify the value of the self.comfyui_server variable.

xxxxxxxxxx
self.comfyui_server = '127.0.0.1:8188'

4. Running the Code

Open a new system terminal and start the ComfyUI service.

xxxxxxxxxx
cd ~/ComfyUI
source venv/bin/activate
python main.py

Open another new system terminal, start the virtual environment, and start the main program for telling stories.

xxxxxxxxxx
cd ~/showing/
source venvs/bin/activate
cd animation/
python animation.py

5. Experimental Results

After the program starts, it will call a large model file to render the animation. The program will take a relatively long time to run; the default configuration takes about 16 minutes.

When the progress bar reaches 100%, it indicates completion.

After the animation is generated, the MP4 video file is stored in the ~/ComfyUI/output/video/ directory.