Image to Animation

1. Function Introduction

This example program can generate a 5-second animation from a single image.

2. Hardware Connection and Setup

This example program requires not only a power adapter but also a DP monitor, mouse, and keyboard for displaying the interface and performing operations, or for logging into a remote desktop.

3. Code Analysis

Source code path:

xxxxxxxxxx
/home/jetson/showing/animation/animation.py

If you need to replace the image, please replace the image in the path ~/ComfyUI/input/image.jpg.

If you need to modify the prompt message, you can modify the PROMPT variable.

xxxxxxxxxx
TEXT_PROMPT = '''The background is a cyberpunk-style Tang Dynasty Never Sleeping City, with flying eaves and holographic lanterns. The camera slowly zooms in and circles around. The image shows a robotic dog, and a person is slowly waving a yellow bone in front of it. The robotic dog follows the bone's sway, and the movements are natural and seamless, with a smooth transition between real and virtual elements. \nConstraint:\nDo not change the robotic dog's appearance.'''

If you need to change the output file name, modify the OUTPUT_NAME variable. The final output will be in the ~/ComfyUI/output/video/ directory.

xxxxxxxxxx
OUTPUT_NAME = "myvideo"

If you need to change the ComfyUI port, modify the value of the self.comfyui_server variable.

xxxxxxxxxx
self.comfyui_server = '127.0.0.1:8188'

4. Running the code

Open a new system terminal and start the ComfyUI service.

xxxxxxxxxx
cd ~/ComfyUI
source venv/bin/activate
python main.py

Next, open a new system terminal, activate the virtual environment, and launch the main storytelling program.

xxxxxxxxxx
cd ~/showing/
source venvs/bin/activate
cd animation/
python animation.py

5. Experimental Results

Once the program starts, it will invoke the large model files to render the animation. The execution process takes some time; with the default configuration, it requires approximately 8 minutes.

Monitor the ComfyUI terminal; when the progress bar reaches 100%, it indicates that the process is complete.

After the animation is generated, the resulting MP4 video file is stored in the ~/ComfyUI/output/video/ directory.

xxxxxxxxxx
cd ~/ComfyUI/output/video/
ls