3. Visual and text interaction VLM

1. Introduction

An interactive computer vision application using the Hailo Visual Language Model (VLM) for real-time image analysis and question answering.

2. Startup

For USB cameras, run the following command:

python -m hailo_apps.python.gen_ai_apps.vlm_chat.vlm_chat --input usb

If it's a Raspberry Pi CSI camera, run the following command.

xxxxxxxxxx
python -m hailo_apps.python.gen_ai_apps.vlm_chat.vlm_chat --input rpi

Note: After starting the command, you will see a live camera window.

Interactive Mode

• Click on the terminal and press Enter to capture the current frame and enter question-and-answer mode.

  

• Enter a question about capturing images (or press Enter to use the default question).

• Press Enter to submit your question and get a VLM response.Press Enter again to continue with a new round of questions.

  

• Press Enter again to continue with a new round of questions.

  

• Click on the camera view window, then press the 'q' key in the video window to exit.

For more information on interactive graphic and text content, please refer to: hailo-apps/hailo_apps/python/gen_ai_apps/vlm_chat at main · hailo-ai/hailo-apps · GitHub