OpenClaw AI Diet Calorie Estimation Tutorial

Before starting, please complete [01-Peripheral Hardware Configuration], [02-OpenClaw Jetson-KEY Configuration], [03-OpenClaw Model Switching], [04-AI Voice Interaction Configuration], and [05-Three Dialogue Solution Configuration Tests] in the Pre-use Preparation directory. If you are only using TUI / WhatsApp, you can skip 04 for now. This article assumes the basic environment is already prepared and only covers the diet calorie estimation project itself.

1. Tutorial Objective

This article demonstrates OpenClaw's natural language capabilities in dietary analysis scenarios through "photo recognition + LLM estimation".

• Learn how to use the camera and vision model for plate analysis
• Interact with OpenClaw through three methods: WhatsApp, TUI, and voice code
• Complete four project scenarios: plate photography, food recognition, calorie estimation, health advice

2. Hardware Preparation

• OpenClaw main control board
• CSI camera module
• OLED display
• [Optional] AI voice module + speaker
• Wiring diagram

3. Software Preparation

Code entry points used in this tutorial

• Camera capture command: /home/jetson/hardware_hal_demo/camera
• Capture CLI: /home/jetson/hardware_hal_demo/cli/seewhat
• Image output directory: /home/jetson/hardware_hal_demo/cli
• Voice entry: /home/jetson/voice_to_openclaw/src/main.py

In this project, the actual "calorie estimation" is mainly performed by the large language model (LLM), so it is recommended to first ensure the following two links are working:

• The camera can capture photos
• The vision model can describe images

Note: Calorie estimation is more suitable as a demonstration and daily reference, not as a medical-grade or nutritionist-level precise measurement result.

4. Testing Basic Project Capabilities

Before starting this project, please complete an openclaw tui basic dialogue self-check first; if you haven't done so, see [05-Three Dialogue Solution Configuration Tests] first. After passing, use TUI to verify the basic capabilities in the project.

First enter OpenClaw TUI:

openclaw tui

Then you can start with basic capture and recognition tests, for example:

• Take a photo of my lunch
• Check what's on this plate

5. Three Solutions for Communicating with OpenClaw

If you have selected the AI voice module + speaker, Feishu dialogue, tui, and whatsapp dialogue need to broadcast OpenClaw's replies. You can run the following program in the terminal. The AI voice module dialogue does not need to be run repeatedly.

• Modify the broadcast toggle parameter

xxxxxxxxxx
/home/jetson/voice_to_openclaw/.env

xxxxxxxxxx
ENABLE_TTS=true  #This parameter controls whether to broadcast. After modification, save and run the following program in the terminal

• Enter in terminal

xxxxxxxxxx
cd ~/voice_to_openclaw
source .venv/bin/activate
python -m src.openclaw_session_tts

5.1 WhatsApp Solution

For WhatsApp integration and bot creation steps, please refer directly to [05-Three Dialogue Solution Configuration Tests]. After setting up the bot, you can remotely initiate dietary analysis in WhatsApp, for example:

• Take a photo to check the current food, estimate food calories, if over 1000 calories light red otherwise light green, servo turn to 90 degrees means not recommended for consumption, 0 degrees means recommended for consumption

5.2 TUI Solution

If you have completed the TUI self-check in the Prerequisites section, directly enter terminal text dialogue:

xxxxxxxxxx
openclaw tui

Then input project-related commands, for example:

• Take a photo to check the current food, estimate food calories, if over 1000 calories light red otherwise light green, servo turn to 90 degrees means not recommended for consumption, 0 degrees means recommended for consumption

5.3 Voice Code Solution

xxxxxxxxxx
Requires optional AI voice interaction module + speaker

For voice module installation, environment configuration, and basic wake-up process, please refer directly to [04-AI Voice Interaction Configuration]. After wake-up, you can directly speak control commands, for example:

• Food calorie recognition

6. Project Implementation

Note: The following demonstrations are all conducted using the openclaw tui method. You can also choose WhatsApp or voice methods to complete them.

6.1 Scenario 1: Plate Photography

Experiment Objective

First, capture a plate image to obtain an image that can be used for subsequent analysis.

Effect Description

After a successful capture, a plate image will be saved locally and an alias for the most recent image will be generated. Subsequent recognition and estimation will continue based on this image.

Example Commands

• Take a photo of my lunch
• Take a plate photo now

6.2 Scenario 2: Food Recognition

Experiment Objective

Have OpenClaw identify the main foods on the plate.

Effect Description

The focus of this step is not to immediately provide calories, but to first identify the main food items such as rice, meat, and vegetables, and beverages, preparing for calorie estimation.

Example Commands

• Check what foods are in this meal
• Help me identify the plate contents
• List the main ingredients

6.3 Scenario 3: Calorie Estimation

Experiment Objective

Based on the identified food items, have OpenClaw provide a rough calorie estimate.

Effect Description

OpenClaw can provide total calories or itemized calories based on image content and typical calorie values for common foods. This result is more suitable for demonstrations and references, not as strict nutritional calculations.

Example Commands

• Estimate the total calories of this meal
• Estimate by item: rice, eggs, meat, and vegetables
• How many kilocalories is this lunch approximately

6.4 Scenario 4: Health Advice

Experiment Objective

Have OpenClaw provide a brief suggestion after estimating calories.

Effect Description

In addition to reporting numbers, OpenClaw can also provide advice that is closer to everyday communication based on goals such as "fat loss, muscle gain, light meals, late-night snacking".

Example Commands

• If I'm trying to lose fat, give me a suggestion
• Is this meal balanced
• Help me summarize the results in one sentence

Extension Ideas

• Display total calories on OLED
• Make it into a daily diet check-in assistant
• Combine with voice narration for a lifestyle demonstration