OpenClaw AI Calorie Estimation Tutorial

Before starting, please complete [01-Peripheral Hardware Configuration], [02-OpenClaw API-KEY Configuration], [03-OpenClaw Model Switching], [04-AI Voice Interaction Configuration], and [05-Three Dialogue Configuration Tests] in the Preparation Before Use directory. If you are only using TUI/WhatsApp, you can skip 04. This article assumes the basic environment is ready and only covers the diet calorie estimation project itself.

1. Tutorial Objectives

This article demonstrates OpenClaw's natural language capability in the diet analysis scenario through "photo recognition + large model estimation". After completing this tutorial, readers will be able to:

• Understand how to use cameras and visual models for plate analysis
• Communicate with OpenClaw via WhatsApp, TUI, and voice commands
• Complete four project scenarios: plate photo, food recognition, calorie estimation, health advice

2. Hardware Preparation

• OpenClaw main control board
• CSI camera module
• OLED display screen
• [Optional] AI voice module + speaker
• Wiring diagram

3. Software Preparation

Code entry points used in this tutorial:

• Camera capture command: /home/pi/hardware_hal_demo/camera
• Capture script: /home/pi/.openclaw/workspace/skills/pi-vision/scripts/csi_snap
• Visual analysis script: /home/pi/.openclaw/workspace/skills/pi-vision/scripts/csi_describe
• Image output directory: /home/pi/.openclaw/media/csi-camera/
• Voice entry: /home/pi/voice_to_openclaw/src/main.py

In this project, the real "calorie estimation" is mainly done by the large model, so it is recommended to first ensure the following two chains are normal:

• Camera can complete photo capture
• Visual model can complete image description

Important note: Calorie estimation is more suitable as a demonstration and daily reference, not suitable for medical-grade, nutritionist-grade precise measurement results.

4. Testing Project Basic Capabilities

Before starting this project, please complete an openclaw tui basic conversation self-check first; if you haven't done this yet, see [05-Three Dialogue Configuration Tests]. After passing, use TUI to separately verify the basic capabilities in the project.

First enter OpenClaw TUI:

openclaw tui

Then you can first do the most basic photo capture and recognition test, for example:

• Take a photo of my lunch for me
• See what's in this plate

5. Three Ways to Communicate with OpenClaw

If you have the optional AI voice module + speaker, the TUI and WhatsApp conversations need to broadcast OpenClaw's replies. You can run the following program in the terminal. This is not needed for AI voice module conversations.

• Modify the TTS enable parameter

xxxxxxxxxx
/home/pi/voice_to_openclaw/.env

xxxxxxxxxx
ENABLE_TTS=true  # This parameter controls whether to broadcast replies. After saving, run the program below in the terminal.

• Terminal input

xxxxxxxxxx
cd ~/voice_to_openclaw
source .venv/bin/activate
python -m src.openclaw_session_tts

5.1 WhatsApp Solution

For WhatsApp integration and bot creation steps, please refer directly to [05-Three Dialogue Configuration Tests]. After creating, you can remotely initiate diet analysis in WhatsApp, for example:

• Take a photo to see the current food, estimate the food calories. If it exceeds 1000 calories, light up red light; otherwise light up green light. Servo rotates to 90 degrees to indicate not recommended for consumption, 0 degrees indicates recommended for consumption

5.2 TUI Solution

If you have already completed the TUI self-check from chapter 0, directly enter terminal text dialogue:

xxxxxxxxxx
openclaw tui

Then input project-related commands, for example:

• Take a photo to see the current food, estimate the food calories. If it exceeds 1000 calories, light up red light; otherwise light up green light. Servo rotates to 90 degrees to indicate not recommended for consumption, 0 degrees indicates recommended for consumption

5.3 Voice Code Solution

xxxxxxxxxx
Requires optional AI voice interaction module + speaker

For voice module installation, environment configuration, and basic wake-up flow, please refer directly to [04-AI Voice Interaction Configuration]. After waking up, you can directly speak control content, for example:

• Food calorie recognition

6. Project Implementation

Note: All demonstrations below use the openclaw tui method for conversation. You can also choose WhatsApp or voice to complete.

6.1 Scenario 1: Plate Photo

Experiment Objective

First complete plate image capture to get an image that can be used for subsequent analysis.

Effect Description

After successful photo capture, a plate image will be saved locally and a recent image alias will be generated. Subsequent recognition and estimation will continue based on this image.

Example Commands

• Take a photo of my lunch for me
• Take a plate photo now

6.2 Scenario 2: Food Recognition

Experiment Objective

Have OpenClaw recognize what main foods are in the plate.

Effect Description

The focus of this step is not immediately giving calories, but first recognizing the main content such as rice, meat, vegetables, and drinks to build a foundation for subsequent estimation.

Example Commands

• See what foods are in this meal
• Help me recognize the plate contents
• List the main ingredients

6.3 Scenario 3: Calorie Estimation

Experiment Objective

Based on the recognized food content, have OpenClaw give an approximate calorie estimate.

Effect Description

OpenClaw can, based on the image content and common food experience values, give total calories or itemized calories. This result is more suitable for demonstrations and references, not strict nutritional calculations.

Example Commands

• Estimate the total calories of this meal
• Estimate by category for rice, eggs, meat, and vegetables
• How many calories is this lunch approximately

6.4 Scenario 4: Health Advice

Experiment Objective

Have OpenClaw give a brief suggestion after estimating calories.

Effect Description

Besides reporting numbers, OpenClaw can also give a sentence closer to daily communication based on goals like "fat loss", "muscle gain", "light meal", "late-night snack".

Example Commands

• Give me a suggestion if I'm in fat loss mode
• Is this meal combination balanced
• Help me summarize the results into one sentence

Extensible Play

• Display total calories on OLED
• Make a daily diet check-in assistant
• Combine with voice broadcast to form a daily life scenario demonstration