OpenClaw AI Diet Calorie Estimation Tutorial

Before starting, please complete the [01-Peripheral Hardware Configuration], [02-OpenClaw API-KEY Configuration], [03-OpenClaw Model Switching], [04-OpenClaw Prompts], [05-AI Voice Interaction Configuration], and [06-Three Conversation Solutions Configuration Testing] under the Preparation directory. If you are only using TUI/WhatsApp for now, you can skip 05. This document assumes that the basic environment has been prepared and focuses solely on the diet calorie estimation project itself.

1. Tutorial Objectives

This tutorial demonstrates OpenClaw's natural language capabilities in dietary analysis scenarios through "photo recognition + large model estimation".

• Learn how to use cameras and vision models for meal tray analysis
• Interact with OpenClaw through three methods: WhatsApp, TUI, and voice code
• Complete four project scenarios: meal tray photography, food recognition, calorie estimation, and health advice

2. Hardware Preparation

• OpenClaw main control board
• CSI camera module
• OLED display screen
• [Optional] AI voice module + speaker
• Wiring diagram

3. Software Preparation

Code entry points used in this tutorial

• Camera photo command: /home/sunrise/hardware_hal_demo/camera
• Photo script: /home/sunrise/.openclaw/workspace/skills/sunrise-vision/scripts/csi_snap
• Vision analysis script: /home/sunrise/.openclaw/workspace/skills/sunrise-vision/scripts/csi_describe
• Image output directory: /home/sunrise/.openclaw/media/csi-camera/
• Voice entry point: /home/sunrise/voice_to_openclaw/src/main.py

In this project, the actual "calorie estimation" is mainly completed by the large model, so it is recommended to ensure that the following two links work properly first:

• The camera can complete photo capture
• The vision model can complete image description

Note: Calorie estimation is more suitable as a demonstration and daily reference, not as a medical-grade or nutritionist-level precise measurement result.

4. Testing Basic Project Capabilities

Before starting the current project, please complete a basic openclaw tui conversation self-check; if you haven't done so, first refer to [06-Three Conversation Solutions Configuration Testing]. After passing, use TUI to verify the basic capabilities in the project.

First enter OpenClaw TUI:

openclaw tui

Then you can start with the most basic photo and recognition tests, such as:

• Help me take a photo of lunch
• See what's in this meal tray

5. Three Solutions for Conversing with OpenClaw

If you have selected the AI voice module + speaker, and WhatsApp conversation, TUI, WhatsApp conversation need to broadcast OpenClaw's reply content, you can run the following program in the terminal. AI voice module conversation does not need to be run repeatedly

• Modify whether to enable broadcasting parameter

xxxxxxxxxx
/home/sunrise/voice_to_openclaw/.env

xxxxxxxxxx
ENABLE_TTS=true  # This parameter controls whether to broadcast, save after modification, then run the program below in terminal

• Terminal input

xxxxxxxxxx
cd ~/voice_to_openclaw
source .venv/bin/activate
python -m src.openclaw_session_tts

5.1 WhatsApp Solution

For WhatsApp integration and bot creation steps, please directly refer to [06-Three Conversation Solutions Configuration Testing]. After creation, you can remotely initiate dietary analysis in WhatsApp, for example:

• Take a photo to view current food, estimate food calories, if over 1000 calories turn on red light otherwise green light, servo turns to 90 degrees to indicate not recommended for consumption, 0 degrees means recommended for consumption

5.2 TUI Solution

If you have completed the TUI self-check in section 6, directly enter terminal text conversation:

xxxxxxxxxx
openclaw tui

Then input project-related commands, for example:

• Take a photo to view current food, estimate food calories, if over 1000 calories turn on red light otherwise green light, servo turns to 90 degrees to indicate not recommended for consumption, 0 degrees means recommended for consumption

5.3 Voice Code Solution

xxxxxxxxxx
Requires AI voice interaction module + speaker

For voice module installation, environment configuration, and basic wake-up process, please directly refer to [05-AI Voice Interaction Configuration]. After waking up, you can directly speak control content, for example:

• Food calorie recognition

6. Project Implementation

Note: The following demonstrations use openclaw tui for conversation, but you can also choose WhatsApp or voice methods to complete them.

6.1 Scenario One: Meal Tray Photography

Experiment Objective

First complete meal tray image capture to obtain an image that can be used for subsequent analysis.

Effect Description

After successful photo capture, a meal tray image will be saved locally, and the most recent image alias will be generated. Subsequent recognition and estimation will continue based on this image.

Example Commands

• Help me take a photo of lunch
• Take a meal tray photo now

6.2 Scenario Two: Food Recognition

Experiment Objective

Let OpenClaw identify what main foods are in the meal tray.

Effect Description

The focus of this step is not to immediately provide calories, but to first identify the main contents such as rice, meat, vegetables, beverages, etc., laying the foundation for subsequent estimation.

Example Commands

• See what foods are in this meal
• Help me identify the meal tray contents
• List the main ingredients

6.3 Scenario Three: Calorie Estimation

Experiment Objective

Based on the identified food content, let OpenClaw provide an approximate calorie estimation.

Effect Description

OpenClaw can provide total calories or itemized calories based on image content and common food experience values. This result is more suitable for demonstration and reference, not as strict nutritional calculation.

Example Commands

• Estimate the total calories of this meal
• Estimate by itemizing rice, eggs, meat, and vegetables
• Approximately how many kilocalories is this lunch

6.4 Scenario Four: Health Advice

Experiment Objective

Let OpenClaw provide a brief suggestion after estimating calories.

Effect Description

In addition to reporting numbers, OpenClaw can also provide a suggestion that is more relevant to daily communication based on goals such as "fat loss, muscle gain, light meals, late-night snacks".

Example Commands

• If I'm losing fat, give me a suggestion
• Is this meal combination balanced
• Summarize the result into one sentence

Expandable Features

• Display total calories on OLED
• Create a daily diet tracking assistant
• Combine with voice broadcasting to form life scenario demonstrations