01
Scale
Place food on the scale — auto-detects stable weight threshold.
Prototype case study
AI Caloric Estimator
CV + Embedded Prototype
ESP32-powered food scale + vision API — ~10% error vs manual logging, sub-3 second inference.
ESP32-S3C++OpenAI APINode.js
- Role
- Embedded + Full Stack Engineer
- Domain
- Health / IoT
- Users
- Personal / demo
- Market
- Colombia
- Stack
- ESP32 · C++ · OpenAI · Express
Problem
Manual calorie tracking is tedious and error-prone — users underestimate portions by 30–50% on average.
Existing apps rely on user input alone. Combining physical weight with computer vision closes the accuracy gap.
The actual friction
Visual guesses alone cannot anchor portion size — weight data has to close the loop.
Understand the painThen see the product response
Solution
Fuse weight and vision instead of asking users to guess portions.
Place food on the scale, capture an image, and let the backend correlate mass with visual portion estimates.
02
Capture
Camera captures top-down image synchronized with weight reading.
03
Analyze
Vision API identifies food items and estimates portion size.
04
Result
Fused calorie estimate displayed on OLED and synced to app.
Capabilities behind the journey
Weight Sensor
HX711 load cell with calibrated gram readings.
Camera Module
OV2640 capture triggered on stable weight.
Vision API
Food identification and portion estimation via OpenAI.
Fusion Logic
Weight + visual data combined for calorie estimate.
Embedded UI
OLED display with real-time feedback on device.
Cloud Sync
Optional logging to Node.js backend for history.
Impact
90% error reduction with ~10% final accuracy.
Fusing load-cell weight with vision-grounded estimates in under 3 seconds.
01 - Friction
Manual calorie logging underestimates portions and breaks down on everyday meals.
02 - Intervention
Fuse camera vision with load-cell weight for grounded portion estimates.
03 - Outcome
90% error reduction with ~10% final accuracy.
Supporting signals
~10%
Accuracy
<3 sec
Processing
Shipped
Hardware
Weight stability
Reliable capture trigger in 95% of tests.
Architecture
How the system is built.
Six views — stack, containers, security, runtime flow, data model, and where it runs.
System Overview
ESP32 reads weight and captures an image; the cloud API runs vision inference and returns a calorie estimate.
ESP32-S3
Node.js API
Vision API
PostgreSQL
OLED / Web
| Layer | Role | Stack |
|---|---|---|
| ESP32-S3 | Load cell + camera capture; sends weight and image to API. | C++ · HX711 · OV2640 |
| Node.js API | Receives payloads, proxies OpenAI Vision, normalizes response. | Node.js · Express |
| Vision API | Identifies food and portion hints. | OpenAI Vision |
| PostgreSQL | Captures, estimates, calibration history. | PostgreSQL |
| OLED / Web | Shows calories and confidence to the user. | OLED |
ESP32-S3C++Node.jsOpenAI VisionPostgreSQL
Decisions
Lessons
Repeat
What is worth carrying into the next product.
- Sensor fusion approach validated accuracy gains.
- FreeRTOS task separation kept firmware maintainable.
Refine
What deserves another iteration.
- On-device fallback for offline use.
- Custom fine-tuned food model for latency.
Transfer
What changed my engineering judgment.
- Physical sensors anchor vision estimates — neither alone is enough.
- Prototype hardware constraints shape API design.
Next step
Want the implementation details?
Explore the repository, try the live flow, or reach out if you want to talk through the architecture behind it.
Keep exploring