Estimating what's on a plate
The question
When an app tells you a meal was 620 kcal, how wrong is that likely to be — and does looking the food up in a nutrition database make it less wrong?
What the evidence says
Two things, and the second one surprises most people.
Estimating food from a photo alone is bad, and telling the app what’s in the meal roughly halves the error. Running ChatGPT-5 over 195 dishes, energy error fell from 30.5% on an image alone, to 24.4% once non-visual details were added, to 13.9% given a list of ingredients with weights. Fat is the worst macro throughout — 52.9% down to 24.2%. Carbohydrate is the most reliable. Real shipped apps show the same shape: an NIH group weighed 102 metabolic-kitchen meals to 0.1 g and found photo-based apps underestimated by 250–345 kcal per meal, with roughly 30 g of that in fat alone.
So the single most useful thing anyone can do is say what was in the food and roughly how much. That is not a limitation of one model; it is where the information actually is.
Looking the food up in a database does not reliably help. This is contested, and the disagreement is worth stating rather than smoothing over. NutriBench, testing 11,857 human-verified meal descriptions, has a section headed “RAG Does Not Always Improve Performance” — for meals described in ordinary language, retrieval-augmented answers were consistently worse than the model reasoning on its own. Open-KNEAD (2026) reports the same: a modern multimodal model’s direct estimate now matches or beats a full retrieval pipeline, and naive retrieval made energy error worse on two of three datasets.
The strongest result on the other side is real: grounding an image model in the USDA FNDDS database cut error to 47.7 kcal per dish, against 168–277 for ungrounded baselines. But those test images were built from the top 1,000 FNDDS food codes — the right answer was guaranteed to be inside the database being searched, which is not what happens when someone types “leftover stir fry.”
The mechanism behind the failures is mundane and specific. Food databases store raw and cooked forms as separate rows with no field marking which is which — the state exists only as English words in the description. Cooked brown rice is 123 kcal per 100 g; raw is 367. Matching a plate of cooked rice to the raw row triples it. And text matching is blind to exactly that word: for the query “brown rice”, standard string similarity ranks the raw entry above the cooked one. The error also flips direction depending on the food — starches absorb water and meats lose it — so there is no single correction that fixes it.
Portion is a major source of error. Doulah and colleagues identified both portion-size estimation and mismatched portion definitions between food and nutrient sources as important contributors. Their breakdown does not support combining those categories into a claim that nearly all error is about quantity rather than food composition. Consistent with the broader uncertainty, a study of 16 manual-logging apps, using exact specified weights, trained researchers and commercial databases, still landed 11–17% off on daily energy.
Key studies
| Study | Year | Design | n | Finding |
|---|---|---|---|---|
| Rodríguez-Jiménez et al., Nutrients 17(22):3613 | 2025 | Model evaluation vs. known composition | 195 dishes | Energy error 30.5% image-only → 13.9% with weighted ingredient list; fat 52.9% → 24.2% |
| Hengist et al. (NIH/NIDDK), presented NUTRITION 2026 | 2026 | Metabolic-kitchen meals weighed to 0.1 g vs. app output | 102 meals | Apps underestimated 250–345 kcal/meal; ~30 g/meal of it fat. Conference abstract, not yet peer-reviewed |
| NutriBench, arXiv 2407.12843 | 2024 | Benchmark, human-verified meal descriptions | 11,857 | For natural-language servings, chain-of-thought alone beat retrieval-augmented generation |
| Open-KNEAD, arXiv 2607.12911 | 2026 | Retrieval pipeline vs. direct estimate | 3 datasets | Direct MLLM estimate matched or surpassed full retrieval; naive RAG worsened MAE on 2 of 3 |
| DietAI24, Commun Med (PMC12589391) | 2025 | MLLM + retrieval over FNDDS | ASA24 image set | 47.7 kcal/dish MAE vs 168–277 baselines — but images built from the same 1,000 food codes the retriever searched |
| Doulah et al., Int J Obes 46(11) | 2022 | Error-source decomposition | — | Portion estimation and mismatched portion definitions were major sources of error |
| Li et al., Nutrients (PMC11314244) | 2024 | 16 logging apps, exact weights, trained users | — | Daily energy still 11–17% off with commercial databases |
| Hieronimus et al., Nutr Res (PMID 39102765) | 2024 | AI-generated meal plans assessed | 108 plans | Plans “should not be relied on”; no significant difference between prompts or tools |
| O’Hara et al. (PMID 40004936) | 2025 | Micronutrient accuracy | — | Vitamin D −100%, potassium −49.5%, folate −38.6%, calcium −27.8% |
Confidence rating and why
Moderate. The direction is consistent across independent lines — controlled model evaluations, a metabolic-kitchen comparison, a large human-verified benchmark, and an error-source decomposition all point the same way. But the field is young and moving: the strongest pro-database result and the strongest anti-database result were published within a year of each other, the NIH app study is still a conference abstract with no per-app breakdown, and no single study cleanly separates nutrition-table error from portion error inside one design. The “portion dominates” conclusion is inference across studies rather than a measured decomposition, and it is labelled as such.
What Eat On Pace does with this
- The estimate is itemised, and every amount is editable. This is not a convenience feature. Naming the ingredients and their weights is the single measured intervention that roughly halves the error, and it is the reason Eat On Pace shows a breakdown instead of one number.
- No food database (ADR 0022). On the input Eat On Pace actually takes — food described in ordinary language — retrieval is not measured to help and carries a large, silent failure mode in the cooked/raw distinction.
- Portion gets a direct correction path. A recipe’s finished weight is measured once on a scale rather than guessed on every estimate, and a packaged ingredient’s numbers are read off its label rather than inferred from a photo.
- The app says plainly what an estimate is. No accuracy claim is made that these studies do not support.
- Micronutrients are out. LLM estimates of them are not imprecise, they are directionally broken. Eat On Pace surfaces none, and this doc does not cover them.
What we do NOT claim
Being straight about the limits matters as much as the estimate itself, so here is what these numbers do and do not support.
- That Eat On Pace’s numbers are accurate. The best measured configuration in the literature still sits around 14% off on energy, and that is with the ingredients and their weights supplied.
- That correcting the ingredient rows makes an estimate exact. It roughly halves the error. Halving is not removing.
- That Eat On Pace is more accurate than logging manually against a food database. Nobody has measured that comparison. What the evidence supports is narrower: retrieval is not measured to reliably help on food described in ordinary language, and careful manual logging with exact weights and a licensed database still lands 11 to 17% off on daily energy.
- That a photo alone is good enough. It is the worst of the measured configurations at 30.5%, and it is a starting point rather than an answer.
- Anything about micronutrients. LLM estimates of them are directionally broken, Eat On Pace surfaces none, and none of the numbers here apply to them.
- That we can tell you how often an estimate is internally inconsistent. No published study measures how often model outputs violate the basic calorie-from-macros arithmetic, and we do not claim a rate.