Recipe, meal planning & nutrition assistant

"What do we cook tonight?" is one of the most frequent household decisions. An AI assistant turns the contents of the fridge, a diet profile and a budget into recipes, a weekly plan and a shopping list, adapts recipes (vegan, gluten-free, half portions) and can read steps aloud while the user cooks. It is a convenience and inspiration tool, not a medical dietitian: calorie and nutrient numbers are estimates, and allergy-critical decisions must stay with the human.

Problem & value

• Today: people browse recipe sites full of ads, copy ingredients by hand, forget what is in the fridge, throw food away and repeat the same five dishes. Households with allergies, intolerances, religious or ethical diets, or a medical diet (diabetes, coeliac disease, kidney disease) spend considerable effort adapting recipes.
• AI value: one conversational interface for ideation, adaptation, planning and list-making; reads recipes from photos or links; reduces food waste by planning around what is on hand; keeps variety and budget in view.
• Evidence: UNEP's Food Waste Index shows households are the largest source of food waste, so planning is a relevant lever [1]. Treat vendor saving claims as unproven until measured.

Variants

• Everyday cooking: "cook with what I have" from a fridge photo, quick weeknight dinners.
• Weekly planner: plan for 7 days, family of 4, vegetarian twice a week, 80 EUR budget, batch-cooking and leftovers reuse.
• Special diets and allergies: gluten-free, lactose-free, vegan, halal/kosher, low-FODMAP; allergen filtering against the 14 allergens named in EU food-information law [2].
• Nutrition tracking: photo or text meal logging with estimated calories/macros; link to fitness goals (PERS-002).
• Medical diets: only as support next to a dietitian or doctor.
• Voice and smart kitchen: hands-free step reading and timers.

How it works

1. profile (diet, allergies, dislikes, household, budget, equipment) + inputs (fridge/pantry photo, receipt scan, recipe URL/screenshot, free text)
2. ingredient recognition & normalisation (VLM / OCR
3. canonical ingredient list)
4. recipe retrieval or generation (own recipe base via RAG + LLM adaptation)
5. constraint check (allergens, diet rules, budget, time, equipment)
6. nutrition estimation (food-composition database lookup, not LLM guess)
7. weekly plan + merged shopping list (pantry stock subtracted)
8. output: app/chat, voice guidance, optional push to grocery service

• Ingredient and dish recognition: a vision-language model identifies items from photos; quantities and portion sizes are the weak point and should be confirmed by the user.
• Recipe generation and adaptation: an LLM scales, substitutes and rewrites; retrieval over the user's saved recipes keeps suggestions personal and grounded.
• Nutrition numbers: best practice is to let the LLM only extract ingredients and grams, and compute nutrients by deterministic lookup in a food-composition table (e.g. Bundeslebensmittelschlüssel BLS in Germany, USDA FoodData Central, Open Food Facts for packaged products) [3][4].
• Planning: an LLM suffices for most; a small solver (cost, variety, nutrients) is more reliable.
• Human-in-the-loop: the user confirms the ingredient list, portions and any allergen claim; the app should show the source of each nutrient value and a clear "estimate" label.

Data preparation

Private users mainly need their own recipe collection and preferences. An SMB (canteen, caterer, grocer, meal-kit shop) needs far more: recipes and production sheets in Excel or Word, product master data and allergen information in the ERP or shop system, price lists, and supplier specifications as PDFs. Typical gaps are inconsistent units and ingredient names, missing allergen and nutrient values, duplicate recipes, photos of cookbook pages, and knowledge that sits in the head of the chef. Preparation steps: convert recipes into one structured format (ingredient, amount, unit, step), map ingredients to article numbers and nutrient databases, flag the 14 allergens per recipe and verify them by a responsible person, build a synonym list (Tomate, Paradeiser) and scan or photograph paper recipes with OCR. Allergen data must be reviewed by a human, never taken from the model. Effort is medium to high for SMB, around 40-50 percent of the total, and small for private use. Related topics are PREP-011 for tidy tables, PREP-015 for terminology lists, PREP-006 for scanning cookbooks, PREP-005 for ERP exports and PREP-013 for a curated recipe base.

Implementation paths

Which path for whom

• Private: A - recipe and meal-planning apps are mature and cheap, and household data is low risk.
• Developer: C - Mealie or Tandoor with a small local model on HW1 gives a controllable base for own extensions.
• SMB: B - a food retailer, canteen or caterer needs the assistant to use its own product range, recipes, allergen data and shop system, and no off-the-shelf product covers that; cloud APIs in an EU region plus own data fit best.

Path A — Buy (recommended for private users)

• General assistants: ChatGPT, Claude, Gemini, Microsoft Copilot and Mistral Le Chat handle recipe ideas, substitutions and meal plans from a prompt; photo input works for fridge contents.
• Recipe and planning apps: Samsung Food (formerly Whisk), Mealime, Yazio (DE), Kitchen Stories (DE), Chefkoch (DE), Cookidoo for Thermomix (Vorwerk, DE), Bring! (CH, shopping lists), Cronometer and MyFitnessPal for nutrition tracking. Check privacy policy, EU hosting and what is shared with advertisers.
• Choose A when: you want results today without technical effort; accept the vendor's data handling.

Path B — Build on cloud APIs

• Models: any frontier multimodal model through API (Claude, Gemini, GPT, Mistral — Mistral is an EU provider) with structured outputs (JSON schema) for ingredient lists, steps and allergen flags.
• Data: Open Food Facts API for packaged products and allergen labels, USDA FoodData Central, BLS for German values; retrieval over a personal recipe collection (schema.org/Recipe JSON-LD parsing is supported by most recipe sites).
• Orchestration: n8n or a small web/mobile app; tool calls for nutrient lookup and shopping list export (e.g. to Bring!, Todoist or a retailer API where available).
• Choose B when: you want a custom family planner, a niche diet app, or a product for a community. Use EU regions and a data processing agreement if you serve other users.

Path C — Self-host (open source)

• Models: open-weight LLMs such as Qwen3, Llama, Gemma or Mistral Small variants (quantised) via Ollama or llama.cpp; open VLMs (Qwen-VL family, Gemma multimodal) for photos; Whisper for voice input; open TTS for reading steps.
• Apps: Mealie and Tandoor Recipes (self-hosted recipe managers with meal plans and shopping lists, with import from URLs) [5][6]; connect to a local LLM for import, cleanup and suggestions.
• Hardware: HW1 (mini-PC or laptop); a 16 GB GPU helps photo understanding. See PERS-005.
• Choose C when: family health data must stay at home.

Path D — Custom training

Rarely sensible for private users. Nutrition apps may fine-tune food-image portion estimation, which needs thousands of meal photos with weighed ground truth.

State of the art (2026)

• Recipe creativity and adaptation are strong: current frontier models substitute ingredients, scale quantities, translate and re-format recipes well, and reasoning models can satisfy multiple constraints in a weekly plan.
• Multimodal fridge and meal photos work for obvious items; hidden ingredients, mixed dishes, oil, sauces and portion sizes make calorie estimation from a single photo unreliable. Research prototypes and apps improve with depth sensors or a reference object, but users should treat output as a rough estimate.
• Tool-grounded nutrition (LLM calls a food-composition database) is clearly better than free-form LLM numbers, which can be confidently wrong.
• Failure modes: hallucinated cooking times/temperatures, unsafe food-handling advice, missed hidden allergens ("may contain"), invented quantities, plans ignoring clinical conditions.

Shared resources & combinations

• Same stack (llm-general + vlm): PERS-006, PERS-008 and PERS-018 can share one home-server VLM (PERS-005).
• PERS-002 health and fitness coach: calorie targets and activity feed the plan.
• PERS-011 shopping and price comparison: turns the shopping list into the cheapest basket.
• PERS-009 family organisation: shared calendar, who cooks when, fridge inventory.
• VIS-033 food waste tracking in commercial kitchens and OPT-031 formulation optimisation: professional cousins with the same ingredient and nutrient data.

Effort, cost & KPIs

• Path A: minutes; free tiers to monthly subscriptions.
• Path B: weekend prototype; production app weeks to months; costs are mainly development and data licences.
• Path C: about a day for Mealie/Tandoor plus a local model; light admin.
• KPIs (personal): share of planned meals cooked, food thrown away, grocery spend per week, variety (distinct dishes per month), satisfaction of household members, nutrient-target adherence where relevant.

People & change

For private use, involve the household: allergies, taste and budget belong to everyone at the table. For an SMB, management decides scope and budget; the department lead (kitchen, product range) owns recipes and allergen rules and should drive the pilot; IT connects shop or ERP data; suppliers supply specifications; and the DPO checks any customer profiles, since diet and allergies can reveal health information (Art. 9 GDPR), which makes consent, an AVV (Art. 28 GDPR) and possibly a DPIA relevant. Works council: depends. If the tool only serves customers, no; if kitchen staff use it and it logs their activity, §87(1) no. 6 BetrVG applies, and §90(1) no. 3 requires early information when work processes are planned with AI. AI literacy: Art. 4 AI Act asks the company to support staff literacy, for example a short workshop on checking allergen output. Typical concerns are liability for allergens and job fear among cooks; address them by keeping allergen statements human-verified and by positioning the tool as a planning aid. Start with one canteen week or product line, with a few champions. Humans decide on every published recipe and allergen label. See GUIDE-004, GUIDE-008 and GUIDE-014.

Risks, regulation & compliance

• EU AI Act: minimal risk for a lifestyle recipe and planning assistant (AI literacy, Art. 4, applies to providers and professional deployers). Chatbot disclosure (Art. 50) applies to products. If a product claims to diagnose, treat or manage a disease (e.g. insulin dosing from meal photos), it can become a medical device under the MDR, and high-risk under the AI Act via Annex I; keep wellness and medical claims separate [7].
• GDPR: diet, allergies, weight and religion-linked diets reveal health or religious beliefs, which are special-category data (Art. 9). Providers need explicit consent or another Art. 9 basis; prefer on-device or EU hosting, no ad-profiling, easy deletion. A household member's data is theirs, including children's [8].
• Food safety and allergens: the EU Food Information to Consumers Regulation requires labelling of 14 allergens; an AI cannot guarantee that a product or dish is allergen-free. Always read the label on the package [2].
• Health harm: disordered eating, crash diets, wrong advice in chronic illness or pregnancy. Mitigate: no extreme targets, "not medical advice" notice, referral to a dietitian.

Getting started

1. Write a one-paragraph household profile (diet, allergies, dislikes, budget, equipment, time) and save it as a standing instruction in your assistant or app.
2. Ask for a 5-day plan with a merged shopping list and "use what is in the fridge first"; check that allergens are really excluded by reading the final ingredient list yourself.
3. Import 10 favourite recipes into Mealie or Tandoor (or Samsung Food) so the assistant can draw on your own collection.
4. If you track nutrition, choose an app that shows the data source (BLS, USDA, Open Food Facts) instead of unexplained numbers.
5. Review after two weeks: food waste, spend, what you actually cooked; adjust the profile.

Sources

1. UN Environment Programme — Food Waste Index Report 2024 — https://www.unep.org/resources/publication/food-waste-index-report-2024
2. Regulation (EU) No 1169/2011 on food information to consumers (14 allergens, Annex II) — https://eur-lex.europa.eu/eli/reg/2011/1169/oj
3. USDA FoodData Central — https://fdc.nal.usda.gov/
4. Open Food Facts — API documentation — https://openfoodfacts.github.io/openfoodfacts-server/api/
5. Mealie — self-hosted recipe manager — https://github.com/mealie-recipes/mealie
6. Tandoor Recipes — https://github.com/TandoorRecipes/recipes
7. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
8. Regulation (EU) 2016/679 (GDPR), Art. 9 — https://eur-lex.europa.eu/eli/reg/2016/679/oj