Real-Time Nutrition Assistant App
Real-Time Nutrition Assistant App
A R T I C L E I N F O A B S T R A C T
Keywords: In an era where intelligent technologies are rapidly shaping our lives, a Real-Time Nutrition Assistant System
Nutrient-based meal recommendation emerges as an essential tool for maintaining a healthy lifestyle and promoting awareness. A Real-Time Nutrition
personalized nutrition Assistant System advances nutrition and healthcare technologies to improve public health by offering quick
Artificial intelligence
insight into the nutritional content of our meals. This study introduces Diet Engine, an innovative smartphone
Convolutional neural network (CNN)
Image segmentation
application powered by machine learning that enhances health outcomes by providing immediate food classi
Machine learning fication and personalized dietary suggestions. The system features modules using deep learning (DL) and Con
Deep learning volutional Neural Networks (CNNs) to detect food, as well as textual analysis and natural language processing
(NLP) to estimate components such as nutritional content. It offers customized food suggestions according to the
user’s dietary preferences and constraints. Diet Engine accurately identifies and evaluates the nutritional value of
food from images. The system employs a client-server architecture, using advanced deep learning techniques like
YOLOv8 (You Only Look Once version 8) and Convolutional Neural Networks (CNNs) optimized for real-time
object detection with 295 layers, for training and processing image requests. Our system outperforms existing
algorithms, achieving an 86 % classification accuracy on food datasets. Moreover, a personalized chatbot pro
vides diet advice, meal recommendations, and fitness suggestions. By seamlessly integrating advanced deep
learning algorithms with user-centric features, this study underscores the transformative potential of Diet Engine
in fostering healthier eating habits, raising nutritional awareness, and contributing to a global shift toward more
informed and sustainable lifestyle choices.
1. Introduction facilitates the process, enhances accuracy, and eliminates the necessity
for manual monitoring (Bahadur et al., 2022; Liu et al., 2018; Yunus
Obesity is now rising rapidly and has a significant impact on com et al., 2019).
munities all over the world. More than 1.5 billion individuals worldwide A real-time food nutrition classifier is a highly efficient device
are classified as overweight, with over 500 million categorized as obese designed to rapidly and accurately evaluate the nutritional composition
(Pérez Rodrigo, 2013; Sen et al., 2016). Overweight refers to a BMI of of food items. It involves using state-of-the-art technologies, such as
25–29.9 kg/m², indicating excess weight, while obesity is a BMI of 30 computer vision and machine learning, to recognize and categorize food
kg/m² or higher, reflecting significant fat accumulation (Gewehr et al., items using images or other sensory inputs (Mitra et al., 2022). The
2020; Smith & Smith, 2016). Maintaining a healthy diet can be chal classifier provides quick information regarding the nutritional content
lenging in the fast-paced world of today. Mobile applications can offer of identified foods, including calories, macronutrients (e.g., protein,
an effective solution in the fight against weight management, enabling carbohydrate, and fat), vitamins, and minerals (Wang, 2022).
users to instantly determine the nutritional value and caloric content of On the other hand, a technologically enabled platform known as a
their meals by utilizing real-time food identification capabilities. This recommender system for personalized nutritional support offers
Abbreviations: AI, Artificial Intelligence; ANN, Artificial Neural Network; BMI, Body Mass Index; CNN, Convolutional Nural Network; DL, Deep Learning; EMA,
Efficient Multi-Scale Attention; ENet, Efficient Neural Network; FCN, Fully Convolutional Network; GFLOPs, Giga Floating Point Operations per Second; IoT, Internet
of Things; ML, Machine Learning; NLP, Natural Language Processing; NN, Neural Network; PPGR, Postprandial Glycaemic Responses; SegNet, Segmentation
Network; SPP, Spatial Pyramid Pooling; SF, Size Factor; SURF, Speeded-Up Robust Features; SVM, Support Vector Machine; YOLO, You Only Look Once.
* Corresponding author at: Department of Mechatronics Engineering, Rajshahi University of Engineering and Technology (RUET), Rajshahi, 6204, Bangladesh.
E-mail address: amsaad1836@[Link] (A.M. Saad).
[Link]
Received 24 September 2024; Received in revised form 15 March 2025; Accepted 5 April 2025
Available online 13 April 2025
2772-753X/© 2025 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY license ([Link]
A.M. Saad et al. Food Chemistry Advances 7 (2025) 100978
individuals customized meal recommendations based on their unique Fig. 1 illustrates the evolution of food analysis research from 2012 to
needs, tastes, and health objectives. These systems integrate a variety of 2022, using three colored dots to differentiate key aspects: blue dots
data sources, including user input, dietary choices, health issues, and represent various model architectures, orange dots denote datasets uti
nutritional requirements, to provide individualized recommendations lized, and green dots highlight major contributions or findings. Orga
(Hovale & G, 2016). nized vertically, the timeline lists research teams (e.g., Bahadur et al.,
In this research, we introduce Diet Engine, a real-time food nutrition Mitra et al.) on the left, with each dot marking a specific development in
assistant system. Diet Engine automatically identifies food items from their work. Over the years, the field has advanced from simple models
images, extracts nutrient information, and provides personalized dietary like Decision Trees in 2012 to sophisticated architectures like Effi
advice based on individual health goals. The system analyzes food items cientDet B0+FPN in 2022, transitioning from basic datasets such as
with high precision using advanced computer vision and machine OpenCV to specialized food datasets. Similarly, contributions have
learning techniques such as convolutional neural networks (CNNs). By evolved from meal planning to more complex applications like
employing the YOLOv8 framework, which incorporates 295 convolu smartphone-based solutions and transfer learning. Notably, the period
tional layers, our model is trained on extensive food datasets and from 2020 to 2022 shows a surge in research activity, reflecting
effectively performs food segmentation, item categorization, and calorie heightened interest in food analysis. The latest developments in 2022
estimation. The primary objective of this study is to develop a system emphasize practical applications, such as nutrient evaluation and mo
that delivers immediate feedback on the caloric content of consumed bile technologies, underscoring a trend toward real-world usability.
foods while offering tailored dietary guidance. Diet Engine demonstrates
the transformative potential of technology in fostering healthier eating
habits and achieving optimal health outcomes. 2.1. Existing food nutrition classifier and recommender systems
The subsequent sections of this article are structured as follows:
Section 2 provides background and related work; Section 3 shows the Food nutrition classifiers and recommender systems have emerged as
methodology of this research; Section 4 provides the results and dis powerful tools that allow users to easily assess the nutritional value of
cussion; Section 5 includes ideas for further work and Section 6 con their meals. Typically, the system processes and classifies food photos
cludes this paper. taken with mobile apps or camera-enabled devices. Training a classifi
cation system necessitates extensive food image databases with diverse
2. Background and the related work food classes and sufficient images per class (Mortazavi &
Gutierrez-Osuna, 2023). To distinguish food items from the background,
The application of computer vision and machine learning technolo reliable segmentation is crucial, regardless of variations in lighting or
gies for the automated identification of food items and the estimation of situations where multiple food items are mixed, making clear separation
their nutritional content from photographic images constitutes funda and identification more challenging (Hovale & G, 2016). Convolutional
mental steps in the development of real-time food nutrition classifica Neural Networks (CNNs) and Artificial Neural Networks (ANNs) are
tion and recommendation systems, facilitating accurate dietary typical machine learning models employed in image-based food cate
assessments and personalized nutritional guidance (Zhu et al., 2011). gorization (Kumar et al., 2021). Additionally, datasets like
Modern food nutrition classification systems use advanced technologies UNIMIB2015, UNIMIB2016, Food201-Segmented, and SUECFood can
like machine learning and computer vision to quickly analyze and evaluate food segmentation and calorie estimation systems (Aslan et al.,
identify nutritional content. On the other hand, current studies 2020).
demonstrate how artificial intelligence (AI) is increasingly being used to A study (Samad et al., 2022) classified mobile apps by their main
enhance eating practices and nutritional tracking. Several studies have purpose and features, including categories such as nutrition, calorie,
investigated various AI approaches to improve the recognition of food, food, diet, fitness, and others. Many nutrition tracker apps track nutri
assess its nutritional value, and provide personalized dietary recom ents, but others also track calories. As shown in Table 2, nutrition
mendations. Table 1 presents an analysis of related systems along with trackers comprised 28.75 % (23 out of 80) of all applications. On the
the dataset used, model architecture, contribution, and limitations of the other hand, 31.25 % (25 out of 80 calorie tracker applications) simply
available literature of the last 10 years (2012–2022). recorded calorie consumption. Food trackers only record names. Only
nutrition or calories were tracked. Only 10 % (8 out of 80) of these apps
Table 1
Related work analysis.
Purpose Dataset used Model architecture Contribution Limitation Refs.
Food image analysis 1000-dimensional feature Deep CNN (FoodDist) Open-source model for food image Potential biases in baseline (Yang et al.,
space analysis selections 2017)
Subcontinental food dataset CNNs Smartphone app for nutritional Depth camera method limitations (Bahadur et al.,
(200 categories) assessment for calorie evaluation 2022)
Food201 dataset GoogLeNet, DeepLab A new dataset for semantic image Poor performance on NFood (Myers et al.,
segmentation images 2015)
FoodLog dataset (623,956 NCT and NN classifiers Combines classifiers for improved Similarity variability complicates (Yu et al., 2019)
images) recognition the recognition
Nutritional Customized dataset (19 EfficientDet D0 Comprehensive nutrient User-friendliness impacts success
evaluation classes evaluation rates (Mitra et al.,
2022)
UEC food dataset Rank-SVM vs. CNN Diabetes meal planning with Uncertainty about the nutritional (Sudo et al., 2020)
healthcare professionals values of meals
Meal planning and Food preference data Not specified Automatic daily meal planning Difficulty in high-calorie meal (Elsweiler &
consultation planning Harvey, 2015)
OpenCV dataset Decision tree analysis Intelligent diet consultation Needs more flexible planning (Lin et al., 2012)
methods
Attribute Dataset with 9 food classes DeepLabV3+, FCN, Evaluation of segmentation and Difficulty in identifying meals (Freitas et al.,
classification SegNet, ENet classification methods with multiple foods 2020)
Common crawl and scraped Inception-v4 and Transfer learning approach for Challenges in user modeling (Shen et al., 2020)
data Inception-v3 food image attributes
2
A.M. Saad et al. Food Chemistry Advances 7 (2025) 100978
3
A.M. Saad et al. Food Chemistry Advances 7 (2025) 100978
4
A.M. Saad et al. Food Chemistry Advances 7 (2025) 100978
features lightweight SEConv for faster processing and introduces the SEF code evaluates the Size Factor (SF): if the ratio falls within a defined
module with Efficient Multi-Scale Attention (EMA) for better multi-scale range (for example, between 4.0 and 6.0), the fruit is classified as "Big."
object detection (Ketkar & Moolayil, 2021; Sudharson et al., 2023). The Otherwise, it is classified as "Small." However, it is noted that this area
model’s performance is enhanced by its comprehensive training strategy ratio calculation is accurate only when the camera is placed about 25 to
and key innovations. YOLOv8 has a wide range of applications across 30 inches away from the food item.
various industries, making it a compelling choice for object detection
tasks. The model architecture utilizes a combination of several key 4. Result and discussion
approaches:
4.1. Application interface of diet engine
• Convolutional Neural Networks (CNNs): This is the foundation of
the model. Convolutional layers are the building blocks, responsible Diet Engine features an intuitive interface that simplifies the process
for ex-tracking features from the input image. of logging and tracking meals, ensuring that even users with limited
• Residual Connections: These blocks allow the network to learn technical expertise can benefit from its capabilities. The image recog
deeper representations by aiding the flow of gradients during nition module employs deep learning algorithms to identify food items
training. This helps address the vanishing gradient problem that can by simply pointing the camera at them.
occur in deep networks. Once identified, the server processes the image to analyze the food’s
• Spatial Pyramid Pooling (SPP): The presence of a spatial pyramid nutritional content. The server-based inspection system enhances user
pooling layer ([Link]) suggests the experience by shifting computation-intensive tasks, such as image pro
model incorporates this technique. SPP increases the network’s cessing, food classification, and nutrient analysis to a high-performance
ability to identify objects of various sizes by enabling it to record server, rather than depending on the limited capabilities of a mobile
features at various scales within the image. device. This approach enables faster processing, improved accuracy, and
• Encoder-Decoder Structure (implicit): While not explicitly greater scalability, ensuring that users receive real-time results regard
mentioned in the table names, the architecture likely follows an less of the task’s complexity. Moreover, server-side processing will
encoder-decoder structure. This is a common pattern in object enable seamless updates and model improvements without the need for
detection models. The "encoder" part (initial convolutional layers changes in the client application, ensuring that users always take
and pooling) extracts features from the image, while the "decoder" advantage of the latest developments in food recognition and analysis
part (upsampling layers and concatenation) increases the resolution technologies.
of the feature maps and combines information from different stages The system lets users create and manage profiles with personal in
to localize objects. formation, dietary preferences, sensitivities, and medical issues. The
system provides accurate and relevant dietary advice by adapting it to
Overall, the model architecture combines these elements to create a user profiles. Fig. 3(a) exhibits the application interface walkthrough.
deep CNN specifically suited for object detection tasks. It leverages The proposed real-time food nutrition assistant, "Diet Engine," provides
convolutional layers for feature extraction, residual connections for users with instant nutritional information on selected foods. The Diet
efficient learning, SPP for multi-scale feature capture, and an implicit Engine effectively categorizes various fruits and presents their total
encoder-decoder structure for object localization. As presented in calories in real-time, as depicted in Fig. 3(b) utilizing image recognition
Table 3, the model architecture is a complex neural network with 295 techniques. By assessing nutritional values, the system provides
layers, consisting of various types of modules. personalized dietary guidance and recommendations based on individ
ual health goals and preferences. In addition, the software allows for the
examination of food images obtained from a private gallery, as shown in
3.6. Determination of the size of the food Fig. 3(c). Diet Engine has a specialized chatbot that enables users to
obtain any information they may need regarding nutritional composi
Unprocessed food can be of different sizes and their calorific content tion and dietary advice, as illustrated in Fig. 3(a) Diet Engine displays
varies with the size of the individual food item. For instance, a small- the user’s Body Mass Index (BMI) to provide them with information
sized apple can have 53 calories, and a large size can have 95 calories. about their weight status, including whether they are within the optimal
Similarly, nutritional content varies according to different sizes and weight range, overweight, or underweight. This data supports the in
shapes of different food items. The mobile application employs a stra dividual in setting their fitness goals.
tegic formula to determine the Size Factor (SF) of the food.
A 4.2. Evaluation of the efficiency of the model
SF = (1)
A∘
Accuracy is a very critical factor for the “Real-time Food Nutrition
Where, A represents the area of the outer box, serving as a fixed
Assistant System” as it directly influences the provision of timely and
reference for size estimation, and A∘ denotes the area of the boundary
accurate dietary recommendations and insights. Ensuring accuracy in
box that encapsulates the detected food item. Since the outer box area
nutritional content estimation is critical for users to efficiently achieve
remains constant, the ratio of these areas provides a standardized
their dietary objectives. The consistent reliability of the system,
method for accurately determining food item size as shown in Fig. 3
demonstrated by its ability to achieve an 86 % classification accuracy on
(b.1). The program categorizes fruits—such as apples, bananas, and
food datasets, fosters user confidence and satisfaction, thereby encour
oranges—using a series of if-else statements. First, it checks if the food
aging adherence to personalized dietary regimens. The consistent pre
item’s name matches one of the specified fruits. If a match is found, the
cision of the system promotes user confidence and contentment, thereby
encouraging compliance with individualized dietary regimens. In the
Table 3 ever-evolving domain of real-time nutrition analysis, users are endowed
Summary of the model architecture and configuration.
with timely and actionable insights that optimize their health and well-
Name of the attributes No. of the attributes being through the provision of high accuracy.
Layers 295 However, the robustness of the proposed occlusion method was
Parameters 25,860,373 evaluated by creating a supplementary dataset of partially occluded
Gradients 25,860,357 fruits. The performance of the model was assessed under different oc
GFLOPs 79.1
clusion levels. Each occlusion dataset included 86 images, and detection
5
A.M. Saad et al. Food Chemistry Advances 7 (2025) 100978
Fig. 3. Overview of Diet Engine, showcasing its user interface flow, real-time food calorie estimation using object detection, and non-real-time calorie tracking via
manual selection.
rates were calculated for 0 % to 70 % for occlusion levels as shown in quantity of epochs, or training cycles, while the y-axis represents the
Fig. 4(g). At 0 % occlusion, the detection rate was 96.51 %, maintaining loss, which serves as a measure of model performance. The plot consists
consistency up to 30 % occlusion, indicating that the model performed of two lines, with the blue line indicating the training loss. The orange
effectively as long as less than one-third of the image was obstructed. A line represents the validation loss curve. Initially, both lines show a
slight decline was observed at 35 % and 40 % occlusion, where detection sharp decline in loss, indicating rapid learning. As epochs increase, both
rates dropped to 95.35 % and 94.19 %, respectively, as a significant lines plateau, suggesting that the model is converging (learning slows
portion of the image became obscured. Interestingly, the detection rate down). As the validation loss stabilizes, the training loss keeps going
increased slightly to 95.35 % at 45 % occlusion, before declining to down slowly, and the graph provides insights into how the model’s
90.70 % as occlusion levels ranged between 50 % and 55 %, eventually performance changes over time during training. It’s essential for opti
reducing further to 86.05 % at 60 % occlusion. At more extreme oc mizing machine learning models.
clusion levels, the model’s performance became significant, with an
accuracy of 79.07 % at 65 % occlusion and 70.93 % at 70 % occlusion. 4.4. Training and validation accuracy
Overall, the model showed robust performance up to 50 % occlusion but
showed a clear decline beyond that point suggesting that at least half of The machine learning training of the model process’s training and
the image is required to accurately identify foods. These results high validation accuracy throughout epochs is depicted in Fig. 4(b) graph.
light the model’s strengths and limitations under occlusion, providing There are now two lines graphed. The blue line of the graph represents
useful insights into its practical use. The high accuracy can be attributed the training accuracy, while the orange line represents the validation
to the advanced architecture of the YOLOv8 model, which utilizes multi- accuracy. Both lines start at a lower value of accuracy and show an
scale feature extraction and convolutional layers to identify key features upward trend as epochs increase, indicating that the model’s perfor
even when parts of an object are occluded. This architecture ensures mance improves over time. In summary, this visualization provides in
reliable classification of food items, demonstrating the model’s effec sights into how the model’s accuracy changes during training. It’s
tiveness under challenging conditions. crucial for assessing model performance and ensuring generalization to
unseen data. The model’s performance evolves as it undergoes addi
tional training epochs. Ideally, both training accuracy and validation
4.3. Training and validation loss accuracy should rise as the model learns. To ensure that the model is
learning efficiently, we should ideally see an increase in both training
The graph in Fig. 4(a) depicts the validation and training loss of a and validation accuracy.
model during epochs in the machine learning model training procedure.
Inferences can be drawn from it as follows: The x-axis displays the
6
A.M. Saad et al. Food Chemistry Advances 7 (2025) 100978
7
A.M. Saad et al. Food Chemistry Advances 7 (2025) 100978
4.8. Interdependence of precision, recall, and confidence levels personalized chatbot for dietary guidance, a user-centric feature absent
in prior models. These advancements establish Diet Engine as a signif
In the Precision-Confidence Curve Fig. 4(d), precision is plotted icant improvement in food recognition and dietary assistance
against the confidence threshold for each class. The graph indicates that technologies.
as the confidence threshold increases, precision improves for most
classes. This is expected, as higher confidence predictions are more 4.10. Discussion
likely to be accurate. Notably, certain classes, such as "bread" and "milk,"
maintain consistently high precision across all confidence levels, while The results show that these systems could help people make healthier
others, like "boiled egg," show greater variability, reflecting the class- food choices. Real-time analysis of food images offers a convenient and
specific challenges in prediction accuracy. user-friendly approach to nutrient intake awareness. The “Real-time
The precision-recall curve of Fig. 4(e), illustrates the trade-off be Food Nutrition Assistant System” (Diet Engine) has 86 % accuracy and
tween precision and recall. The area under the curve for each class advanced architecture, making it robust but computationally intensive.
highlights the balance achieved. For example, "bread" achieves a near- The real-time mobile system is simpler and less demanding, with 83.93
perfect balance with an area of 0.995, whereas "boiled egg" demon % accuracy but suffers from user acceptance issues and a limited dataset.
strates a lower area of 0.773, indicating room for improvement. The The segmentation-assisted system, achieving 56.2 % accuracy, faces
overall mean average precision for all classes is 0.897, showcasing the scalability and segmentation challenges, though it is specialized for di
model’s ability to generalize effectively across the dataset. etary assessment. The results indicate that these systems may assist in
The F1 confidence curve of Fig. 4(f), further refines this analysis by dividuals in selecting healthier foods. Utilizing real-time analysis of food
identifying the confidence threshold where precision and recall are images presents a practical and intuitive method for increasing aware
optimally balanced. The F1-score reaches its peak for most classes at ness of nutrient intake. The accuracy and robustness of classification
specific thresholds, with the overall F1-score for all classes achieving a algorithms concerning a wide variety of food categories and cultural
maximum value of 0.86 at a confidence level of 0.264. These results cuisines continue to be significant obstacles.
demonstrate the model’s optimal performance when both precision and Personalized dietary recommendations face significant hurdles due
recall are given equal importance. to variations in postprandial glycemic responses (PPGRs), even when
individuals consume identical meals (Zeevi et al., 2015). This variability
4.9. Research impact undermines the effectiveness of universal dietary guidelines. Machine
learning techniques, such as neural networks, have been proposed to
The proposed system implements an advanced technique that makes predict individualized PPGRs based on real meal data, moving beyond
the application more effective and user-friendly. Diet Engine is one of generalized recommendations (Mitra et al., 2022). However, a critical
the pioneering mobile applications that automatically detects food items barrier to the advancement of food analysis systems is the lack of an
and displays their nutritional contents It allows the users to gain insight notated large-scale datasets for food segmentation, which limits the
into their dietary intake. Table 4 provides a comparative analysis with ability to evaluate and refine segmentation algorithms effectively
some similar technologies found in the literature. (Franco et al., 2016). Developing comprehensive, well-annotated data
Unlike Zhu et al. (2011), which relied on a small dataset of 30 videos sets tailored to food segmentation is essential for overcoming this lim
with limited generalizability, the model’s ability to generalize across itation and enabling accurate real-world applications (Aslan et al.,
diverse food items, environments, and presentation styles. Diet Engine 2020).
leverages an extensive dataset comprising 6692 labeled images, The concept of real-time in mobile food analysis systems adds
encompassing a diverse array of food items captured under varying another layer of complexity. Real-time functionality refers to the sys
lighting conditions, angles, and contexts. This dataset is categorized into tem’s ability to deliver near-instantaneous feedback, encompassing
seven distinct food classes: apple, banana, boiled egg, bread, fried egg, rapid food recognition, prompt nutrient estimation, and efficient infor
milk, and orange. To ensure robust learning and model reliability, the mation display (Pouladzadeh et al., 2015; Yunus et al., 2019). For
dataset is strategically partitioned into training, validation, and testing instance, advanced segmentation algorithms, such as DeepLabV3+ and
subsets, with the majority allocated to the training set. This approach Mask R-CNN, can handle high intra-class variability (e.g., different
enables the system to achieve high accuracy and adaptability across preparations of the same dish), addressing the challenge of accurate food
diverse real-world scenarios. This diversity allows the model to learn item segmentation (Aslan et al., 2020). Additionally, portion size esti
more robust and generalized features, making it more reliable when mation, often influenced by factors like camera angle or object inter
applied to real-world scenarios where variability is common. While ference, can be improved by integrating depth information from depth
earlier approaches, like Zhu et al. (2011) and Maruyama et al. (2012) cameras or utilizing 3D simulation datasets (Aslan et al., 2020; Lo et al.,
utilized traditional classifiers such as SVM. Diet Engine, on the other 2020).
hand, employs an advanced 295-layer neural network with residual Further enhancements can be achieved through the integration of
connections, which enhances gradient flow and enables deeper feature ubiquitous sensors and mobile applications (Mitra et al., 2022). Sensors
learning, achieving a superior accuracy of 86 %. Additionally, the can passively collect supplementary dietary data, such as meal timing
integration of spatial pyramid pooling (SPP) allows robust multi-scale and frequency, while mobile applications can process this information to
feature extraction, effectively addressing the segmentation challenges provide personalized dietary recommendations (Jia et al., 2017; Lo
noted in Maruyama et al.’s framework. Finally, Diet Engine introduces a et al., 2024). By combining these innovations, real-time food analysis
Table 4
Comparison of Diet Engine with related systems.
Ref. Techniques Features Classifier Performance Limitations
(Zhu et al., 2011) Image-based SURF, color histograms Linear SVM (one-vs- 83.93 % recognition Small dataset (30 videos), limited
ingredient rest) accuracy (top six generalizability
recognition applicants)
(Maruyama et al., Normalized Cuts Color, texture extraction SVM 56.2 % accuracy Performance degrades with
2012) framework increased categories
Proposed System Advanced deep CNNs, residual connections, Deep learning (295- 86 % accuracy Overcomes prior limitations,
(Diet Engine) neural network spatial pyramid pooling (SPP) layer architecture) integrates real-time chatbot support
8
A.M. Saad et al. Food Chemistry Advances 7 (2025) 100978
systems can better meet user needs, promote healthier eating habits, and usability. Despite the current dataset’s limitations, this study success
ultimately improve health outcomes. fully demonstrates the potential of a real-time nutrition assistant.
Therefore, the proposed research has a huge impact on promoting
5. Future work healthier eating habits, client-centered nutrition plans for achieving
individuals’ weight management, and fitness goals, and increasing
The future of real-time food nutrition classification and recommen awareness of nutritional intake in daily life.
dation systems demonstrates significant potential in advancing person
alized health management and promoting healthier lifestyles. Core Ethics approval
advancements will focus on enhancing accuracy in food recognition and
nutrient analysis, particularly for complex and mixed dishes, through Not applicable.
the application of advanced deep learning algorithms, refined computer
vision techniques, and the development of robust, diverse datasets Funding
(Aslan et al., 2020; Yunus et al., 2019).
Systems will increasingly incorporate personalization, leveraging Not applicable.
individual dietary habits, health conditions, and preferences to deliver
tailored recommendations. Real-time analysis, facilitated by mobile
CRediT authorship contribution statement
applications and edge computing, will enable instantaneous feedback,
fostering seamless user engagement (Shah et al., 2022). Integrating
Asim Moin Saad: Writing – review & editing, Writing – original
multimodal data sources, such as image inputs, speech recognition, and
draft, Resources, Methodology, Data curation, Conceptualization. Md.
wearable sensor outputs, will provide comprehensive dietary assess
Raihanul Haque Rahi: Visualization, Validation, Supervision. Md.
ments, while sophisticated feature extraction techniques will enhance
Manirul Islam: Visualization, Supervision. Gulam Rabbani: Writing –
the precision of food recognition and portion estimation (Rewane &
original draft, Data curation.
Chouragade, 2019).
The integration of IoT (Internet of Things) and wearable devices
equipped with advanced sensors introduces a new dimension of Declaration of competing interest
continuous, real-time monitoring of nutritional intake (Ranganathan
et al., 2024). These devices can analyze the nutritional content of food The authors declare that they have no known competing financial
and beverages, delivering personalized health insights and dietary rec interests or personal relationships that could have appeared to influence
ommendations. This technology promises to enhance user convenience the work reported in this paper.
and engagement by offering seamless integration into daily life. While
most current systems focus on identifying food types and nutritional Data availability
content, they lack the capability to detect allergens in real time. Future
advancements could prioritize detecting allergenic elements in meals, Data will be made available on request.
addressing a critical gap in food safety, and providing a vital feature for
individuals with dietary restrictions or severe allergies. Emerging ap References
plications include mobile health platforms for real-time meal analysis,
dietary management tools for addressing conditions like diabetes and Aslan, S., Ciocca, G., Mazzini, D., & Schettini, R. (2020). Benchmarking algorithms for
obesity, and food safety systems for assessing freshness and quality food localization and semantic segmentation. International Journal of Machine
Learning and Cybernetics, 11(12), 2827–2847. [Link]
(Mitra et al., 2022). Broader implementations in public health cam 01153-z
paigns, epidemiological research, clinical settings, and smart kitchen Bahadur, S., Kushwaha, R., Sharma, S., & Anuradha, S. G. (2022). Using deep learning
ecosystems underscore the transformative potential of these systems techniques, a framework for estimating the nutritional value of food in real time.
International Journal of Health Sciences, 7635–7648. [Link]
(Bahadur et al., 2022; Lo et al., 2024; Sempionatto et al., 2021).
v6ns2.6877
However, significant challenges remain, including addressing the Elsweiler, D., & Harvey, M. (2015). Towards automatic meal plan recommendations for
diversity of food types, variability in appearances, and the need for balanced nutrition. In RecSys 2015 - Proceedings of the 9th ACM conference on
extensive, high-quality datasets. Emerging technologies such as AI- recommender systems (pp. 313–316). [Link]
Franco, R. Z., Fallaize, R., Lovegrove, J. A., & Hwang, F. (2016). Popular nutrition-
driven models, edge computing, wearable chemical sensors, IoT inte related mobile apps: A feature assessment. JMIR MHealth and UHealth, 4(3). https://
gration, and cloud-based infrastructures will be pivotal in overcoming [Link]/10.2196/mhealth.5846
these barriers. Freitas, C. N. C., Cordeiro, F. R., & Macario, V. (2020). MyFood: A food segmentation and
classification system to aid nutritional monitoring. [Link]
zenodo.4041488
6. Conclusion Gewehr, M. C. F., Silverio, R., Rosa-Neto, J. C., Lira, F. S., Reckziegel, P., & Ferro, E. S.
(2020). Peptides from natural or rationally designed sources can be used in
overweight, obesity, and type 2 diabetes therapies. Molecules, 25(5). [Link]
Different types of intelligent assistant systems are seamlessly inte 10.3390/molecules25051093
grating into the intricate tasks of our daily lives. In modern days, it is Homenda, W., Luckner, M., & Pedrycz, W. (2016). Classification with rejection: Concepts
necessary to think of a nutrition assistant for personalized dietary and evaluations. Advances in Intelligent Systems and Computing, 364. [Link]
10.1007/978-3-319-19090-7_31
guidance that will keep track of our calorie intake and help us develop a Hovale, S. V., & G, P. (2016). Survey paper on recommendation system using data mining
healthy eating habit. Diet Engine significantly contributes to public techniques. International Journal of Engineering And Computer Science. [Link]
health by offering instant food recognition, calorie estimation, and org/10.18535/ijecs/v5i5.60
Jia, Y., Thong, J., Nguyen, T., Zhang, Q., Karunanithi, M., & Yu, L. (2017). Predicting food
personalized dietary recommendations, effectively bridging the gap nutrition facts using pocket-size near-infrared sensor. doi. 10.0/Linux-x86_64.
between scientific dietary knowledge and practical application. With Ketkar, N., & Moolayil, J. (2021). Deep Learning with Python. In Deep learning with
high accuracy and robust occlusion handling, Diet Engine ensures pre python. [Link]
Kumar, R. D., Julie, E. G., Robinson, Y. H., Vimal, S., & Seo, S. (2021). Recognition of
cise recognition even for partially visible food items, eliminating
food type and calorie estimation using neural network. Journal of Supercomputing, 77
guesswork in food tracking. Its real-time, data-driven recommendations (8), 8172–8193. [Link]
empower users to make informed food choices, positioning it as a crucial Lin, E. Y., Yang, D. L., & Hung, M. C. (2012). System design of an intelligent nutrition
tool in tackling global health challenges like obesity. consultation and recommendation model. In Proceedings - IEEE 9th international
conference on ubiquitous intelligence and computing and IEEE 9th international
Future work should focus on expanding the food database with more conference on autonomic and trusted computing (pp. 740–745). UIC-ATC 2012. https://
diverse and detailed nutritional data to enhance system accuracy and [Link]/10.1109/UIC-ATC.2012.80.
9
A.M. Saad et al. Food Chemistry Advances 7 (2025) 100978
Liu, C., Cao, Y., Luo, Y., Chen, G., Vokkarane, V., Yunsheng, M., Chen, S., & Hou, P. artificial intelligence-based functionalities, features and quality of current apps. In ,
(2018). A new deep learning-based food recognition system for dietary assessment 15. Intelligent systems with applications. [Link]
on an edge computing service infrastructure. IEEE Transactions on Services Computing, Sempionatto, J. R., Montiel, V. R. V., Vargas, E., Teymourian, H., & Wang, J. (2021).
11(2), 249–261. [Link] Wearable and mobile sensors for personalized nutrition. ACS Sensors, 6(5),
Lo, F. P. W., Qiu, J., Wang, Z., Chen, J., Xiao, B., Yuan, W., Giannarou, S., Frost, G., & 1745–1760. [Link]
Lo, B. (2024). Dietary assessment with multimodal ChatGPT: A systematic analysis. Sen, S., Chakraborty, R., & De, B. (2016). Diabesity”: Current situation. Diabetes mellitus
IEEE Journal of Biomedical and Health Informatics. [Link] in 21st century (pp. 45–53). Springer Singapore. [Link]
JBHI.2024.3417280 10-1542-7_6
Lo, F. P. W., Sun, Y., Qiu, J., & Lo, B. (2020). Image-based food classification and volume Shah, M., Degadwala, S., & Vyas, D. (2022). Diet recommendation system based on
estimation for dietary assessment: A review. IEEE Journal of Biomedical and Health different machine learners: A review. In Proceedings of the 2nd international conference
Informatics, 24(7), 1926–1939. [Link] on artificial intelligence and smart energy, ICAIS 2022 (pp. 290–295). [Link]
Maruyama, T., Kawano, Y., & Yanai, K. (2012). Real-time mobile recipe recommendation 10.1109/ICAIS53314.2022.9742919
system using food ingredient recognition. In IMMPD 2012 - Proceedings of the 2012 Shen, Z., Shehzad, A., Chen, S., Sun, H., & Liu, J. (2020). Machine learning based
ACM workshop on interactive multimedia on mobile and portable devices, co-located with approach on food recognition and nutrition estimation. Procedia Computer Science,
ACM multimedia 2012. [Link] 174, 448–453. [Link]
Miao, J., & Zhu, W. (2022). Precision–recall curve (PRC) classification trees. Evolutionary Smith, K. B., & Smith, M. S. (2016). Obesity statistics. Primary Care - Clinics in Office
Intelligence, 15(3). [Link] Practice, 43(1). [Link]
Mitra, A., Goel, S., Mohanty, S. P., Kougianos, E., & Rachakonda, L. (2022). ILog 2.0: A Sudharson, S., Kokil, P., Annamalai, R., & Sai Manoj, N. V. (2023). Enhanced food
novel method for food nutritional value automatic quantification in Smart classification system using YOLO models for object detection algorithm. In 2023 14th
healthcare. In Proceedings - 2022 IEEE international symposium on smart electronic International conference on computing communication and networking technologies,
systems, ISES 2022 (pp. 683–688). [Link] ICCCNT 2023. [Link]
Mortazavi, B. J., & Gutierrez-Osuna, R. (2023). A review of digital innovations for diet Sudo, K., Murasaki, K., Kinebuchi, T., Kimura, S., & Waki, K. (2020). Machine
monitoring and precision nutrition. Journal of Diabetes Science and Technology, 17(1), learning–Based screening of healthy meals from image analysis: System development
217–223. [Link] and pilot study. JMIR Formative Research, 4(10), Article e18507. [Link]
Myers, A., Johnston, N., Rathod, V., Korattikara, A., Gorban, A., Silberman, N., 10.2196/18507
Guadarrama, S., Papandreou, G., Huang, J., & Murphy, K. (2015). Im2Calories: Wang, H. (2022). Design of a food recommendation system using ADNet algorithm on a
Towards an automated mobile vision food diary. In 2015 IEEE international hybrid data mining process. Journal of Soft Computing Paradigm, 3(4). [Link]
conference on computer vision (ICCV) (pp. 1233–1241). [Link] org/10.36548/jscp.2021.4.003
ICCV.2015.146 Yang, L., Hsieh, C. K., Yang, H., Pollak, J. P., Dell, N., Belongie, S., Cole, C., & Estrin, D.
Pérez Rodrigo, C. (2013). Current mapping of obesity. Nutricion Hospitalaria, 28(5), (2017). Yum-me: A personalized nutrient-based meal recommender system. ACM
21–31. Transactions on Information Systems, 36(1). [Link]
Pouladzadeh, P., Shirmohammadi, S., Bakirov, A., Bulut, A., & Yassine, A. (2015). Cloud- Yu, Q., Anzawa, M., Amano, S., & Aizawa, K. (2019). Personalized food image classifier
based SVM for food categorization. Multimedia Tools and Applications, 74(14), considering time-dependent and item-dependent food distribution. IEICE
5243–5260. [Link] Transactions on Information and Systems, E102D(11), 2120–2126. [Link]
Ranganathan, C. S., Riazulhameed, A. A. M. A., Swathi, G., Pavithra, M. R., Senthil, S., & 10.1587/transinf.2019PCP0005
Srinivasan, C. (2024). Optimal dietary management via Data analytics, ANN based Yunus, R., Arif, O., Afzal, H., Amjad, M. F., Abbas, H., Bokhari, H. N., Haider, S. T.,
health insights, and ecosystem for continuous nutrient analysis. In ICEPE 2024 - 6th Zafar, N., & Nawaz, R. (2019). A framework to estimate the nutritional value of food
international conference on energy, power and environment: towards indigenous energy in real time using deep learning techniques. IEEE Access : Practical Innovations, Open
utilization. [Link] Solutions, 7, 2643–2652. [Link]
Raymaekers, J., Rousseeuw, P. J., & Hubert, M. (2022). Class maps for visualizing Zeevi, D., Korem, T., Zmora, N., Israeli, D., Rothschild, D., Weinberger, A., Ben-
classification results. Technometrics : A Journal of Statistics for the Physical, Chemical, Yacov, O., Lador, D., Avnit-Sagi, T., Lotan-Pompan, M., Suez, J., Mahdi, J. A.,
and Engineering Sciences, (2), 64. [Link] Matot, E., Malka, G., Kosower, N., Rein, M., Zilberman-Schapira, G., Dohnalová, L.,
Rewane, R., & Chouragade, P. M. (2019). Food recognition and health monitoring system Pevsner-Fischer, M., & Segal, E. (2015). Personalized nutrition by prediction of
for recommending daily calorie intake. In Proceedings of 2019 3rd IEEE international glycemic responses. Cell, (5), 163. [Link]
conference on electrical, computer and communication technologies. ICECCT. https:// Zhu, F., Bosch, M., Schap, T., Khanna, N., Ebert, D. S., Boushey, C. J., & Delp, E. J.
[Link]/10.1109/ICECCT.2019.8869088. 2019. (2011). Segmentation assisted food classification for dietary assessment.
Samad, S., Ahmed, F., Naher, S., Kabir, M. A., Das, A., Amin, S., & Islam, S. M. S. (2022). Computational Imaging IX, 7873, Article 78730B. [Link]
Smartphone apps for tracking food consumption and recommendations: Evaluating 12.877036
10