Medicare AI

Synopsis
On
“MEDICARE AI – AN INTELLIGENT MEDICAL
ASSISTANCE SYSTEM
”
1. Abstract
MediCare AI is a comprehensive, unified Artificial Intelligence-powered medical assistance
application designed to bridge the gap between complex medical diagnostics and accessible
patient care. In the current healthcare landscape, patients often struggle with interpreting medical
symptoms, accessing early diagnostics for serious conditions like Pneumonia, and maintaining
accurate dietary habits. MediCare AI addresses these challenges by integrating three distinct but
complementary sub-projects into a single mobile platform. The system encompasses Chest X-
Ray Disease Detection utilizing Deep Learning (CNNs), an Intelligent Symptom Checker &
RAG Chatbot using Natural Language Processing (BioBERT), and AI-Based Food Calorie
Estimation using computer vision (segmentation). Built with a scalable architecture featuring a
Flutter frontend and a Python FastAPI backend, the system ensures secure user authentication
and downloadable medical reporting. This project demonstrates the potential of AI to serve as a
reliable, preliminary decision-support tool.

S . V. I . T. S o l a p u r P a g e 1 | 12
 Medicare AI

2. Introduction
The healthcare industry faces significant challenges regarding accessibility, early detection
accuracy, and patient health literacy. Rapid advancements in smartphone technology and
Artificial Intelligence offer a unique opportunity to democratize healthcare access. MediCare AI
is designed to leverage these technologies to create an intelligent medical assistant that resides in
the user's pocket. By combining Computer Vision for image analysis and Natural Language
Processing for symptom understanding, the system aims to provide immediate, reliable, and
educational medical insights.

2.1 Problem Statement

Conventional medical diagnosis is often manual, time-consuming, and expensive.
 Late Diagnosis: Patients in remote areas often lack access to radiologists for timely
Chest X-ray interpretation.
 Misinformation: Users searching for symptoms online often encounter misleading
information or "hallucinations" from generic AI chatbots.
 Dietary Inaccuracy: Existing calorie tracking apps rely on manual input or simple
classification, failing to account for food portion size and volume, leading to inaccurate
nutritional tracking. Therefore, there is a need for a unified, intelligent system that can
accurately identify diseases from images and text while assisting users in managing their
diet effectively.

S . V. I . T. S o l a p u r P a g e 2 | 12
 Medicare AI

2.2 Objectives
 To assist users in early medical awareness using AI.
 To analyze Chest X-ray images for pneumonia detection with high accuracy.
 To provide visual explanations using Grad-CAM heatmaps to build user trust.
 To predict possible diseases from natural language symptoms using NLP models.
 To estimate food calories from images using segmentation for diet awareness.
 To generate downloadable medical reports (PDF) for record-keeping.
 To provide secure user authentication and profile management.

3. Literature Survey:
The development of MediCare AI is grounded in extensive research in the fields of Medical
Imaging and Natural Language Processing.
 Chest X-Ray Analysis: Research by Rajpurkar et al. (CheXNet) demonstrated that
Convolutional Neural Networks (CNNs) could exceed radiologist performance in
detecting pneumonia. Our project builds on this by incorporating Grad-CAM (Selvaraju
et al.) to make these black-box models interpretable.
 Symptom Checking: Traditional symptom checkers use rule-based systems. Recent
advancements, such as BioBERT (Lee et al.), utilize bidirectional transformers pre-
trained on biomedical text, allowing for nuanced understanding of clinical symptoms,
which we apply in our chatbot.
 Calorie Estimation: Standard food recognition uses datasets like Food-101. However,
recent papers on "Deep Learning for Dietary Assessment" highlight the necessity of
Image Segmentation (using architectures like U-Net or SAM) to estimate volume rather
than just class, a methodology we have adopted.

S . V. I . T. S o l a p u r P a g e 3 | 12
 Medicare AI

4. Existing System
In the existing ecosystem, healthcare applications are often fragmented and specialized:
 Manual Diagnosis: Patients rely entirely on physical doctor visits, which can be delayed
due to appointment unavailability.
 Isolated Apps: Users must download one app for diet tracking (e.g., MyFitnessPal),
another for symptom checking (e.g., WebMD), and a completely different system for
storing medical reports.
 Generic Chatbots: General-purpose AI (like standard ChatGPT) is not grounded in
medical literature and can provide unsafe advice ("hallucinations").
 Lack of Explainability: Most AI diagnostic tools give a simple "Yes/No" result without
explaining why, leading to a lack of trust among users and doctors.

5. Proposed System
The proposed "MediCare AI" is a unified Android/iOS-based mobile application that integrates
multiple diagnostic tools into a single cohesive interface.
 Unified Platform: Users can upload X-rays, chat about symptoms, and track calories in
one app.
 Explainable AI: The system uses Grad-CAM to visually highlight infected lung regions,
bridging the gap between AI and human understanding.
 RAG-Based Chatbot: The chatbot uses Retrieval Augmented Generation to fetch
answers from verified medical documents (WHO/CDC), ensuring safety and reliability.
 Advanced Calorie Tracking: By using segmentation (SAM), the system identifies the
exact shape and boundary of food items for precise calorie estimation.
 Multilingual Support: The chatbot supports English and Hindi, making it accessible to a
broader demographic.

S . V. I . T. S o l a p u r P a g e 4 | 12
 Medicare AI

6. System Architecture
The system architecture consists of a client-server model with the following components:
1. Mobile Frontend (Flutter): Handles user interaction, image capture, and result display.
2. Backend API (FastAPI): Manages logic, user authentication (JWT), and routes requests
to AI models.
3. AI Engine:
o Vision Module: TensorFlow/Keras models for X-ray analysis.
o NLP Module: BioBERT & RAG pipeline for the chatbot.
o Segmentation Module: YOLOv8/SAM for food volume estimation.
4. Database: PostgreSQL for user data and Redis for caching.
Block Diagram & Working Flow:
 Input: User uploads Image (X-ray/Food) or Text (Symptoms).
 Processing: API sends data to the specific Model -> Model predicts -> Result processed.
 Output: Diagnosis, Heatmap, Calorie Count, or Chat Response sent back to App.

Block Diagram :

S . V. I . T. S o l a p u r P a g e 5 | 12
 Medicare AI

Working Flow :

7. Used Methodology
1. The methodology is divided into three parallel workflows:
2. A. Chest X-Ray Pipeline:
3. Preprocessing: X-ray images are resized (224x224) and normalized.
4. Classification: A CNN model extracts features and predicts "Pneumonia" or "Normal".
5. Visualization: Grad-CAM extracts the gradients of the final convolutional layer to
generate a heatmap overlay.
6. B. Symptom Chatbot Pipeline:
7. Vectorization: Medical guidelines are converted into vector embeddings.

S . V. I . T. S o l a p u r P a g e 6 | 12
 Medicare AI

8. Retrieval: User queries trigger a search for the most relevant medical documents.
9. Generation: The LLM generates an answer based only on the retrieved documents.
10. C. Calorie Estimation Pipeline:
11. Segmentation: The Segment Anything Model (SAM) creates a binary mask of the food
item.
12. Volume Estimation: The area of the mask is calculated and mapped to volume using
depth estimation heuristics.
13. Calorie Calculation: Volume × Caloric Density of the food class = Total Calories.
14.

8. Hardware Requirements
 Android Smartphone:
o Minimum 4 GB RAM.
o Camera (12 MP or higher recommended).
o Active Internet Connection.
 Server/Training PC:
o Processor: Intel Core i5/i7 or AMD Ryzen.
o RAM: 16 GB (Minimum).
o GPU: NVIDIA RTX 3050 or higher (for CUDA acceleration).

9.. Software Requirements

 Mobile Development: Android Studio / VS Code, Flutter SDK (Dart).
 Backend: Python 3.10, FastAPI, Uvicorn.
 Machine Learning: TensorFlow, PyTorch, Ultralytics (YOLO), Hugging Face
Transformers.
 Database: SQLAlchemy, Redis.
 Third-Party Services: Twilio (OTP), Gemini API.

S . V. I . T. S o l a p u r P a g e 7 | 12
 Medicare AI

10. Advantages
 Integrated Solution: Combines diagnostics, diet, and education in one app.
 Cost-Effective: Reduces the need for unnecessary preliminary doctor visits.
 High Accuracy: Uses state-of-the-art Deep Learning models (CNN, BERT, SAM).
 User Trust: Provides visual proof (heatmaps) and source-backed answers (RAG).
 Portable: Accessible anywhere via smartphone.

11. Disadvantages
 Hardware Dependency: Requires a smartphone with a decent camera and internet
connectivity.
 AI Limitations: Accuracy depends on the quality of the uploaded image; poor lighting or
blur can affect results.
 Not a Doctor Replacement: The system is for educational/preliminary support only and
cannot perform clinical interventions or prescribe medication.

12. Applications
 Remote Health Monitoring: Useful for patients in rural areas.
 Medical Education: Helps students understand X-ray analysis and symptoms.
 Diet Management: Assists users in tracking nutrition more accurately.
 Telemedicine: Serves as a triage tool before consulting a doctor.

S . V. I . T. S o l a p u r P a g e 8 | 12
 Medicare AI

13. Results
The system successfully integrates all three modules.
 X-Ray: Achieves >90% accuracy in Pneumonia detection on the test dataset, with clear
heatmap generation.
 Chatbot: Provides accurate, non-hallucinated responses citing medical guidelines.
 Diet: Successfully segments food items from backgrounds and provides closer calorie
approximations than standard classification methods.
 Performance: The app performs smoothly on standard Android devices with a response
latency of under 3 seconds.

S . V. I . T. S o l a p u r P a g e 9 | 12
 Medicare AI

14. Future Scope

 Doctor Consultation: Integration with live doctor appointment booking.
 Wearable Integration: Syncing data with smartwatches for real-time vitals.
 More Diseases: expanding the X-ray model to detect Tuberculosis and COVID-19.
 Offline Mode: Optimizing models (TensorFlow Lite) to run locally without internet.

S . V. I . T. S o l a p u r P a g e 10 | 12
 Medicare AI

[Link]

 Rajpurkar, P., et al. "CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays

with Deep Learning." arXiv preprint.
 Selvaraju, R. R., et al. "Grad-CAM: Visual Explanations from Deep Networks via
Gradient-based Localization." ICCV.
 Lee, J., et al. "BioBERT: a pre-trained biomedical language representation model for
biomedical text mining." Bioinformatics.
 Bossard, L., et al. "Food-101 – Mining Discriminative Components with Random
Forests." ECCV.

S . V. I . T. S o l a p u r P a g e 11 | 12
 Medicare AI

16. Conclusion
MediCare AI provides an efficient, affordable, and user-friendly solution for modern healthcare
assistance. By integrating machine learning, image processing, and secure backend technologies,
the application empowers users to take control of their health. It bridges the gap between raw
medical data and actionable insights, proving that AI can be a safe and effective partner in
personal healthcare management.

S . V. I . T. S o l a p u r P a g e 12 | 12