Sant Dnyaneshwar Shikshan Sanstha

ANNASAHEB DANGE COLLEGE ENGINEERING AND TECHNOLOGY,ASHTA

( AN AUTONOMUS INSTITUTE )

MICRO PROJECT

Project Name: Hospital Data Analytics

Big Data Analytics

Under the guidance of: -

Mrs. Snehal V. Zargad

Submitted By: -

SR NO NAME OF THE STUDENT URN NO ROLL NO

1 Sanika Udaysinh Mohite 23092003 4039

1
 Index

Sr. No. Contents Page No.

1 Abstract 3

5 Introduction 4

6 methodology 5-6

7 Results 7

9 Conclusion 8

10 Future Scope 9

2
ABSTARCT

Hospitals generate massive amounts of data every day from patient admissions, diagnostics,
treatments, and billing activities. Managing and analyzing this large-scale data with traditional
systems becomes challenging due to issues of scalability, speed, and integration. This project
proposes a hospital data analytics solution built on Apache Hive, a powerful data warehousing
tool on the Hadoop ecosystem. Hive enables efficient storage, querying, and processing of
structured hospital data while supporting SQL-like operations, making it suitable for healthcare
analytics tasks such as patient trend analysis, doctor performance evaluation, and disease
pattern identification.

The project involves collecting hospital datasets, preprocessing them, storing the cleaned data
in Hive tables, and performing analytical operations to extract insights that improve operational
efficiency and clinical decision-making. By leveraging distributed processing, Hive handles
large datasets with faster execution times and improved reliability. The results demonstrate
how Hive-based analytics can support hospitals in resource planning, enhancing service
quality, and enabling data-driven strategies for improved patient care and administrative
management.

3
INTRODUCTION

Hospitals operate in a complex environment where vast amounts of data are generated every
minute through patient registrations, consultations, laboratory tests, radiology reports,
prescriptions, surgeries, and billing transactions. Traditionally, this information is stored in
separate systems, making it difficult to integrate and analyze effectively. As healthcare
institutions grow in size and service range, the need for centralized and scalable data
management becomes essential. This challenge has led to the adoption of big data technologies
capable of handling large datasets efficiently while supporting advanced analytics.

Apache Hive, built on the Hadoop ecosystem, offers a robust framework for managing and
querying structured healthcare data at scale. Its SQL-like HiveQL makes data analytics more
accessible to hospital administrators and analysts who are familiar with relational database
concepts. Hive transforms large datasets into meaningful insights by enabling complex queries,
aggregations, and trend analyses. With the ability to process data stored in distributed systems,
Hive ensures high performance even when dealing with millions of records related to patient
demographics, diagnoses, treatments, or financial operations.

Implementing Hive in a hospital data analytics system enables more informed decision-
making, improved operational efficiency, and enhanced patient care. Hospitals can analyze
patient flow patterns, optimize resource allocation, identify seasonal disease outbreaks,
evaluate doctor performance, and monitor revenue trends. By integrating hospital data into
Hive, healthcare organizations gain a unified platform for comprehensive analysis, ultimately
helping them transition from reactive operations to data-driven and proactive healthcare
management.

4
METHODOLOGY

1. Data Collection
o Hospital datasets are collected from various sources such as patient records,
admission logs, doctor details, diagnostic reports, prescriptions, billing
information, and discharge summaries.
o Data is gathered in formats like CSV, Excel, JSON, or SQL exports.
2. Data Preprocessing
o Raw data is cleaned by removing duplicates, handling missing values,
correcting inconsistent entries, and standardizing data formats.
o Data is validated to ensure accuracy and completeness before loading into the
Hadoop ecosystem.
o The cleaned data is converted into structured formats like CSV or Parquet for
efficient storage.
3. Loading Data into HDFS
o Preprocessed datasets are uploaded into Hadoop Distributed File System
(HDFS).
o Directory structures are created for patient data, clinical data, transaction/billing
data, and administrative data.
4. Hive Table Creation
o Internal or external Hive tables are created based on the storage strategy.
o Tables include:
▪ Patient Table
▪ Doctor Table
▪ Admission Table
▪ Diagnosis Table
▪ Lab Reports Table
▪ Billing Table
o Data is mapped to the appropriate Hive schema using HiveQL.
5. Data Loading into Hive
o Using LOAD DATA or external table references, the HDFS data is loaded into
Hive tables.
o Partitioning and bucketing techniques are applied for faster query performance.

5
6. Data Analysis Using HiveQL
o SQL-like queries are executed to extract insights such as:
▪ Patient admission trends
▪ Disease frequency
▪ Department-wise revenue
▪ Bed occupancy rates
▪ Doctor workload analysis
o Aggregations, joins, filters, and group-by operations are used to generate
meaningful analytics.
7. Visualization & Reporting
o Hive is connected with BI tools like Tableau, Power BI, or Apache Superset for
graphical dashboards.
o Reports are generated for administrators to support decision-making.

6
RESULTS

• Step1 : Database Creation

• Step 2: Create External Table and load CSV File

• Step 3: Run Hive Queries (Data Analysis)

7
Count total records:

Find average time spent in hospital by age group:

Check readmission rate by medical specialty:

8
Top 5 diagnoses leading to readmission:

• Step 4: Export Data for Machine Learning

9
Python File:

import pandas as pd
from sklearn.model_selection import train_test_split
from [Link] import RandomForestClassifier
from [Link] import accuracy_score, classification_report

# Load exported Hive data

# Drop or fill missing values

# Encode categorical data

# Train-test split
X = [Link]('readmitted', axis=1)
y = df['readmitted']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42)

# Model
model = RandomForestClassifier(random_state=42)
[Link](X_train, y_train)
y_pred = [Link](X_test)

print("Accuracy:", accuracy_score(y_test, y_pred))

Output:

10
FUTURE SCOPE

• Real-Time Data Processing

Integrate Apache Spark, Kafka, or Flink for real-time streaming of patient vitals,
emergency updates, and live bed availability.

• Predictive Analytics & Machine Learning

Build ML models for predicting disease outbreaks, patient admission rates, readmission
risks, and treatment effectiveness.

• IoT and Wearable Device Integration

Collect patient health data from IoT sensors and smart devices for continuous monitoring
and advanced analytics.

• Advanced Visualization Dashboards

Connect Hive with Power BI, Tableau, or Apache Superset to create interactive dashboards
for doctors and administrators.

11
• Cloud-Based Deployment

Deploy the system on cloud platforms like AWS EMR, Google Dataproc, or Azure
HDInsight for better scalability and reduced infrastructure costs.

CONCLUSION
The Hospital Data Analytics Project using Apache Hive demonstrates how big data
technologies can transform traditional healthcare data management. By storing and processing
large volumes of structured hospital data in a distributed environment, Hive ensures faster
querying, improved scalability, and greater accuracy in analytical results. The project
successfully highlights how hospital datasets can be integrated into a unified system, enabling
comprehensive analysis of patient trends, doctor performance, disease patterns, and financial
activities.

Overall, the project shows that Hive is an effective solution for hospitals seeking data-driven
operational improvements. It provides a foundation for making informed decisions, optimizing
resource allocation, enhancing service quality, and improving patient outcomes. With Hive,
healthcare institutions can transition from fragmented data handling to a centralized analytics-
driven approach, supporting long-term growth and better management practices.

12