BCA-76T-313

Introduction to Data Science

Comprehensive Study Notes
University of Rajasthan, Jaipur
Mid-Semester Exam Preparation | 6th Semester BCA
UNIT - I: Introduction to Data Science

1. Concept of Data Science

Data Science is an interdisciplinary field that uses scientific methods, processes, algorithms, and systems to
extract knowledge and insights from structured and unstructured data. It combines statistics, computer science,
domain knowledge, and data analysis to make data-driven decisions.

📌 Remember: Data Science = Statistics + Computer Science + Domain Knowledge + Data Analysis

In simple words: Data Science is the art of turning raw data into useful information to help solve real-world
problems.

2. Need for Data Science

Why do we need Data Science? Every day, massive amounts of data are generated. Without tools and
techniques to analyse this data, it is useless. Data Science helps:
• Make better business decisions based on evidence (not just gut feeling)
• Identify patterns and trends in large datasets
• Predict future outcomes (e.g., stock prices, customer behaviour)
• Automate processes and improve efficiency
• Detect fraud and security threats
• Personalise user experiences (like Netflix recommendations)

3. Components of Data Science

Data Science has 5 major components:

Component Description
Statistics & Mathematics Foundation for analysis — probability,
distributions, linear algebra
Machine Learning Algorithms that learn from data to make
predictions
Data Engineering Collection, storage, and management of large
datasets
Domain Knowledge Understanding the industry/context to interpret
results correctly
Data Visualisation Presenting data insights visually using charts,
 graphs, dashboards

4. Big Data
Big Data refers to extremely large datasets that cannot be processed using traditional data processing tools. It is
defined by the 5 V's:
• Volume: Huge amounts of data (e.g., petabytes)
• Velocity: Speed at which data is generated and processed
• Variety: Different types — text, images, videos, logs, etc.
• Veracity: Accuracy and reliability of the data
• Value: Usefulness of the data for decision-making

📌 Remember: Think of Big Data as water from a giant flood — you need special pipes (tools like
Hadoop, Spark) to handle it.

5. Facets of Data (Types of Data)

Data can appear in many different forms:

A. Structured Data
Data that is organised in a fixed format (rows and columns). Easy to search and analyse.
• Example: Excel spreadsheets, relational databases (SQL), bank records
• Tools: MySQL, Oracle, MS Excel

B. Unstructured Data
Data that does not have a predefined format or organisation. Harder to process.
• Example: Emails, social media posts, Word documents, PDFs, images, videos
• Tools: Hadoop, NoSQL databases (MongoDB)

C. Machine-Generated Data
Data automatically produced by machines/computers without human involvement.
• Example: Web server logs, sensor readings (IoT), GPS tracking data, satellite data

D. Graph-Based / Network Data

Data that represents relationships and connections between entities.
• Example: Social networks (Facebook friends), road maps, protein interaction networks
• Represented using nodes (entities) and edges (relationships)
E. Audio, Image, and Video Data
Multimedia data captured in digital form.
• Audio: Music files, voice recordings, call centre data
• Image: Photos, X-rays, satellite imagery, scanned documents
• Video: CCTV footage, YouTube videos, medical scan recordings
• Requires special techniques like Computer Vision and Natural Language Processing (NLP)

F. Streaming Data
Continuously generated data in real-time from various sources.
• Example: Live stock market prices, Twitter feeds, live sensor data, online transactions
• Cannot be stored first — must be processed immediately
• Tools: Apache Kafka, Apache Spark Streaming

📌 Remember: Key Difference: Structured data = neatly organised table. Unstructured data = messy
pile of documents. Streaming data = water flowing in a pipe — process it as it comes!

6. The Need for Business Analytics

Business Analytics is the use of data, statistical analysis, and machine learning to make business decisions.

Why is it needed?
• Helps organisations understand past performance
• Identifies what is working and what is not
• Predicts future trends and customer behaviour
• Reduces costs and increases revenue
• Gives competitive advantage in the market

Types of Business Analytics:

• Descriptive Analytics: What happened? (Sales reports, dashboards)
• Diagnostic Analytics: Why did it happen? (Root cause analysis)
• Predictive Analytics: What will happen? (Forecasting using ML)
• Prescriptive Analytics: What should we do? (Optimisation, recommendations)

7. Data Science Life Cycle

The Data Science Life Cycle is a step-by-step process to solve a data problem. It has 6 stages:

Stage What Happens Here

1. Business Understanding Define the problem. What question are we trying
 to answer?
2. Data Collection Gather relevant data from databases, APIs, web
scraping, surveys
3. Data Preparation Clean, format, and organise the data (remove
errors, fill missing values)
4. Exploratory Data Analysis (EDA) Explore data using statistics and visualisation to
find patterns
5. Modelling Build and train ML/statistical models to make
predictions
6. Deployment & Communication Deploy the model and present findings to
stakeholders

📌 Remember: Remember the cycle: Understand → Collect → Prepare → Explore → Model → Deploy.
It is often repeated in loops!

8. Applications of Data Science

Data Science is used across almost every industry:
• Healthcare: Disease prediction, drug discovery, patient diagnosis (e.g., cancer detection using AI)
• Finance: Fraud detection, credit scoring, stock market prediction
• E-Commerce: Product recommendations (Amazon, Flipkart), customer segmentation
• Social Media: Content personalisation, sentiment analysis, ad targeting
• Transport: Route optimisation, self-driving cars, traffic prediction
• Agriculture: Crop yield prediction, weather analysis, soil health monitoring
• Education: Personalised learning, dropout prediction, student performance analysis
• Government: Crime prediction, public policy analysis, census data analysis
UNIT - II: Data Science Process & Data Analytics

1. Overview of Data Science Process

The Data Science Process is a structured approach to solving problems using data. It is also known as the CRISP-
DM (Cross-Industry Standard Process for Data Mining) model. The key steps are:

Step 1: Setting the Research Goal

This is the most important first step. You must clearly define:
• What problem you are trying to solve
• What the expected outcome is
• What data you need
• How you will measure success
Example: Goal = 'Predict which customers will stop using our service (churn) in the next 3 months'

Step 2: Retrieving Data

Collect data from relevant sources:
• Internal databases (company records, transaction logs)
• External sources (web scraping, public datasets, APIs)
• Surveys and forms
• IoT sensors, logs, social media feeds
Key consideration: Data should be relevant, sufficient, and trustworthy.

Step 3: Cleansing Data

Raw data is always messy! Data cleansing (also called data cleaning or data wrangling) involves:
• Removing duplicate records
• Handling missing values (fill with mean/median, or remove)
• Fixing incorrect data (wrong dates, typos)
• Removing outliers (extreme values that skew results)
• Standardising formats (e.g., date formats, capitalisation)

📌 Remember: Garbage In = Garbage Out. If the input data is bad, no matter how good your model is,
the output will be wrong!

Step 4: Integrating and Transforming Data

After cleaning, data from different sources is merged and transformed:
• Data Integration: Combining data from multiple sources into one unified dataset
 • Data Transformation: Converting data into a format suitable for analysis
• Normalisation: Scaling numerical data to a common range (e.g., 0 to 1)
• Encoding: Converting text categories into numbers (e.g., Male=0, Female=1)
• Feature Engineering: Creating new meaningful variables from existing ones

Step 5: Exploratory Data Analysis (EDA)

EDA is about understanding your data before building any model. It involves:
• Summary statistics (mean, median, mode, standard deviation)
• Data distribution analysis (histograms, box plots)
• Finding correlations between variables (correlation matrix, scatter plots)
• Identifying patterns, trends, and anomalies
• Visualisation using tools like Matplotlib, Seaborn (Python)

📌 Remember: EDA is like a doctor examining a patient before prescribing medicine — you must
understand the data before applying any model.

Step 6: Data Modelling

Choosing and building a mathematical/statistical model on the cleaned data:
• Select the right algorithm (regression, classification, clustering)
• Split data into Training set (to train model) and Test set (to evaluate)
• Train the model on training data
• Evaluate using metrics like accuracy, precision, recall, RMSE
• Tune the model (improve performance)

Step 7: Presentation and Automation

Final step — communicate findings to non-technical stakeholders:
• Create dashboards and visualisations (Power BI, Tableau)
• Write clear reports with actionable insights
• Automate the pipeline so it runs regularly without manual effort
• Deploy the model in production systems (web apps, APIs)

2. Data Analytics

Types of Analytics
There are 4 main types of Data Analytics:

Type of Analytics Purpose & Example

 Descriptive Analytics Describes what has already happened. Example:
Monthly sales report, website traffic summary
Diagnostic Analytics Explains WHY something happened. Example:
Why did sales drop in March?
Predictive Analytics Predicts what WILL happen. Example: Predicting
next month's demand using ML
Prescriptive Analytics Recommends what TO DO. Example: Optimise
delivery routes to reduce cost

📌 Remember: Easy way to remember: Descriptive = Rear-view mirror | Diagnostic = Mechanic

inspecting the car | Predictive = Weather forecast | Prescriptive = GPS navigation (tells you what to
do)

3. Data Analytics Lifecycle

The Data Analytics Lifecycle is a detailed framework with 6 phases:

Phase 1: Discovery
The starting point of any analytics project:
• Understand the business problem and define objectives
• Identify what data is available and what is needed
• Assess resources — team, tools, technology
• Frame hypotheses (educated guesses) to test
• Determine success criteria

Phase 2: Data Preparation

Prepare your data workspace and data:
• Collect raw data from all sources (databases, files, APIs)
• Set up an analytics sandbox (working environment)
• Perform ETLT: Extract, Transform, Load, and Transform again
• Explore raw data to understand its structure
• Clean and pre-process (handle missing values, outliers, duplicates)
• Create derived variables (feature engineering)

Phase 3: Model Planning

Plan which analytical methods and tools to use:
• Explore relationships between variables
• Select variables (features) most relevant to the outcome
• Choose ML algorithms based on the problem type:
 ◦ Classification problem → use Logistic Regression, Decision Tree, Random Forest
◦ Regression problem → use Linear Regression, Ridge, Lasso
◦ Clustering problem → use K-Means, Hierarchical Clustering
• Plan the model workflow from input to output

Phase 4: Model Building

Actually build and train the model:
• Split data: Training set (70-80%) and Test set (20-30%)
• Apply the chosen algorithm to the training data
• Build multiple models and compare them
• Use cross-validation to prevent overfitting
• Fine-tune hyperparameters for better performance

📌 Remember: Overfitting = Model memorises training data but performs badly on new data. Like a
student who memorises answers but cannot apply concepts!

Phase 5: Communicate Results

Present findings to stakeholders clearly:
• Summarise insights in simple, non-technical language
• Use visualisations — charts, graphs, dashboards
• Highlight the business impact of the findings
• Give recommendations based on the analysis
• Document the entire process for reproducibility

Phase 6: Operationalise
Deploy and automate the model in a real-world system:
• Deploy the model into the production environment
• Monitor model performance over time
• Retrain with new data when performance drops
• Automate data pipelines for continuous updates

4. Regression Analysis
Regression is a statistical method to find the relationship between a dependent variable (output) and one or
more independent variables (inputs).

Types of Regression:
• Simple Linear Regression: One input, one output. Equation: Y = mX + c. Example: Predicting house price
based on area.
 • Multiple Linear Regression: Multiple inputs, one output. Example: Predicting salary based on
experience, education, and age.
• Logistic Regression: Used for classification (0 or 1 output). Example: Will a patient get diabetes?
(Yes/No)
• Polynomial Regression: Non-linear relationship between variables.

Key Terms:
• Dependent Variable (Y): The outcome we want to predict
• Independent Variable (X): The input features we use for prediction
• Coefficient (m): How much Y changes when X changes by 1
• Intercept (c): Value of Y when X = 0

5. Classification Techniques
Classification is a supervised ML technique where we predict which category/class an input belongs to.

Common Classification Algorithms:

• Logistic Regression: Binary classification — output is 0 or 1 (Yes/No, Spam/Not Spam)
• Decision Tree: Uses a tree structure with if-else conditions. Easy to understand and visualise.
• Random Forest: Combines many decision trees to improve accuracy and reduce overfitting.
• K-Nearest Neighbour (KNN): Classifies based on the majority class of K closest data points.
• Naive Bayes: Based on probability. Fast and works well for text classification.
• Support Vector Machine (SVM): Finds the best boundary (hyperplane) to separate classes.

Evaluation Metrics for Classification:

• Accuracy = Correct predictions / Total predictions
• Precision = True Positives / (True Positives + False Positives)
• Recall = True Positives / (True Positives + False Negatives)
• F1 Score = 2 * (Precision * Recall) / (Precision + Recall)

6. Clustering
Clustering is an unsupervised ML technique that groups similar data points together without using labels.

📌 Remember: Key Difference: Classification uses labelled data (supervised). Clustering finds groups
WITHOUT labels (unsupervised).
Types of Clustering:
• K-Means Clustering: Divides data into K groups. Each point belongs to the cluster with the nearest
centroid (centre). Steps: 1) Choose K, 2) Assign points to nearest centroid, 3) Recalculate centroids, 4)
Repeat until stable.
• Hierarchical Clustering: Creates a tree-like structure (dendrogram) showing how clusters merge. Two
types: Agglomerative (bottom-up) and Divisive (top-down).
• DBSCAN: Density-Based Spatial Clustering. Groups points in dense regions. Good for irregular shapes
and detecting outliers.

Applications of Clustering:
• Customer segmentation (grouping customers by buying behaviour)
• Document grouping (organising news articles by topic)
• Image segmentation
• Anomaly detection (finding unusual data points)

7. Association Rules Analysis

Association Rule Mining finds interesting relationships (associations) between variables in large datasets. It is
mainly used in Market Basket Analysis.

Classic Example: If a customer buys bread and butter, they are likely to also buy milk. This is an association rule:
{Bread, Butter} → {Milk}

Key Terms:
• Support: How frequently the itemset appears in the dataset. Support({A, B}) = Transactions with A & B /
Total transactions
• Confidence: How often the rule is correct. Confidence(A → B) = Support(A & B) / Support(A)
• Lift: Measures the strength of the rule. Lift > 1 means items are positively associated.

Popular Algorithm: Apriori Algorithm

• Finds all frequent itemsets above a minimum support threshold
• Generates association rules with minimum confidence
• Used in retail (Amazon, Flipkart), recommendation systems

Applications:
• Retail: Product placement and promotions (Walmart uses this!)
• Medical: Finding which symptoms co-occur with diseases
• Web usage mining: What pages users visit together
QUICK REVISION SUMMARY

Unit I - Key Points

• Data Science = Statistics + CS + Domain Knowledge
• Big Data: 5 V's = Volume, Velocity, Variety, Veracity, Value
• 6 Types of Data: Structured, Unstructured, Machine-Generated, Graph-Based, Multimedia, Streaming
• 4 Types of Business Analytics: Descriptive, Diagnostic, Predictive, Prescriptive
• Data Science Life Cycle: Understand → Collect → Prepare → EDA → Model → Deploy

Unit II - Key Points

• Data Science Process: Research Goal → Retrieve → Cleanse → Integrate → EDA → Model → Present
• Analytics Lifecycle: Discovery → Data Preparation → Model Planning → Model Building → Communicate
→ Operationalise
• Regression: Linear (continuous output), Logistic (binary output)
• Classification (supervised): Decision Tree, Random Forest, SVM, KNN
• Clustering (unsupervised): K-Means, Hierarchical, DBSCAN
• Association Rules: Support, Confidence, Lift — used in market basket analysis

📌 Remember: Exam Tip: Most questions ask you to DEFINE + EXPLAIN + GIVE EXAMPLE. Always
include a real-world example in your answer for extra marks!

Likely Exam Questions

Short Answer (2-5 marks):

• Define Data Science. What are its components?
• What is Big Data? Explain its 5 V's.
• Differentiate between Structured and Unstructured data.
• What is streaming data? Give two examples.
• Explain the need for Business Analytics.
• What is EDA? Why is it important?
• Define Support, Confidence, and Lift in Association Rules.
• What is clustering? How is it different from classification?

Long Answer (10-15 marks):

• Explain the Data Science Life Cycle with a diagram.
• Describe the Data Analytics Lifecycle in detail.
• Explain the different facets of data with examples.
• What are the types of Business Analytics? Explain with examples.
• Explain the steps in the Data Science Process.
• Write a detailed note on Regression Analysis and its types.
• Explain K-Means Clustering with an example.
• What is Association Rule Mining? Explain Apriori Algorithm.

All the Best for Your Exams, Tanvi! ✨