UNIT-1

Introduction to Data Science & Linear Algebra for Data Science

1. Introduction to Data Science

Data Science is an interdisciplinary field that uses scientific methods, algorithms, and
systems to extract knowledge and insights from structured and unstructured data.

Key Components:

- Data Collection
- Data Cleaning
- Exploratory Data Analysis (EDA)
- Model Building
- Model Evaluation
- Deployment

Applications include fraud detection, recommendation systems, self-driving cars, and

predictive maintenance.

Example: A data scientist at Amazon analyzes customer purchasing behavior to

recommend products in real time using machine learning algorithms trained on historical
data.

2. Data Science Process

The Data Science Life Cycle typically includes:

- Problem Definition: Understand what needs to be solved.

- Data Collection: Gather data from sources like APIs, sensors, web scraping.
- Data Cleaning: Handle missing values, outliers, and errors.
- Exploratory Data Analysis (EDA): Understand data using statistics and
 visualization.
- Feature Engineering: Derive new variables from raw data.
- Model Building: Apply algorithms like regression, decision trees.
- Model Evaluation: Use metrics like RMSE, precision, recall.
- Deployment: Deploy model using tools like Flask, AWS.

3. Need for Data Science

The Need for Data Science arises from the exponential growth of digital data and the
demand for intelligent, data-driven decisions across industries to improve efficiency,
personalization, automation, and forecasting.

Why It’s Needed

1. Explosion of Data

 Every click, swipe, purchase, and sensor reading generates data.

 Traditional methods can’t handle this volume — data science makes sense of it.

2. Better Decision-Making

 Replaces guesswork with evidence-based strategies.

 Helps businesses choose the right product, price, or customer segment.

3. Automation of Tasks

 Chatbots, fraud detection, and recommendation engines run without human input.

 Saves time and reduces errors.

4. Personalization

 Apps like Spotify or Amazon tailor content based on your behavior.

  Increases user satisfaction and engagement.

5. Prediction and Forecasting

 Weather apps, stock market tools, and healthcare systems use data science to
predict future events.

 Helps in planning and risk management.

Real-Life Examples

Industry Use of Data Science

E-commerce Product recommendations, customer segmentation

Healthcare Disease prediction, treatment personalization

Finance Fraud detection, credit scoring

Agriculture Crop yield forecasting, soil analysis

Education Student performance tracking, adaptive learning

Traditional tools like Excel and SQL fail when dealing with large, complex, or real-time
data. Data Science enables better decision-making, predictive analysis, automation, and
personalization.

Example: Spotify uses data science to generate personalized playlists by analyzing user
listening patterns.

4. Linear Algebra for Data Science

Linear Algebra is the mathematics of vectors and matrices, and its core to almost
everything in data science: machine learning, image processing, NLP, deep learning, etc.

Linear Algebra: Linear algebra is a branch of mathematics that deals with:

 Vectors

 Matrices

 Linear transformations

 Systems of linear equations

In data science, data is often represented in matrix or vector form (think of an Excel
spreadsheet — rows and columns = matrices!).
Linear Algebra in Data Science:

Use Case Linear Algebra Role

Machine Learning Training models using matrix operations

Computer Vision Representing images as matrices

NLP (Natural Language

Word embeddings (vectors), sentence similarity
Processing)

Matrix factorization (like Netflix movie

Recommendation Systems
suggestions)

Dimensionality Reduction PCA using eigenvectors and eigenvalues

Key Concepts with Examples:

A. Vectors

A vector is just an ordered list of numbers (1D array). It can represent features of data.

Example: Student scores=[80,90,85]

Here, each value is a feature (Math, English, Science). This vector represents one student.

Operations:

 Addition: [1,2]+[3,4]=[4,6]

 Scalar multiplication: 2×[1,3]=[2,6]

B. Matrices

A matrix is a 2D array — rows and columns of numbers. In data science, datasets are
typically matrices.

Example: 3 students’ scores across 3 subjects:

Each row = one student
Each column = one subject

Operations:

 Matrix multiplication

 Transpose

 Inverse

C. Linear Equations

A linear equation is a rule that tells you how two or more quantities (variables) are
related, and when you draw it on a graph, it forms a straight line.

Real-Life Example (Simple Analogy)

Imagine you are buying chocolates.

 Each chocolate costs ₹10.

 If you buy x chocolates, the total price (₹) will be:

Total=10x

This is a linear equation because:

 The cost is directly proportional to the number of chocolates.

 It has the form y = m × x, where m is the price per chocolate (10).

A 2-Variable Example

Let’s say you’re buying:

 x apples (₹20 each)

 y mangoes (₹10 each)

If you spend ₹100, the equation is:

20x+10y=100

This equation tells you all combinations of apples and mangoes that cost ₹100.

System of Linear Equations

A system consists of multiple linear equations working together.

Example:

2x+3y=8 x−y=2

Goal: Find values of x and y that satisfy both equations.

Representing in Matrix Form

Any system of linear equations can be written as:

A⋅X=B

For the above example:

Where:

 Matrix A = coefficients

 Vector X = variables

 Vector B = right-hand side

Methods to Solve Linear Equations

1. Substitution Method

From x−y=2

x=y+2
Substitute into the first equation:

2(y+2)+3y=8

⇒2y+4+3y=8

⇒5y=4

⇒y=0.82

Then:
x=0.8+2=2.8
2. Elimination Method

We want to eliminate one of the variables by making their coefficients the same in both
equations. Let's eliminate x by aligning coefficients.

We already have:

 Equation (1): x−y=2

 Equation (2): 2x+3y=8

Multiply Equation (1) by 2 so that the coefficient of xxx becomes 2 in both equations:

Multiply (1) by 2: 2x−2y=4

Now, our two equations are:

 (2x−2y=4) — (Modified Equation 1)

 (2x+3y=8) — (Original Equation 2)

Subtract the equations

Subtract the first from the second:

(2x+3y)−(2x−2y)=8−4

Simplify left-hand side and right-hand side:

2x+3y−2x+2y=4

⇒5y=4

⇒y=0.8

Substitute back to find x

Use Equation (1):

x−y=2x

Substitute y=0.8

x−0.8=2

⇒x=2.8

x=2.8, y=0.8x
3. Matrix Method (Using Inverse)

Solve the system of equations:

1) 2x + 3y = 8

2) x - y = 2

Matrix Representation:

We represent the system as: A * X = B

Where:

A = [[2, 3], [1, -1]] (Coefficient Matrix)

X = [x, y]^T (Unknown Variables)

B = [8, 2] (Output Vector)

Matrix Equation:

To find X, we use the formula: X = A⁻¹ * B

Python Code:

import numpy as np

A = [Link]([[2, 3], [1, -1]])

B = [Link]([8, 2])

X = [Link](A, B)

print(X)

# Output: [2.8 ,0.8]

Explanation:

We are using NumPy's '[Link]()' function which efficiently solves the linear system.

This method is preferred over directly calculating A⁻¹ because it's numerically stable.

The output means: x = 2.8 and y = 0.8

Linear equations model relationships between variables.

Example:
2x+3y=8 and x−y=2

In matrix form:

We can solve for x using:

 Gaussian elimination

 Matrix inverse

 Python: [Link](A, b)

Use of Linear Equation in Data Science — Simple Example

Problem:

A company wants to predict the salary of a person based on their years of experience.

They observe:

 For 1 year → ₹35,000

 For 2 years → ₹40,000

 For 3 years → ₹45,000

...and so on.

Linear Equation Form:

Salary=5000⋅Years+30000

This is a linear equation of the form:

y=mx+c

Where:

 y = salary (output)

 x = years of experience (input)

 m=5000 = increase per year

 c=30000 = base salary

Example Prediction:

Question: What will be the salary for 4 years of experience?

Salary=5000⋅4+30000=20000+30000=₹50,000

Answer: ₹50,000

6. Distance (Euclidean Distance)

Euclidean distance is the most common way to calculate the distance between two points
in Euclidean space. It is derived from the Pythagorean theorem and is used to measure the
straight-line distance between two vectors.

Formula (2D):
Distance = sqrt((x2 - x1)^2 + (y2 - y1)^2)

Example:
Points A = (2, 3), B = (5, 7)
Distance = sqrt((5-2)^2 + (7-3)^2) = sqrt(9 + 16) = 5

Applications in Data Science:

1. K-Nearest Neighbors (K-NN) Algorithm

Purpose: Used for classification or regression.

Example:
You want to predict whether a patient has diabetes based on attributes like age, BMI, and
glucose level.
Suppose a new patient has:
- Age = 45, BMI = 28, Glucose = 140
You compare this with known patients using Euclidean distance:
Distance = sqrt((Age1 − 45)^2 + (BMI1 − 28)^2 + (Glucose1 − 140)^2)
The K-NN algorithm finds the K closest patients and takes a majority vote (for
classification) or average value (for regression).

Why Euclidean Distance?

It helps the algorithm measure similarity — closer neighbors are assumed to have similar
outcomes.

2. K-Means Clustering

Purpose: To group similar data points into clusters (unsupervised learning).

Example:
A retail company wants to segment its customers based on:
- Annual income
- Spending score
Each customer is a point in 2D space (e.g., Income = ₹5L, Score = 60).
The algorithm assigns customers to nearest cluster centers using Euclidean distance,
updates centers, and repeats.

Why Euclidean Distance?

It decides which cluster center a point belongs to, based on how close the data point is to
the center.

3. Outlier Detection

Purpose: To identify anomalies or unusual data [Link]:

Detecting fraudulent transactions in banking.
Transactions far from normal (based on amount, time, location) have high Euclidean
distance from mean transaction.
Distance = sqrt((Amount - Mean)^2 + (Time - MeanTime)^2 + ...)

Why Euclidean Distance?

High distance indicates the point is an outlier → potentially suspicious or invalid.

4. Recommender Systems

Purpose: Suggest products, movies, or music to users.

Example:
Netflix recommends movies based on user ratings.
User A = [4, 5, 3, 1], User B = [5, 5, 2, 1]
Euclidean distance = sqrt((4-5)^2 + (5-5)^2 + (3-2)^2 + (1-1)^2) = sqrt(2)
Smaller distance = more similar tastes → Recommend User B's liked movies to User A.

Why Euclidean Distance?

Measures similarity in user preferences.

5. Image Recognition / Face Detection

Purpose: Match new images to known images (like face unlock in phones).

Example:
Each image is represented as a high-dimensional vector of pixel values.
Euclidean distance is used to find the closest stored image to the new one.
Minimum distance → best match.

Why Euclidean Distance?

Helps in matching or recognizing visual similarities between images.
7. Eigenvalues and Eigenvectors
Eigenvalues and eigenvectors are fundamental in understanding matrix transformations.
An eigenvector of a matrix is a vector that only changes in scale when a linear
transformation is applied to it. The scalar by which it is scaled is the eigenvalue.

Matrix Equation:
A*v=λ*v

Where:
- A is a square matrix
- v is the eigenvector
- λ (lambda) is the eigenvalue
- The direction of v remains unchanged after the transformation A is applied

Imagine you're pushing a rubber sheet from all sides (this is your transformation). Most
points on the sheet will move in new directions. But there are some directions (vectors)
where the point will just stretch or shrink, not rotate.
These directions are eigenvectors, and the stretch amount is the eigenvalue.

Let’s say you have a matrix:

A = [[2, 0],
[0, 3]]

Try multiplying it with a vector v = [1, 0]:

A * v = [2*1 + 0*0, 0*1 + 3*0] = [2, 0]

  The direction [1, 0] didn't change, but it got stretched by 2.
 So:
→ [1, 0] is an eigenvector
→ 2 is its eigenvalue

Applications in Data Science:

1. Principal Component Analysis (PCA)

Purpose: Reduce the number of features (dimensionality) while preserving the most
important information in data.

Real-time Example:

You have a dataset with 100 features (like customer age, income, spending score, etc.).

➡PCA uses eigenvectors to identify the most important directions (called principal
components) where the data varies the most.
➡Eigenvalues tell us how much variance each component (direction) explains.

Use Case:
In a marketing campaign, you might reduce 100 features down to 2 or 3 for visualization
or faster machine learning, without losing much information.

2. Face Recognition – Eigenfaces

Purpose: Represent and recognize faces efficiently.

Real-time Example:

When you use Face ID to unlock your phone:

 The system stores your face as a combination of eigenfaces (special facial

patterns).

 These are computed from a large set of faces using eigenvectors of the face
dataset.

 When you present your face, it checks how closely it matches the stored
eigenfaces.

Use Case:
Used in facial recognition systems like Apple Face ID, Facebook photo tagging, or
surveillance systems.
3. Latent Semantic Analysis (LSA) – Text Mining

Purpose: Understand relationships between words and documents.

Real-time Example:

You search on Google for “AI in medicine” but the article has the title "Artificial
Intelligence revolutionizes healthcare".

 Eigenvectors help extract concepts hidden in the text even when the exact
keywords aren't present.

 This is done through matrix decomposition techniques like SVD (Singular Value
Decomposition), which uses eigenvalues/vectors.

Use Case:
Search engines, topic modeling, recommendation systems like YouTube or Netflix.

4. Image Compression

Purpose: Reduce image size without losing key features.

Real-time Example:

Suppose you want to store 1000 face images using less memory:

 Convert the image to a matrix.

 Use eigenvectors to represent the most important patterns (edges, light areas,
shadows).

 Discard less important parts (small eigenvalues).

Use Case:
Used in image storage, transmission (e.g., medical images, CCTV footage), and apps like
Google Photos for optimization.

5. Stability Analysis in Deep Learning

Purpose: Analyze and control how the neural network learns.

Real-time Example:

In training a neural network, sometimes the model becomes unstable (loss goes to NaN or
gradient explodes).
  Eigenvalues of weight matrices or the Hessian matrix can tell you whether the
model is learning efficiently or diverging.

 If eigenvalues are too large or negative, adjustments are made (like learning rate
tuning).

Use Case:
Used in designing optimizers, debugging training issues in TensorFlow or PyTorch.
 UNIT-II

Descriptive Statistics:

Descriptive statistics summarize and organize features of a dataset using numbers, charts, and
graphs.

Examples:

o Mean: Average value

o Median: Middle value
o Mode: Most frequent value
o Range: Difference between max and min
o Standard deviation: Spread of the data

1. Measure of Central Tendency

These values represent the center or typical value in a dataset.

Mean (Average):

Formula: Mean=Sum of all values/Number of values

 Example:
Dataset = [10, 20, 30, 40]
Mean = (10+20+30+40)/4 = 25

Median:

 The middle value when data is ordered.

 If even number of values: take average of two middle values.
 Example:
Dataset = [10, 20, 30, 40, 50]
Median = 30
 Dataset = [10, 20, 30, 40]
Median = (20+30)/2 = 25

Mode:

 The most frequent value in the dataset.

 Example:
Dataset = [10, 20, 20, 30, 40]
Mode = 20

2. Measure of Dispersion (Spread)

These values show how spread out or scattered the data is.

Range:

 Formula:

Range=Maximum−Minimum

 Example:
Dataset = [10, 20, 30, 40]
Range = 40 - 10 = 30

Variance:

 Measures the average squared deviation from the mean.

Standard Deviation:

 The square root of variance.

 Tells us how much the values deviate from the mean.
 Example:
Dataset = [10, 20, 30, 40]
Mean = 25
Deviations = [-15, -5, 5, 15]
Squared = [225, 25, 25, 225]
Variance = 500 / 4 = 125
Std. Deviation = √125 ≈ 11.18

Real-Life Example: Sales Data

Let’s say you’re analyzing daily sales (in ₹) for a store over one week:

[800, 1000, 950, 1100, 1050, 990, 1200]

  Mean = (800 + 1000 + 950 + 1100 + 1050 + 990 + 1200)/7 = 1013.57
 Median = 1000 (after sorting)
 Mode = No mode (no repeated values)
 Range = 1200 - 800 = 400
 Standard Deviation = ≈ 121.4 (calculated using formula)

This tells the store owner:

 Typical sales are ~₹1000

 Sales vary by around ₹120
 ₹800 was the lowest, ₹1200 was the highest

Summary Table

Measure Meaning Use Case

Mean Average of values Average salary, marks, temperature
Median Middle value Income analysis (less affected by outliers)
Mode Most frequent value Product popularity, survey choices
Range Spread from min to max Knowing variation in data
Std Deviation Spread around the mean Consistency in test scores, prices, etc.

Data Preparation:

Data Preparation is the essential step in the data science workflow that comes before
analysis or modeling. Raw data is often incomplete, inconsistent, or messy, and cannot be
used directly for insights or machine learning.

Goal:
Transform raw data into a clean, structured, and machine-readable format.

Steps in Data Preparation (with Examples)

1. Handling Missing Values

Real-world datasets often have missing entries, like blank cells or NaN.

Techniques:

 Remove rows/columns with too many missing values

 Impute missing values with:
o Mean/Median (for numerical values)
o Mode (for categorical values)
o Forward-fill or backward-fill (time series)

Example:

Customer_Age = [25, 28, None, 30, 27]

→ Impute missing age with mean = (25+28+30+27)/4 = 27.5

→ Updated: [25, 28, 27.5, 30, 27]

2. Removing Duplicates

Duplicate entries can skew analysis or lead to data leakage in machine learning models.

Example:

Name Email
John john@[Link]
John john@[Link]

Remove duplicate row → Keep only one.

3. Converting Data Types

Ensure each column has the correct type: integers, floats, strings, dates, etc.

Example:

 "2025-07-24" should be a Date, not a String

 "Age" column with '25', '30' as text → Convert to Integer

This is important for:

 Calculations (age differences, totals)

 Sorting (dates)
  Machine learning models (which need numerical input)

4. Normalization or Standardization

When features (columns) have different scales, we need to scale them so that no single
feature dominates.

Normalization:

Scales values between 0 and 1

x_normalized = (x - min) / (max - min)

Standardization:

Converts values to z-scores (mean = 0, std = 1)

z = (x - mean) / std

Example:

 Income ranges from ₹30,000 to ₹2,00,000

 Age ranges from 18 to 70
→ Income will dominate unless scaled

5. Encoding Categorical Variables

Convert text labels into numerical codes for ML models.

Example:

Gender column: ['Male', 'Female', 'Female', 'Male']

→ Label Encoding:
Male = 1, Female = 0
→ [1, 0, 0, 1]

Real-Time Use Case: Customer Data for ML

Let’s say you're building a churn prediction model for a telecom company. You receive the
following raw data:

Customer ID Age Gender Plan Monthly Charges Last Login

Customer ID Age Gender Plan Monthly Charges Last Login
101 25 Male A 1200 2025-06-01
102 Female A 2025-06-02
101 25 Male A 1200 2025-06-01
103 30 Female B 950 not recorded

After Data Preparation:

 Missing Age → filled using median or mean

 Missing Monthly Charges → filled with average charges for Plan A
 "not recorded" → replaced with NaT (Not a Time) or a default date
 Duplicate customer ID 101 → removed
 "Gender" and "Plan" → encoded to numeric format
 "Monthly Charges" → normalized

Result: Clean, ready-to-use data for training your model to predict churn accurately.

Summary Table

Step Purpose Real-life Example

Replace blank salary with average in
Handle Missing Values Fill or remove blanks
employee data
Avoid redundant
Remove Duplicates Delete repeated user registration entries
information
Enable correct
Convert Data Types Change age from text to integer
calculations

Prepare features for Scale income and expenses for credit

Normalize/Standardize
modeling scoring
Encode Categorical
Convert text to numbers Change "Plan A" to 1, "Plan B" to 2, etc.
Data

Exploratory Data Analysis (EDA):

Exploratory Data Analysis (EDA) is a crucial step in data science and analytics. It's a process
of visually and statistically summarizing the main characteristics of a dataset to uncover
patterns, find anomalies, and guide further analysis.

Why EDA is Important

EDA is foundational to any data-driven project for several reasons:

  Understanding the data: It gives you a first look at the dataset's structure, data types,
and value distribution. This is essential for a complete understanding of the
information you have.
 Pattern recognition: It helps you find hidden patterns and relationships between
different variables that might not be obvious in raw data.
 Outlier detection: You can easily spot errors or unusual data points (outliers) that
could skew your analysis or negatively affect a model's performance.
 Feature engineering and selection: The insights from EDA help you identify the
most important features for a model and guide you on how to transform them for
better performance.
 Informed modeling: By understanding your data's characteristics, you can make
informed decisions about which machine learning models or statistical tests are most
suitable.

Types of EDA

EDA is often categorized by the number of variables being analyzed at once.

1. Univariate Analysis

This focuses on analyzing a single variable to understand its characteristics.

 Goal: Describe the data and find patterns within a single feature.
 Techniques:
o Histograms: Show the distribution of a numerical variable.
o Box Plots: Visualize the spread and detect outliers.
o Bar Charts: Used for categorical data to show frequencies.
o Summary Statistics: Measures like mean, median, mode, and standard
deviation describe central tendency and spread.
2. Bivariate Analysis

This examines the relationship between two variables.

 Goal: Find connections, correlations, and dependencies between two variables.

 Techniques:
o Scatter Plots: Visualize the relationship between two continuous variables.
o Correlation Coefficient: A number that measures the strength and direction
of a relationship (e.g., Pearson's correlation for linear relationships).
o Cross-tabulation (Contingency Tables): Shows the frequency distribution of
two categorical variables.
o Line Graphs: Useful for showing the relationship between a continuous
variable and a time-based variable.

3. Multivariate Analysis

This explores the relationships among three or more variables.

  Goal: Understand how multiple variables interact with each other.
 Techniques:
o Pair Plots: A grid of scatter plots showing the relationships between multiple
variables at once.
o Principal Component Analysis (PCA): A dimensionality reduction
technique that simplifies complex datasets while retaining key information.
o Spatial Analysis: Uses maps to visualize the geographical distribution of data.
o Time Series Analysis: Focuses on patterns and trends in time-based data.

Key Steps for Performing EDA

1. Understand the Problem and the Data: Before you start, you need to have a clear
understanding of the business or research question you are trying to solve. You should
also familiarize yourself with the dataset's variables, data types, and any potential
limitations.
2. Import and Inspect the Data: Load the data into your analysis environment (e.g.,
Python with Pandas). Inspect its size (rows and columns), check for missing values,
and identify data types for each variable.
3. Handle Missing Data: Decide how to manage missing values. You can either remove
the data points or impute (fill in) the values using a suitable method like the mean or
median.
4. Explore Data Characteristics: Calculate summary statistics (mean, median, standard
deviation, etc.) for numerical variables and create frequency tables for categorical
variables. This provides a clear overview of your data's properties.
5. Visualize Data Relationships: Use plots like histograms, box plots, scatter plots, and
correlation matrices to visually explore the data. This is where you'll find most of the
patterns and insights.
 6. Handle Outliers: Identify and manage outliers, which are data points that are
significantly different from the rest. Outliers can be detected using methods like the
Interquartile Range (IQR) or Z-scores. You can then decide whether to remove,
adjust, or keep them, depending on the context.
7. Perform Data Transformation: If necessary, transform your data to prepare it for
modeling. This could involve scaling numerical variables, encoding categorical
variables, or applying mathematical functions to fix skewness.
8. Communicate Findings and Insights: The final step is to summarize and present
your discoveries in a clear and compelling way. Use visualizations to support your
findings and highlight key insights, limitations, and suggestions for the next steps.

Data Summarization:
Data summarization is the process of condensing large and complex datasets into smaller,
more meaningful pieces of information without losing the essence of the data. It’s like
reading the highlights of a long book instead of reading every page.

2. Purpose of Data Summarization

We summarize data to:

• Quickly understand the data’s main characteristics.

• Spot patterns and trends without reading every data point.

• Prepare data for deeper analysis like hypothesis testing or modeling.

• Communicate results to decision-makers in a clear way.

3. Types of Data Summarization

A. Numerical Summarization (Statistical Measures)

Used when dealing with quantitative (numeric) data.

1. Central Tendency → Shows where most values lie:

- Mean (average)

- Median (middle value)

- Mode (most frequent value)

2. Spread / Variability → Shows how spread out values are:

- Range (max – min)

- Variance

- Standard Deviation
3. Shape of Data:

- Skewness (asymmetry)

- Kurtosis (peakedness)

Example:
Dataset: [10, 20, 30, 40, 50]

- Mean = 30

- Median = 30

- Range = 50 – 10 = 40

- Std. Dev. ≈ 15.8

B. Categorical Summarization

Used when data is qualitative (categories, labels).

• Frequency Table: Counts of each category.

• Percentage/Proportion: Share of each category.

C. Graphical Summarization

Visual representation to help interpret quickly:

• Histogram → Distribution of numeric data.

• Bar Chart → Comparison of categories.

• Box Plot → Spread + outliers.

• Pie Chart → Proportions of categories.

• Heatmap → Relationship between multiple variables.

4. Example

Raw Dataset:

CustomerID Age Region Purchase

1 25 East 2000

2 45 West 3000

3 35 East 1500

4 28 North 4000
Summarized Data:

- Numerical Summary: Mean Age = 33.25, Mean Purchase = ₹2,625, Max Purchase = ₹4,000

- Categorical Summary: Region Counts → East: 2, West: 1, North: 1

- Graphical Summary: Bar chart showing purchase per region.

Data Distribution :
Data distribution refers to the way values in a dataset are spread or arranged across possible values.
It describes the frequency or probability of occurrence of each value (or range of values) and is
fundamental in understanding data characteristics.

Importance in Data Science

- Guides Statistical Analysis: Many statistical models assume specific data distributions (e.g., normal
distribution in parametric tests).
- Detects Anomalies: Outliers and unusual patterns can be identified by observing the spread.
- Data Preprocessing: Skewed or non-normal distributions may require transformations.
- Model Selection: Certain machine learning algorithms perform better with specific data
distributions.

Types of Data Distributions

A. Based on Shape

1. Normal (Gaussian) Distribution

- Symmetrical bell-shaped curve.
- Mean = Median = Mode.
- Many natural phenomena follow this distribution (e.g., height, weight).

2. Uniform Distribution
- Equal probability for all values in the range.
- Example: Rolling a fair die.

3. Skewed Distribution
- Positively Skewed (Right Skew): Long tail on the right; mean > median.
- Negatively Skewed (Left Skew): Long tail on the left; mean < median.

4. Bimodal and Multimodal Distributions

- Two or more peaks in the data.
- Example: Test scores of two different student groups.
B. Based on Probability Type

1. Discrete Distributions (data takes specific, separate values)

- Binomial Distribution: Number of successes in a fixed number of trials.
- Poisson Distribution: Number of events occurring in a fixed time/space.

2. Continuous Distributions (data can take any value within a range)

- Normal Distribution: Common in natural and social phenomena.
- Exponential Distribution: Time until an event occurs.

Methods to Represent Data Distribution

- Histogram: Displays frequency counts for grouped intervals (bins).
- Boxplot: Shows median, quartiles, and outliers.
- Density Plot: Smooth curve showing probability density.
- Violin Plot: Combines boxplot with a mirrored density plot.

Example
Exam Scores Data:
- Mean = 72, Median = 74
- Slightly left-skewed (negative skew) → Most students scored high, but a few low scores reduced
the mean.

Summary Table
Distribution Type Shape Example Applications
Normal Symmetrical bell Human height Parametric tests,
curve regression
Uniform Flat, equal Dice rolls Random sampling
probability
Positive Skew Long tail right Income levels Wealth distribution
analysis
Negative Skew Long tail left Age at retirement Demographic studies
Bimodal Two peaks Test scores from two Population
batches segmentation
Poisson Skewed, discrete Number of Event counting
emails/day
Exponential Continuous, skewed Time to service Reliability analysis
completion

Measuring Asymmetry
In data science, asymmetry (or skewness) refers to the degree to which the distribution of
data deviates from perfect symmetry around its central value (mean or median). A symmetric
distribution has equal spread on both sides, while an asymmetric distribution shows more
concentration of values on one side.

Importance in Data Science

Measuring asymmetry is important for:

• Understanding Data Shape – Identifying whether data is symmetric or skewed.
• Selecting Appropriate Models – Many machine learning algorithms assume normally
distributed data.
• Feature Engineering – Skewed variables may require transformation before model training.
• Business Insights – Detecting skew can reveal unusual patterns, such as extreme spending
or unusual customer behavior.

Types of Asymmetry

a) Positive Skew (Right Skewed)

• Tail extends more towards the right side of the distribution.
• Mean > Median > Mode.
• Example: Distribution of income levels in a country.

b) Negative Skew (Left Skewed)

• Tail extends more towards the left side of the distribution.
• Mean < Median < Mode.
• Example: Age at retirement for a specific population.

Measures of Asymmetry

Moment Coefficient of Skewness

Formula:
Skewness = [ Σ(xi - x̄)³ ] / [ n * s³ ]
Where:
• xi = individual data values
• x̄ = mean of data
• s = standard deviation
• n = number of observations

Pearson’s Coefficients of Skewness

1. First Coefficient: ( x̄ – Mode ) / s

2. Second Coefficient: 3( x̄ – Median ) / s

Bowley’s (Quartile) Coefficient of Skewness

Formula:
Skewness = ( Q3 + Q1 – 2Q2 ) / ( Q3 – Q1 )
Where:
• Q1 = First Quartile
• Q2 = Median
• Q3 = Third Quartile

Interpretation of Skewness Values

Skewness Value Interpretation

0 Perfectly symmetric

0 to 0.5 or -0.5 to 0 Approximately symmetric

0.5 to 1 Moderately positively skewed

-1 to -0.5 Moderately negatively skewed

>1 Highly positively skewed

< -1 Highly negatively skewed

Detecting Asymmetry

• Histogram – Shows shape and tail direction.

• Boxplot – Longer whisker indicates skew direction.
• Density Plot – Reveals deviation from symmetry.

Handling Skewness in Data Science

• Log Transformation – Commonly used for right-skewed data.

• Square Root Transformation – Useful for moderate skew.
• Box-Cox Transformation – General transformation method.
• Use of Robust Models – Decision Trees and Random Forests handle skew naturally.

Example

Dataset: Exam scores = {45, 50, 52, 53, 55, 60, 95}
• Mean = 58.57
• Median = 53
• Skewness (calculated) ≈ 1.40 → Positive Skew.

Sample Mean and Estimated Mean

Sample Mean

The sample mean is the arithmetic average of values from a sample, not the entire population.

Formula:
x̄ = Σ(xi) / n
Where:
• x̄ = sample mean
• xi = each value in the sample
• n = number of observations in the sample

Properties

• Serves as an unbiased estimator of the population mean (μ).

• Sensitive to extreme values (outliers).
• Simple to compute and widely used.

Example

Sample data: {10, 12, 15, 18, 20}

x̄ = (10 + 12 + 15 + 18 + 20) / 5 = 75 / 5 = 15
The sample mean is 15.

Estimated Mean

The estimated mean refers to the value obtained by using the sample mean to approximate the
unknown population mean (μ).

Since we cannot compute the exact population mean without having all data points, we
estimate it using the sample mean:
μ̂ ≈ x̄
Where:
• μ̂ = estimated population mean
• x̄ = sample mean

Relationship

• Sample mean is a statistic (calculated from data).

• Estimated mean is an estimation of a parameter (population mean) based on that statistic.

Example in Data Science Context

Suppose we want the average monthly spending of all customers in a city:

• Population size: 100,000 customers (unknown μ).
• We take a sample of 200 customers and find the sample mean = ₹ 5,200.

Estimated mean:
μ̂ ≈ 5,200
This is our best guess for the true population mean.

Importance in Data Science

• Used in descriptive statistics to summarize data.

• Provides a basis for inference — many statistical models assume the mean is known or
estimated.
• Critical in hypothesis testing and confidence interval calculation.

Difference Between Sample Mean and Estimated Mean

Aspect Sample Mean (x̄) Estimated Mean (μ̂)

Definition Arithmetic mean of sample Approximation of the

data population mean using
sample data

Data Basis Calculated directly from Based on the sample mean

the sample

Purpose Describes the sample Predicts the population

mean

Symbol x̄ μ̂

• Sample mean is computed from observed data.

• Estimated mean uses the sample mean to infer the population mean.

Variance and Standard Score

In data science and statistics, variance and standard score (also called z-score) are essential
concepts used to measure the spread of data and standardize data points for comparison.
Variance indicates how much the data points differ from the mean, while the standard score
tells us how far a specific value is from the mean in terms of standard deviations.

Variance

Variance is a measure of the dispersion of a set of values. It calculates the average of the
squared differences between each value and the mean.
A high variance indicates that the data points are spread out widely from the mean, while a
low variance means they are closer to the mean.

Formula for population variance: σ² = Σ (xᵢ - μ)² / N

Formula for sample variance: s² = Σ (xᵢ - x̄)² / (n - 1)

Example:

Consider the dataset: 5, 7, 3

Mean = (5 + 7 + 3) / 3 = 5
Variance = [(5-5)² + (7-5)² + (3-5)²] / 3 = (0 + 4 + 4) / 3 = 2.67

Standard Score (Z-score)

The standard score (z-score) is a statistical measure that describes a value's position relative
to the mean of a group of values, measured in terms of standard deviations. It is especially
useful for comparing values from different datasets or distributions.

Formula: z = (x - μ) / σ

Example:

If a student scored 85 on a test where the mean score was 75 and the standard deviation was
5:
z = (85 - 75) / 5 = 2
Interpretation: The student scored 2 standard deviations above the mean.

Applications in Data Science

1. Variance is used in statistical modeling to understand variability and detect features with
high or low variability.
2. Z-scores are used in anomaly detection, standardizing data for machine learning models,
and in hypothesis testing.

Statistical Inference (Frequency Approach)

Definition:

Statistical inference is the process of drawing conclusions about a population based on

information from a sample.
The frequentist (frequency) approach defines probability as the long-run frequency of an
event occurring after repeated trials.

 Example: If you flip a fair coin many times, the probability of heads = 0.5 means that
in the long run, 50% of flips will show heads.

Key Idea:

 Population → The entire group (e.g., all students in a college).

 Sample → A small part of the population (e.g., 50 students chosen randomly).

  Inference → Using the sample to estimate or test something about the population (like
the average marks of all students).

Example:

Suppose you want to know the average height of college students.

 You cannot measure all 5,000 students.

 Instead, you take a sample of 100 students.

 Using their heights, you estimate the average for the whole college.

This process = statistical inference

Applications:

 Predicting election results by surveying a small group.

 Estimating the failure rate of machines in a factory.

 Medical trials – testing a drug on a small group before general use.

Variability of Estimates:

When we take different random samples from the same population, the estimates (like sample
mean, variance, or proportion) will not be exactly the same. This variation is called
sampling variability or variability of estimates.

Example:

Imagine you want to estimate the average mark of students in a class of 500.

 Sample 1 (50 students): Mean = 68

 Sample 2 (50 students): Mean = 72
 Sample 3 (50 students): Mean = 70

Each sample gives a slightly different mean → This is variability of estimates.

Because we need to know how reliable our sample estimate is.

 If variability is small → The estimate is stable and reliable.

 If variability is large → The estimate is uncertain.

Applications:

 In quality control, measuring consistency in production.

 In finance, understanding how different samples of stock data may affect predictions.
  In medicine, testing how reliable treatment outcomes are when repeated with different
groups.

Hypothesis Testing using Confidence Intervals:

A confidence interval (CI) gives a range of values within which the true population
parameter is likely to lie with a certain level of confidence (usually 95%).

Instead of just giving a single estimate (like a sample mean), CI provides a range.

Example:

Suppose a sample of 100 students has an average height = 160 cm, with a 95% confidence
interval of [158 cm, 162 cm].

 This means we are 95% confident that the true average height of all students lies
between 158 and 162 cm.

Hypothesis Testing with CI:

We test claims (hypotheses) using CI:

 Null Hypothesis (H₀): The population mean = 160 cm

 Alternative Hypothesis (H₁): The population mean ≠ 160 cm

If the claimed value (160) lies inside the confidence interval, we do not reject H₀.
If it lies outside the interval, we reject H₀.

Applications:

 Checking whether a new teaching method improves student scores significantly.

 Estimating whether a factory machine produces items within acceptable size limits.

 Medical research to test if a drug’s effect differs from standard treatment.

Using p-values
Definition:

The p-value is the probability of observing results as extreme as (or more extreme than) the
actual sample result, if the null hypothesis is true.

 A small p-value (≤ 0.05) → Strong evidence against H₀ → Reject H₀.

 A large p-value (> 0.05) → Weak evidence against H₀ → Do not reject H₀.

Example:
Suppose you test whether a coin is fair.

 Null Hypothesis (H₀): Coin is fair (p = 0.5).

 You flip it 20 times and get 17 heads.

 You calculate a p-value = 0.01.

Since 0.01 < 0.05, the result is very unlikely under H₀ → You reject H₀ and conclude the coin
is probably biased.

Applications:

 Determining whether a medicine has a real effect or just a random outcome.

 Testing whether a new machine produces better results than the old one.

 Validating scientific claims in experiments.

 UNIT-3 1. Data Analysis – Cleaning,  Limited support for
summarizing, and exploring enterprise-scale
data. applications.
Introduction to R Programming
2. Data Visualization –
Creating graphs, plots, and
R is an open-source programming dashboards. Getting Started with R
language and software environment 3. Statistical Modeling –
designed for statistical computing, Regression, hypothesis 2.1 Installation of R Software
data analysis, and data testing, ANOVA.
visualization. It was developed by 4. Machine Learning – R is an open-source statistical
Ross Ihaka and Robert Gentleman Classification, clustering, computing software available for
in the early 1990s at the University of prediction models. free. To start programming in R, it
Auckland, New Zealand. 5. Finance & Business must first be installed on your
Analytics – Risk modeling, computer.
Features of R stock market prediction,
sales forecasting. Steps for Installation
6. Healthcare &
 Open Source – Free to use Bioinformatics – Genome 1. Visit CRAN Website
and modify. sequencing, medical
 Cross-Platform – Works research. o Go to the
on Windows, Linux, and 7. Text Mining & NLP – Comprehensive
MacOS. Sentiment analysis, topic R Archive
Network
 Statistical Support – modeling.
(CRAN):
Provides a wide range of 🔗 [Link]
statistical and mathematical Example Program in R [Link]
techniques.
 Data Visualization – # Simple R Program o CRAN hosts R
Produces high-quality numbers <- c(10, 20, 30, 40, 50) # distributions for
graphics and plots. Create a vector all major
 Extensible – Thousands of average <- mean(numbers) # operating
packages are available via Calculate mean systems.
CRAN (Comprehensive R print(average) # Print
Archive Network). output 2. Select Operating System
 Integration – Supports
integration with other Output: o Windows Users:
languages like C, C++, Click “Download
Python, and Java. R for Windows”
[1] 30 → Select latest
 Large Community –
Actively supported by base version.
researchers, statisticians, Advantages of R
and data scientists o MacOS Users:
worldwide. Click “Download
 Free and open-source. R for macOS” →
 Rich set of libraries for data Select the
Importance of R analysis. package that
 Strong data visualization matches your OS
 Widely used in Data capabilities. version.
Science, Artificial  Supports both structured
Intelligence, and Machine and unstructured data. o Linux Users:
Choose
Learning.  Widely used in academics, “Download R for
 Supports hypothesis research, and industries. Linux” and
testing, regression, and
follow the
predictive analytics.
Limitations of R instructions for
 Provides powerful data your distribution
visualization libraries such (Debian, Ubuntu,
as ggplot2 and plotly.  Slower execution speed Fedora, etc.).
 Suitable for academic compared to C++/Java.
research, business  Memory intensive for very 3. Download the Installer
analytics, and large datasets.
bioinformatics.  Steep learning curve for o Download the
beginners. latest stable
Applications of R release.
 o File format: .exe tasks, but limited in o Plots: Display
for Windows, advanced project visualizations.
.pkg for Mac, management.
and system- o Packages:
specific Example in Console: Install, load, and
instructions for manage R
Linux. 2+5 packages.

4. Run the Installer and Output: o Help: Access

Setup documentation
[1] 7 for R functions.
o Windows/Mac:
Double-click the B. RStudio IDE Advantages of RStudio
installer and
follow the RStudio is a powerful Integrated  Provides debugging tools
default Development Environment (IDE) to find errors easily.
instructions designed specifically for R. It provides
(Next → Next → a user-friendly graphical interface and
Finish). additional features.
 Supports project
management for large-
scale data analysis.
o Linux: Use Key Features of RStudio IDE
package
managers (e.g., 1. Script Editor (Source  Integrated with GitHub,
apt-get install r- Pane): Shiny Apps, and
base on Ubuntu). Markdown for reports.
o Write, edit, and
5. Verify Installation save R programs  Makes R more accessible
(.R files). for beginners and
o Open R Console researchers.
from the Start o Supports syntax
Menu (Windows) highlighting and
or Applications auto-completion.
(Mac). R Installation requires downloading
2. Console Pane: from CRAN and setting up based on
o Type: the operating system.
o Runs R
o version commands R can be used through: R Console
directly. (basic, command-line execution).
o The output will
show the o Displays outputs RStudio IDE (feature-rich
installed R and error environment with advanced tools for
version and messages. coding, visualization, and package
system details. management).
3. Environment/History
After installation, R is ready to use. Pane:
However, for better productivity,
many users prefer working in RStudio o Shows all Variables and Data Types in R
IDE. defined
variables, data Variables in R
2.2 Using the Interface frames, and
objects. A variable is a named storage
Once installed, R can be used in two location that holds data values. In R,
primary ways: o Keeps track of variables are created when values are
previously assigned using the assignment
A. R Console executed operators <-, =, or ->.
commands.
Rules for Variables:
 The basic command-line
interface that comes with 4. Files/Plots/Packages/Help
R. Pane: 1. Variable names are case-
sensitive (Age and age are
o different).
 Allows direct typing and Files: View
execution of commands. working
directory 2. Must start with a letter or a
contents. dot (.), but not with a digit.
 Suitable for quick
calculations and small
 3. Cannot use reserved print(int_var) Checking Data Types
keywords (e.g., if, else,
TRUE). typeof(int_var) # integer Functions used:

4. Avoid spaces in variable 3. Character  typeof(x) → tells the type

names (use underscore _ or of object.
dot . instead).  Represents text or string
values (enclosed in quotes).  class(x) → gives the class
Example:
of an object.
name <- "Data Science"
# Creating variables
 [Link](x),
typeof(name) # character [Link](x), [Link](x)
x <- 10 # using <- operator
→ checks the type.
4. Logical (Boolean)
y = 20 # using = operator
Example:
 Stores TRUE or FALSE.
30 -> z # using -> operator
a <- 10
is_valid <- TRUE
b <- "Hello"
typeof(is_valid) # logical
# Display values
c <- TRUE
5. Complex
print(x)

print(y)  Used for complex numbers

(with imaginary part). typeof(a) # numeric
print(z)
comp <- 3 + 2i class(b) # character
Output:
typeof(comp) # complex [Link](c) # TRUE
[1] 10
6. Raw R Objects
[1] 20
 Represents raw bytes (used In R, everything is treated as an
[1] 30 rarely, e.g., binary data). object — data, variables, functions, or
even expressions.
Data Types in R r <- charToRaw("R") An object is simply a data structure
that holds values and defines how they
R supports several basic data types, print(r) # 52 are stored, accessed, and manipulated.
which form the foundation of all data
structures. Type Conversion R provides a wide range of basic and
complex objects to handle different
1. Numeric Sometimes data needs to be converted types of data.
from one type to another using
Types of Objects in R
 Represents decimal values functions like:
or real numbers. R objects can be broadly classified
 [Link]()
into:
 Default type for numbers
with decimals.  [Link]() 1. Atomic Objects (Basic Building
Blocks)
num <- 3.14  [Link]()
 Contain only one type of
print(num)  [Link]() data.
typeof(num) # numeric Example:  Includes: Vector, Matrix,
Array.
2. Integer x <- "100"
2. Non-Atomic Objects (Can contain
 Represents whole numbers. num_x <- [Link](x) multiple types of data)

 Defined by adding L at the print(num_x + 50) # 150

 Includes: List, Data
end of the number. Frame, Factors.
int_var <- 25L Basic R Objects
 1. Vectors y <- c(1, 2, 3) [3,] 3 6 9

 A vector is a sequence of 3. Arrays

data elements of the same
basic type (homogeneous). # Arithmetic operations  Similar to matrices but can
have more than 2
 In R, vectors are the most
x + y # Element-wise addition dimensions.
common data structure and
form the basis for most x - y # Subtraction  Created using array()
operations. function.
x * y # Multiplication
Types of Vectors Example:
x / y # Division
1. Numeric Vector – stores arr <- array(1:12, dim = c(3, 2, 2))
numbers.
print(arr)
2. Character Vector – stores # Logical operations
text/strings. 4. Lists
x > 15 # Returns TRUE/FALSE for
3. Logical Vector – stores each element
A list is an ordered collection of
TRUE/FALSE values.
y == 2 # Checks equality elements that can be of different
data types. Unlike vectors, lists
4. Integer Vector – stores
Important Functions for Vectors can contain heterogeneous data
integers specifically.
(numbers, strings, vectors,
length(x) # Number of elements matrices, functions, even other
5. Complex Vector – stores lists).
complex numbers.
sum(x) # Sum of all elements
Creating Vectors Creating Lists
mean(x) # Average # Simple list
Vectors can be created using the c() my_list <- list(101, "R Programming",
function (combine function). max(x) # Maximum value TRUE, 3.14)

# Numeric vector min(x) # Minimum value # List with named elements

student <- list(Name="Nandini",
num_vec <- c(1, 2, 3, 4, 5) sort(x) # Sort the vector Age=21, Marks=c(85, 90, 88))

Accessing List Elements

# Using index
# Character vector 2. Matrices my_list[[1]] # 101
my_list[[2]] # "R Programming"
char_vec <- c("R", "Python", "Java")  Two-dimensional array-like
structure. # Using names
student$Name # "Nandini"
 Can store only one data
student$Marks # c(85, 90, 88)
# Logical vector type (all numeric or all
character). # Extract specific mark
log_vec <- c(TRUE, FALSE, TRUE) student$Marks[2] # 90
 Created using matrix() Modifying List Elements
function. student$Age <- 22 # Update value
# Integer vector student$Course <- "[Link]" # Add
Example: new element
int_vec <- c(1L, 2L, 3L, 4L) mat <- matrix(1:9, nrow = 3, ncol = 3) Nested Lists
print(mat)
Lists can contain other lists:
# Complex vector Output:
nested_list <- list(Name="R",
comp_vec <- c(2+3i, 4+5i) [,1] [,2] [,3] Details=list(Year=2025,
Level="Advanced"))
Vector Operations [1,] 1 4 7 nested_list$Details$Year # 2025
x <- c(10, 20, 30) [2,] 2 5 8
 5. Data Frames Example: class(d) # logical

 Table-like structure (like x <- c(1,2,3,4)  Complex – Numbers with

Excel). imaginary parts.
class(x) # "numeric"
 Stores data in rows and e <- 3 + 4i
columns. typeof(x) # "double"
class(e) # complex
length(x) # 4
 Each column can hold a
different data type.  Factor – Used for
categorical data (nominal or
ordinal).
 Created using [Link]() Classes in R
function.
gender <- factor(c("Male",
In R, classes represent the type or "Female", "Male"))
Example: nature of an object. A class determines
how R treats the object and which class(gender) # factor
df <- [Link]( functions can be applied to it.
Classes are the foundation of Object-
ID = c(1, 2, 3), Oriented Programming (OOP) in R,  Date and Time – Special
enabling structured data handling and classes for handling dates.
Name = c("Alice", "Bob", "Charlie"), method implementation.
today <- [Link]()
Score = c(85, 90, 95) Checking Class
class(today) # Date
) You can check the class of an object
using: Object-Oriented Classes in R
print(df)
x <- 10 R supports different object-oriented
systems:
class(x)
6. Factors (a) S3 Classes
Output:
 Used to handle categorical  Informal and widely used
data. [1] "numeric" system.

 Stores values as levels (e.g., Common Classes in R  Objects are given a class
Male/Female, Pass/Fail). attribute.
 Numeric – Represents
 Created using factor() numbers (integers or  Example:
function. decimals).
 person <- list(name="John",
Example: a <- 23.5 age=25)
class(a) # numeric
gender <- factor(c("Male", "Female",  class(person) <- "Student"
"Male", "Female"))
 Integer – Whole numbers.
 print(person)
print(gender)
b <- 10L
(b) S4 Classes
Checking Object Type
class(b) # integer
Functions used:  More formal with explicit
definitions.
 Character – Text or string
 typeof(object) → data type values.
of object.  Example:
c <- "Hello"
 class(object) → object  setClass("Student",
class. class(c) # character
 slots =
 length(object) → number of  Logical – Boolean values list(name="character",
elements. (TRUE or FALSE). age="numeric"))

 str(object) → structure of
d <- TRUE  s <- new("Student",
object. name="Alice", age=22)
  class(s) # "Student" Used to execute statements based on "c" = "Third")
conditions.
(c) Reference Classes (R5) print(result)
(a) if Statement
 Also known as RC classes,
used for mutable objects. Executes a block if the condition is
true. 2. Looping Structures (Iteration)
 Example: x <- 10 Used for repeating a block of code
multiple times.
 Person <- if(x > 5){
setRefClass("Person", (a) for Loop
print("x is greater than 5")
 fields = Executes a block for each element in a
list(name="character", } sequence.
age="numeric"))
(b) if-else Statement for(i in 1:5){
 p1 <-
Person$new(name="John", Executes one block if condition is print(i)
age=30) true, another if false.
}
x <- 3
 p1$age # Access field
(b) while Loop
if(x > 5){
Importance of Classes
Executes as long as the condition is
print("x is greater than 5") true.
 Helps in data
organization. } else { x <- 1

 Enables polymorphism print("x is less than or equal to 5") while(x <= 5){
(same function behaves
differently for different } print(x)
classes).
(c) if-else ladder x <- x + 1
 Forms the basis for custom
objects in advanced R Multiple conditions can be checked. }
programming.
x <- 0 (c) repeat Loop

Classes in R define the type and if(x > 0){ Executes repeatedly until a break
behavior of objects. Apart from built- condition is encountered.
in classes (numeric, integer, character, print("Positive")
etc.), R also supports object-oriented x <- 1
programming systems (S3, S4, and } else if(x < 0){
Reference Classes) that allow repeat {
creating user-defined structures. print("Negative")
print(x)
} else {
x <- x + 1
R-Programming Structures print("Zero")
if(x > 5){
Programming structures in R are the }
control mechanisms that determine break
how instructions are executed. (d) switch Statement
They help in decision-making, }
iteration, and code organization, Chooses one case among many.
making programs more flexible and }
efficient. x <- "b"
3. Control Statements
Types of Structures in R result <- switch(x,
Used to alter the flow inside loops.
1. Conditional Structures (Decision- "a" = "First",
Making) break → Terminates the loop.
"b" = "Second",
for(i in 1:10){
 if(i == 5) break Arithmetic operators are used for Op Me Ex
basic mathematical Res
era ani am
print(i) calculations. ult
tor ng ple

} O
Ex Les
O u FA
a s 5<
next → Skips the current pe Descri t < LS
m tha 3
iteration. rat ption p E
pl n
or u
for(i in 1:5){ e
t
Eq 5
TR
if(i == 3) next == ual ==
10 UE
Additi 1 to 5
+ +
print(i) on 5
5
No
} t 5
Subtra 10 TR
- 5 != equ !=
ction -5 UE
4. Functions (User-Defined al 3
Structures) to
Multip 10
5
Functions group a set of instructions * licatio * c(T
0
for reuse. n 5 RU
E,
add <- function(a, b){ Divisi 10 Ele FA
/ 2 (T
on /5 me LS
return(a + b) RU
nt- E)
E,
Modul 10 & wis &
} FA
% o % e c(T
1 LS
% (remai % AN RU
print(add(5, 3)) E)
nder) 3 D E,
TR
Importance of Programming UE
Structures Power )
^ 2
(expo
or ^ 8
 Helps in decision-making. **
nentiat
3 `c(
ion) Ele
TR
 Reduces repetition of code
me
UE
through loops. These operations also work on vectors nt-
` ` ,
element-wise. wis
FA
e
 Makes programs organized
a <- c(2, 4, 6) OR
LS
and efficient. E)
b <- c(1, 2, 3)
 Enables modularity via
NO
!T FA
functions. a + b # (3, 6, 9) ! RU LS
T
E E
R-Programming structures a * b # (2, 8, 18)
include conditional statements, Used in filtering data,
looping constructs, control Logical Operations conditional checks, and
statements, and functions. They comparisons.
provide mechanisms to control Logical operators return TRUE
the flow of execution and make or FALSE values depending on Matrix Operations
R programs more structured and conditions.
powerful. Matrices in R support special
Op Me Ex operations useful in data science
Operations in R Res and linear algebra.
era ani am
ult
tor ng ple
In R, operations can be Matrix Multiplication (%*%)
performed on numbers, vectors,
matrices, and logical values. Gr A <- matrix(c(1,2,3,4), nrow=2)
They are broadly divided into eat
5> TR
Arithmetic, Logical, and > er B <- matrix(c(5,6,7,8), nrow=2)
3 UE
Matrix operations. tha
n A %*% B
Arithmetic Operations
 → Performs matrix We use the function [Link]() to Function Description
multiplication (not element- create a data frame.
wise).
str(df) Structure of data frame
# Example: Creating a data frame
Transpose of a Matrix (t())
students <- [Link]( Summary statistics of
summary(df)
A <- matrix(c(1,2,3,4), nrow=2) columns
ID = c(1, 2, 3, 4),
t(A) nrow(df) Number of rows
Name = c("Alice", "Bob", "Charlie",
→ Converts rows into columns. "David"),
ncol(df) Number of columns
Inverse of a Matrix (solve()) Age = c(20, 21, 19, 22),
colnames(df) Names of columns
A <- matrix(c(2,1,1,2), nrow=2) Marks = c(85, 90, 78, 88),
rownames(df) Names of rows
solve(A) Passed = c(TRUE, TRUE, FALSE,
TRUE) head(df) First few rows
→ Finds the inverse of matrix A
(only for square, non-singular )
matrices). tail(df) Last few rows

Applications
print(students)
 Arithmetic operations → Modifying a Data Frame
Basic computations in Output:
statistics, finance, 1. Add a new column
simulations. ID Name Age Marks Passed
students$Grade <- c("A", "A+", "B",
"A")
 Logical operations → Data 1 1 Alice 20 85 TRUE
filtering, condition checks,
2 2 Bob 21 90 TRUE 2. Add a new row
classification.
3 3 Charlie 19 78 FALSE new_row <- [Link](ID=5,
 Matrix operations → Name="Eva", Age=20, Marks=92,
Linear regression, machine 4 4 David 22 88 TRUE Passed=TRUE, Grade="A+")
learning algorithms, image
processing. students <- rbind(students, new_row)
Accessing Data from a Data Frame
Data Frames in R 3. Remove a column
1. By column name ($)
A Data Frame in R is a two- students$Grade <- NULL
students$Name
dimensional table-like structure
where: Applications of Data Frames
# Output: "Alice" "Bob" "Charlie"
"David"
 Data is stored in rows  Storing datasets (CSV,
(observations) and 2. By indexing ([row, Excel, SQL tables).
columns (variables). column])
 Performing data analysis,
 Each column can have students[1, 2] # Row 1, Column 2 cleaning, and
different data types → "Alice" manipulation.
(numeric, character, logical,
factor). students[ , 3] # Entire 3rd column
(Age)
 Input format for statistical
modeling and machine
 It is similar to a learning.
spreadsheet (Excel) or a students[2:4, ] # Rows 2 to 4
database table. Data Frames in R are tabular
3. By column names
data structures that allow
Data frames are the most commonly storage of heterogeneous data
used data structure in data analysis students[ , "Marks"]
types across columns. They are
with R. essential for data manipulation,
Useful Functions for Data Frames
exploration, and modeling.
Creating a Data Frame
Function Description Functions in R
A function in R is a block of 1. Function without arguments # Anonymous function inside apply()
reusable code that performs a specific
task. greet <- function() { sapply(1:5, function(x) x^2)

 It helps avoid repetition. print("Hello, welcome to R # Output: 1 4 9 16 25

programming!")
 Improves readability and
Scope of Variables in Functions
modularity. }
 Local variables: Declared
 Takes input inside the function (exist
(arguments/parameters) only within it).
greet()
and may return output
(result).  Global variables: Declared
2. Function with arguments
outside the function
In R, even built-in commands like (accessible everywhere).
add <- function(a, b) {
sum(), mean(), print() are functions.
x <- 10 # global
return(a + b)
Structure of a Function in R
myFunc <- function() {
}
General syntax:
x <- 5 # local
function_name <- function(arg1, arg2,
...) { return(x)
add(5, 3) # Output: 8
# Body of function }
3. Function with default arguments
# Perform computations myFunc() # 5
power <- function(x, y = 2) {
return(result) # optional x # 10
return(x^y)
}
}
Applications of Functions in R

Types of Functions
power(4) # 16 (default square)  Breaking large programs
into smaller modular units.
Built-in Functions power(4, 3) # 64
Functions already provided by R.  Performing repeated tasks
Examples: efficiently.
4. Function returning multiple
values
sum(c(2, 3, 5)) # 10
 Used in data cleaning,
calculate <- function(a, b) { analysis, visualization,
mean(c(10, 20, 30)) # 20
modeling.
sum_val <- a + b
sqrt(25) #5
prod_val <- a * b
User-defined Functions Control Structures in R
Functions created by the user for
return(list(Sum = sum_val, Product =
specific tasks.
prod_val)) Control Structures in R are
statements that control the flow of
# Example: Function to calculate
} execution in a program.
square
They help in decision-making and
repetition of tasks, making R
square <- function(x) { programs more flexible and powerful.
calculate(4, 5)
return(x^2)
Types of Control Structures
# Output: $Sum = 9, $Product = 20
} 1. Conditional Statements
Anonymous Functions (Lambda
Functions) (a) if statement
square(6) # Output: 36
In R, we can create functions without Executes a block of code only if a
names, often used in quick operations. condition is TRUE.
Examples of User-defined Functions
Syntax: } else { result <- switch(operation,

if (condition) { # code if none are TRUE "Add" = 2 + 3,

# code to execute if condition is } "Subtract" = 5 - 2,

TRUE
Example: "Multiply" = 4 * 3)
}
marks <- 75 print(result) # 12
Example:
if (marks >= 90) { 2. Looping Structures
x <- 10
grade <- "A" Used for repetition of tasks.
if (x > 5) {
} else if (marks >= 75) { (a) for loop
print("x is greater than 5")
grade <- "B" Iterates over a sequence.
}
} else if (marks >= 50) { Syntax:
(b) if...else statement
grade <- "C" for (variable in sequence) {
Executes one block if the condition is
TRUE, otherwise executes another } else { # code
block.
grade <- "Fail" }
Syntax:
} Example:
if (condition) {
print(grade) for (i in 1:5) {
# code if TRUE
(d) switch statement print(i^2)
} else {
Used to select one value among many. }
# code if FALSE
Syntax: (b) while loop
}
switch(expression, Repeats code while condition is
Example: TRUE.
case1 = { code },
x <- 3 Syntax:
case2 = { code },
if (x %% 2 == 0) { while (condition) {
...
print("Even number") # code
)
} else { }
Example:
print("Odd number") Example:
choice <- 2
} i <- 1
result <- switch(choice,
(c) if...else if...else (ladder) while (i <= 5) {
"Add" = 2 + 3,
Used when multiple conditions must print(i)
be checked. "Subtract" = 5 - 2,
i <- i + 1
Syntax: "Multiply" = 4 * 3)
}
if (condition1) { print(result) # NULL (no match since
choice=2) (c) repeat loop
# code if condition1 TRUE
⚡ Tip: Usually works with strings Repeats code indefinitely until a
} else if (condition2) { better: break statement is used.

# code if condition2 TRUE operation <- "Multiply" Syntax:

repeat {  Conditional execution f(10)
(e.g., grade assignment,
# code eligibility checks). traceback()

if (condition) {  Iteration for data browser()

processing (e.g., looping Pauses execution at a point,
break through dataset rows). allowing step-by-step checking.

} test_fun <- function(x) {

 Automating repetitive
} tasks. browser()

Example:  Decision-making in y <- x^2

algorithms.
i <- 1 return(y)

repeat { }
Debugging and Simulation in R
print(i) test_fun(4)
R provides powerful features for
i <- i + 1 debugging programs and running debug() and undebug()
simulations. Debugging helps identify Runs functions in debug mode.
if (i > 5) { and fix errors in code, while
simulation allows modeling real-world debug(sum)
break processes using random numbers and
probability distributions. sum(1:5)
}
Debugging in R undebug(sum)
}
Debugging is the process of detecting, recover()
3. Loop Control Statements analyzing, and fixing errors (bugs) in Helps navigate through error
R programs to ensure correct locations in nested functions.
execution.
 break → exits from a loop
immediately. options(error = recover)
Common Errors in R
try() and tryCatch()
 next → skips the current 1. Syntax Errors – Errors in Helps handle errors without
iteration and moves to the typing commands. stopping the program.
next.
o Example: x <- result <- try(log("text"),
Example: c(1 2 3) → silent=TRUE)
missing comma.
for (i in 1:10) {
print("Program continues despite
2. Runtime Errors – Occur error")
if (i == 5) { while executing (e.g.,
invalid operation).
next # skip printing 5
o Example: tryCatch(
} dividing by zero.
{ log("abc") },
if (i == 8) { 3. Logical Errors – Code
runs but produces incorrect error = function(e) {
break # stop loop at 8 results. print("Caught an error!") }
} Debugging Tools in R )
print(i) traceback() Simulation in R
Shows the sequence of function
} calls after an error. Simulation is the process of
generating artificial data using random
Applications of Control Structures f <- function(x) { g(x) } numbers to model real-life phenomena
in R
or test statistical methods.
g <- function(y) { stop("Error in
g()") } Applications of Simulation
  Testing algorithms and Simulation in R models real-world
models. processes using probability
distributions (runif(), rnorm(),
 Predicting outcomes under
rbinom()), and methods like Monte
Carlo.
uncertainty.
Together, debugging ensures
 Monte Carlo methods correctness of code, while simulation
(repeated random provides insights into uncertain
sampling). systems.

 Risk analysis and

forecasting.

Random Number Generation in R

runif(n, min, max) – Uniform

distribution.

runif(5, min=0, max=10)

rnorm(n, mean, sd) – Normal

distribution.

rnorm(5, mean=50, sd=10)

sample() – Random sampling.

sample(1:10, 5, replace=TRUE)

rbinom(), rpois(), rexp() – Other

probability distributions.

rbinom(5, size=10, prob=0.5) #

Binomial

rpois(5, lambda=3) #
Poisson

rexp(5, rate=1) #
Exponential

Example: Monte Carlo Simulation

Estimating the value of π using

random points:

N <- 100000

x <- runif(N, -1, 1)

y <- runif(N, -1, 1)

inside <- (x^2 + y^2) <= 1

pi_estimate <- 4 * mean(inside)

pi_estimate

Debugging in R helps identify and fix

errors using tools like traceback(),
browser(), debug(), tryCatch().
 Unit-4

Predictive Modeling
Predictive modeling is a statistical and machine learning approach used to analyze historical
data and make forecasts about future events. It involves building mathematical models that
identify patterns and relationships between input variables (independent variables) and output
variables (dependent variables).

The core idea is: 'If we know how things behaved in the past, we can predict how they will
behave in the future.'

Predictive models may use statistical techniques such as linear regression, logistic regression,
decision trees, random forests, or neural networks depending on the complexity of the data
and the problem.

Purpose
The main purposes of predictive modeling include:
1. Forecasting – Estimating continuous numerical outcomes (e.g., predicting sales,
temperature, or revenue).
2. Classification – Determining categories or classes (e.g., whether a customer will buy a
product – Yes/No).
3. Risk Assessment – Measuring the likelihood of future events such as fraud detection, loan
defaults, or insurance claims.
4. Decision Support – Helping businesses and governments make better decisions using data-
driven insights.

Applications of Predictive Modeling

 Banking: Predicting loan defaults.

 Healthcare: Predicting disease risk.

 Business: Forecasting sales revenue.

 Weather: Predicting rainfall or temperature

Types of Predictive Models

1. Regression Models – Predict numerical outcomes (e.g., stock price prediction).
2. Classification Models – Predict categorical outcomes (e.g., disease present: Yes/No).
3. Clustering Models – Group similar data points (e.g., customer segmentation).
4. Time Series Models – Predict values based on time (e.g., predicting electricity demand).

Linear Regression
Linear regression is a statistical and machine learning technique used to model the
relationship between a dependent variable (Y) and one or more independent variables (X). It
assumes that this relationship can be represented using a straight line (linear relationship).

- If there is one independent variable (X) → Simple Linear Regression

- If there are multiple independent variables (X1, X2, X3, …) → Multiple Linear Regression

Equation (Simple Linear Regression):

Y = β₀ + β₁X + ε

Where:
- Y = Dependent variable (the outcome we want to predict)
- X = Independent variable (the predictor or input)
- β₀ = Intercept (value of Y when X = 0)
- β₁ = Slope coefficient (how much Y changes for one unit increase in X)
- ε = Error term (difference between actual and predicted values, accounts for
randomness/noise)

How It Works (Intuition):

1. We have data points (X, Y) plotted on a graph.

2. Linear regression tries to fit the best possible straight line through these points.
3. "Best line" means the one that minimizes the total error (distance of each point from the
line).
- This is usually done using the Least Squares Method.

Real-time Example: Predicting House Price Based on Size

Suppose we are predicting house price (Y) using house size (X).

Equation (trained model): Price = 50,000 + 200 × Size

- β₀ = 50,000 → Intercept (base cost of the house, even if size is 0 [Link].)

- β₁ = 200 → Slope (for each additional [Link]., price increases by ₹200)

Example Prediction: If Size = 1000 [Link].

Price = 50,000 + 200 × 1000 = 2,50,000

So, the predicted price of a 1000 [Link]. house = ₹2,50,000

Applications of Linear Regression

- Predicting house prices based on size, location, number of rooms, etc.

- Forecasting sales based on advertising spend.
- Estimating salary based on years of experience.
- Predicting demand based on price.
- Relationship analysis in economics, finance, and healthcare.

Simple Linear Regression Model Building

Simple Linear Regression (SLR) is a statistical technique that uses one independent variable
(X) to predict the value of a dependent variable (Y). It assumes a linear relationship between
X and Y. The goal is to find the best-fitting straight line that represents the relationship
between the two variables.

Steps in Model Building

1. Data Collection – Gather relevant data that contains both independent and dependent
variables.
Example: Salaries (Y) vs. Years of Experience (X).

2. Exploratory Analysis – Visualize the data using scatter plots to check whether a linear
pattern exists.

3. Model Fitting – Fit a regression line using the Least Squares Method, which minimizes the
error between actual and predicted values.

4. Evaluation – Evaluate the model using performance metrics such as R² (coefficient of

determination) to determine how well the regression line explains the variation in the
dependent variable.

Real-time Example: Predicting Student Exam Score Based on Study Hours

Equation (trained model): Score = 20 + 5 × Hours

- β₀ = 20 → Intercept (minimum score even if study hours = 0).

- β₁ = 5 → Slope (for every extra hour of study, the exam score increases by 5 marks).

Example Prediction: If Hours = 6

Score = 20 + 5 × 6 = 50

So, if a student studies for 6 hours, the predicted exam score is 50 marks.

Applications of Simple Linear Regression

- Predicting exam scores based on study hours.

- Estimating salary based on years of experience.
- Forecasting sales revenue based on advertising spend.
- Predicting crop yield based on rainfall.
- Estimating energy consumption based on temperature.

Multiple Linear Regression (MLR)

Definition

Multiple Linear Regression (MLR) is an extension of simple linear regression where the
dependent variable (Y) is predicted using two or more independent variables (X₁, X₂, X₃, …,
Xn). It helps to understand how different factors collectively influence an outcome.

Unlike simple linear regression (which uses only one predictor), MLR considers multiple
predictors simultaneously, making the model more realistic for solving real-world problems.

Equation (General Form):

Y = β₀ + β₁X₁ + β₂X₂ + … + βnXn + ε

Where:
- Y = Dependent variable (the output we want to predict)
- X₁, X₂, …, Xn = Independent variables (inputs or predictors)
- β₀ = Intercept (value of Y when all X = 0)
- β₁, β₂, …, βn = Coefficients (indicate how much Y changes when a specific X increases by
1, keeping other variables constant)
- ε = Error term (difference between actual and predicted values)

How It Works (Intuition):

1. Collect data with multiple predictors.

Example: House prices based on size, number of bedrooms, and location rating.

2. Fit a regression plane (or hyperplane in higher dimensions) that best represents the data.

3. The model estimates coefficients (β values) that minimize the difference between actual
and predicted values using the Least Squares Method.

4. Evaluate the accuracy using metrics such as R², Adjusted R², RMSE, and MAE.

Real-time Example: Predicting House Price

Equation (trained model): Price = 30,000 + 150 × Size + 20,000 × Bedrooms + 10,000 ×
Location

- β₀ = 30,000 → Base cost of the house (when all predictors = 0).

- β₁ = 150 → For each additional [Link]., the price increases by ₹150 (keeping bedrooms &
location constant).
- β₂ = 20,000 → Each additional bedroom increases price by ₹20,000.
- β₃ = 10,000 → For each unit increase in location rating, price increases by ₹10,000.

Example Prediction: If Size = 1000 [Link]., Bedrooms = 2, Location = 3

Price = 30,000 + (150 × 1000) + (20,000 × 2) + (10,000 × 3)

Price = 30,000 + 150,000 + 40,000 + 30,000 = 2,50,000

So, the predicted house price = ₹2,50,000.

Applications of Multiple Linear Regression

- Economics: Predicting GDP based on investment, exports, and population.

- Business: Forecasting sales based on advertising budget, price, and product quality.
- Education: Predicting student performance using study hours, attendance, and participation.
- Healthcare: Estimating patient recovery time based on age, treatment type, and health .

Simulation in R
Simulation in R refers to the process of generating artificial (random) data to represent real-
world scenarios. It helps in studying system behavior, testing models, and analyzing
outcomes under different conditions.

Example: Instead of collecting marks from students, we can simulate exam scores using
probability distributions.

Common Functions Used in Simulation

1. rnorm(n, mean, sd)

Generates random numbers from a Normal distribution.

Parameters:
- n → Number of values to generate
- mean → Central value (average)
- sd → Standard deviation (spread of data)
Example:
rnorm(5, mean=50, sd=5)

2. runif(n, min, max)

Generates random numbers from a Uniform distribution.

Parameters:
- n → Number of values to generate
- min, max → Range of values
Example:
runif(5, min=0, max=100)

3. sample(x, size, replace=FALSE)

Randomly selects values from a given set/vector.

Parameters:
- x → Source of values
- size → Number of samples
- replace → Sampling with/without replacement
Example:
sample(1:6, size=10, replace=TRUE) # Rolling a dice 10 times

Importance of Simulation

- Prediction of outcomes without real-world data collection.

- Risk analysis in finance, economics, and business.
- Decision making under uncertainty.
- Teaching and research through controlled experimentation.
- Monte Carlo Simulation: Uses repeated random sampling to estimate probabilities and
expected values (e.g., estimating π).
 UNIT –V
Classification

1. Classification
Classification is a supervised learning technique used to predict qualitative (categorical)
outcomes. It classifies data into predefined categories such as spam/not spam, disease/no
disease, or pass/fail. The goal is to learn a decision boundary that separates classes.

Types of Classification:

- Binary Classification: Two classes (0/1, Yes/No).

- Multi-Class Classification: More than two classes.
- Multi-Label Classification: Instances may belong to multiple classes.

Classification Process:
1. Collect and label dataset.

2. Split data into training and testing sets.

3. Train the classification algorithm.
4. Predict classes for unseen data.
5. Evaluate model performance.

Popular Classification Algorithms:

Logistic Regression, KNN, Naive Bayes, Decision Trees, Random Forest, SVM, Neural
Networks.

Applications:

Medical diagnosis, spam filtering, sentiment analysis, document classification.

2. Performance Measures
Performance metrics evaluate the effectiveness of classification models.

Confusion Matrix:
TP – True Positive

TN – True Negative
FP – False Positive
FN – False Negative

Accuracy = (TP + TN) / (TP + TN + FP + FN)

Precision = TP / (TP + FP)

Recall = TP / (TP + FN)

F1 Score = 2 * (Precision * Recall) / (Precision + Recall)
Specificity = TN / (TN + FP)

ROC Curve: Graph of TPR vs FPR at different thresholds.

AUC: Measures area under ROC. Higher AUC indicates better model.

Need for Multiple Metrics:

Accuracy alone is not reliable for imbalanced datasets.

3. Logistic Regression
Logistic Regression is used for binary classification. It predicts probability using the
sigmoid function:

h(x) = 1 / (1 + e^-(b0 + b1x))

Decision rule:
If h(x) > 0.5 → Class 1
If h(x) < 0.5 → Class 0

Log-Odds:
log(p / (1 – p)) = b0 + b1x

Assumptions:
- Binary dependent variable
- No multicollinearity
- Linearity in log-odds
- Independent observations

Applications:
Medical diagnosis, credit scoring, fraud detection, marketing analysis.

R Implementation:
model <- glm(Species ~ [Link] + [Link], data=iris_binary, family=binomial)

Explanation:
 We are creating a logistic regression model and saving it in the variable model.
 glm() is the function used to build the model.
 Species ~ [Link] + [Link] means:
o Species is what we want to predict.
o We are using Sepal Length and Sepal Width to make the prediction.

 data = iris_binary means the model uses the dataset called iris_binary.
 family = binomial tells R to perform logistic regression (because the output has
two classes: 0 or 1).
4. K-Nearest Neighbours (KNN)
KNN is a non-parametric, instance-based algorithm. Classification is based on majority
voting of K nearest neighbors.

Process:
1. Choose value of K.
2. Compute Euclidean distance.
3. Select K nearest points.

4. Assign class by majority voting.

Advantages:
Simple, no training phase, effective for small datasets.

Disadvantages:

Slow for large data, sensitive to noise, requires feature scaling.

R Example:
pred <- knn(train[,1:4], test[,1:4], train$Species, k=3)

Explanation:
 We are using the KNN algorithm to predict the species of flowers in the test
data.
 train[,1:4] → the input features from training data
 test[,1:4] → the input features from test data
 train$Species → the correct species of the training flowers
 k=3 → the algorithm looks at the 3 nearest neighbors to decide the class
 The predicted species are stored in pred.


5. Clustering – K-Means Algorithm

K-Means is an unsupervised learning algorithm grouping data into K clusters.

Steps:
1. Choose K.
2. Initialize centroids.
3. Assign points to nearest centroid.
4. Recalculate centroids.

5. Repeat until stability.

Objective:
Minimize within-cluster sum of squares.

Applications:

Market segmentation, image compression, pattern recognition.

R Example:
km <- kmeans(iris[,1:4], centers=3)

Meaning of Each Part

1. iris[,1:4]
 Uses the first 4 columns:
o [Link]
o [Link]
o [Link]
o [Link]
 These are the features used for clustering.

2. centers = 3
 We are asking k-means to create 3 clusters.
 Because the iris dataset has 3 types of flowers.

3. km <-
 Stores the clustering result (cluster numbers, centers, etc.) in km.
6. Time Series Analysis
Time series is a sequence of observations recorded over time.

Components:
- Trend: Long-term direction.
- Seasonality: Regular repeating patterns.
- Cyclic variations.
- Random noise.
Models:

AR, MA, ARMA, ARIMA, SARIMA.

Applications:
Weather forecasting, stock market prediction, sales forecasting.

R Example:
model <- [Link](AirPassengers)

Meaning of Each Part

1. [Link]()
 Automatically checks many ARIMA models.
 Selects the best one based on accuracy.
 Saves you from manually testing p, d, q values.

2. AirPassengers
 A built-in time series dataset in R.

 Contains monthly airline passenger counts from 1949–1960.

3. model <-
 Saves the final selected ARIMA model into the variable model.
7. Social Network Analysis
Social Network Analysis (SNA) is a method used to study the relationships,
connections, and interaction patterns among individuals, groups, or organizations.
It represents these relationships as nodes (people or objects) and edges (connections or
interactions).
SNA helps understand how information flows, who is influential, how communities are
formed, and how groups behave.

Example
Consider a WhatsApp group:
 Each member is a node
 Each message or interaction between members is an edge
 A person who talks to most people has high degree centrality
 A person who connects two sub-groups has high betweenness centrality

This small social network can be analyzed to find influencers and communication
patterns.

Advantages of Social Network Analysis

1. Identifies key influencers and important people in a network.
2. Helps understand information flow and communication patterns.
3. Detects communities, clusters, and subgroups.
4. Useful for predicting behavior based on connections.
5. Helps organizations improve team communication and structure.
6. Useful for analyzing large data from social media platforms.

7. Reveals hidden patterns not visible in traditional analysis.

Disadvantages of Social Network Analysis

1. Requires large, accurate, and complete data to give meaningful results.
2. Analysis becomes complex for very large networks.
 3. Privacy concerns when collecting personal relationship data.
4. Networks are dynamic (change over time), so results may become outdated.
5. Requires specialized tools and skills to interpret graphs.
6. Missing data or noise can affect accuracy.

Applications of Social Network Analysis

a) Social Media
b) Business & Management
c) Health & Medicine
d) Crime & Security

e) Education
f) Marketing

6. Tools Used for Social Network Analysis

In R
 igraph

 statnet
 sna
In Python
 NetworkX
 Graph-tool
 PyVis

Standalone Visualization Tools

 Gephi
 Cytoscape
 NodeXL
Graph Databases
  Neo4j
 OrientDB
 ArangoDB

8. Reading Data from MySQL in R

Definition
Reading data from MySQL in R means connecting R to a MySQL database and
importing tables into R for data analysis.
This is done using a database connection package such as RMySQL or DBI.

Explanation of Code
library(RMySQL)

conn <- dbConnect(MySQL(),

user='root',
password='1234',
dbname='company')

data <- dbGetQuery(conn, 'SELECT * FROM employees')

Step-by-step (Simple Explanation)

1. library(RMySQL)
– Loads the RMySQL package so R can talk to MySQL.

2. dbConnect()
– Creates a connection between R and the MySQL server.
– You give username, password, and database name.

3. dbGetQuery()
– Sends an SQL query to MySQL.
– Here, "SELECT * FROM employees" means:
Get all the rows and columns from the employees table.
4. data
– Stores the imported table as a data frame in R.

Example:
Suppose we have a MySQL database company with a table employees:

id name salary

1 Mani 50000

2 Nandini 60000

After running:

data <- dbGetQuery(conn, 'SELECT * FROM employees')

R will contain:
id name salary
1 1 Mani 50000
2 2 Nandini 60000

Advantages:
1. Fast data transfer from MySQL to R.
2. Can run SQL queries directly in R.
3. Good for large datasets stored in databases.
4. Secure connection using username & password.
5. Useful for real-time data analysis.

Disadvantages:
1. Requires MySQL installed and running.
2. Passwords in code may be unsafe if not handled carefully.
3. RMySQL package may need additional configuration on some systems.
4. Large queries may take time or cause memory usage in R.

Applications:
1. Business data analysis (sales, employees, inventory).

2. Machine learning models using stored data.

3. Automated report creation from database tables.
4. Real-time dashboards built with R Shiny.
 5. Data cleaning and statistical analysis for research.

Tools Used
 RMySQL (R package)

 DBI (Database Interface package)

 MySQL Server
 MySQL Workbench (optional)
 RStudio (for writing R code)

9. Reading Data from MongoDB in R :

Definition
Reading data from MongoDB in R means using R to connect to a MongoDB NoSQL
database and import collections (documents) into R for data analysis.
We commonly use the mongolite package in R to do this.

Simple Explanation
MongoDB stores data as documents (JSON-like format) instead of tables.
To read data from MongoDB into R:
1. Connect to MongoDB

2. Select the database and collection

3. Run a query
4. Store the data in an R data frame

Example Code :
library(mongolite)

# Connect to MongoDB collection

conn <- mongo(collection = "employees",
db = "company",
url = "mongodb://localhost")
# Read the data
data <- conn$find()

Step-by-step Explanation

 library(mongolite)
Loads the MongoDB package in R.

 mongo(...)
Connects R to MongoDB.
o collection = "employees" → choose the collection
o db = "company" → choose the database
o url = "mongodb://localhost" → MongoDB runs on local system

 conn$find()
Means:
Get all documents from the employees collection
and store them in R as a data frame.

Example
Suppose MongoDB contains:

{ "id": 1, "name": "Mani", "salary": 50000 }

{ "id": 2, "name": "Nandini", "salary": 60000 }
After running:
data <- conn$find()
R will show:

id name salary

1 Mani 50000

2 Nandini 60000

Advantages
1. Easy handling of JSON-like data.
 2. Great for unstructured or semi-structured data.
3. Fast reading and writing operations.
4. Flexible queries using MongoDB syntax.
5. Scales well for large data.

Disadvantages
1. Needs MongoDB installed and running.
2. No fixed schema → may cause inconsistent data.
3. Large collections may need powerful memory in R.
4. Fewer R packages available compared to SQL.