Abstract

Data analytics plays a vital role in extracting meaningful insights from raw data.
In this project, employee-related variables such as age, experience, salary,
performance, and project involvement are analyzed using Python libraries like
Pandas, NumPy, Matplotlib, and Scikit-learn.

Exploratory Data Analysis (EDA) techniques are applied to understand patterns,

relationships, and anomalies within the dataset. Visualization techniques such as
histograms, boxplots, scatter plots, and heatmaps help interpret the distribution
and correlation between variables.

INTRODUCTION

Data analytics plays a crucial role in modern organizations. Employee data contains valuable information
that can help organizations improve productivity, optimize operations, and make strategic decisions.

Data analytics plays a crucial role in modern organizations. Employee data contains valuable information
that can help organizations improve productivity, optimize operations, and make strategic decisions.
Python has become one of the most popular programming languages for data analytics due to its
simplicity and powerful libraries

Data analytics plays a crucial role in modern organizations. Employee data contains valuable information
that can help organizations improve productivity, optimize operations, and make strategic decisions.
Python has become one of the most popular programming languages for data analytics due to its
simplicity and powerful libraries. In this project, employee-related data is analyzed using statistical
techniques and machine learning models. The project demonstrates how raw data can be transformed
into useful insights.

OBJECTIVES OF THE PROJECT

• Understand employee dataset structure

• Perform exploratory data analysis

• Handle missing values and outliers

• Analyze statistical distribution of data

• Build regression and classification models

• Evaluate model performance using standard metrics

Tasks [Link] UNDERSTANDING:

Data analytics plays a vital role in extracting meaningful insights from raw data.
In this project, employee-related variables such as age, experience, salary,
performance, and project involvement are analyzed using Python libraries like
Pandas, NumPy, Matplotlib, and Scikit-learn.

Data analytics plays a vital role in extracting meaningful insights from raw data.
In this project, employee-related variables such as age, experience, salary,
performance, and project involvement are analyzed using Python libraries like
Pandas, NumPy, Matplotlib, and Scikit-learn.

Exploratory Data Analysis (EDA) techniques are applied to understand patterns,

relationships, and anomalies within the dataset. Visualization techniques such as
histograms, boxplots, scatter plots, and heatmaps help interpret the distribution
and correlation between variables

EmployeeID Age Experience Department Salary Quantity Profitable

0 21 22 Marketing 42570 6 0
1 30 20 Sale 23000 2 1
2 23 10 hr 100000 3 0

TASK 2: EXPLORATORY DATA ANALYSIS

Data analytics plays a vital role in extracting meaningful insights from raw data.
In this project, employee-related variables such as age, experience, salary,
performance, and project involvement are analyzed using Python libraries like
Pandas, NumPy, Matplotlib, and Scikit-learn.

Exploratory Data Analysis (EDA) techniques are applied to understand patterns,

relationships, and anomalies within the dataset. Visualization techniques such as
histograms, boxplots, scatter plots, and heatmaps help interpret the distribution
and correlation between variables.

Regression models are used to predict sales performance based on

independent variables such as projects and performance ratings. Classification
techniques such as logistic regression are used to categorize employees as
profitable or
not profitable. Model evaluation metrics including Mean Squared Error (MSE),
Mean Absolute Error (MAE), R² Score, accuracy, and confusion matrix are
used to evaluate model performance.

Data analytics plays a vital role in extracting meaningful insights from raw data.
In this project, employee-related variables such as age, experience, salary,
performance, and project involvement are analyzed using Python libraries like
Pandas, NumPy, Matplotlib, and Scikit-learn.

Exploratory Data Analysis (EDA) techniques are applied to understand patterns,

relationships, and anomalies within the dataset. Visualization techniques such as
histograms, boxplots, scatter plots, and heatmaps help interpret the distribution
and correlation between variables.
Data analytics plays a vital role in extracting meaningful insights from raw data.
In this project, employee-related variables such as age, experience, salary,
performance, and project involvement are analyzed using Python libraries like
Pandas, NumPy, Matplotlib, and Scikit-learn.

Scatter Plot

A Scatter Plot shows the relationship between two numerical variables.

Each point represents one observation in the dataset.

Purpose

 Identify relationships between variables

 Detect correlation
 Find clusters
 Detect outliers
 Example

Suppose we analyze:

 Experience (years)
 Salary

Each point represents one employee.

If points move upward diagonally, it shows positive correlation.

Types of Relationships
Relationship Pattern

Positive correlation Points rise upward

Negative correlation Points slope downward

No correlation Random points

Correlation Matrix

A Correlation Matrix is a table showing the correlation coefficient between variables.

The correlation coefficient ranges from:

−1≤r≤1-1 \leq r \leq 1−1≤r≤1

Value Meaning

+1 Perfect positive correlation

0 No correlation

-1 Perfect negative correlation

Pair Plot

A Pair Plot shows the relationship between multiple variables simultaneously.

It creates a grid of scatter plots for every pair of variables.

What Pair Plot Shows

 Scatter plots for each variable pair

 Histogram or KDE plot on the diagonal

Heatmap

A Heatmap is a graphical representation where values are displayed as colors.

It is commonly used to visualize the correlation matrix

TASK 3: HANDLING MISSING DATA AND OUTLIERS

[Link] Missing Values

Missing values occur when no data is stored for a variable in an observation. These
values are usually represented as:

 NaN (Not a Number)

 Null
 Empty cells
 Missing entries

For example:

Employee ID Age Salary

101 25 30000

102 NaN 35000

103 28 NaN

104 30 40000

In this dataset:

 Age is missing for Employee 102

 Salary is missing for Employee 103

Missing values can occur due to several reasons such as:

 Data entry errors

 System failures
 Survey respondents skipping questions
 Data corruption
 Incomplete data collection

Why Missing Values Are a Problem

Missing values can negatively affect data analysis because:

 1. They reduce dataset quality.
2. Machine learning algorithms cannot process null values.
3. They may lead to incorrect statistical results.
4. They reduce the accuracy of predictive models.

Therefore, identifying missing values is a critical step in data preprocessing.

Identifying Missing Values Using Python

In data analytics, missing values are usually detected using libraries like Pandas.

Example:

import pandas as pd

[Link]().sum()

This command counts how many missing values exist in each column.

Another method:

[Link]().sum()

This helps analysts understand which features contain missing data.

[Link] Missing Values

Once missing values are identified, they must be handled properly. There are multiple techniques
for dealing with missing values.

The most common methods include:

 Mean
 Median
 Mode

These techniques replace missing values with appropriate statistical measures.

Mean Method

The mean is the average value of a dataset.

Formula:

Mean=∑xnMean = \frac{\sum x}{n}Mean=n∑x

Where:

 x = values in the dataset

 n = number of observations

Example

Consider the following salary data:

Salary

30000

35000

NaN

40000

Mean calculation:

Mean=(30000+35000+40000)/3Mean = (30000 + 35000 + 40000) / 3Mean=(30000+35000+40000)/3

Mean=35000Mean = 35000Mean=35000

The missing value is replaced with 35000.

Python Example
df['Salary'].fillna(df['Salary'].mean(), inplace=True)
When to Use Mean

Mean is best used when:

 Data distribution is normal

 No extreme outliers exist
 The dataset is numerical
 Median Method

The median is the middle value when data is arranged in ascending order.

Example
Salary

30000

35000

NaN

50000

Sorted values:

30000, 35000, 50000

Median = 35000

The missing value is replaced with 35000.

Python Example
df['Salary'].fillna(df['Salary'].median(), inplace=True)
When to Use Median

Median is useful when:

 Data contains outliers

 Data is skewed
 Extreme values affect the

mean Median is more robust than

mean.

Mode Method

The mode is the value that occurs most frequently in a dataset.

Example
Department

HR

IT

IT

NaN

IT

Mode = IT

The missing value is replaced with IT.

Python Example
df['Department'].fillna(df['Department'].mode()[0], inplace=True)
When to Use Mode

Mode is used when:

 Data is categorical
 Values represent categories or labels

Example:

 Gender
 Department
 City
 Product type

[Link] Outliers
What Are Outliers?

Outliers are data points that significantly differ from other observations in the dataset.

They are extremely high or extremely low values compared to the rest of the data.

Example:
 Salary

30000

32000

35000

37000

900000

Here, 900000 is an outlier.

Outliers can occur due to:

 Measurement errors
 Data entry mistakes
 Natural variations
 Fraudulent data

Detecting Outliers Using Boxplots

A Boxplot (Box-and-Whisker Plot) is one of the most common techniques used to detect
outliers.

A boxplot displays:

 Minimum value
 First Quartile (Q1)
 Median
 Third Quartile (Q3)
 Maximum value

Interquartile Range (IQR)

IQR=Q3−Q1IQR = Q3 - Q1IQR=Q3−Q1

Outliers are usually detected using:

Lower bound:

Q1−1.5×IQRQ1 - 1.5 \times IQRQ1−1.5×IQR

Upper bound:

Q3+1.5×IQRQ3 + 1.5 \times IQRQ3+1.5×IQR

Values outside this range are considered outliers.

Example

Dataset:

10, 12, 14, 16, 18, 20, 200

Here:

 Q1 = 12
 Q3 = 18

IQR=18−12=6IQR = 18 - 12 = 6IQR=18−12=6

Lower limit:

12−1.5(6)=312 - 1.5(6) = 312−1.5(6)=3

Upper limit:

18+1.5(6)=2718 + 1.5(6) = 2718+1.5(6)=27

Value 200 lies outside this range, so it is an outlier.

Python Example
import seaborn as sns

[Link](x=df['Salary'])

The outliers appear as individual points outside the boxplot whiskers.

[Link] of Outliers

Outliers can significantly affect data analysis and machine learning results.
Impact on Mean

Outliers can distort the average value.

Example:

Dataset:
10, 12, 14, 16, 18

Mean = 14

Add outlier:

10, 12, 14, 16, 18, 200

New Mean:

Mean=45Mean = 45Mean=45

The mean becomes misleading.

Impact on Machine Learning Models

Outliers can reduce the performance of models such as:

 Linear Regression
 Logistic Regression
 K-Nearest Neighbors
 Neural Networks

Because models may try to fit abnormal values.

Impact on Visualization

Outliers can distort graphs such as:

 Histograms
 Scatter plots
 Boxplots

They may hide important patterns in the data.

Handling Outliers

Common techniques include:

[Link] Outliers

Remove extreme data points when they are caused by errors.

[Link]

Apply mathematical transformations such as:

 Log transformation
 Square root transformation

[Link] (Winsorization)

Replace extreme values with upper or lower limits.

[Link] Median Instead of Mean

Median is less affected by outliers.

Task 4: Spread of Data

Understanding the spread of data is an important step in Exploratory Data Analysis (EDA). It
helps analysts understand how values are distributed in a dataset and whether the data follows a
normal distribution or skewed distribution. Statistical measures such as mean, median,
standard deviation, skewness, and kurtosis help describe the distribution and variability of
data.

Analyzing data spread is essential because it helps identify patterns, variability, outliers, and
overall data behavior, which improves the accuracy of statistical analysis and machine learning
models.
[Link] Data Distribution

Data distribution describes how values are spread across the dataset. Two common types of
distributions are:

 Normal Distribution
 Skewed Distribution

Normal Distribution

A normal distribution (also called Gaussian distribution) is a symmetrical distribution where

most of the data points are concentrated around the mean.

Characteristics of Normal Distribution

1. The curve is bell-shaped.

2. Mean, median, and mode are equal.
3. Data is symmetrically distributed around the center.
4. Most values lie close to the average.

Example

Heights of people in a population usually follow a normal distribution.

Example values:

150, 155, 160, 165, 170, 175, 180

Here:

 Mean ≈ Median ≈ Mode

Visualization

A histogram of normally distributed data looks like a bell-shaped curve.

Importance

Normal distribution is important because many statistical models and machine learning
algorithms assume that data follows a normal distribution.
Skewed Distribution

A skewed distribution occurs when the data is not symmetrical and one side of the distribution
is longer than the other.

There are two types:

[Link] Skew (Right Skew)

In a positively skewed distribution, the tail extends towards the right side.

Characteristics:

Mean > Median > Mode

Example: Income distribution

Most people earn moderate salaries, but a few earn extremely high incomes.

Example values:

20000, 22000, 25000, 28000, 30000, 90000

The value 90000 creates right skew.

[Link] Skew (Left Skew)

In a negatively skewed distribution, the tail extends towards the left

side. Characteristics:

Mean < Median < Mode

Example: Exam scores where most students score high but a few score very low.

Example values:

20, 50, 80, 85, 90, 95

The value 20 creates left skew.

[Link] Measures of Data Spread

To understand the distribution and spread of data, several statistical measures are calculated.

These include:

 Mean
 Median
 Standard Deviation
 Skewness
 Kurtosis

Mean (Average)

The mean is the average value of a dataset.

Formula
Mean=∑XNMean = \frac{\sum X}{N}Mean=N∑X

Where:

X = individual data values

N = total number of observations

Example

Dataset:

10, 15, 20, 25, 30

Mean=10+15+20+25+305Mean = \frac{10 + 15 + 20 + 25 + 30}{5}Mean=510+15+20+25+30

Mean = 20

Interpretation

 Represents the central value of the dataset.

 Sensitive to outliers.
 Useful for normally distributed data.

Python Example
df['Salary'].mean()
Median

The median is the middle value when the dataset is arranged in ascending order.

Example

Dataset:

10, 15, 20, 25, 30

Median = 20

If the dataset has an even number of values:

Example:

10, 15, 20, 25

Median = (15 + 20) / 2 = 17.5

Interpretation

 Represents the center of the dataset.

 Less affected by outliers.
 Useful for skewed data.

Python Example
df['Salary'].median()

Standard Deviation

Standard deviation measures how much data values deviate from the mean.

It indicates the spread or variability of the dataset.

Formula
σ=∑(x−μ)2N\sigma = \sqrt{\frac{\sum (x - \mu)^2}{N}}σ=N∑(x−μ)2

Where:
x = individual value
μ = mean
N = number of observations

Example

Dataset:

10, 12, 14, 16, 18

Mean = 14

Standard deviation measures how far each value is from 14.

Interpretation

Low Standard Deviation:

 Data points are close to the mean

High Standard Deviation:

 Data points are spread far from the mean

Python Example
df['Salary'].std()

Skewness

Skewness measures the asymmetry of the data distribution.

It indicates whether the data is skewed to the left or right.

Skewness Values
Skewness Value Distribution

0 Perfectly normal distribution

>0 Positive skew

<0 Negative skew

Example

Skewness = 1.2

This means the distribution is positively skewed.

Interpretation

Positive Skew:

 Long tail on the right

Negative Skew:

 Long tail on the left

Python Example
df['Salary'].skew()

Kurtosis

Kurtosis measures the peakedness or flatness of a distribution.

It shows how heavy the tails are compared to a normal

distribution. Types of Kurtosis

1. Mesokurtic

Kurtosis ≈ 0
Distribution similar to normal distribution.

2. Leptokurtic

Kurtosis > 0
Distribution has sharp peak and heavy tails.

3. Platykurtic

Kurtosis < 0
Distribution is flatter with lighter tails.
Interpretation

High Kurtosis:

 More outliers

Low Kurtosis:

 Fewer extreme values

Python Example
df['Salary'].kurt()

[Link] Results

After calculating these statistical measures, the results must be interpreted properly.

Mean vs Median

If:

Mean ≈ Median
→ Data is normally distributed

If:

Mean > Median

→ Data is positively skewed

If:

Mean < Median

→ Data is negatively skewed

Standard Deviation

Small Standard Deviation:

Data points are close to the mean

Large Standard Deviation:

Data values are widely spread

Skewness Interpretation

Skewness ≈ 0
→ Symmetrical distribution

Skewness > 0
→ Right-skewed data

Skewness < 0
→ Left-skewed data

Kurtosis Interpretation

Kurtosis ≈ 0
→ Normal distribution

High Kurtosis
→ More extreme values (outliers)

Low Kurtosis
→ Flatter distribution

Task 6: Regression Analysis

Regression analysis is a statistical method used to identify relationships between variables and
predict a dependent variable using one or more independent variables.

In this project, regression analysis is used to analyze how sales depend on factors such as
quantity and discount.

Identifying Variables
Dependent Variable

The dependent variable is the variable that we want to predict or explain.

In this project:
Dependent Variable = Sales

Sales change depending on other factors such as quantity and discount.

Independent Variables

Independent variables are the variables that influence or affect the dependent variable.

In this project:

Independent Variables:

 Quantity
 Discount

These variables help determine the value of sales.

Example dataset:

Quantity Discount Sales

10 5 500

20 10 900

30 15 1200

Here:

Sales depend on quantity and discount.

Simple Linear Regression

Simple linear regression is used when there is one independent variable and one dependent
variable.

Equation

Y = a + bX

Where:
Y = Dependent variable (Sales)
X = Independent variable (Quantity)
a = Intercept
b = Slope

Example:

Sales = 200 + 50 × Quantity

If Quantity = 10

Sales = 200 + 50(10) = 700

Covariance Analysis

Covariance measures how two variables change together.

Formula

Cov(X,Y) = Σ[(Xi − X̄ )(Yi − Ȳ)] / n

Interpretation

Positive Covariance
→ Both variables increase together.

Negative Covariance
→ One variable increases while the other decreases.

Zero Covariance
→ No relationship between variables.

Example:

If Quantity increases and Sales also increase → Positive Covariance.

Correlation Analysis

Correlation measures the strength and direction of the relationship between two variables.
Correlation Range

-1 ≤ r ≤ 1

Value Meaning

+1 Perfect positive correlation

0 No correlation

-1 Perfect negative correlation

Example:

If correlation between Quantity and Sales = 0.85

This means strong positive relationship.

Task 7: Supervised Learning – Regression Model

Supervised learning is a machine learning technique where the model learns from labeled data.

In regression problems, the goal is to predict numerical values.

Splitting Dataset

Before training a model, the dataset is divided into two parts:

Training Data

Used to train the machine learning model.

Usually 70%–80% of the data.

Testing Data

Used to evaluate the model performance.

Usually 20%–30% of the data.

Example:

Total records = 1000

Training data = 800

Testing data = 200

Linear Regression using Scikit-Learn

Python provides the Scikit-learn library to build machine learning models.

Example code:

from sklearn.model_selection import train_test_split

from sklearn.linear_model import LinearRegression

X = df[['Quantity','Discount']]
y = df['Sales']

X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2)

model = LinearRegression()
[Link](X_train,y_train)

Overfitting

Overfitting occurs when a model learns the training data too well, including noise and
unnecessary details.

Characteristics

 Very high training accuracy

 Poor testing accuracy

Example:

Training accuracy = 98%

Testing accuracy = 60%

This means the model does not generalize well.

Underfitting

Underfitting occurs when the model is too simple to capture patterns in the data.

Characteristics

 Low training accuracy

 Low testing accuracy

Example:

Training accuracy = 60%

Testing accuracy = 58%

The model fails to learn the relationship.

Model Training Process

Step 1 – Train Model

The model learns patterns using training data.

[Link](X_train, y_train)

Step 2 – Validation

Validation data checks model performance during training.

It helps prevent overfitting.

Step 3 – Testing

Testing data evaluates the final model performance.

[Link](X_test)
Types of Regression Models
(i) Simple Linear

Regression One

independent variable.

Example:

Sales = a + b(Quantity)

(ii)Multiple Linear

Regression Two or more

independent variables. Example:

Sales = a + b1(Quantity) + b2(Discount)

This model predicts sales using both quantity and discount.

(iii) Logistic Regression

Logistic regression is used when the dependent variable is categorical instead of numerical.

Example:

Profit:

 Profitable
 Not Profitable

Logistic regression predicts the probability of a class.

Task 9: Overfitting and Underfitting Analysis

Understanding overfitting and underfitting helps improve model performance.

Comparing Training and Testing Error
Model Training Error Testing Error

Good Model Low Low

Overfitting Very Low High

Underfitting High High

Effect of Model Complexity

Simple model → Underfitting

Complex model → Overfitting
Balanced model → Best performance

Increasing model complexity improves accuracy initially, but too much complexity causes
overfitting.

Task 10: Classification Task

Sometimes regression problems can be converted into classification problems.

Example:

Instead of predicting profit amount, classify as:

 Profitable
 Not

Profitable Example

rule:

If Profit > 0 → Profitable

If Profit ≤ 0 → Not Profitable

Building Classification Model

Logistic regression can be used.

Example:

from sklearn.linear_model import LogisticRegression

model = LogisticRegression()
[Link](X_train,y_train)

Accuracy

Accuracy measures how many predictions are correct.

Formula

Accuracy = Correct Predictions / Total Predictions

Example:

Correct predictions = 90
Total predictions = 100

Accuracy = 90%

Confusion Matrix

A confusion matrix evaluates classification results.

Actual / Predicted Positive Negative

Positive True Positive False Negative

Negative False Positive True Negative

Interpretation

True Positive
Correctly predicted positive class

True Negative
Correctly predicted negative class
False Positive
Incorrect positive prediction

False Negative
Incorrect negative prediction

Task 11 & 12: Model Evaluation

Model evaluation measures how well a machine learning model performs.

Important metrics include:

 Mean Squared Error (MSE)

 Mean Absolute Error (MAE)
 R² Score

Mean Squared Error (MSE)

MSE measures the average squared difference between actual and predicted values.

Formula

MSE = Σ(Actual − Predicted)² / n

Interpretation

Lower MSE = Better model

Mean Absolute Error (MAE)

MAE measures the average absolute difference between actual and predicted values.

Formula

MAE = Σ|Actual − Predicted| / n

Interpretation

Lower MAE indicates better prediction accuracy.

R² Score

R² score measures how well the regression model explains the variability of the data.

Range

0 ≤ R² ≤ 1

Value Meaning

0 Model explains none of the variance

1 Perfect prediction

Example:

R² = 0.85

This means 85% of sales variation is explained by the model.

Comparing Regression and Classification

Feature Regression Classification

Output Continuous value Category

Example Sales prediction Profit / Loss

Algorithms Linear Regression Logistic Regression

Task 13: Data Visualization

Data visualization helps understand patterns and relationships in the dataset.

Univariate Analysis

Univariate analysis studies one variable at a time.

Examples:

 Histogram
 Boxplot
 Density

plot Purpose:

Understand distribution and spread of a single variable.

Bivariate Analysis

Bivariate analysis studies relationship between two variables.

Examples:

 Scatter plot
 Correlation plot
 Line graph

Example:

Sales vs Quantity

This helps identify relationships between variables.

Multivariate Analysis

Multivariate analysis studies three or more variables simultaneously.

Examples:

 Pair plots
  Heatmaps
 3D scatter plots

Purpose:

Understand complex relationships between multiple variables.

Data Normalization Plots

Normalization adjusts data values so they fall within a specific range.

Common normalization techniques:

 Min-Max Scaling
 Z-score Standardization

Normalization helps:

 Improve machine learning performance

 Ensure fair comparison between variables

Normal Distribution Plot

A normal distribution plot shows whether the data follows a bell-shaped curve. If

data is normally distributed:

Mean ≈ Median ≈ Mode

Skewness Visualization

Skewness plots help determine if data is:

Positive skew → tail on right

Negative skew → tail on left

These plots help analysts decide whether data transformation is needed.\