0% found this document useful (0 votes)
9 views8 pages

Central Tendency and Data Preprocessing

The document outlines two experiments focused on statistical analysis and data preprocessing techniques. Experiment 01 computes central tendency measures (mean, median, mode) and measures of dispersion (variance, standard deviation) using a dataset of student scores. Experiment 02 applies preprocessing techniques such as attribute selection, handling missing values, discretization, and elimination of outliers on a dataset containing demographic and spending information.

Uploaded by

kodurupavithra61
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
9 views8 pages

Central Tendency and Data Preprocessing

The document outlines two experiments focused on statistical analysis and data preprocessing techniques. Experiment 01 computes central tendency measures (mean, median, mode) and measures of dispersion (variance, standard deviation) using a dataset of student scores. Experiment 02 applies preprocessing techniques such as attribute selection, handling missing values, discretization, and elimination of outliers on a dataset containing demographic and spending information.

Uploaded by

kodurupavithra61
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Experiment -01

Aim: Compute Central Tendency Measures: Mean, Median and Mode and Measures of
Dispersion: Variance and Standard Deviation.

Dataset:

Studen
t Score
A 85
B 90
C 78
D 92
E 85
F 95
G 78
H 85
I 90
J 88

Program:

from statistics import mean, median, mode

from statistics import variance, stdev

import pandas as pd

data=pd.read_csv("D:/KRMCSEKHAR/ML/[Link]")

print(data)

# Calculate central tendency measures

mean_value = mean(data['Score'])

median_value = median(data['Score'])

mode_value = mode(data['Score'])

# Calculate Variance

variance_value = variance(data['Score'])
# Calculate Standard Deviation

std_dev_value = stdev(data['Score'])

# Display the results

print(f"Mean: {mean_value}")

print(f"Median: {median_value}")

print(f"Variance: {variance_value}")

print(f"Standard Deviation: {std_dev_value}")

print(f"Mode: {mode_value}")

O/P:

runcell(0, 'D:/ANAND/DV LAB/[Link]')

Student Score

0 A 85

1 B 90

2 C 78

3 D 92

4 E 85

5 F 95

6 G 78

7 H 85

8 I 90

9 J 88

Mean: 86.6

Median: 86.5

Mode: 85

Variance: 31.15555555555556
Standard Deviation: 5.581716183715861

Experiment -02
Aim: Apply the following Pre-processing techniques for a given dataset.
a. Attribute selection
b. Handling Missing Values
c. Discretization
d. Elimination of Outliers

a. Attribute selection
Dataset:

Age Income Gender Education Marital_Status Spending_Score Target


25 50000 Male Bachelor Single 45 1
30 60000 Female Master Married 78 0
22 40000 Female Bachelor Single 34 1
35 80000 Male PhD Married 65 0
28 55000 Female Master Single 50 1
40 100000 Male PhD Married 90 0
23 48000 Female Bachelor Single 42 1
45 110000 Male PhD Married 85 0
31 72000 Female Master Single 67 1
50 120000 Male PhD Married 95 0

Program:

import pandas as pd
from [Link] import LabelEncoder
from sklearn.feature_selection import SelectKBest, f_classif
# Load the dataset
df = pd.read_csv('D:/KRMCSEKHAR/ML/r23/[Link]')
print(df)
O/P:
******************************************************************
print(df)
Age Income Gender Education Marital_Status Spending_Score Target
0 25 50000 Male Bachelor Single 45 1
1 30 60000 Female Master Married 78 0
2 22 40000 Female Bachelor Single 34 1
3 35 80000 Male PhD Married 65 0
4 28 55000 Female Master Single 50 1
5 40 100000 Male PhD Married 90 0
6 23 48000 Female Bachelor Single 42 1
7 45 110000 Male PhD Married 85 0
8 31 72000 Female Master Single 67 1
9 50 120000 Male PhD Married 95 0
******************************************************************

# ---- Categorical Encoding (Label Encoding) ----


label_encoder = LabelEncoder()

# Apply Label Encoding to 'Gender' and 'Education' columns


df['Gender_Encoded'] = label_encoder.fit_transform(df['Gender'])
df['Education_Encoded'] = label_encoder.fit_transform(df['Education'])
df['Marital_Status_Encoded'] = label_encoder.fit_transform(df['Marital_Status'])
print("\nDataset with Label Encoding:")
print(df)
O/P:

print(df)
Age Income ... Education_Encoded Marital_Status_Encoded
0 25 50000 ... 0 1
1 30 60000 ... 1 0
2 22 40000 ... 0 1
3 35 80000 ... 2 0
4 28 55000 ... 1 1
5 40 100000 ... 2 0
6 23 48000 ... 0 1
7 45 110000 ... 2 0
8 31 72000 ... 1 1
9 50 120000 ... 2 0

[10 rows x 10 columns]


****************************************************************

#dropping Non-numerical data


df=[Link](columns=['Gender','Education','Marital_Status'])
print(df)

O/P:
print(df)
Age Income ... Education_Encoded Marital_Status_Encoded
0 25 50000 ... 0 1
1 30 60000 ... 1 0
2 22 40000 ... 0 1
3 35 80000 ... 2 0
4 28 55000 ... 1 1
5 40 100000 ... 2 0
6 23 48000 ... 0 1
7 45 110000 ... 2 0
8 31 72000 ... 1 1
9 50 120000 ... 2 0

[10 rows x 7 columns]


************************************************************

# Split features and target


X = [Link](columns=['Target']) # Feature variables
print(X)
*****************************************************************************
O/P:
print(X)
Age Income ... Education_Encoded Marital_Status_Encoded
0 25 50000 ... 0 1
1 30 60000 ... 1 0
2 22 40000 ... 0 1
3 35 80000 ... 2 0
4 28 55000 ... 1 1
5 40 100000 ... 2 0
6 23 48000 ... 0 1
7 45 110000 ... 2 0
8 31 72000 ... 1 1
9 50 120000 ... 2 0

[10 rows x 6 columns]


**********************************************************************

y = df['Target'] # Target variable


print(y)
***********************************************************************
O/P:
print(y)
0 1
1 0
2 1
3 0
4 1
5 0
6 1
7 0
8 1
9 0
Name: Target, dtype: int64
*****************************************************************

# Filter Method: Correlation with target (numerical features only)


print("Correlation with Target:")
correlations = [Link](y)
print(correlations)
**************************************************************
O/P:
print(correlations)
Age -0.789425
Income -0.769867
Spending_Score -0.852604
Gender_Encoded -0.600000
Education_Encoded -0.842701
Marital_Status_Encoded 1.000000
dtype: float64
********************************************************************

# 2. Filter Method: SelectKBest using ANOVA F-statistic


print("\nTop Features using ANOVA F-statistic:")
selector_anova = SelectKBest(score_func=f_classif, k=3) # Select top 5 features
X_new_anova = selector_anova.fit_transform(X, y)
selected_features_anova = [Link][selector_anova.get_support()]
print(selected_features_anova)
*******************************************************************
O/P:
print(selected_features_anova)
Index(['Spending_Score', 'Education_Encoded', 'Marital_Status_Encoded'], dtype='object')
*******************************************************************

You might also like