ML Project - Adult Income Classification
Objective
To build ML classification models to predict whether a person earns >50K or <=50K
per year using the Adult Income dataset.
We will perform:
Dataset loading
Cleaning & preprocessing
Exploratory Data Analysis (EDA)
Feature encoding & scaling
Model training (Logistic Regression, Decision Tree, Random Forest)
10-fold cross-validation
ROC curve
Model comparison
In [1]: import pandas as pd
import numpy as np
import [Link] as plt
import seaborn as sns
from sklearn.model_selection import train_test_split, GridSearchCV, cross_val_score, KFold
from [Link] import StandardScaler, LabelEncoder, OneHotEncoder
from [Link] import ColumnTransformer
from [Link] import Pipeline
from [Link] import accuracy_score, precision_score, recall_score, f1_score, roc_auc_s
from sklearn.linear_model import LogisticRegression
from [Link] import DecisionTreeClassifier
from [Link] import RandomForestClassifier
[Link](style="whitegrid")
Load the Adult Income Dataset
In [2]: # Correct column names for Adult dataset
column_names = [
'age','workclass','fnlwgt','education','education_num','marital_status',
'occupation','relationship','race','sex','capital_gain','capital_loss',
'hours_per_week','native_country','income'
]
# Load the extracted file
df = pd.read_csv(
r"C:\Users\hp\Downloads\[Link]",
header=None,
names=column_names,
skipinitialspace=True
)
[Link]()
Out[2]: age workclass fnlwgt education education_num marital_status occupation
Adm-
0 39 State-gov 77516 Bachelors 13 Never-married
clerical
Self-emp- Married-civ- Exec-
1 50 83311 Bachelors 13
not-inc spouse managerial
Handlers-
2 38 Private 215646 HS-grad 9 Divorced
cleaners
Married-civ- Handlers-
3 53 Private 234721 11th 7
spouse cleaners
Married-civ- Prof-
4 28 Private 338409 Bachelors 13
spouse specialty
In [3]: [Link]()
<class '[Link]'>
RangeIndex: 32561 entries, 0 to 32560
Data columns (total 15 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 age 32561 non-null int64
1 workclass 32561 non-null object
2 fnlwgt 32561 non-null int64
3 education 32561 non-null object
4 education_num 32561 non-null int64
5 marital_status 32561 non-null object
6 occupation 32561 non-null object
7 relationship 32561 non-null object
8 race 32561 non-null object
9 sex 32561 non-null object
10 capital_gain 32561 non-null int64
11 capital_loss 32561 non-null int64
12 hours_per_week 32561 non-null int64
13 native_country 32561 non-null object
14 income 32561 non-null object
dtypes: int64(6), object(9)
memory usage: 3.7+ MB
Clean Missing Values (Replace '?' with NaN)
In [4]: [Link]("?", [Link], inplace=True)
[Link]().sum()
Out[4]: age 0
workclass 1836
fnlwgt 0
education 0
education_num 0
marital_status 0
occupation 1843
relationship 0
race 0
sex 0
capital_gain 0
capital_loss 0
hours_per_week 0
native_country 583
income 0
dtype: int64
In [5]: df = [Link]()
[Link]
Out[5]: (30162, 15)
Exploratory Data Analysis (EDA)
1. Income class distribution
In [6]: [Link](figsize=(6,4))
[Link](x=df['income'])
[Link]("Income Distribution")
[Link]()
2. Education Levels
In [7]: [Link](figsize=(10,5))
df['education'].value_counts().head(10).plot(kind='bar')
[Link]("Top Education Levels")
[Link]()
In [8]: [Link](figsize=(10,7))
[Link](df[['age','education_num','capital_gain','capital_loss','hours_per_week']].corr(), ann
[Link]("Correlation Heatmap")
[Link]()
Preprocessing (Encoding + Scaling)
In [10]: X = [Link]("income", axis=1)
y = df["income"]
num_features = X.select_dtypes(include=['int64','float64']).columns
cat_features = X.select_dtypes(include=['object']).columns
preprocessor = ColumnTransformer([
('num', StandardScaler(), num_features),
('cat', OneHotEncoder(handle_unknown='ignore'), cat_features)
])
In [11]: X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42
)
X_train.shape, X_test.shape
Out[11]: ((24129, 14), (6033, 14))
Model 1 — Logistic Regression
In [12]: log_clf = Pipeline([
('prep', preprocessor),
('model', LogisticRegression(max_iter=200))
])
log_clf.fit(X_train, y_train)
log_pred = log_clf.predict(X_test)
In [13]: print("Accuracy:", accuracy_score(y_test, log_pred))
print("Precision:", precision_score(y_test, log_pred, pos_label='>50K'))
print("Recall:", recall_score(y_test, log_pred, pos_label='>50K'))
print("F1:", f1_score(y_test, log_pred, pos_label='>50K'))
Accuracy: 0.8534725675451682
Precision: 0.7588141025641025
Recall: 0.6189542483660131
F1: 0.681785457163427
Model 2 — Decision Tree
In [14]: tree_clf = Pipeline([
('prep', preprocessor),
('model', DecisionTreeClassifier(max_depth=10))
])
tree_clf.fit(X_train, y_train)
tree_pred = tree_clf.predict(X_test)
In [15]: print("Accuracy:", accuracy_score(y_test, tree_pred))
print("Precision:", precision_score(y_test, tree_pred, pos_label='>50K'))
print("Recall:", recall_score(y_test, tree_pred, pos_label='>50K'))
print("F1:", f1_score(y_test, tree_pred, pos_label='>50K'))
Accuracy: 0.8546328526437925
Precision: 0.7836663770634231
Recall: 0.5895424836601307
F1: 0.6728832525177173
Model 3 — Random Forest
In [16]: rf_clf = Pipeline([
('prep', preprocessor),
('model', RandomForestClassifier(n_estimators=200))
])
rf_clf.fit(X_train, y_train)
rf_pred = rf_clf.predict(X_test)
In [18]: print("Accuracy:", accuracy_score(y_test, rf_pred))
print("Precision:", precision_score(y_test, rf_pred, pos_label='>50K'))
print("Recall:", recall_score(y_test, rf_pred, pos_label='>50K'))
print("F1:", f1_score(y_test, rf_pred, pos_label='>50K'))
Accuracy: 0.8521465274324548
Precision: 0.7398496240601504
Recall: 0.6431372549019608
F1: 0.6881118881118881
In [19]: cm = confusion_matrix(y_test, rf_pred)
[Link](cm, annot=True, fmt="d", cmap="Blues")
[Link]("Random Forest Confusion Matrix")
[Link]()
In [20]: rf_prob = rf_clf.predict_proba(X_test)[:,1]
fpr, tpr, _ = roc_curve((y_test=='>50K').astype(int), rf_prob)
[Link](fpr, tpr)
[Link]("ROC Curve – Random Forest")
[Link]("False Positive Rate")
[Link]("True Positive Rate")
[Link]()
Model Comparison Summary
In [21]: results = {
"Model": ["Logistic Regression", "Decision Tree", "Random Forest"],
"Accuracy": [
accuracy_score(y_test, log_pred),
accuracy_score(y_test, tree_pred),
accuracy_score(y_test, rf_pred)
],
"F1 Score": [
f1_score(y_test, log_pred, pos_label='>50K'),
f1_score(y_test, tree_pred, pos_label='>50K'),
f1_score(y_test, rf_pred, pos_label='>50K')
]
}
[Link](results)
Out[21]: Model Accuracy F1 Score
0 Logistic Regression 0.853473 0.681785
1 Decision Tree 0.854633 0.672883
2 Random Forest 0.852147 0.688112