Machine Learning: Logistic Regression
By,
Ayush Sinha, 23030124161, Div-E
Definition:
Logistic regression is a supervised machine learning algorithm that is used to solve binary
classification problems. It predicts the probability that a data instance belongs to a
particular class by modelling the relationship between input features and the output using
the logistic or sigmoid function. The output probability is always between 0 and 1, making it
suitable for classification tasks like whether a customer will churn or stay.
Problem Statement:
In this project, we aim to predict customer churn, which is whether a customer will leave or
continue using the service. The dataset contains various customer attributes such as age,
credit score, geography, and account balance. The challenge is to build a model that can use
these features to estimate the likelihood of churn and identify key factors influencing
customer decisions
Code:
# Step 1: Import Libraries
import pandas as pd
import numpy as np
import [Link] as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from [Link] import LabelEncoder, StandardScaler
from sklearn.linear_model import LogisticRegression
from [Link] import (
accuracy_score, confusion_matrix, classification_report, roc_auc_score, roc_curve
)
# Step 2: Load Dataset
file_path = "/[Link]" # Path to uploaded file
df = pd.read_csv(file_path)
print("Dataset loaded successfully!")
print("Columns:", [Link]())
print("First few rows:\n", [Link]())
# Step 3: Exploratory Data Analysis (EDA)
print("\nChecking for missing values:")
print([Link]().sum())
print("\nDistribution of target variable 'Exited':")
print(df['Exited'].value_counts())
print("\nPercentage distribution:")
print(df['Exited'].value_counts(normalize=True) * 100)
# Step 4: Data Cleaning & Preprocessing
# Drop unnecessary columns
df = [Link](['RowNumber', 'CustomerId', 'Surname'], axis=1)
# Identify categorical columns
cat_cols = df.select_dtypes(include='object').[Link]()
# Clean categorical columns if necessary
for col in cat_cols:
df[col] = df[col].astype(str).[Link]()
# Label encode binary categorical columns
binary_cols = [col for col in cat_cols if df[col].nunique() == 2]
for col in binary_cols:
df[col] = LabelEncoder().fit_transform(df[col])
print(f"Label encoded {col}")
# One-hot encode remaining categorical columns
multi_cat_cols = [col for col in cat_cols if col not in binary_cols]
df = pd.get_dummies(df, columns=multi_cat_cols)
print(f"One-hot encoded columns: {multi_cat_cols}")
# Feature scaling
scaler = StandardScaler()
X = [Link]('Exited', axis=1)
y = df['Exited']
X_scaled = scaler.fit_transform(X)
print("\nFeatures scaled successfully!")
# Step 5: Train-Test Split
X_train, X_test, y_train, y_test = train_test_split(
X_scaled, y, test_size=0.2, stratify=y, random_state=42
)
print("Data split into training and testing sets.")
# Step 6: Train Logistic Regression Model
model = LogisticRegression(max_iter=1000)
[Link](X_train, y_train)
print("Logistic Regression model trained.")
# Step 7: Model Evaluation
y_pred = [Link](X_test)
y_prob = model.predict_proba(X_test)[:, 1]
print("\nModel Evaluation Metrics:")
print("Accuracy:", accuracy_score(y_test, y_pred))
print("\nClassification Report:\n", classification_report(y_test, y_pred))
print("ROC-AUC Score:", roc_auc_score(y_test, y_prob))
# Step 8: Plot Confusion Matrix
cm = confusion_matrix(y_test, y_pred)
[Link](figsize=(6,5))
[Link](cm, annot=True, fmt="d", cmap="Blues")
[Link]("Confusion Matrix")
[Link]("Predicted")
[Link]("Actual")
[Link]()
# Step 9: Plot ROC Curve
fpr, tpr, thresholds = roc_curve(y_test, y_prob)
[Link](figsize=(6,5))
[Link](fpr, tpr, marker='.', label='Logistic Regression')
[Link]([0, 1], [0, 1], linestyle='--', label='No Skill')
[Link]("False Positive Rate")
[Link]("True Positive Rate")
[Link]("ROC Curve")
[Link]()
[Link]()
Conclusion:
By applying logistic regression, we were able to model customer churn effectively. The
algorithm provided not only predictions but also interpretable coefficients that help us
understand the impact of each feature. The model evaluation using accuracy, ROC-AUC
score, and confusion matrix confirmed that the model performs well. Additionally, feature
importance analysis revealed which factors most influence churn, helping businesses take
targeted actions to retain customers.