Using Logistic Regression from Scikit-learn
1. Import Required Libraries
• Scikit-learn provides Logistic Regression in the linear_model module
• NumPy and pandas are used for data handling
import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from [Link] import accuracy_score
2. Load and Prepare the Dataset
• Load the dataset
• Separate features (X) and target (y)
• Convert data into numerical format if needed
X = data[['feature1', 'feature2']]
y = data['target']
3. Split Data into Training and Testing Sets
• Data is divided to evaluate model performance
• Common split: 80% training, 20% testing
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
4. Create and Train Logistic Regression Model
• Create Logistic Regression object
• Train model using training data
model = LogisticRegression()
[Link](X_train, y_train)
5. Make Predictions
• Predict class labels for test data
y_pred = [Link](X_test)
6. Evaluate the Model
• Check accuracy of the model
• Other metrics like precision and recall can also be used
accuracy = accuracy_score(y_test, y_pred)
print("Accuracy:", accuracy)
Using Decision Tree for Continuous-Valued Features
Decision Trees in Scikit-learn can handle both continuous and categorical features. For continuous
features, the tree finds the best threshold to split the data at each node.
1. Import Libraries
import pandas as pd
from sklearn.model_selection import train_test_split
from [Link] import DecisionTreeClassifier
from [Link] import accuracy_score
2. Load and Prepare Dataset
• Suppose we have a dataset with continuous features like age, income, or scores.
# Example dataset
data = [Link]({
'Age': [22, 25, 47, 52, 46, 56, 55, 60],
'Income': [20000, 25000, 50000, 60000, 52000, 70000, 68000, 72000],
'Buy': [0, 0, 1, 1, 1, 1, 1, 1] # Target variable
})
# Features and target
X = data[['Age', 'Income']] # continuous features
y = data['Buy']
3. Split Dataset
X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.25, random_state=42)
4. Create and Train Decision Tree
• Scikit-learn automatically handles continuous features.
• criterion='gini' or 'entropy' can be used for splitting.
# Create Decision Tree classifier
clf = DecisionTreeClassifier(criterion='entropy', random_state=42)
# Train the model
[Link](X_train, y_train)
5. Make Predictions
y_pred = [Link](X_test)
print("Predicted Labels:", y_pred)
6. Evaluate Model
accuracy = accuracy_score(y_test, y_pred)
print("Accuracy:", accuracy)
Random Forest in Scikit-learn
Random Forest is an ensemble learning algorithm that combines multiple decision trees to improve
accuracy and reduce overfitting. It can be used for classification and regression problems.
1. Import Libraries
import pandas as pd
from sklearn.model_selection import train_test_split
from [Link] import RandomForestClassifier
from [Link] import accuracy_score
2. Load and Prepare Dataset
• Example dataset with features and target variable:
# Sample dataset
data = [Link]({
'Age': [22, 25, 47, 52, 46, 56, 55, 60],
'Income': [20000, 25000, 50000, 60000, 52000, 70000, 68000, 72000],
'Buy': [0, 0, 1, 1, 1, 1, 1, 1] # Target variable
})
# Features and target
X = data[['Age', 'Income']]
y = data['Buy']
3. Split Dataset
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)
4. Create and Train Random Forest
• n_estimators = number of trees in the forest
• criterion = measure of split quality ('gini' or 'entropy')
# Create Random Forest classifier
rf_model = RandomForestClassifier(n_estimators=100, criterion='entropy', random_state=42)
# Train the model
rf_model.fit(X_train, y_train)
5. Make Predictions
y_pred = rf_model.predict(X_test)
print("Predicted Labels:", y_pred)
6. Evaluate Model
accuracy = accuracy_score(y_test, y_pred)
print("Accuracy:", accuracy)
. Using KNN in Scikit-learn (Step by Step)
Step 1: Import Libraries
import pandas as pd
from sklearn.model_selection import train_test_split
from [Link] import KNeighborsClassifier
from [Link] import accuracy_score
from [Link] import StandardScaler
Step 2: Load and Prepare Dataset
# Sample dataset with continuous features
data = [Link]({
'Age': [22, 25, 47, 52, 46, 56, 55, 60],
'Income': [20000, 25000, 50000, 60000, 52000, 70000, 68000, 72000],
'Buy': [0, 0, 1, 1, 1, 1, 1, 1] # Target variable
})
X = data[['Age', 'Income']]
y = data['Buy']
Step 3: Feature Scaling
• KNN is distance-based, so scaling features is important.
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
Step 4: Split Dataset
X_train, X_test, y_train, y_test = train_test_split(X_scaled, y, test_size=0.25, random_state=42)
Step 5: Create KNN Model
knn = KNeighborsClassifier(n_neighbors=3, metric='minkowski') # K=3, Euclidean distance
[Link](X_train, y_train)
Step 6: Make Predictions
y_pred = [Link](X_test)
print("Predicted Labels:", y_pred)
Step 7: Evaluate Model
accuracy = accuracy_score(y_test, y_pred)
print("Accuracy:", accuracy)
Using SVM in Scikit-learn (Step by Step)
Step 1: Import Libraries
import pandas as pd
from sklearn.model_selection import train_test_split
from [Link] import SVC
from [Link] import accuracy_score, confusion_matrix
from [Link] import StandardScaler
Step 2: Load and Prepare Dataset
# Sample dataset
data = [Link]({
'Age': [22, 25, 47, 52, 46, 56, 55, 60],
'Income': [20000, 25000, 50000, 60000, 52000, 70000, 68000, 72000],
'Buy': [0, 0, 1, 1, 1, 1, 1, 1] # Target variable
})
X = data[['Age', 'Income']]
y = data['Buy']
Step 3: Feature Scaling
• SVM is distance-based, so scaling features improves performance.
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
Step 4: Split Dataset
X_train, X_test, y_train, y_test = train_test_split(X_scaled, y, test_size=0.25, random_state=42)
Step 5: Create SVM Model
• kernel can be 'linear', 'poly', 'rbf', 'sigmoid'
svm_model = SVC(kernel='linear', C=1.0, random_state=42)
svm_model.fit(X_train, y_train)
Step 6: Make Predictions
y_pred = svm_model.predict(X_test)
print("Predicted Labels:", y_pred)
Step 7: Evaluate Model
accuracy = accuracy_score(y_test, y_pred)
conf_matrix = confusion_matrix(y_test, y_pred)
print("Accuracy:", accuracy)
print("Confusion Matrix:\n", conf_matrix)
Explain the implementation of SVM on Iris Using Sklearn?
Step 1: Import Libraries
# Data handling
import pandas as pd
from [Link] import load_iris
# Train-test split and scaling
from sklearn.model_selection import train_test_split
from [Link] import StandardScaler
# SVM model
from [Link] import SVC
# Evaluation metrics
from [Link] import accuracy_score, confusion_matrix, classification_report
Step 2: Load the Iris Dataset
# Load dataset
iris = load_iris()
# Features and target
X = [Link]
y = [Link]
# Feature names
print(iris.feature_names)
# Target names
print(iris.target_names)
Step 3: Split the Dataset
• Split data into training and testing sets (usually 70–30 or 80–20 split)
X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.3, random_state=42)
Step 4: Feature Scaling
• SVM is distance-based, so scaling features improves accuracy
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = [Link](X_test)
Step 5: Create and Train SVM Model
• For multi-class classification, SVM automatically uses one-vs-one approach
# Create SVM classifier with linear kernel
svm_model = SVC(kernel='linear', C=1.0, random_state=42)
# Train the model
svm_model.fit(X_train, y_train)
Step 6: Make Predictions
y_pred = svm_model.predict(X_test)
print("Predicted Labels:", y_pred)
Step 7: Evaluate the Model
# Accuracy
accuracy = accuracy_score(y_test, y_pred)
print("Accuracy:", accuracy)
# Confusion matrix
conf_matrix = confusion_matrix(y_test, y_pred)
print("Confusion Matrix:\n", conf_matrix)
# Classification report
print("Classification Report:\n", classification_report(y_test, y_pred))