TYPES OF
MACHINE
LEARNING
DO I REALLY NEED IT ?
• In machine learning, supervised
learning, is where a model is
trained with input objects and the
desired output known or named,
such as human-made labels.
• In supervised learning, the input
and the output are known, and the
machine learns and creates
everything in-between
PRACTICAL EXAMPLES OF SUPERVISED LEARNING
Few practical examples of supervised machine learning across various industries:
•Fraud Detection in Banking: Utilizes supervised learning algorithms on historical
transaction data, training models with labeled datasets of legitimate and fraudulent
transactions to accurately predict fraud patterns.
•Parkinson Disease Prediction: Parkinson’s disease is a progressive disorder that affects
the nervous system and the parts of the body controlled by the nerves.
•Customer Churn Prediction: Uses supervised learning techniques to analyze historical
customer data, identifying features associated with churn rates to predict customer
retention effectively.
•Cancer cell classification: Implements supervised learning for cancer cells based on their
features, and identifying them if they are ‘malignant’ or ‘benign.
•Stock Price Prediction: Applies supervised learning to predict a signal that indicates
whether buying a particular stock will be helpful or not.
TYPES OF SUPERVISED LEARNING IN MACHINE
LEARNING
EXAMPLE 1
CLASSIFICATION
WHAT IS
CLASSIFICATION?
• The process of recognition and grouping
of objects and ideas into categories.
• classification is a form of “pattern
recognition”, applied to the training data
find the same pattern in future data sets.
• In classification, the machine uses the
dataset or observations provided (by the
human) to learn how to categorize new
observations into various classes.
• Features and Patterns (e.g., shapes, color,
size). are what the classifier uses to
determine relations and indicators (e.g.,
cats have whiskers, dogs have bigger
ears) for the various classes.
Types of Classification
TRAINING
DATA
VALIDATION DATA
TEST DATA
Advantages of Supervised Learning
The power of supervised learning lies in its ability to accurately predict patterns and make
data-driven decisions across a variety of applications. Here are some advantages
of supervised learning listed below:
•Supervised learning excels in accurately predicting patterns and making data-driven decisions.
•Labeled training data is crucial for enabling supervised learning models to learn input-output
relationships effectively.
•Supervised machine learning encompasses tasks such as supervised learning
classification and supervised learning regression.
•Applications include complex problems like image recognition and natural language processing.
•Established evaluation metrics (accuracy, precision, recall, F1-score) are essential for
assessing supervised learning model performance.
•Advantages of supervised learning include creating complex models for accurate predictions on
new data.
•Supervised learning requires substantial labeled training data, and its effectiveness hinges on
data quality and representativeness.
Disadvantages of Supervised
Learning
Despite the benefits of supervised learning methods, there are
notable disadvantages of supervised learning:
[Link]: Models can overfit training data, leading to poor
performance on new data due to capturing noise in supervised machine
learning.
[Link] Engineering : Extracting relevant features is crucial but can be
time-consuming and requires domain expertise in supervised learning
applications.
[Link] in Models: Bias in the training data may result in unfair predictions
in supervised learning algorithms.
[Link] on Labeled Data: Supervised learning relies heavily on
labeled training data, which can be costly and time-consuming to obtain,
posing a challenge for supervised learning techniques.
DECISION
TREES
• Decision trees are among the most
popular machine learning algorithms
given their intelligibility and simplicity.
• In Decision tree learning the goal is to create a
model that predicts the value of a target variable
based on several input variables.
• Algorithms for constructing decision
trees usually work top-down
Example 2:
1. Read and print the data set:
import pandas
df = pandas.read_csv("[Link]")
print(df)
2. Change string values into numerical
values:
d = {'UK': 0, 'USA': 1, 'N': 2}
df['Nationality'] =
df['Nationality'].map(d)
d = {'YES': 1, 'NO': 0}
df['Go'] = df['Go'].map(d)
print(df)
3. X is the feature columns, y is the
target column:
features =
['Age', 'Experience', 'Rank',
'Nationality']
X = df[features]
y = df['Go']
print(X)
print(y)
import pandas
from sklearn import tree
from [Link] import DecisionTreeClassifier
import [Link] as plt
df = pandas.read_csv("[Link]")
d = {'UK': 0, 'USA': 1, 'N': 2}
df['Nationality'] = df['Nationality'].map(d)
d = {'YES': 1, 'NO': 0}
df['Go'] = df['Go'].map(d)
features =
['Age', 'Experience', 'Rank', 'Nationality']
X = df[features]
y = df['Go']
dtree = DecisionTreeClassifier()
dtree = [Link](X, y)
tree.plot_tree(dtree, feature_names=features)
We can use the Decision Tree to predict new values.
Example: Should I go see a show starring a 40 years old American comedian, with 10 years of experience, and a comedy
ranking of 7? print([Link]([[40, 10, 7, 1]]))
import pandas
from sklearn import tree
from [Link] import DecisionTreeClassifier
df = pandas.read_csv("[Link]")
d = {'UK': 0, 'USA': 1, 'N': 2}
df['Nationality'] = df['Nationality'].map(d)
d = {'YES': 1, 'NO': 0}
df['Go'] = df['Go'].map(d)
features = ['Age', 'Experience', 'Rank', 'Nationality']
X = df[features]
y = df['Go']
dtree = DecisionTreeClassifier()
dtree = [Link](X, y)
print([Link]([[40, 10, 7, 1]]))
print("[1] means 'GO'")
print("[0] means 'NO'")
RANDOM
FOREST
• Random forests or random decision
forests is an ensemble learning method
for classification, regression and other
tasks that works by creating a multitude
of decision trees during training.
• For classification tasks, the output of the
random forest is the class selected by
most trees. For regression tasks, the
output is the average of the predictions
of the trees.
• Random forests are a way of averaging
multiple deep decision trees, trained on
different parts of the same training set,
with the goal of reducing the variance
Example 3: Predicting Loan Approval (Classification)
Problem Statement
A bank wants to predict whether a customer’s loan application should be approved
based on features like income, credit score, employment status, and loan amount.
Steps:
[Link] Collection:
•Features (Independent Variables): Income, Credit Score, Employment Status, Loan Amount
•Target (Dependent Variable): Approved (Yes/No)
[Link] the Random Forest:
•Multiple decision trees are trained on random subsets of data.
•Each tree predicts independently.
[Link]:
•New applicant: Income = $80k, Credit Score = 700, Employed = Yes, Loan Amount = $50k
•Trees predict: [Yes, No, Yes, Yes, Yes]
•Final Prediction: Yes (Majority Voting)
Step 1: Import Required Libraries
from [Link] import RandomForestClassifier
from sklearn.model_selection import train_test_split
import pandas as pd
import warnings
Step 2: Sample Data
data = {
'Income': [50, 80, 30, 90, 60],
'Credit_Score': [600, 700, 500, 800, 650],
'Employed': [1, 1, 0, 1, 1],
'Loan_Amount': [20, 50, 10, 60, 30],
'Approved': [0, 1, 0, 1, 1] # Fixed typo in column name (was 'Approved')
}
df = [Link](data)
Step 3: Features and Target
X = [Link]('Approved', axis=1)
y = df['Approved']
Step 4: Train-Test Split
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2, random_state=42)
Step 5: Train Random Forest
model = RandomForestClassifier(n_estimators=100)
[Link](X_train, y_train)
Step 6: Correct way to make a prediction (preserving feature names)
new_applicant = [Link]({
'Income': [80],
'Credit_Score': [700],
'Employed': [1],
'Loan_Amount': [50]
})
prediction = [Link](new_applicant)
print("Loan Approved?", "Yes" if prediction[0] == 1 else "No")
COMPARISON BETWEEN DECISION TREE AND
RANDOM FOREST
COMPARISON BETWEEN
comparison of various types of classification algorithms
REGRESSION
WHAT IS
REGRESSION?
• The process of using machine learning to finds
correlations between dependent and independent
variables( is used when both the input and the
predicted data are numerical.)
• Regression algorithms work on finding the
mapping function so we can understand the
numerical input “x” on the continuous output “y.”
• Regression algorithms plot the line or curve that
best fits between the data.
• These best fit lines can be represented by equations,
which allows us to predict the value of future
inputs.
• House prices, market trends, weather patterns, oil
and gas prices are all applications of Regression
TRAINING
DATA
VALIDATION
DATA
TEST DATA
Example 4: Predicting House Prices (Regression)
Problem Statement :
A real estate company wants to predict house prices based on features like:
•Size (sq. ft.)
•Bedrooms
•Location (1=Urban, 0=Rural)
•Age of House (years)
STEPS:
[Link] Collection:
•Features (Independent Variables): Size, Bedrooms, Location, Age
•Target (Dependent Variable): Price ($)
2. Training the Random Forest Regressor:
Multiple decision trees predict house prices independently.
•Final prediction = Average of all tree predictions
3. Prediction
•New house: Size = 1800, Bedrooms = 3, Location = 1 (Urban), Age = 5
•Individual tree predictions: Each of the 100 trees (n_estimators=100) in your Random Forest
makes its own prediction for the house price.
• Here’s a simulated example of what the first 10 trees might have predicted:
Step 1: Import Required Libraries
from [Link] import RandomForestRegressor
from sklearn.model_selection import train_test_split
from [Link] import mean_squared_error, r2_score
import pandas as pd
import numpy as np
Step 2: Sample Data
data = {
'Size': [1200, 1500, 1800, 2000, 2400, 1600, 1900, 2100, 2300, 2500],
'Bedrooms': [2, 3, 3, 4, 4, 2, 3, 4, 3, 4],
'Location': [0, 1, 1, 0, 1, 0, 1, 0, 1, 1], # 0=Rural, 1=Urban
'Age': [10, 5, 8, 15, 3, 7, 9, 12, 4, 2],
'Price': [250000, 320000, 350000, 400000, 450000, 310000, 370000, 410000, 440000, 470000]
}
df = [Link](data)
Step 3: Features & Target
X = [Link]('Price', axis=1)
y = df['Price']
Step 4: Train-Test Split – now with more data and proper test size
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
Step 5: Train Random Forest Regresso
model = RandomForestRegressor(n_estimators=100, random_state=42)
[Link](X_train, y_train)
Step 6: Make predictions (preserving feature names)
new_house = [Link]({
'Size': [1800],
'Bedrooms': [3],
'Location': [1],
'Age': [5]
})
predicted_price = [Link](new_house)
print(f"Predicted House Price: ${predicted_price[0]:,.2f}")
Step 7: Evaluate Model
y_pred = [Link](X_test)
mse = mean_squared_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)
print(f"MSE: {mse:,.2f}, R² Score: {r2:.2f}")
Step 8: Feature Importance
importances = model.feature_importances_
features = [Link]
print("\nFeature Importances:")
for feature, importance in zip(features, importances):
print(f"{feature}: {importance:.2f}")
OUTPUT
LINEAR
REGRESSION
• Linear regression estimates the linear
relationship between a dependent
variable (Y) and one or more
independent variable (X).
• Linear regression maps the data
points to the most optimized linear
functions that can be used for
prediction on new datasets.
• Normal Linear regression models
have a number of assumptions about
the fitted data, numerous extensions
of linear regression have been
developed, which allow some or all
the assumptions underlying the basic
model to be relaxed.
POLYNOMIAL
REGRESSION
• polynomial regression estimates the
relationship between a dependent
variable (Y) and one or more
independent variable (X) as a model
of the nth degree of polynomial in X.
• Polynomial regression is considered
to be a special case of linear
regression.
• It is often difficult to interpret the
individual coefficients in a
polynomial regression fit, since the
underlying monomials can be highly
correlated.
EXAMPLE 5: LINEAR AND
POLYNOMIAL REGRESSION
SUPPORT
VECTOR
MACHINES
• SVMs are helpful in text and
hypertext categorization
• The popularity of SVMs is
flexibility in being applied to a
wide variety of tasks.
• SVMs have better predictive
performance than other linear
models, such as logistic
regression and linear
regression.
EXAMPLE 6: SVM FOR CLASSIFICATION
[Link] Goal:
•To classify iris flowers into one of three species (setosa, versicolor, virginica) based on their sepal
and petal measurements.
[Link] Steps & Objectives:
•Load Data: Use the Iris dataset (features: sepal length/width, petal length/width; target: species
label).
•Train-Test Split: Divide data into 70% training and 30% testing to evaluate generalization.
•Model Selection: Use SVC (Support Vector Classifier) with an RBF kernel to handle non-linear
boundaries.
•Hyperparameters:
•C=1.0: Balances margin width vs. classification errors.
•gamma='scale': Controls kernel flexibility (auto-adjusted based on data variance).
•Evaluation:
•Accuracy: Measure overall correctness of predictions.
•Classification Report: Show precision, recall, and F1-score per class to assess per-species
performance.
Step 1: Import the required libraries
from sklearn import datasets
from sklearn.model_selection import train_test_split
from [Link] import SVC
from [Link] import accuracy_score,
classification_report
Step 2: Load iris dataset
iris = datasets.load_iris()
X = [Link]
y = [Link]
Step 3: Split data
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.3, random_state=42)
Step 4: Create SVM classifier
svm_classifier = SVC(kernel='rbf', C=1.0, gamma='scale') # RBF kernel
svm_classifier.fit(X_train, y_train)
Step 5: Predict
y_pred = svm_classifier.predict(X_test)
Step 6: Evaluate
print("Accuracy:", accuracy_score(y_test, y_pred))
print("\nClassification Report:\n", classification_report(y_test, y_pred))
Example 7: SVM for Regression with Python and Scikit Learn
We are using the Diabetes Dataset and achieving the following objective to predict the
progression of diabetes:
[Link] Goal:
•To predict the progression of diabetes (a continuous value) based on patient features (age, BMI,
blood pressure, etc.).
[Link] Steps & Objectives:
•Load Data: Use the Diabetes dataset (features: 10 physiological variables; target: disease
progression score).
•Feature Scaling: Standardize features (critical for SVM regression to ensure equal weighting).
•Train-Test Split: 70-30 split for reliable performance estimation.
•Model Selection: Use SVR (Support Vector Regressor) with a linear kernel (assuming linear
relationships).
•Hyperparameters:
•C=1.0: Penalty for deviations beyond the margin.
•epsilon=0.1: Defines the “tolerance margin” where errors are ignored (larger = more tolerant).
•Evaluation:
•Mean Squared Error (MSE): Quantifies average prediction error (lower = better).
•R² Score: Measures how well the model explains variance in the target (1 = perfect fit).
Step 1: Import the required libraries
from sklearn import datasets
from sklearn.model_selection import train_test_split
from [Link] import SVR
from [Link] import mean_squared_error, r2_score
from [Link] import StandardScaler
Step 2: Load diabetes dataset
diabetes = datasets.load_diabetes()
X = [Link]
y = [Link]
Step 3: Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
Step 4: Scale features
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = [Link](X_test)
Step 5: Create SVM regressor
svm_regressor = SVR(kernel='linear', C=1.0, epsilon=0.1) # Linear kernel
svm_regressor.fit(X_train, y_train)
Step 6: Predict
y_pred = svm_regressor.predict(X_test)
Step 7: Evaluate
print("Mean Squared Error:", mean_squared_error(y_test, y_pred))
print("R2 Score:", r2_score(y_test, y_pred))
OUTPUT
COMPARING BOTH EXAMPLES
EXAMPLE 8: SVM FOR CLASSIFICATION WITH PLOT
Objective: Visualize the decision boundary for iris classification (2D projection for
simplicity).
Step 1: Import the required libraries
import numpy as np
import [Link] as plt
from sklearn import datasets
from [Link] import SVC
from [Link] import PCA
Step 2: Load iris data
iris = datasets.load_iris()
X = [Link][:, :2] # Use only 2 features (sepal length/width) for visualization
y = [Link]
Step 3: Train SVM
svm = SVC(kernel='linear', C=1.0)
[Link](X, y)
Step 4: Create a meshgrid for decision boundary
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
xx, yy = [Link]([Link](x_min, x_max, 0.02),
[Link](y_min, y_max, 0.02))
Z = [Link](np.c_[[Link](), [Link]()])
Z = [Link]([Link])
Step 5: Plot
[Link](figsize=(10, 6))
[Link](xx, yy, Z, alpha=0.8, cmap=[Link])
[Link](X[:, 0], X[:, 1], c=y, edgecolors='k', cmap=[Link])
[Link]('Sepal Length (cm)')
[Link]('Sepal Width (cm)')
[Link]('SVM Decision Boundary (Iris Classification)')
[Link](label='Class')
[Link]()
OUTPUT
EXAMPLE 8: SVM FOR REGRESSION WITH PLOT
Objective: Visualize how SVR fits a continuous target (diabetes progression).
Step 1: Import required libraries
import numpy as np
import [Link] as plt
from sklearn import datasets
from [Link] import SVR
from [Link] import StandardScaler
Step 2: Load diabetes data
diabetes = datasets.load_diabetes()
X = [Link][:, [Link], 2] # Use only BMI feature
(column index 2)
y = [Link]
Step 3: Scale data (critical for SVR)
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
Step 4: Train SVR model
svr = SVR(kernel='linear', C=1.0, epsilon=0.2)
[Link](X_scaled, y)
Step 5: Predict
y_pred = [Link](X_scaled)
Step 6: Plot results
[Link](figsize=(10, 6))
[Link](X_scaled, y, color='blue', label='Actual Data')
[Link](X_scaled, y_pred, color='red', linewidth=2, label='SVR Prediction')
plt.fill_between(X_scaled.flatten(),
y_pred - [Link],
y_pred + [Link],
color='gray', alpha=0.2, label='Epsilon Margin')
[Link]('BMI (Standardized)')
[Link]('Disease Progression')
[Link]('Support Vector Regression (SVR) on Diabetes Dataset')
[Link]()
[Link]()
OUTPUT
COMPARISON BETWEEN DIFFERENT ALGORITHMS OF REGRESSION
LIBRARIES AND PACKAGES
•To understand machine learning, you need to have basic knowledge of
Python programming. In addition, there are a number of libraries and
packages generally used in performing various machine learning tasks
as listed below:
• numpy - is used for its N-dimensional array objects
• pandas – is a data analysis library that includes dataframes
• matplotlib – is 2D plotting library for creating graphs and plots
• scikit-learn - the algorithms used for data analysis and data mining tasks
• seaborn – a data visualization library based on matplotlib
ML IN PYTHON
•Machine learning in Python is implemented using various libraries and
frameworks that simplify the process of building and deploying machine
learning models. Here’s a step-by-step overview of how machine learning is
typically implemented in Python:
### 1. **Set Up Your Environment**
•- **Install Python**: Ensure Python is installed on your system.
•- **Install Libraries**: Use pip to install essential libraries. Common libraries
include:
• ```bash
• pip install numpy pandas matplotlib scikit-learn tensorflow keras
• ### 2. **Import Libraries**
• ```python
• import numpy as np
• import pandas as pd
• import [Link] as plt
• from sklearn.model_selection import train_test_split
• from [Link] import StandardScaler
• from [Link] import accuracy_score
• from sklearn.linear_model import LogisticRegression
• ```
•### 3. **Load and Prepare Data**
•- **Load Data**: Use libraries like pandas to load data.
• ```python
• data = pd.read_csv(‘[Link]')
• ```
•- **Explore Data**: Check the structure and summary statistics.
• ```python
• print([Link]())
• print([Link]())
• ```
•- **Preprocess Data**: Handle missing values, encode categorical variables, and split data.
• ```python
• X = [Link]('target', axis=1)
• y = data['target']
• X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
• ```
• ### 4. **Feature Scaling**
• - **Standardize Features**: Normalize or scale the features for better performance.
• ```python
• scaler = StandardScaler()
• X_train_scaled = scaler.fit_transform(X_train)
• X_test_scaled = [Link](X_test)
• ### 5. **Build the Model**
• - **Select Model**: Choose an algorithm, e.g., Logistic
Regression.
• ```python
• model = LogisticRegression()
• ### 6. **Train the Model**
• - **Fit Model**: Train the model on the training data.
• ```python
• [Link](X_train_scaled, y_train)
• ```
•### 7. **Evaluate the Model**
•- **Make Predictions**: Use the model to predict on the test data.
• ```python
• y_pred = [Link](X_test_scaled)
• ```
•- **Evaluate Performance**: Check the accuracy and other metrics.
• ```python
• accuracy = accuracy_score(y_test, y_pred)
• print(f'Accuracy: {accuracy * 100:.2f}%')
• ```
•### 8. **Improve the Model**
•- **Hyperparameter Tuning**: Use GridSearchCV or RandomizedSearchCV to optimize
hyperparameters.
• ```python
• from sklearn.model_selection import GridSearchCV
• param_grid = {'C': [0.1, 1, 10, 100]}
• grid_search = GridSearchCV(LogisticRegression(), param_grid, cv=5)
• grid_search.fit(X_train_scaled, y_train)
• print(f'Best parameters: {grid_search.best_params_}')
• ```
•### 9. **Deploy the Model**
•- **Save the Model**: Use joblib or pickle to save the trained model.
• ```python
• import joblib
• [Link](model, '[Link]')
• ```
•- **Load and Use the Model**: Load the model to make predictions on
new data.
• ```python
• model = [Link]('[Link]')
• predictions = [Link](new_data)
• ```
• ### Additional Tools and Libraries
• - **TensorFlow/Keras**: For deep learning.
• - **PyTorch**: Another popular deep learning library.
• - **XGBoost, LightGBM, CatBoost**: Libraries for gradient boosting.
• This framework provides a general guide to building and deploying machine learning models
using Python. Each step can be customized based on the specific problem and dataset you are
working with.