DATA SCIENCE IN PYTHON

Unsupervised
Learning
With Expert Data Science Instructor Alice Zhao

*Copyright Maven Analytics, LLC

 ABOUT THIS SERIES

This is Part 4 of a 5-Part series designed to take you through several applications of data science
using Python, including data prep & EDA, regression, classification, unsupervised learning & NLP

PART 1 PART 2 PART 3 PART 4 PART 5

Data Prep & EDA Regression Classification Unsupervised Natural Language
Learning Processing

*Copyright Maven Analytics, LLC

 COURSE STRUCTURE

This is a project-based course for students looking for a practical, hands-on approach to
learning data science and applying unsupervised learning models with Python

Additional resources include:

Downloadable PDF to serve as a helpful reference when you’re offline or on the go

Quizzes & Assignments to test and reinforce key concepts, with step-by-step solutions

Interactive demos to keep you engaged and apply your skills throughout the course

*Copyright Maven Analytics, LLC

 COURSE OUTLINE

Introduce the fields of data science and machine learning, review

1 Intro to Data Science essential skills, and introduce each phase of the data science workflow

Review the basics of unsupervised learning, including key concepts, types of

2 Unsupervised Learning 101 techniques and applications, and its place in the data science workflow

Recap the data prep & EDA steps required to apply unsupervised learning
3 Pre-Modeling Data Prep models, including restructuring data, engineering features, and more

Apply several clustering techniques in Python and learn to interpret the

4 Clustering results using metrics, visualizations, and domain expertise

Apply three clustering techniques on a data set, interpret the outputs,

5 PROJECT: Clustering and compare the results to make a final recommendation

Understand where anomaly detection fits in the data science workflow,

6 Anomaly Detection and apply several anomaly detection techniques in Python

*Copyright Maven Analytics, LLC

 COURSE OUTLINE

Understand where dimensionality reduction fits in the data science

7 Dimensionality Reduction workflow, apply several techniques in Python, and interpret the results

Recognize the variety of approaches for creating recommenders and the

8 Recommenders required data structures, then apply multiple techniques in Python

Use matrix factorization to create a recommender, interpret the results,

9 PROJECT: Recommenders and make recommendations

Review and compare unsupervised learning algorithms, including their

10 Unsupervised Learning Review strengths and weaknesses, as well as common applications and use cases

Apply multiple unsupervised learning techniques including clustering and

11 FINAL PROJECT dimensionality reduction on a single data set to make recommendations

*Copyright Maven Analytics, LLC

 INTRODUCING THE COURSE PROJECT

THE You’ve just been hired as an Associate Data Scientist for the HR Analytics team at a
SITUATION medium-sized software company that’s trying to increase employee retention

You have access to the company’s employee database, including demographic info,
THE performance history, tenure at the company, attrition, and more
ASSIGNMENT Your task is to use unsupervised learning techniques to define employee
segments and make recommendations to increase retention within each one

1. Prepare the data for unsupervised modeling

THE
2. Segment the employees using clustering
OBJECTIVES
3. Visualize the clusters using dimensionality reduction
4. Explore the employees within each cluster
5. Recommend next steps to increase retention

*Copyright Maven Analytics, LLC

 SETTING EXPECTATIONS

This course covers clustering & dimensionality reduction techniques

• We will review both the theory and application of unsupervised learning models, including K-Means
Clustering, Hierarchical Clustering, DBSCAN, Principal Component Analysis (PCA), and t-SNE

We’ll also dive into popular applications: anomaly detection & recommenders
• Anomaly detection and recommender techniques include statistical and supervised learning approaches as
well, but we will be focusing on popular unsupervised learning approaches in this course

We’ll use Jupyter Notebook as our primary coding environment

• Jupyter Notebook is free to use, and the industry standard for conducting data analysis with Python

You do NOT need to be a Python expert to take this course

• It is strongly recommended that you complete the first course in this series, Data Prep & EDA, but we will
teach the relevant math and Python code for applying unsupervised learning models in this course

*Copyright Maven Analytics, LLC

INSTALLATION & SETUP

*Copyright Maven Analytics, LLC

 INSTALLATION & SETUP

In this section we’ll install Anaconda and introduce Jupyter Notebook, a user-friendly
coding environment where we’ll be coding in Python

TOPICS WE’LL COVER: GOALS FOR THIS SECTION:

• Install Anaconda and launch Jupyter Notebook

Installing Anaconda Launching Jupyter
• Get comfortable with the Jupyter Notebook
environment and interface

*Copyright Maven Analytics, LLC

 INSTALL ANACONDA (MAC)

1) Go to [Link]/products/distribution and click

3) Follow the installation steps

(default settings are OK)
Installing
Anaconda

Launching
Jupyter

2) Launch the downloaded Anaconda pkg file

*Copyright Maven Analytics, LLC

 INSTALL ANACONDA (PC)

1) Go to [Link]/products/distribution and click

3) Follow the installation steps

(default settings are OK)
Installing
Anaconda

Launching
Jupyter

2) Launch the downloaded Anaconda exe file

*Copyright Maven Analytics, LLC

 LAUNCHING JUPYTER

1) Launch Anaconda Navigator 2) Find Jupyter Notebook and click

Installing
Anaconda

Launching
Jupyter

*Copyright Maven Analytics, LLC

 YOUR FIRST JUPYTER NOTEBOOK

1) Once inside the Jupyter interface, create a folder to store your notebooks for the course

Installing
Anaconda

Launching
Jupyter

NOTE: You can rename your folder by clicking “Rename” in the top left corner

2) Open your new coursework folder and launch your first Jupyter notebook!

NOTE: You can rename your notebook by clicking on the title at the top of the screen

*Copyright Maven Analytics, LLC

 THE NOTEBOOK SERVER

NOTE: When you launch a Jupyter notebook, a terminal window may pop up as
well; this is called a notebook server, and it powers the notebook interface

Installing
Anaconda

Launching
Jupyter

If you close the server window,

your notebooks will not run!

Depending on your OS, and method

of launching Jupyter, one may not
open – as long as you can run your
notebooks, don’t worry!

*Copyright Maven Analytics, LLC

INTRO TO DATA SCIENCE

*Copyright Maven Analytics, LLC

 INTRO TO DATA SCIENCE

In this section we’ll introduce the field of data science, discuss how it compares to
other data fields, and walk through each phase of the data science workflow

TOPICS WE’LL COVER: GOALS FOR THIS SECTION:

• Compare data science and machine learning with

What is Data Science? Essential Skills other common data analytics fields
• Introduce supervised and unsupervised learning,
Machine Learning Data Science Workflow and examples of each technique
• Review the machine learning landscape and
commonly used algorithms
• Discuss essential skills, and review each phase of
the data science workflow

*Copyright Maven Analytics, LLC

 WHAT IS DATA SCIENCE?

Data science is about using data to make smart decisions

What is Data
Science?
Wait, isn’t that business
predictive
data analysis
intelligence
analytics ?
Essential Skills

Yes! The differences lie in the types of problems you solve, and tools and
Machine Learning
techniques you use to solve them:
Data Science
Workflow
What happened? What’s going to happen?
• Descriptive Analytics • Predictive Analytics
• Data Analysis • Data Mining
• Business Intelligence • Data Science

*Copyright Maven Analytics, LLC

 DATA SCIENCE SKILL SET

Data science requires a blend of coding, math, and domain expertise

What is Data
Science?

The key is in applying these along

Essential Skills with soft skills like:
Machine
Learning • Communication
Coding Math
Machine Learning • Problem solving
Data • Curiosity & creativity
Science
Data Science • Grit
Workflow Danger Traditional
Zone! Research • Googling prowess

Data scientists & analysts approach problem

Domain solving in similar ways, but data scientists will
Expertise often work with larger, more complex data sets
and utilize advanced algorithms

*Copyright Maven Analytics, LLC

 WHAT IS MACHINE LEARNING?

Data scientists use machine learning algorithms to enable computers to learn

and make decisions from data
What is Data
Science?
Machine learning algorithms fall into two broad categories:

Essential Skills Supervised Learning Unsupervised Learning

Using historical data to predict the future Finding patterns and relationships in data

Machine Learning

Data Science
Workflow What will house prices look like How can I segment my
for the next 12 months? customers?

How can I flag suspicious emails Which TV shows should I

as spam? recommend to each user?

*Copyright Maven Analytics, LLC

 COMMON ALGORITHMS

These are some of the most common machine learning algorithms that data
scientists use in practice
What is Data
Science?

MACHINE LEARNING
Essential Skills

Supervised Learning Unsupervised Learning Another category of machine

Machine Learning
learning algorithms is called
K-Means Clustering
reinforcement learning, which
Hierarchical Clustering is commonly used in robotics
Data Science Regression Classification and gaming
Workflow DBSCAN
Fields like deep learning and
Linear Regression KNN Isolation Forests natural language processing
Regularized Regression Logistic Regression Principal Component Analysis
utilize both supervised and
unsupervised learning
Time Series Tree-Based Models Singular Value Decomposition techniques
Naïve Bayes (NLP) Topic Modeling (NLP)

Support vector machines, neural networks, Factor analysis, association

deep learning, etc. rule mining, etc.

*Copyright Maven Analytics, LLC

 DATA SCIENCE WORKFLOW

The data science workflow consists of scoping the project, gathering, cleaning
What is Data
and exploring the data, applying models, and sharing insights with end users
Science?

1 2 3 4 5 6
Essential Skills

Machine Learning

Data Science
Workflow
Scoping a Gathering Cleaning Exploring Modeling Sharing
Project Data Data Data Data Insights

This is not a linear process! You’ll likely go back to further gather, clean and explore your data

*Copyright Maven Analytics, LLC

 STEP 1: SCOPING A PROJECT

What is Data
Science?

Essential Skills
1 2 3 4 5 6
Scoping a
Project
Gathering
Data
Cleaning
Data
Exploring
Data
Modeling
Data
Sharing
Insights

Machine Learning
Projects don’t start with data, they start with a clearly defined scope:
Data Science
Workflow • Who are your end users or stakeholders?
• What business problems are you trying to help them solve?
• Is this a supervised or unsupervised learning problem? (do you even need data science?)
• What data do you need for your analysis?

*Copyright Maven Analytics, LLC

 STEP 2: GATHERING DATA

What is Data
Science?

Essential Skills
1 2 3 4 5 6
Scoping a
Project
Gathering
Data
Cleaning
Data
Exploring
Data
Modeling
Data
Sharing
Insights

Machine Learning
A project is only as strong as the underlying data, so gathering the right data is
Data Science essential to set a proper foundation for your analysis
Workflow

Data can come from a variety of sources, including:

• Files (flat files, spreadsheets, etc.)
• Databases
• Websites
• APIs

*Copyright Maven Analytics, LLC

 STEP 3: CLEANING DATA

What is Data
Science?

Essential Skills
1 2 3 4 5 6
Scoping a
Project
Gathering
Data
Cleaning
Data
Exploring
Data
Modeling
Data
Sharing
Insights

Machine Learning
A popular saying within data science is “garbage in, garbage out”, which means that
Data Science
cleaning data properly is key to producing accurate and reliable results
Workflow
Data cleaning tasks may include: Building models
The flashy part of data science
• Resolving formatting issues
• Correcting data types Cleaning data
• Imputing missing data Less fun, but very important
(Data scientists estimate that around
• Restructuring the data 50-80% of their time is spent here!)

*Copyright Maven Analytics, LLC

 STEP 4: EXPLORING DATA

What is Data
Science?

Essential Skills
1 2 3 4 5 6
Scoping a
Project
Gathering
Data
Cleaning
Data
Exploring
Data
Modeling
Data
Sharing
Insights

Machine Learning
Exploratory data analysis (EDA) is all about exploring and understanding the
Data Science data you’re working with before applying models or algorithms
Workflow

EDA tasks may include:

• Slicing & dicing the data A good number of the final insights that you share
will come from the exploratory analysis phase!
• Profiling the data
• Visualizing the data

*Copyright Maven Analytics, LLC

 STEP 5: MODELING DATA

What is Data
Science?

Essential Skills
1 2 3 4 5 6
Scoping a
Project
Gathering
Data
Cleaning
Data
Exploring
Data
Modeling
Data
Sharing
Insights

Machine Learning
Modeling data involves structuring and preparing data for specific modeling
Data Science techniques, and applying those models to make predictions or discover patterns
Workflow

Modeling tasks include:

With fancy new algorithms introduced every
year, you may feel the need to learn and apply
• Feature selection & engineering
the latest and greatest techniques
• Fitting models In practice, simple is best; businesses &
leadership teams appreciate solutions that are
• Interpreting results easy to understand, interpret and implement

*Copyright Maven Analytics, LLC

 STEP 6: SHARING INSIGHTS

What is Data
Science?

Essential Skills
1 2 3 4 5 6
Scoping a
Project
Gathering
Data
Cleaning
Data
Exploring
Data
Modeling
Data
Sharing
Insights

Machine Learning
The final step of the workflow involves summarizing your key findings and sharing
Data Science insights with end users or stakeholders:
Workflow
• Reiterate the problem Even with all the technical work
• Summarize the results of your analysis that’s been done, it’s important to
remember that the focus here is
• Share recommendations and next steps on non-technical solutions

• Focus on potential impact, not technical details

NOTE: Another way to share results is to deploy your model, or put it into production

*Copyright Maven Analytics, LLC

 UNSUPERVISED LEARNING

What is Data
Science?

Essential Skills
1 2 3 4 5 6
Scoping a
Project
Gathering
Data
Cleaning
Data
Exploring
Data
Modeling
Data
Sharing
Insights

Machine Learning
DATA PREP & EDA
Data Science
Workflow UNSUPERVISED LEARNING

Unsupervised learning is used to discover patterns & relationships within data

These techniques can be utilized during the cleaning, exploratory or modeling
phase of the data science workflow, and all findings can be shared as insights

*Copyright Maven Analytics, LLC

 KEY TAKEAWAYS

Data science is about using data to make smart decisions

• Supervised learning techniques use historical data to predict the future, and unsupervised learning
techniques use algorithms to find patterns and relationships

Data scientists have both coding and math skills along with domain expertise
• In addition to technical expertise, soft skills like communication, problem-solving, curiosity, creativity, grit,
and Googling prowess round out a data scientist’s skillset

The data science workflow starts with defining a clear scope

• Once the project scope is defined, you can move on to gathering and cleaning data, performing exploratory data
analysis, preparing data for modeling, applying algorithms, and sharing insights with end users

Unsupervised learning techniques are used for finding patterns in data

• Data scientists are often tasked with finding patterns and relationships in data, which can happen during the
cleaning, exploratory or modeling phase of the data science workflow

*Copyright Maven Analytics, LLC

UNSUPERVISED LEARNING 101

*Copyright Maven Analytics, LLC

 UNSUPERVISED LEARNING 101

In this section we’ll cover the basics of unsupervised learning, including key concepts,
techniques & applications, and where it can be used within the data science workflow

TOPICS WE’LL COVER: GOALS FOR THIS SECTION:

• Introduce the basics of unsupervised learning

Unsupervised Learning Techniques & Applications
• Review key terminology and concepts
• Understand the different techniques and
Data Science Workflow applications of unsupervised learning
• Revisit the data science workflow and identify
where unsupervised learning fits within it

*Copyright Maven Analytics, LLC

 UNSUPERVISED LEARNING 101

Unsupervised learning is about finding insights & patterns hidden in the data
• Unlike regression or classification, we don’t care about splitting our data into train / test sets
and making predictions, we just care about understanding the relationships in our data
Unsupervised
Learning

Techniques & MACHINE LEARNING

Applications

Data Science Supervised Learning Unsupervised Learning

Workflow

Has a “label” Does NOT have a “label”

Regression Classification Used to DESCRIBE or ORGANIZE

data in some non-obvious way

Used to predict Used to predict

values for labels for
NUMERICAL CATEGORICAL
variables variables

*Copyright Maven Analytics, LLC

 UNSUPERVISED LEARNING 101

EXAMPLE Clustering customers based on listening behavior

Unsupervised These are features (what goes into the model)

Learning

Music Streaming Podcast Listening

Techniques & Customer Note that there is NO target
Hours Hours
Applications
Each row Aria 46 9
represents a
Data Science customer Chord 38 10
Workflow
Harmony 44 17
How can we segment
Melody 19 50
these customers?
Reed 7 44

Viola 16 52

Rock 5 19

Piper 10 11

Allegra 17 9

*Copyright Maven Analytics, LLC

 UNSUPERVISED LEARNING 101

EXAMPLE Clustering customers based on listening behavior

Unsupervised These are features (what goes into the model)

Learning

Music Streaming Podcast Listening

Techniques & Customer Note that there is NO target
Hours Hours
Applications
Each row Aria 46 9
represents a Cluster 1
Data Science customer Chord 38 10
Workflow
Music lovers
Harmony 44 17

Melody 19 50
Cluster 2
Reed 7 44
Podcast enthusiasts
Viola 16 52

Rock 5 19
Cluster 3
Piper 10 11
Casual listeners
Allegra 17 9

*Copyright Maven Analytics, LLC

 UNSUPERVISED LEARNING 101

EXAMPLE Clustering customers based on listening behavior

Unsupervised These are the customers in our data

Learning

Techniques &
Applications We can clearly segment our
MUSIC STREAMING HOURS customers into three clusters:
Data Science • Music lovers
Workflow • Podcast enthusiasts
• Casual listeners

These are
features
PODCAST LISTENING HOURS

*Copyright Maven Analytics, LLC

 UNSUPERVISED LEARNING TECHNIQUES

There are two popular categories of unsupervised learning techniques:

Unsupervised
Learning

Techniques & Clustering Dimensionality Reduction

Applications

Identifying groups (or clusters) of data Reducing the number of columns (or
Data Science points that are similar to one another but dimensions) in a data set while losing as
Workflow distinct from other groups little information as possible

Common techniques: Common techniques:

• K-Means Clustering • PCA (Principal Component Analysis)
• Hierarchical Clustering • t-SNE (t-Stochastic Neighbor Embedding)
• DBSCAN (Density-Based Clustering) • SVD (Singular Value Decomposition)

Applications: Applications:
• Clustering / Segmentation • Feature Extraction
• Anomaly Detection • Data Visualization
• Recommenders • Recommenders

*Copyright Maven Analytics, LLC

 UNSUPERVISED LEARNING APPLICATIONS

These are two common applications of unsupervised learning techniques:

Unsupervised
Learning

Techniques & Anomaly Detection Recommenders

Applications

Identifying rare points in a data set that Suggesting items to users based on their
Data Science deviate significantly from the rest preferences or behaviors
Workflow

Unsupervised learning techniques: Unsupervised learning techniques:

• Clustering Techniques • Clustering Techniques
• Isolation Forests • Dimensionality Reduction Techniques

Other techniques: Other techniques:

• Statistical Analysis • Distance Measures
• Time Series Analysis • Supervised Learning

*Copyright Maven Analytics, LLC

 STRUCTURE OF THIS COURSE

We’ll cover unsupervised learning techniques & applications in this order:

Unsupervised
Learning Category Unsupervised Learning Techniques Application

Techniques &
K-Means Clustering
Applications
Clustering / Segmentation
Hierarchical Clustering Anomaly Detection
Data Science Recommenders
Workflow Clustering
DBSCAN (Density-Based Clustering)

Isolation Forests Anomaly Detection

Feature Extraction
PCA (Principal Component Analysis)
Data Visualization

t-SNE (t-Distributed Stochastic Neighbor Embedding) Data Visualization

Dimensionality
Reduction SVD (Singular Value Decomposition) Recommenders

*Copyright Maven Analytics, LLC

 UNSUPERVISED LEARNING WORKFLOW

1 2 3 4 5 6
Scoping a Gathering Cleaning Exploring Modeling Sharing
Unsupervised
project data data data data insights
Learning

Techniques & Data Prep Algorithm Tuning Selection

Applications

Get your data ready to be Apply an unsupervised Evaluate & tune the model Pick the best results and
Data Science input into a model learning technique using metrics and intuition identify any insights
Workflow

• Single table, non-null, • Clustering • Metrics (i.e. inertia) • Business objective

numeric data • Dimensionality • Data visualization • Domain expertise
• Feature engineering, Reduction • Interpret the results
selection, and scaling

The main differences compared to supervised learning are the lack of data splitting (having
training and test sets) and the focus on evaluation based on intuition (versus mainly metrics)

*Copyright Maven Analytics, LLC

 UNSUPERVISED LEARNING WORKFLOW

1 2 3 4 5 6
Scoping a Gathering Cleaning Exploring Modeling Sharing
Unsupervised
project data data data data insights
Learning

Techniques & Anomaly Detection PCA / t-SNE PCA

Applications

Can be used as a machine learning Can be used as a machine learning Can be used as an ML alternative to
Data Science alternative to Outlier Detection alternative to Data Visualization Feature Selection / Engineering
Workflow

• Outlier detection is typically • Data is typically visualized using • Feature selection or engineering
done using statistics or plots two dimensions (x and y-axes) is typically done manually by
removing features or applying
• By using anomaly detection, you • By using PCA or t-SNE, high-
transformations to create them
can catch unusual data points dimensional data can also be
and patterns before applying visualized in two dimensions • By using PCA, multiple fields can
other ML algorithms be mathematically reduced to
fewer fields, which is called
feature extraction

*Copyright Maven Analytics, LLC

 KEY TAKEAWAYS

Unsupervised learning is used to find patterns & relationships in data

• There are no predictions or labels with unsupervised learning – we are just trying to better understand the
data’s non-obvious structure, organization, and relationships between data points

Unsupervised learning has a different mindset than supervised learning

• Unlike supervised learning, unsupervised learning does not require splitting the data into a training and test
set, and the evaluation is based on a heavy-dose of domain expertise in addition to metrics

There are multiple applications for unsupervised learning techniques

• While the two main categories of unsupervised learning techniques fall under clustering and dimensionality
reduction, these techniques can be applied to segmentation, anomaly detection, recommenders, and more

The techniques can be used at multiple steps of the data science workflow
• In addition to using unsupervised learning techniques during the modeling step of the data science workflow, select
techniques can also be used during the data cleaning, exploration, and feature engineering phases

*Copyright Maven Analytics, LLC

PRE-MODELING DATA PREP

*Copyright Maven Analytics, LLC

 PRE-MODELING DATA PREP

In this section we’ll review the data prep steps required before applying unsupervised
learning algorithms, including making sure the rows and columns are set up properly

TOPICS WE’LL COVER: GOALS FOR THIS SECTION:

• Learn Python techniques to adjust row granularity

Data Prep Steps Row Granularity and make sure all values are non-null and numeric
• Apply feature engineering techniques for
Column Preparation Feature Engineering unsupervised learning models
• Understand why identifier columns should be
excluded from modeling, but used as reference
Feature Selection Feature Scaling
• Identify situations where feature scaling is needed

*Copyright Maven Analytics, LLC

 PRE-MODELING DATA PREP

These are common data prep steps required to transform your source data into a
format that can be directly input into an unsupervised learning model:
Data Prep Steps

Row Granularity 1 Setting the correct row granularity

Column 2 Ensuring each column is non-null and numeric

Preparation

Feature
3 Engineering features for modeling
Engineering

4 Selecting features and excluding identifier columns

Feature Selection

5 Scaling features for distance-based unsupervised learning algorithms

Feature Scaling

*Copyright Maven Analytics, LLC

 SETTING THE CORRECT ROW GRANULARITY

To set the correct row granularity, think about the question you’re trying to
answer and determine what one row (observation) of your table will look like
Data Prep Steps
GOAL Cluster customers based on listening behavior

Row Granularity
Customer Genre # Songs Customer # Pop Songs # Indie Songs # Rock Songs

Aria Pop 50 Aria 50 48 1

Column
Preparation Aria Indie 48 Chord 15 36 0

Aria Rock 1 Harmony 10 5 3

Feature
Engineering Chord Pop 15 Melody 0 0 2

Chord Indie 36 Reed 0 0 5

Feature Selection Harmony Pop 10

Harmony Indie 5 This table has the correct data, but it

needs to be reshaped so that each
Harmony Rock 3 row represents one customer
Feature Scaling
(that’s what we’re clustering!)
Melody Rock 2

Reed Rock 5

*Copyright Maven Analytics, LLC

 RESHAPING DATA

Common ways to reshape data to the correct row granularity are:

• Using .groupby() to combine all records with the same ID column into a single row
Data Prep Steps

Row Granularity

Column
Preparation

Feature
Engineering
Now each row represents a customer!

Feature Selection

Feature Scaling

*Copyright Maven Analytics, LLC

 RESHAPING DATA

Common ways to reshape data to the correct row granularity are:

• Using .groupby() to combine all records with the same ID column into a single row
Data Prep Steps • Using .pivot() to transform data from a “long” format to a “wide” format

Row Granularity

Column
Preparation

Feature
Engineering
Now each row represents a customer, and
we still have the genre information!

Feature Selection

The opposite of .pivot() is

Feature Scaling .melt(), which is often used to
transform data from a “wide”
format to a “long” format

*Copyright Maven Analytics, LLC

 ASSIGNMENT: SET THE CORRECT ROW GRANULARITY

Key Objectives
NEW MESSAGE
March 4, 2024 1. Read the Excel file into a Pandas DataFrame
From: Cindy Cinema (Lead Data Scientist) 2. Check the number of rows and columns
Subject: Please format data for analysis
3. Determine the row granularity needed
Hi, 4. Apply the correct DataFrame transformation
I hear you’re the new associate data scientist on the team – 5. Save the transformation as a new DataFrame
welcome!
6. Check the number of rows and columns
We’re currently working on a project to segment students
based on their entertainment preferences.
Could you format the data in this spreadsheet so that it’s at Hint: The new DataFrame should have 150 rows
the correct row granularity for student-level analysis?

Thanks!
Cindy

[Link]

*Copyright Maven Analytics, LLC

 PREPARING COLUMNS FOR MODELING

After setting the row granularity, you need to prepare your columns for modeling:

Data Prep Steps

1• All values should be non-null
Row Granularity • Identify missing (or null) values using [Link]() or [Link]()
• Resolve them by either removing them or imputing the values
Column
Preparation
2• All values should be numeric
Feature • Convert fields from text data types to numeric data types
Engineering
• Turn fields into numeric fields using conditional logic with [Link]()
• Turn categorical fields into numeric fields using dummy variables
Feature Selection

Feature Scaling PRO TIP: There are some algorithms that can handle null and non-numeric values, including tree-
based models and some classification models, but it’s still best practice to prepare the data this way

*Copyright Maven Analytics, LLC

 IDENTIFYING MISSING DATA

You can identify missing data with the .info() or .isna() methods

Data Prep Steps

Row Granularity

This is True for

Column any null values
Preparation

Feature
Engineering

Compare the total entries with the

Feature Selection non-null count for each column

You can use .any(axis=1) to

Feature Scaling
return the rows with null values

*Copyright Maven Analytics, LLC

 HANDLING MISSING DATA

There are multiple ways to handle missing data in a DataFrame:

• The .dropna() method removes rows or columns with missing data
Data Prep Steps

Row Granularity

Column
Preparation

Feature
Engineering

Note the index values are skipping the dropped rows,

Feature Selection but you can fix this by chaining on a .reset_index()

Feature Scaling
PRO TIP: The .dropna() method defaults to dropping rows, but
you can drop columns instead by specifying .dropna(axis=1)

*Copyright Maven Analytics, LLC

 HANDLING MISSING DATA

There are multiple ways to handle missing data in a DataFrame:

• The .dropna() method removes rows or columns with missing data
Data Prep Steps
• The .fillna() method imputes missing data with an appropriate value

Row Granularity

Column
Preparation

Feature
Engineering

Feature Selection

Feature Scaling
What values can we fill in here?
• The median age would remove the impact of outliers
• Experience tells us most customers have 0 followers

*Copyright Maven Analytics, LLC

 CONVERTING TO NUMERIC

Pandas will often read in numeric fields as text (object) data types
You can use pd.to_numeric() to convert them to numeric fields
Data Prep Steps

Row Granularity

Column
Preparation

Feature
Engineering

Feature Selection

Feature Scaling The dollar sign and comma

cause this to be read in as text

*Copyright Maven Analytics, LLC

 CONVERTING TO NUMERIC

Pandas will often read in numeric fields as text (object) data types
You can use pd.to_numeric() to convert them to numeric fields
Data Prep Steps

Row Granularity

Column
Preparation

Feature
Engineering

Feature Selection

Feature Scaling Even though we removed the punctuation,

we still need to convert to numeric

*Copyright Maven Analytics, LLC

 CONVERTING TO NUMERIC

Pandas will often read in numeric fields as text (object) data types
You can use pd.to_numeric() to convert them to numeric fields
Data Prep Steps

Row Granularity

Column
Preparation

Feature
Engineering

Feature Selection

Feature Scaling

*Copyright Maven Analytics, LLC

 CONVERTING TO DATETIME

Pandas will often read in date fields as text (object) data types as well
You can use pd.to_datetime() to convert them to datetime fields
Data Prep Steps

Row Granularity

Column
Preparation

Feature
Engineering

Feature Selection

Feature Scaling These look like dates, but they’re

actually being read in as text

*Copyright Maven Analytics, LLC

 CONVERTING TO DATETIME

Pandas will often read in date fields as text (object) data types as well
You can use pd.to_datetime() to convert them to datetime fields
Data Prep Steps

Row Granularity

Column
Preparation

Feature
Engineering

Feature Selection

Feature Scaling
While a datetime field can’t be input into a model, its components can be extracted
as numeric values and saved as new fields for modeling (more on this next)

*Copyright Maven Analytics, LLC

 EXTRACTING DATETIME COMPONENTS
You can use datetime methods to extract components from datetime fields

Data Prep Steps

Row Granularity

Column
Preparation

Feature
Engineering

Feature Selection

Feature Scaling
Even though month and day of week look like numeric values, it’s inaccurate to say that 8>5 (August is better than May).
These values often go through an additional process called binning, which we’ll cover in the feature engineering section.

*Copyright Maven Analytics, LLC

 CALCULATING BASED ON A CONDITION

Use [Link]() to create a numeric column based on a logical condition

Data Prep Steps

Row Granularity

Column
Preparation

Feature
Engineering

Feature Selection

Feature Scaling
PRO TIP: [Link]() is quite versatile and can also be used to set values other than 1/0 (other
numeric values, text values, etc.), set more than two values, set values for another column, and more

*Copyright Maven Analytics, LLC

 DUMMY VARIABLES

A dummy variable is a field that only contains ones and zeros to represent the
presence (1) or absence (0) of a value, also known as one-hot encoding
Data Prep Steps
• They are used to transform categorical fields into multiple numeric fields

Row Granularity
These dummy variables are numeric
representations of the “Education Level” field
Column
Preparation
Graduate High
Name Age Education Level Name Age Education Level College
School School
Feature
Engineering Aria 25 College Aria 25 College 1 0 0

Chord 19 High School Chord 19 High School 0 0 1

Feature Selection Harmony 26 Graduate School Harmony 26 Graduate School 0 1 0

Melody 47 College Melody 47 College 1 0 0

Feature Scaling Reed 52 High School Rock 52 High School 0 0 1

*Copyright Maven Analytics, LLC

 DUMMY VARIABLES

Use pd.get_dummies() to create dummy variables in Python

Data Prep Steps

Row Granularity

Column
Preparation

Feature
Engineering

Feature Selection

Feature Scaling

*Copyright Maven Analytics, LLC

 DUMMY VARIABLES

Use pd.get_dummies() to create dummy variables in Python

Data Prep Steps

Row Granularity

Column Use .astype(int) to convert

Preparation TRUE/FALSE values to 1/0

Feature
Engineering

Feature Selection

Feature Scaling

*Copyright Maven Analytics, LLC

 ASSIGNMENT: PREPARE COLUMNS FOR MODELING

Key Objectives
NEW MESSAGE
March 5, 2024 1. Find the missing values
From: Cindy Cinema (Lead Data Scientist) 2. Fill in the missing values with zeros
Subject: Clean up columns
3. Create a new column called video_game_lover
Hi again, • Set the value to 1 if a student played more
Thanks for getting the data in the correct format earlier. than 7 hours of videos games each week
• Set the value to 0 otherwise
Can you also help me with some column clean up?

There are some null values that I think make sense to fill in
with zeros, since those are fields that the students left blank.

Can you also create a new column that flags students who
really love video games (those who play over 7 hours a week)?
Thanks!

*Copyright Maven Analytics, LLC

 FEATURE ENGINEERING

Feature engineering is the process of creating columns that you think will be
helpful inputs for improving a model (help segment, recommend, etc.)
Data Prep Steps

Row Granularity
This DataFrame:
• Is at the correct row granularity
Column • Contains non-null and numeric values
Preparation • Includes newly engineered features
• Is ready for modeling
Feature
Engineering

Common feature engineering techniques:

Feature Selection
Once you have prepared the rows & columns,
• All the data prep steps so far the data is technically ready for modeling
• Applying calculations However, being deliberate about engineering
Feature Scaling new features can be the difference between a
• Binning values good model and a great one
• Identifying proxy variables

*Copyright Maven Analytics, LLC

 FEATURE ENGINEERING DURING DATA PREP

Everything we’ve done so far is technically a feature engineering technique,

since we’ve created new columns, or features, along the way
Data Prep Steps

Row Granularity
1• Setting the correct row granularity
Column • Feature aggregation: aggregating multiple rows into a single row
Preparation

Feature
Engineering 2• Ensuring each column is non-null and numeric
• Handling missing data: imputing missing values
Feature Selection
• Categorical encoding: turning categorical values into numeric values

Feature Scaling

*Copyright Maven Analytics, LLC

 APPLYING CALCULATIONS

You can combine columns using calculations to create new features

Data Prep Steps

Row Granularity

Column
Preparation

Feature
Engineering

Feature Selection

Feature Scaling

*Copyright Maven Analytics, LLC

 BINNING VALUES

You can group numerical features into bins or discrete categories if it makes
more sense for your analysis
Data Prep Steps

Row Granularity

Column
Preparation

Feature
Engineering

Feature Selection

Feature Scaling

*Copyright Maven Analytics, LLC

 PROXY VARIABLES

A proxy variable is a feature meant to approximately represent another variable

• They are used when a feature is either difficult to gather or engineer into a new feature
Data Prep Steps

“Sign Up Month” is numeric, but

Row Granularity may not be good for modeling
(August is not better than May)
Instead of turning it into dummy
Column variables or binning the months, you
Preparation can use a proxy variable like the
average temperature each month

Feature
Engineering

You may not be able to engineer

proxy variables from existing
Feature Selection
data, but they can be gathered
from external sources

Feature Scaling

*Copyright Maven Analytics, LLC

 FEATURE ENGINEERING TIPS

1.
1 Anyone can apply an algorithm, but only someone with domain expertise
Data Prep Steps
can engineer relevant features, which is what makes a great model

Row Granularity 2.
2 You want your data to be long, not wide (many rows, few columns), so
remember to:
Column
Preparation • Try to collect as many observations (rows) of data as you can
Feature
• Only select the most meaningful features for modeling
Engineering

3.
3 Once you start modeling, you’re bound to find things you missed during
Feature Selection data prep and will continue to engineer features

Feature Scaling

*Copyright Maven Analytics, LLC

 ASSIGNMENT: FEATURE ENGINEERING

Key Objectives
NEW MESSAGE
March 6, 2024 1. Create a column called total_entertainment
From: Cindy Cinema (Lead Data Scientist) that sums up all the types of entertainment for
each student
Subject: Engineer new features
2. Create a column called pct_screen that
Hi! calculates the percent of entertainment that’s on
screens (everything except for books) for each
We’d like to add a few features before we begin modeling.
student
Can you create the following:

• A column that sums up the total hours of entertainment

each student consumes weekly
• A column that calculates the percent of entertainment
consumed that’s on screens
Thanks again,
Cindy

*Copyright Maven Analytics, LLC

 EXCLUDING IDENTIFIERS

Columns containing names or IDs should be excluded from modeling, but

remembered for interpretation down the line
Data Prep Steps

features / inputs features / inputs

Row Granularity (what goes into a model) (what goes into a model)

Column Customer Age Discount Pct_Pop House ID Price Bedrooms

Preparation
Aria 25 1 51 1 350,000 2
Feature Chord 19 1 29 2 500,000 3
Engineering
Harmony 26 0 56 3 180,000 0
Melody 47 0 0 4 270,000 2
Feature Selection
Reed 52 1 0 5 245,000 1

Feature Scaling
While it’s important to remember these fields, they should
not be included when the data is input into a model

*Copyright Maven Analytics, LLC

 FEATURE SELECTION

More features does not always mean a better model, so it’s important to select a
subset of the features for modeling
Data Prep Steps

Row Granularity
How do we select the “right” number of features?
• Use your intuition and think about the goal for your analysis – which features would do
Column the best job predicting / segmenting / etc.?
Preparation
• Start simple with perhaps two or three features for your model, then assess the results,
and continue to make the model more complex and assess
Feature
Engineering • Don’t worry about getting this right the first time – modeling is a lot of trial and error!

Feature Selection

There are also feature selection and extraction techniques that use machine learning
– we will be covering some of them in the dimensionality reduction section
Feature Scaling

*Copyright Maven Analytics, LLC

 FEATURE SELECTION

More features does not always mean a better model, so it’s important to select a
subset of the features for modeling
Data Prep Steps

Row Granularity

Column
Preparation

Feature
Engineering

Feature Selection

My domain expertise & gut feel is telling me that the best way to differentiate customers is by
their age, the total number of songs they listen to and the types of songs they listen to
Feature Scaling
These three features are what I’ll include in my first round of modeling

*Copyright Maven Analytics, LLC

 ASSIGNMENT: FEATURE SELECTION

Key Objectives
NEW MESSAGE
March 7, 2024 1. Save the name column of the DataFrame as its
From: Cindy Cinema (Lead Data Scientist) own Series for reference
Subject: Narrow down features 2. Save the three new columns you engineered as
its own DataFrame for modeling –
Hi again, video_game_lover, total_entertainment and
pct_screen
We’re almost ready for modeling! Just a few more steps.

Can you save the student name column as its own Series that
we can use to reference later on?
Can you create a final modeling DataFrame that includes only
the three new features that we engineered –
video_game_lover, total_entertainment and pct_screen?

Thanks!
Cindy

*Copyright Maven Analytics, LLC

 SCALING FEATURES

Scaling, as the name implies, requires setting all input features on a similar scale
• Scaling is an optional feature engineering step which depends on the ML algorithm
Data Prep Steps
• Common techniques for scaling include normalization and standardization

Row Granularity

Column
Preparation

Feature
Engineering

Feature Selection

Feature Scaling PRO TIP: Many unsupervised learning techniques use distance-based
calculations, which makes scaling a required data prep step before modeling

*Copyright Maven Analytics, LLC

 NORMALIZATION

Normalization transforms all columns to be between 0 and 1 (or between -1 and 1)

• This is typically used when the column distributions are unknown or mixed
Data Prep Steps

Row Granularity

Column
Preparation

Use sklearn’s MinMaxScaler function

Feature and the .fit_transform method to
Engineering normalize DataFrame columns

Feature Selection Normalization equation

𝑥 − 𝑥𝑚𝑖𝑛
Feature Scaling 𝑥𝑚𝑎𝑥 − 𝑥𝑚𝑖𝑛

*Copyright Maven Analytics, LLC

 STANDARDIZATION

Standardization transforms all columns to have a mean of 0 and standard deviation of 1

• This is typically used when the column distributions are normal (bell curve)
Data Prep Steps

Row Granularity

Column
Preparation

Use sklearn’s StandardScaler function

Feature and the .fit_transform method to
Engineering standardize DataFrame columns

Feature Selection Standardization equation

𝑥 − 𝑥𝑚𝑒𝑎𝑛
Feature Scaling 𝑥𝑠𝑡𝑑

*Copyright Maven Analytics, LLC

 ASSIGNMENT: FEATURE SCALING

Key Objectives
NEW MESSAGE
March 8, 2024 1. Scale the features in the modeling DataFrame so
From: Cindy Cinema (Lead Data Scientist) they all have a mean of 0 and a standard
deviation of 1
Subject: Feature scaling request
2. Save the output as a final DataFrame that’s ready
Hi, I have one final request for you. for modeling
We plan on clustering the data using a distance-based
algorithm, so the data needs to be scaled.

Using the DataFrame with the three features for modeling,

can you scale the columns so that they all have a mean of 0
and a standard deviation of 1?

Once you do that, we should be all set with our data prep
steps and ready for modeling.

Thanks for your help this week!

*Copyright Maven Analytics, LLC

 KEY TAKEAWAYS

Data prep is required before applying unsupervised learning techniques

• This includes setting the correct row granularity, making sure all columns are non-null and numeric,
engineering useful features, selecting the best ones, and scaling them

Both the rows & columns of a DataFrame must be prepared for modeling
• There are several techniques that are commonly used to prepare rows and columns for modeling, including
using .groupby(), .pivot(), .fillna(), [Link]() and pd.get_dummies()

Feature engineering can be the difference between a good and a great model
• There are many techniques for feature engineering, including aggregating values, handling missing data,
categorical encoding, applying calculations, binning data, using proxy variables – and then selecting features

Feature scaling is required for models that use distance-based calculations

• Many unsupervised learning techniques use distance-based calculations, and it’s a good idea to try multiple
scaling techniques, including normalization and standardization, to see which one performs best

*Copyright Maven Analytics, LLC

CLUSTERING

*Copyright Maven Analytics, LLC

 CLUSTERING

In this section we’ll introduce the fundamentals of clustering and compare three popular
clustering techniques: K-Means Clustering, Hierarchical Clustering, and DBSCAN

TOPICS WE’LL COVER: GOALS FOR THIS SECTION:

• Learn how clustering models fundamentally work

Clustering Basics K-Means Clustering
• Use Python to apply different clustering models
and interpret their results
Hierarchical Clustering DBSCAN
• Compare & contrast popular clustering techniques

Comparing Models

*Copyright Maven Analytics, LLC

 CLUSTERING BASICS

Clustering allows you to find concentrations or groups of observations which are

similar to one another but distinct from other groups
Clustering Basics

K-Means
Clustering

Hierarchical
Clustering

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 CLUSTERING WORKFLOW

The general clustering workflow consists of the following steps:

Clustering Basics
Data Prep Modeling Tuning Selection
K-Means
Clustering
Get your data ready to be Apply a clustering Evaluate & tune the model Pick the best results and
input into an ML model algorithm using metrics and intuition identify any insights
Hierarchical
Clustering
• Single table, non-null • K-Means Clustering • Metrics (i.e. inertia) • Business objective
and numeric data • Hierarchical • Data visualization • Domain expertise
• Feature engineering, Clustering
DBSCAN • Interpret the results
selection, and scaling • DBSCAN

Comparing
Models

Remember, there’s no “right” answer or single optimization metric when it comes to clustering; the best outputs
are the ones which help you answer the question at hand and make practical, data-driven business decisions

*Copyright Maven Analytics, LLC

 K-MEANS CLUSTERING

K-Means Clustering is a popular algorithm which assigns each observation in a

data set to a specific cluster, where “K” represents the number of clusters
Clustering Basics

K-Means
Clustering

Hierarchical
Clustering

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 K-MEANS CLUSTERING

K-Means Clustering is a popular algorithm which assigns each observation in a

data set to a specific cluster, where “K” represents the number of clusters
Clustering Basics

Here’s how it works:

K-Means
Clustering 1. Select “K” arbitrary locations in a scatter plot as cluster centers (or centroids),
and assign each observation to a cluster based on the closest centroid
Hierarchical
Clustering 2. Recalculate and relocate each centroid to the mean of the observations assigned
to it, then reassign each observation to its new closest centroid

DBSCAN 3. Repeat the process until observations no longer change clusters

Comparing
Models Example use cases:
• Identifying customer segments for targeted marketing campaigns
• Clustering store locations based on factors like sales, ratings, size, etc.

*Copyright Maven Analytics, LLC

 K-MEANS CLUSTERING

STEP 1: Determine what you think might be an appropriate number of clusters

(in this case 2), and select arbitrary locations as initial centroids
Clustering Basics

K-Means
Clustering

Hierarchical
Clustering

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 K-MEANS CLUSTERING

STEP 2: Assign each observation to a cluster, based on the closest centroid

Clustering Basics

K-Means
Clustering

Hierarchical
Clustering

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 K-MEANS CLUSTERING

STEP 3: Relocate each centroid to the mean of its assigned observations, and
reassign each observation to the new closest centroid
Clustering Basics

K-Means
Clustering

Hierarchical
Clustering

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 K-MEANS CLUSTERING

STEP 3: Relocate each centroid to the mean of its assigned observations, and
reassign each observation to the new closest centroid
Clustering Basics

K-Means
Clustering

Hierarchical
Clustering

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 K-MEANS CLUSTERING

STEP 4: Continue to relocate each centroid to the mean of its assigned

observations, until the clusters no longer change
Clustering Basics

K-Means
Clustering

Hierarchical
Clustering

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 K-MEANS CLUSTERING

STEP 4: Continue to relocate each centroid to the mean of its assigned

observations, until the clusters no longer change
Clustering Basics

K-Means
Clustering

Hierarchical
Clustering

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 K-MEANS CLUSTERING

STEP 4: Continue to relocate each centroid to the mean of its assigned

observations, until the clusters no longer change
Clustering Basics

K-Means
Clustering

Hierarchical
Clustering

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 K-MEANS CLUSTERING

STEP 4: Continue to relocate each centroid to the mean of its assigned

observations, until the clusters no longer change
Clustering Basics

K-Means
Clustering

Hierarchical
Clustering

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 K-MEANS CLUSTERING IN PYTHON

You can use sklearn’s KMeans() function to perform K-Means Clustering

Clustering Basics

K-Means
Clustering

Hierarchical
Clustering

The “k” number of The number of models to fit Setting a random_state value
DBSCAN
clusters to identify with different initial centroids, guarantees the same results
(default is 8) returning the best result each time the model is fit
(”auto” will fit one model)
Comparing
Models

PRO TIP: It’s typically a good idea to start with 2 clusters and build up from there, comparing the results

*Copyright Maven Analytics, LLC

 K-MEANS CLUSTERING IN PYTHON

Clustering high school students based on entertainment preferences so the

EXAMPLE local library can use targeted ads to encourage teenagers to read more

Clustering Basics

K-Means
Clustering

Some students spend almost

Hierarchical
no time reading each week
Clustering

Students prefer to spend most of

DBSCAN their time playing video games

Comparing
Models

*Copyright Maven Analytics, LLC

 K-MEANS CLUSTERING IN PYTHON

Clustering high school students based on entertainment preferences so the

EXAMPLE local library can use targeted ads to encourage teenagers to read more

Clustering Basics

K-Means
Clustering

Hierarchical
Clustering
Once the model is fit, you can view the cluster that each row
as been assigned to using the .labels_ attribute
DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 VISUALIZING K-MEANS CLUSTERING

You can visualize the results of a K-Means model using the .labels_ attribute and
data visualizations libraries in Python
Clustering Basics

K-Means
Clustering

Hierarchical
Clustering

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 INTERPRETING K-MEANS CLUSTERING

You can interpret the results of a K-Means model using the .cluster_centers_
attribute and your intuition
Clustering Basics

K-Means
Clustering

Hierarchical
Clustering

DBSCAN
Students in the first cluster spend on average: Students in the second cluster spend on average:
• 4.2 hours reading books • 0.6 hours reading books
Comparing • 4.3 hours watching TV shows • 5.1 hours watching TV shows
Models • 6.3 hours playing video games • 5 hours playing video games

These students consume a good amount of These students don’t read many books, so
each type of content, so we could name them we could name them “non-readers”
“consumers of all types of entertainment”

*Copyright Maven Analytics, LLC

 VISUALIZING CLUSTER CENTERS
Visualizing the cluster centers in a heat map can help interpret them

Clustering Basics

K-Means
Clustering

Hierarchical
Clustering

DBSCAN

Comparing
Models

Cluster 0 students don’t read many books

Cluster 1 students consume a lot of entertainment
Cluster 2 students prefer video games to books

*Copyright Maven Analytics, LLC

 ASSIGNMENT: K-MEANS CLUSTERING

Key Objectives
NEW MESSAGE
March 11, 2024
1. Read in the [Link] file
From: Clyde Clusters (Sr. Data Scientist)
2. Prep the data by dropping the name and
Subject: K-Means help
manufacturer columns

Hi there! 3. Fit a K-Means Clustering model with 2 clusters

Our client, Maven Supermarket, would like to set up cereal 4. Interpret the cluster centers
displays around their store based on various niches of cereals
(i.e. geared towards kids, those with dietary restrictions, etc.).

As a starting point, can you fit a K-Means Clustering model on

the attached cereal data set? I recommend using two clusters.
Let me know how you decide to interpret the two clusters.

Thanks!
Clyde

[Link]

*Copyright Maven Analytics, LLC

 INERTIA

How do we know what’s the “right” number of clusters (K)?

• While there is no “right” or “wrong” number of clusters, you can use the inertia
Clustering Basics (aka within-cluster sum of squares or WCSS) to help inform your decision

K-Means
Clustering

Hierarchical
Clustering

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 INERTIA

How do we know what’s the “right” number of clusters (K)?

• While there is no “right” or “wrong” number of clusters, you can use the inertia
Clustering Basics (aka within-cluster sum of squares or WCSS) to help inform your decision

K-Means
Clustering Square these distances and sum them to calculate inertia for two clusters (K=2)

Hierarchical
Clustering

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 INERTIA

How do we know what’s the “right” number of clusters (K)?

• While there is no “right” or “wrong” number of clusters, you can use the inertia
Clustering Basics (aka within-cluster sum of squares or WCSS) to help inform your decision

K-Means
Clustering

Hierarchical
Clustering
INERTIA

DBSCAN

Comparing
Models

2 3 4 5 6 7 8
NUMBER OF CLUSTERS (K)

*Copyright Maven Analytics, LLC

 INERTIA

How do we know what’s the “right” number of clusters (K)?

• While there is no “right” or “wrong” number of clusters, you can use the inertia
Clustering Basics (aka within-cluster sum of squares or WCSS) to help inform your decision

K-Means
Clustering

Hierarchical Rerun the model with additional clusters, and

Clustering plot the inertia for each value of K
INERTIA

DBSCAN

Comparing
Models

2 3 4 5 6 7 8
NUMBER OF CLUSTERS (K)

*Copyright Maven Analytics, LLC

 INERTIA

How do we know what’s the “right” number of clusters (K)?

• While there is no “right” or “wrong” number of clusters, you can use the inertia
Clustering Basics (aka within-cluster sum of squares or WCSS) to help inform your decision

K-Means
Clustering

Hierarchical Look for an “elbow” or inflection point, where

Clustering adding another cluster has a relatively small
impact on inertia (in this case where K=5)
INERTIA

DBSCAN

PRO TIP: Think of this as

Comparing a guideline, not a strict rule
Models

2 3 4 5 6 7 8
NUMBER OF CLUSTERS (K)

*Copyright Maven Analytics, LLC

 PLOTTING INERTIA IN PYTHON

You can plot inertia in Python by fitting K-Means models with different values for
“k”, storing the inertia for each one, and visualizing the results
Clustering Basics

Fit models using 2 to 15 clusters and

K-Means append their inertia values to a list
Clustering

Hierarchical
Clustering

DBSCAN
The “elbow” seems
to be at 3 clusters
Comparing
Models

*Copyright Maven Analytics, LLC

 ASSIGNMENT: INERTIA PLOT

Key Objectives
NEW MESSAGE
March 12, 2024 1. Write a loop to fit K-Means Clustering models
From: Clyde Clusters (Sr. Data Scientist) with 2 to 15 clusters
Subject: More K-Means help 2. Create a plot with the number of clusters on the
x-axis and the inertia on the y-axis
Hi again!
3. Identify the elbow of the plot
Thanks for sharing the results of your 2-cluster model – they
make a lot of sense, so let’s continue down this path. 4. Fit a K-Means Clustering model on the specific
number of clusters at the elbow
Can you fit 14 K-Means Clustering models, using 2-15
clusters, and plot their inertia values? 5. Interpret the cluster centers using a heat map
Once you find the “elbow”, let me know how many clusters we
should use and what you recommend naming those clusters.

Thanks!
Clyde

*Copyright Maven Analytics, LLC

 TUNING A K-MEANS MODEL

Part of the clustering workflow is to go back to various data prep and modeling
steps to tune a model before selecting the best one
Clustering Basics
Data Prep
K-Means • Data cleaning: Removing outliers, etc.
Clustering
• Feature engineering: Creating relevant features, etc.
Hierarchical • Feature selection: More features does not mean a better model!
Clustering
• Scaling: K-Means is a distance-based algorithm, so it’s a good idea to scale the data

DBSCAN
Modeling
• Trying a different number of clusters: If your clusters are very different with each run,
Comparing
Models K-Means with that specific number of clusters may not be the best fit for your data, so try
using a different number of clusters
• Trying other clustering models: K-Means works best when the clusters are mostly
circular in shape, but algorithms like Hierarchical Clustering (up next!) can address this

*Copyright Maven Analytics, LLC

 ASSIGNMENT: TUNING A K-MEANS MODEL

Key Objectives
NEW MESSAGE
March 13, 2024
1. Remove the ”Fat” column
From: Clyde Clusters (Sr. Data Scientist)
2. Standardize the remaining columns
Subject: K-Means Fine Tuning
3. Repeat the steps from the previous inertia plot
Hi again, assignment
I realized that we could potentially improve our model by a) Write a loop to fit K-Means Clustering models
selecting a subset of the features and scaling them. with 2 to 15 clusters

Can you: b) Create a plot with the number of clusters on the

x-axis and the inertia on the y-axis
• Remove the “Fat” column
c) Identify the elbow of the plot
• Standardize the four remaining columns
d) Fit a K-Means Clustering model on the specific
• Run the same code fitting 14 different models, plot their number of clusters at the elbow
inertia values, and then interpret the “best” clusters
e) Interpret the cluster centers using a heat map
Thanks!

*Copyright Maven Analytics, LLC

 SELECTING THE BEST MODEL

There is no definitive best model when clustering, but a good model has clusters
that make sense, capture patterns, and help solve the business problem
Clustering Basics

K-Means
Clustering
You can explore the clusters using data-based approaches such as:
• Comparing the cluster assignments for the data set and individual rows of data
Hierarchical
Clustering
• Comparing metrics of the various models – inertia, silhouette score (coming soon!), etc.
• Testing the clustering models on unseen data (more on this at the end of this section!)

DBSCAN

Comparing PRO TIP: Sometimes the exact cluster assignments don’t matter as much as the
Models
actionable recommendations you can make based on the clustering models

*Copyright Maven Analytics, LLC

 SELECTING THE BEST MODEL

Clustering high school students based on entertainment preferences so the

EXAMPLE local library can use targeted ads to encourage teenagers to read more

Clustering Basics

K-Means
Clustering

Hierarchical
Clustering
Model 1 has 3 evenly-sized
clusters, while model 2 has
more specific clusters
DBSCAN

Comparing
Models
The “less screens” and
“entertainment enthusiasts”
groups are quite similar and
could possibly be combined
into a single cluster

*Copyright Maven Analytics, LLC

 ASSIGNMENT: SELECTING THE BEST K-MEANS MODEL

Key Objectives
NEW MESSAGE
March 14, 2024
1. Compare two models:
a) Label each row in your original data set with a
From: Clyde Clusters (Sr. Data Scientist)
cluster name from the unstandardized data model
Subject: Final K-Means Model and a cluster name from standardized data model
b) How many cereals fall into each cluster?
Hello, thanks for all your help with modeling so far!
c) Decide on the best model for our client
As a reminder, our original goal was to help our client, Maven
Supermarket, set up cereal displays around their store based 2. Recommend a specific number of displays and
on various niches of cereals. suggest a few cereals that should be shown in
Looking at the models that you built, can you compare them each display
and let me know which clusters make the most sense?

Once you do, I’ll pass along your recommendations to the

Maven Supermarket team.

Thanks!

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

Hierarchical Clustering is a clustering technique that creates clusters by

grouping similar data points together
Clustering Basics

K-Means
Clustering

2 clusters
Hierarchical
Clustering p5
p6

DISTANCE
p2

DBSCAN p3

p1
Comparing p4
Models

p1 p2 p3 p4 p5 p6

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

Hierarchical Clustering is a clustering technique that creates clusters by

grouping similar data points together*
Clustering Basics

Here’s how it works:

K-Means
Clustering 1. Within a scatter plot, find the 2 closest points, and group them into a cluster

Hierarchical
2. Then find the next two closest points or clusters, and group them to a cluster
Clustering
3. Repeat the process of combining the closest pairs of points or clusters until you
eventually end up with one single cluster
DBSCAN

Comparing
This process is visualized using a tree diagram called a dendrogram, which shows
Models the hierarchical relationship between clusters

*This is known as agglomerative or “bottom-up” clustering (vs. divisive or “top-down” clustering, which is much less common)

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 1: Find the two closest points, and group them into a cluster

Clustering Basics

K-Means
Clustering How do you define “closest”?

Hierarchical p5
p6
Clustering
p2

p3
DBSCAN
p1
p4
Comparing
Models

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 1: Find the two closest points, and group them into a cluster

Clustering Basics

K-Means
Clustering How do you define “closest”?
• Most commonly, the
Hierarchical
Clustering
p5
p6 Euclidean distance is used
p2

p3
DBSCAN
p1
p4
Comparing
Models

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 1: Find the two closest points, and group them into a cluster

Clustering Basics

K-Means
Clustering How do you define “closest”?
• Most commonly, the
Hierarchical
Clustering
p5
p6 Euclidean distance is used
p2

p3
• Alternatively, there’s
DBSCAN Manhattan distance
p1
p4
Comparing
Models

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 1: Find the two closest points, and group them into a cluster

Clustering Basics

K-Means
Clustering How do you define “closest”?
• Most commonly, the
Hierarchical
Clustering
p5
p6 Euclidean distance is used
p2

p3
• Alternatively, there’s
DBSCAN Manhattan distance
p1 • And Cosine distance
p4
Comparing
Models

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 1: Find the two closest points, and group them into a cluster

Clustering Basics

K-Means
Clustering

Hierarchical p5
p6
Clustering

DISTANCE
p2

p3
DBSCAN
p1
p4
Comparing
Models

p1 p2 p3 p4 p5 p6

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 1: Find the two closest points, and group them into a cluster

Clustering Basics

K-Means
Clustering

Hierarchical p5
p6
Clustering

DISTANCE
p2

p3
DBSCAN
p1
p4
Comparing
Models 5 clusters

p1 p2 p3 p4 p5 p6

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 2: Find the next two closest points/clusters, and group them together

Clustering Basics

K-Means
Clustering

Hierarchical p5
p6
Clustering

DISTANCE
p2

p3
DBSCAN
p1
p4
Comparing
Models 5 clusters

p1 p2 p3 p4 p5 p6

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 2: Find the next two closest points/clusters, and group them together

Clustering Basics

K-Means
Clustering

Hierarchical p5
p6
Clustering

DISTANCE
p2

p3
DBSCAN
p1
p4
4 clusters
Comparing
Models

p1 p2 p3 p4 p5 p6

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 3: Repeat the process until all points are part of the same cluster

Clustering Basics

K-Means
Clustering

Hierarchical p5
p6
Clustering

DISTANCE
p2

p3
DBSCAN
p1
p4
4 clusters
Comparing
Models

p1 p2 p3 p4 p5 p6

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 3: Repeat the process until all points are part of the same cluster

Clustering Basics

K-Means
Clustering

Hierarchical p5
p6
Clustering

DISTANCE
p2

p3 3 clusters
DBSCAN
p1
p4
Comparing
Models

p1 p2 p3 p4 p5 p6

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 3: Repeat the process until all points are part of the same cluster

Clustering Basics

K-Means
Clustering How do you define distance
between clusters?
Hierarchical p5
p6
Clustering
p2

p3
DBSCAN
p1
p4
Comparing
Models

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 3: Repeat the process until all points are part of the same cluster

Clustering Basics

K-Means
Clustering How do you define distance
between clusters?
Hierarchical p5
Clustering p6 • Single linkage (closest)
p2 Which is
shorter?
p3
DBSCAN
p1
p4
Comparing
Models

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 3: Repeat the process until all points are part of the same cluster

Clustering Basics

K-Means
Clustering How do you define distance
between clusters?
Hierarchical p5
Clustering p6 • Single linkage (closest)
p2 Which is
shorter? • Complete linkage (furthest)
p3
DBSCAN
p1
p4
Comparing
Models

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 3: Repeat the process until all points are part of the same cluster

Clustering Basics

K-Means
Clustering How do you define distance
between clusters?
Hierarchical p5
Clustering Which average p6 • Single linkage (closest)
p2 is smaller?
• Complete linkage (furthest)
p3
DBSCAN • Average linkage (all pairs)
p1
p4
Comparing
Models

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 3: Repeat the process until all points are part of the same cluster

Clustering Basics

K-Means
Clustering How do you define distance
between clusters?
Hierarchical p5
Clustering p6 • Single linkage (closest)
p2
• Complete linkage (furthest)
p3
DBSCAN • Average linkage (all pairs)
p1 • Ward’s method (variance)
p4
Comparing
Models

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 3: Repeat the process until all points are part of the same cluster

Clustering Basics

K-Means
Clustering How do you define distance
between clusters?
Hierarchical Which variances
p5
Clustering changed the least? p6 • Single linkage (closest)
p2
• Complete linkage (furthest)
p3
DBSCAN • Average linkage (all pairs)
p1 • Ward’s method (variance)
p4
Comparing
Models

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 3: Repeat the process until all points are part of the same cluster

Clustering Basics

K-Means
Clustering

Hierarchical p5
p6
Clustering

DISTANCE
p2

p3 3 clusters
DBSCAN
p1
p4
Comparing
Models

p1 p2 p3 p4 p5 p6

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 3: Repeat the process until all points are part of the same cluster

Clustering Basics

K-Means
Clustering
2 clusters

Hierarchical p5
p6
Clustering

DISTANCE
p2

p3
DBSCAN
p1
p4
Comparing
Models

p1 p2 p3 p4 p5 p6

*Copyright Maven Analytics, LLC

 HIERARCHICAL CLUSTERING

STEP 3: Repeat the process until all points are part of the same cluster

Clustering Basics

K-Means
Clustering

Hierarchical p5
p6
Clustering

DISTANCE
p2

p3
DBSCAN
p1
p4
Comparing
Models

p1 p2 p3 p4 p5 p6

*Copyright Maven Analytics, LLC

 DENDROGRAMS IN PYTHON

The dendrogram() function within the scipy library in Python allows you to
visualize Hierarchical Clustering clusters
Clustering Basics

K-Means
Clustering

Hierarchical
Clustering

DBSCAN

These colors are set by default, but you can set

Comparing the color_threshold argument to update them
Models for a specific number of clusters

There seem to be 3 clusters in the dendrogram,

so let’s update the color_threshold to 3

*Copyright Maven Analytics, LLC

 AGGLOMERATIVE CLUSTERING IN PYTHON

The AgglomerativeClustering() function within sklearn’s cluster module allows

you to perform Hierarchical Clustering in Python
Clustering Basics

K-Means
Clustering

Hierarchical
Clustering
The number of The method used to measure The method used to measure
clusters to identify the distance between points the distance between clusters
DBSCAN (default is 2) (default is “euclidean”) (default is “ward”)
Other distances: Other methods:
• “manhattan” • “single”
Comparing
Models • “cosine” • “complete”
PRO TIP: While you can tune • “precomputed” • “average”
these parameters, the defaults
are by far the most common

*Copyright Maven Analytics, LLC

 AGGLOMERATIVE CLUSTERING IN PYTHON

The AgglomerativeClustering() function within sklearn’s cluster module allows

you to perform Hierarchical Clustering in Python
Clustering Basics

K-Means
Clustering

Hierarchical
Clustering
Once the model is fit, you can view the cluster
assignments using the .labels_ attribute
DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 CLUSTER MAPS IN PYTHON

Seaborn’s clustermap() function adds a heatmap on top of the dendrogram,

which helps visually interpret each cluster
Clustering Basics

K-Means
Clustering
You also get the relationships between features (tv
& video game hours are more closely related!)
Hierarchical
Clustering

These students don’t read many books

DBSCAN

These students consume an average

Comparing
amount of each type of entertainment
Models

These students play lots of video games

*Copyright Maven Analytics, LLC

 ASSIGNMENT: HIERARCHICAL CLUSTERING

NEW MESSAGE
Key Objectives
March 18, 2024
1. Create a dendrogram using the 5 numeric fields
From: Clyde Clusters (Sr. Data Scientist)
of the cereal data set
Subject: Hierarchical Clustering Help
2. Visually identify the ”best” number of clusters
Hi again! and adjust the color_threshold

Thanks for your help applying K-Means to the cereal data set. 3. Create a dendrogram using the 4 standardized
fields (excluding “Fat”) of the cereal data set
To get another perspective, please create dendrograms for
both the original and standardized cereal data sets and let me 4. Visually identify the ”best” number of clusters
know how many clusters you can visually detect in each! and adjust the color_threshold
Finally, fit a hierarchical model on the “best” results from the 5. Fit a hierarchical clustering model on the “best”
standardized data set, visualize the clusters using a cluster map results from the last step and view the labels
and interpret the clusters.

Thanks!
6. Create a cluster map of the “best” results and
Clyde interpret the clusters

*Copyright Maven Analytics, LLC

 DBSCAN

DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is a

clustering technique that identifies clusters based on the density of data points
Clustering Basics

K-Means
Clustering

Hierarchical
Clustering

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is a

clustering technique that identifies clusters based on the density of data points
Clustering Basics
Here’s how it works*:
K-Means 1. Select a radius (eps) and a minimum number of points (min_samples)
Clustering
2. Within a scatter plot, label each point as one of the following:
Hierarchical
Clustering • Core point – has the minimum number of points within its radius (in a dense region)
• Border point – does not have the minimum number of points within its radius, but has
at least one core point within its radius (on the outskirts of clusters)
DBSCAN
• Noise point (outlier) – does not have a core point within its radius (isolated points)

Comparing
Models
DBSCAN allows for irregular-shaped clusters and can also identify outliers

*This is a rough summary of the steps – more detailed steps are included at the end of the DBSCAN section

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 1: Select a radius (eps) and minimum number of points (min_samples)

Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 2: Select a point at random and count the points within its radius
• If it’s greater than or equal to the “min_samples”, then start a cluster, label the point as a
Clustering Basics
core point, and mark the points within its radius as a neighbor

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering
3
2

DBSCAN 1

4 5

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 2: Select a point at random and count the points within its radius
• If it’s greater than or equal to the “min_samples”, then start a cluster, label the point as a
Clustering Basics
core point, and mark the points within its radius as a neighbor

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering
3
2

DBSCAN 1

4 5

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 3: Move to a neighbor and count the points within its radius
• If it’s greater than or equal to the “min_samples”, label it as a core point, and mark its neighbors
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering
2
3
1

DBSCAN 4 5

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 3: Move to a neighbor and count the points within its radius
• If it’s greater than or equal to the “min_samples”, label it as a core point, and mark its neighbors
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering
2
3
1

DBSCAN 4 5

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 3: Move to a neighbor and count the points within its radius
• If it’s less than “min_samples”, but at least one of them is a core point, label it as a border point
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical 2
Clustering
1

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 3: Move to a neighbor and count the points within its radius
• If it’s less than “min_samples”, but at least one of them is a core point, label it as a border point
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical 2
Clustering
1

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 3: Move to a neighbor and count the points within its radius
• If it’s less than “min_samples”, and none of them is a core point, label it as a noise point
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical 1
Clustering
2

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 3: Move to a neighbor and count the points within its radius
• If it’s less than “min_samples”, and none of them is a core point, label it as a noise point
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical 1
Clustering
2

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering
1
2

DBSCAN 3

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering
1
2

DBSCAN 3

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN 1

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN 1

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN 2 3

1 4

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN 2 3

1 4

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN 3
4
2 1

6
Comparing 5

Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 6
Hierarchical
Clustering

DBSCAN 3
4
2 1

6
Comparing 5

Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN
1
2
4
3
Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN
1
2
4
3
Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN
3
2

1
Comparing 4

Models 6
5

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN
4
2

1
Comparing 4

Models 6
5

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN
2

3
Comparing 1

Models
4

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN
2

3
Comparing 1

Models
4

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN

3
Comparing 2

Models 4
1

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN

3
Comparing 2

Models 4
1

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN

4
3
Comparing 5
Models 1
2

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN

4
3
Comparing 5
Models 1
2

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN
2
1

Comparing 4
Models 3

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN
2
1

Comparing 4
Models 3

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN

Comparing 1
4
Models 3

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN

Comparing 1
4
Models 3

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN

Comparing 2
1
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 4: Move on to another neighbor and continue this process until all the
points are labeled within the cluster
Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN

Comparing 2
1
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical 2 3
Clustering 1
6
4
5

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical 2 3
Clustering 1
6
4
5

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical 1
Clustering 3

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical 1
Clustering 3

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical 2
Clustering 3

1
4

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical 2
Clustering 3

1
4

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering 3
4
2
1

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering 3
4
2
1

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical 4
Clustering 3
1

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical 4
Clustering 3
1

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
2 min_samples = 4
Hierarchical 1
Clustering 3
4

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
2 min_samples = 4
Hierarchical 1
Clustering 3
4

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
1 min_samples = 4
Hierarchical 2
Clustering

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
1 min_samples = 4
Hierarchical 2
Clustering

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN
1

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN
1

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN

Comparing
Models
1

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN

Comparing
Models
1

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN
1

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN
1

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 DBSCAN

STEP 5: Move on to another random point and repeat the same steps

Clustering Basics

2 clusters
K-Means
Clustering
eps = 0.75
min_samples = 4
Hierarchical
Clustering

DBSCAN

Comparing
Models

4 outliers

*Copyright Maven Analytics, LLC

 DBSCAN DETAILED STEPS

Here are some more detailed steps of how DBSCAN works:

Clustering Basics
1. Select a radius (eps) and a minimum number of points (min_samples)

2. Within a scatterplot, select a point at random and count the points within its radius
K-Means • If it’s greater than or equal to the “min_samples”, then start a cluster, label the point as a core point,
Clustering and mark the points within its radius as a neighbor
• If it’s less than the “min_samples”, label the point as a noise point and move to step 5
Hierarchical
Clustering 3. Move to a neighbor and count the points within its radius
• If it’s greater than or equal to the “min_samples”, label it as a core point, and mark its neighbors
DBSCAN • If it’s less than the “min_samples”, but at least one of them is a core point, label it as a border point
• If it’s less than the “min_samples”, and none of them is a core point, label it as a noise point

Comparing 4. Continue with another neighbor until all the points are labeled within the cluster
Models
5. Move on to another random point and repeat the same steps

*Copyright Maven Analytics, LLC

 DBSCAN IN PYTHON

You can use sklearn’s DBSCAN() to perform DBSCAN clustering in Python

Clustering Basics

K-Means
Clustering

Hierarchical
Clustering
The radius, or maximum distance Minimum points in the radius
between neighboring points needed to become a core point,
(default is 0.5) including the point itself
DBSCAN
(default is 5)

Comparing
Models
You may have noticed that there is no random_state parameter, even though DBSCAN starts at a random point.
This is because sklearn’s implementation of DBSCAN starts at the first point in the data set and moves on from there.
You could attempt to get different labels by shuffling the data, but DBSCAN’s labels don’t vary as much as K-Mean
Clustering’s labels, so this isn’t as big of an issue.

*Copyright Maven Analytics, LLC

 DBSCAN IN PYTHON

You can use sklearn’s DBSCAN() to perform DBSCAN clustering in Python

Clustering Basics

K-Means
Clustering

Hierarchical
Clustering You can view the cluster assignments using the .labels_ attribute
(-1 values represent noise points)

DBSCAN

Comparing
Models There are too many noise
points (-1) here, let’s tweak
eps and min_samples to
mainly see clusters instead

*Copyright Maven Analytics, LLC

 SILHOUETTE SCORE

While intuition is a large part of comparing clustering models, you can also use metrics
like silhouette score to help compare models

Clustering Basics
• Scores range from -1 to +1, with higher values meaning data points are highly matched to
their own cluster and poorly matched to other clusters (which is a good thing!)

K-Means
Clustering

Hierarchical
Clustering

DBSCAN

Comparing
Models Silhouette score = positive Silhouette score ~ zero Silhouette score = negative
The data point fits well in its cluster The data point is near the boundary The data point doesn't fit well in its
and is distant from other clusters between two clusters own cluster and might belong to a
neighboring cluster

*Copyright Maven Analytics, LLC

 SILHOUETTE SCORE

The following formula is used to calculate the silhouette score for each data point:

Smallest average distance to Average distance to other

Clustering Basics
points in a different cluster points in the same cluster

K-Means 𝑏(𝑖) − 𝑎(𝑖)

Clustering 𝑆𝑖 =
max{𝑎 𝑖 , 𝑏 𝑖 }
Hierarchical Silhouette score for
the ith data point The denominator normalizes the difference so the
Clustering silhouette score ends up between -1 and 1

DBSCAN
1.04
1.08 𝟏. 𝟎𝟒 − 𝟎. 𝟑𝟐 The data point fits
= 𝟎. 𝟔𝟗 well in its cluster!
Comparing 𝟏. 𝟎𝟒
Models

To calculate the overall silhouette score for a

0.32 particular model, average all the silhouette scores

*Copyright Maven Analytics, LLC

 SILHOUETTE SCORE IN PYTHON

You can use sklearn’s silhouette_score() function to calculate the silhouette

score for the results of a clustering model in Python
Clustering Basics

K-Means
Clustering

Hierarchical
Clustering

DBSCAN
The data you fit The labels your Distance calculation used to Size of the random subset of data
the model on model generated calculate the silhouette score used to calculate the score
Comparing
(required) (required) (optional, default is “euclidean”) (optional, default is None)
Models

*Copyright Maven Analytics, LLC

 ASSIGNMENT: DBSCAN

Key Objectives
NEW MESSAGE
March 19, 2024 1. Loop through multiple “eps” and “min_samples”
From: Clyde Clusters (Sr. Data Scientist) values to fit multiple DBSCAN models
Subject: DBSCAN Help 2. Apply the function on both the original and
standardized datasets
Hi again!
3. Find the highest silhouette score and note down
To get one more perspective, please create DBSCAN models on the “eps” and “min_samples” values
the original and standardized data sets using these ranges:
4. Fit a final DBSCAN model with those “eps” and
• eps: 0.1 to 2, with steps of 0.1 “min_samples” values and review the labels
• min_samples: 2 to 10, with steps of 1
Which combination of these values ends up with the highest
silhouette score?
Thanks again for all your help this week!
Clyde

*Copyright Maven Analytics, LLC

 CLUSTERING ALGORITHMS RECAP

Each clustering algorithm has pros and cons to keep in mind:

Clustering Basics

Model Pros Cons In Practice

K-Means
Clustering Must specify the number of clusters
Popular clustering model
❌

K-Means ✓ Easy to understand & interpret Different initial centroids lead to

Typically the first choice when it
❌

Clustering ✓ Scales well with large datasets different results

comes to clustering
Hierarchical ❌ Assumes clusters are roughly spherical
Clustering
Does not scale well with large datasets Good for visualization
Hierarchical ✓ No need to predefine “k” upfront
❌

Data points can connect to outliers, Dendrogram allows you to

Clustering ✓ Can work with complex datasets
❌

which distorts distance calculations explore the clusters visually

DBSCAN

Good for outliers

✓ No need to predefine “k” upfront Does not scale well with large datasets
DBSCAN Effectively identifies & handles
❌

Comparing ✓ Can work with complex datasets Hyperparameter tuning is challenging

noise points in the data
❌

Models

*Copyright Maven Analytics, LLC

 CLUSTERING ALGORITHMS RECAP

Here is how each clustering algorithms compares on different data sets:

Clustering Basics

K-Means
K-Means Clustering
Clustering

Hierarchical
Clustering
Hierarchical
Clustering
DBSCAN

Comparing
Models
DBSCAN

Sou rce: [Link]

*Copyright Maven Analytics, LLC
 CLUSTERING NEXT STEPS

Once you’ve fit & interpreted your initial models, the potential next steps are to:

Clustering Basics
1.
1 Compare clustering models
K-Means • Silhouette Score: Higher scores closer to 1 mean that clusters are better defined
Clustering
• Intuition: At the end of the day, the main objective is to answer your business question, so you
want to pick the clusters that make the most sense for you to make decisions
Hierarchical
Clustering
2.
2 Label unseen data
DBSCAN • Data Prep: Before labeling unseen data, you must apply the same transformations you applied
to the original data set (feature scaling, etc.) when creating the clusters
Comparing • K-Means Clustering: The .predict method lets you assign unseen data points to a cluster
Models
• Hierarchical Clustering & DBSCAN: While there is no .predict method for these, you can
label unseen data points by re-fitting the models (you may need to update the parameters)

*Copyright Maven Analytics, LLC

 KEY TAKEAWAYS

Clustering lets you find groups of observations similar to one another

• Common clustering techniques include K-Means Clustering, Hierarchical Clustering, and DBSCAN

In practice, follow the clustering workflow

1) Prepare your data for clustering (correct row granularity, non-null & numeric values, feature scaling, etc.)
2) Start with K-Means Clustering, then tune and select a “best” model
3) If you want better results, try Hierarchical Clustering or DBSCAN, then tune and select a “best” model
4) Compare models using silhouette scores & intuition, and select the one that best answers your business question
CLUSTERING PROJECT

*Copyright Maven Analytics, LLC

 PROJECT: CLUSTERING CLIENTS

THE You work as a Data Scientist for Northwind Traders, a wholesale distributor that
SITUATION supplies gourmet food products to restaurants, cafes, and specialty food retailers

You have access to their yearly client report, which includes the channel & region for
THE each client, as well as their total spend by product categories in the last year
ASSIGNMENT Your task is to use clustering techniques to understand the different client segments
so the company can better cater to and support each type

THE 1. Scale the data

OBJECTIVES 2. Apply 3 different clustering techniques
3. Compare the segments using silhouette score
& intuition and select the “best” segments
4. Predict the cluster of a new client

*Copyright Maven Analytics, LLC

ANOMALY DETECTION

*Copyright Maven Analytics, LLC

 ANOMALY DETECTION

In this section we’ll introduce the concept of anomaly detection and cover two popular
unsupervised learning techniques for finding anomalies: Isolation Forests and DBSCAN

TOPICS WE’LL COVER: GOALS FOR THIS SECTION:

Anomaly Detection • Review where anomaly detection sits within the data
Isolation Forests science workflow
Basics

• Learn how popular unsupervised techniques for

DBSCAN Comparing Models anomaly detection fundamentally work

• Apply and interpret the results of Isolation Forests

and DBSCAN models for anomaly detection

• Compare & contrast unsupervised techniques for

anomaly detection

*Copyright Maven Analytics, LLC

 ANOMALY DETECTION BASICS

Anomaly detection, also known as outlier detection, is used to identify

observations in a data set that are significantly different from the others
Anomaly
Detection Basics

Isolation Forests

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 ANOMALY DETECTION BASICS

Anomaly detection, also known as outlier detection, is used to identify

observations in a data set that are significantly different from the others
Anomaly
Detection Basics

This anomaly could not be detected via 2D scatter

plots alone, but is clearly an anomaly in 3D
Isolation Forests

DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 ANOMALY DETECTION IN THE DS WORKFLOW

1 2 3 4 5 6
Anomaly
Detection Basics Scoping a Gathering Cleaning Exploring Modeling Sharing
project data data data data insights
Isolation Forests

Data Issues Insights

DBSCAN

Identify data points to exclude to Identify data points to look into to

avoid skewing future analysis provide meaningful information
Comparing
Models
• Typically called outlier detection • Typically called anomaly detection
• Done using statistics or plots • Done using either supervised or
unsupervised learning models
• Can use models to detect outliers
that are difficult or impossible to • Can use unsupervised learning
detect with statistics or plots models if you don’t have any
alone information on past anomalies
(data is unlabeled)

*Copyright Maven Analytics, LLC

 ANOMALY DETECTION APPROACHES

Popular approaches to anomaly detection include:

Anomaly • Statistical analysis: flagging data points more than 3 standard deviations away
Detection Basics
from the mean

Isolation Forests • Data visualization: visually identifying distant points by creating histograms,
scatterplots, or boxplots
DBSCAN
• Supervised learning: if you have information on past anomalies, you can apply
classification algorithms like Random Forests to identify new ones
Comparing
Models • Unsupervised learning: if you have unlabeled data, you need algorithms like
Isolation Forests and DBSCAN

Many machine learning algorithms can be modified for anomaly detection including K-Nearest
Neighbors, One-Class Support Vector Machines, Time Series Analysis, etc.

*Copyright Maven Analytics, LLC

 UNSUPERVISED LEARNING TECHNIQUES

You can perform anomaly detection using unsupervised learning techniques

Anomaly
Detection Basics
Type of Technique Unsupervised Learning Techniques Common Applications

Isolation Forests
K-Means Clustering

Clustering
DBSCAN Hierarchical Clustering Clustering
Techniques
DBSCAN (Density-Based Clustering)
Comparing Anomaly Detection
Models Isolation Forests

PCA (Principal Component Analysis)

Dimensionality
Reduction T-SNE (t-Distributed Stochastic Neighbor Embedding)
Techniques
SVD (Singular Value Decomposition)

*Copyright Maven Analytics, LLC

 ANOMALY DETECTION WORKFLOW

The general anomaly detection workflow using unsupervised machine learning

techniques consists of the following steps:
Anomaly
Detection Basics

Data Prep Modeling Tuning Selection

Isolation Forests

Get your data ready to be Apply an algorithm for Evaluate & tune the model Pick the best results and
input into an ML model anomaly detection using plots and intuition identify any insights
DBSCAN

• Single table, non-null • Isolation Forests • Data visualization • Business objective

and numeric data • DBSCAN • Interpret the results • Domain expertise
Comparing
Models • Feature engineering,
selection and scaling

Remember, there’s no “right” answer or single optimization metric when it comes to anomaly detection; the best
outputs are the ones which help you answer the question at hand and make practical, data-driven business decisions

*Copyright Maven Analytics, LLC

 ISOLATION FORESTS

An Isolation Forest detects anomalies by building decision trees that split the
data randomly and measure how quickly specific data points can be isolated
Anomaly
Detection Basics

Isolation Forests

TV Shows

<3.6 >3.6
DBSCAN hours hours

Comparing
Models
Books

<0.7 >0.7
hours hours

*Copyright Maven Analytics, LLC

 ISOLATION FORESTS

An Isolation Forest detects anomalies by building decision trees that split the
data randomly and measure how quickly specific data points can be isolated
Anomaly
Detection Basics Here’s how it works:
1. Randomly select a feature from the data and split the observations using a random threshold
Isolation Forests
• If a value is under the threshold, it goes to one branch; otherwise, it goes to the other

DBSCAN 2. Continue splitting the branches using random features and thresholds, creating a tree
structure, until every data point is isolated or a maximum depth is reached

Comparing 3. Repeat these steps using different features and splits to create multiple trees (aka a forest)
Models
4. Calculate the anomaly score for each observation by averaging the number of splits it took to
isolate (path length) in each tree – the lowest scores are anomalies!

Example use cases:

• Detect fraudulent activities in financial data or credit card transactions
• Detect unexpected malfunctions in sensor data or unusual test results in patient data

*Copyright Maven Analytics, LLC

 ISOLATION FORESTS

STEP 1: Randomly select a Features

feature from the data and Books

Anomaly split the observations using TV Shows

TV Shows
Detection Basics
a random threshold <3.6
hours
>3.6
hours
Video Games

Isolation Forests

Anomaly score =1
DBSCAN

Comparing
Models

*Copyright Maven Analytics, LLC

 ISOLATION FORESTS

STEP 2: Continue splitting Features

using random features and Books

Anomaly thresholds until every data TV Shows

TV Shows
Detection Basics
point is isolated or a max <3.6
hours
>3.6
hours
Video Games

depth is reached
Isolation Forests

Anomaly score =1 Books

DBSCAN
<0.7 >0.7
hours hours

These two students

Comparing are likely anomalies
Models

Anomaly score =2 Video Games

<5.5 >5.5
hours hours

These would keep splitting until each

point is isolated or a maximum depth is
reached (i.e. number of splits = 3)

*Copyright Maven Analytics, LLC

 ISOLATION FORESTS

STEP 3: Repeat using different features and splits to create multiple trees

Anomaly
Detection Basics

Isolation Forests
SPLIT 1 SPLIT 2 SPLIT N

Books Books Books

DBSCAN TV Shows TV Shows TV Shows

Video Games Video Games Video Games

Comparing
Models

*Copyright Maven Analytics, LLC

 ISOLATION FORESTS

STEP 3: Repeat using different features and splits to create multiple trees

Anomaly
Detection Basics

Isolation Forests
SPLIT 1 SPLIT 2 SPLIT N

Books Books Books

DBSCAN TV Shows TV Shows TV Shows

Video Games Video Games Video Games

Comparing
Models

*Copyright Maven Analytics, LLC

 ISOLATION FORESTS

STEP 3: Repeat using different features and splits to create multiple trees

Anomaly
Detection Basics

Isolation Forests
SPLIT 1 SPLIT 2 SPLIT N

Books Books Books

DBSCAN TV Shows TV Shows TV Shows

Video Games Video Games Video Games

Comparing
Models

*Copyright Maven Analytics, LLC

 ISOLATION FORESTS

STEP 3: Repeat using different features and splits to create multiple trees

Anomaly
Detection Basics

Isolation Forests
SPLIT 1 SPLIT 2 SPLIT N

Books Books Books

DBSCAN TV Shows TV Shows TV Shows

Video Games Video Games Video Games

Comparing
Models

*Copyright Maven Analytics, LLC

 ISOLATION FORESTS

STEP 4: Calculate the anomaly score for each observation by averaging the
number of splits it took to isolate (path length) in each tree
Anomaly
Detection Basics TREE 1
ANOMALY SCORE This is for a single data point

Isolation Forests 1
SPLIT 1 SPLIT 2 SPLIT N
TREE 2
Books Books Books ANOMALY SCORE

DBSCAN FINAL
TV Shows TV Shows TV Shows 3 ANOMALY SCORE

1.75
Video Games Video Games Video Games
TREE 3
Comparing ANOMALY SCORE

Models
2
TREE N Since the score is so
ANOMALY SCORE
low, this value is
likely an anomaly!
1

*Copyright Maven Analytics, LLC

 ISOLATION FORESTS IN PYTHON

You can use sklearn’s IsolationForest() to apply Isolation Forests in Python

Anomaly
Detection Basics

Isolation Forests

DBSCAN

The percentage of data points Setting a random_state value

Comparing you’d like to flag as an anomaly guarantees the same results
Models (optional, between 0 - 0.5) each time the model is fit

PRO TIP: Even though there is an option to set the contamination percentage
upfront, you can also decide the number of anomalies after fitting the model

*Copyright Maven Analytics, LLC

 ISOLATION FORESTS IN PYTHON

You can use sklearn’s IsolationForest() to apply Isolation Forests in Python

Anomaly
Detection Basics

Isolation Forests
Once the model is fit, you can view the scaled anomaly scores
(-0.5 to 0.5) using the .decision_function()
DBSCAN

Comparing
Models

Use .predict() to flag anomalies (-1) as

determined by the contamination you set

*Copyright Maven Analytics, LLC

 VISUALIZING ANOMALIES

You can visualize anomalies by using the anomaly flags as the “hue” in a pair plot

Anomaly
Detection Basics

Isolation Forests

DBSCAN

Comparing
Models Visually, this student watches less TV
and reads fewer books than the rest

*Copyright Maven Analytics, LLC

 VISUALIZING ANOMALIES

You can visualize anomalies by using the anomaly flags as the “hue” in a pair plot

Anomaly
Detection Basics

Isolation Forests

DBSCAN

Comparing
Models The student that looked like an
anomaly in two dimensions was not
flagged using Isolation Forests
Instead, the anomalies were the
students that watched a lot of TV
and played a lot of video games

*Copyright Maven Analytics, LLC

 TUNING & INTERPRETING ISOLATION FORESTS

After you view the results of your first Isolation Forest model, you can:
• Tune the model by updating the contamination percent to flag more or fewer anomalies
Anomaly
Detection Basics
• Interpret the results by filtering on the anomaly labels or sorting the anomaly scores

Isolation Forests

DBSCAN

Comparing
Models

Based on your objective, you may decide

to either exclude the anomalies before
doing any further analysis or dig into Some “anomaly” students consumed a ton of
them to discover insights entertainment, while others read almost no books

*Copyright Maven Analytics, LLC

 ASSIGNMENT: ISOLATION FORESTS

Key Objectives
NEW MESSAGE
March 20, 2024 1. Open the tripadvisor_reviews.csv file, remove the
From: Molly Anomaly (Researcher) user_id column and view the range of each rating
Subject: Identify Tourist Anomalies 2. Visualize the data using a seaborn pair plot

3. Fit an Isolation Forest model using a

Hi there!
contamination of 0.01
We just received the results of our quarterly tourist survey,
where we have tourists rate their experiences at our town’s 4. Visualize the anomalies on the seaborn pair plot
museums, parks, restaurants, and nightlife locations.
5. Notice where there are anomalies in the pair plot
There are always a few tourists whose feedback really skews
our results. Could you identify them using that anomaly 6. Modify the contamination to 0.005, visualize the
detection technique you mentioned earlier (Isolation anomalies and note the differences
Forests)?

Thanks!
Molly

tripadvisor_reviews.csv

*Copyright Maven Analytics, LLC

 DBSCAN FOR ANOMALY DETECTION

DBSCAN (Density-Based Spatial Clustering of Applications with Noise), which is

a clustering technique we covered, can also be used for anomaly detection
Anomaly
Detection Basics

Isolation Forests

DBSCAN

Comparing
Models

4 outliers

*Copyright Maven Analytics, LLC

 DBSCAN FOR ANOMALY DETECTION IN PYTHON

You can use sklearn’s DBSCAN() to apply the DBSCAN algorithm in Python

Anomaly
Detection Basics

Isolation Forests

DBSCAN
The radius, or maximum distance Minimum points in the radius
between neighboring points needed to become a core point,
Comparing (default is 0.5) including the point itself
Models
(default is 5)

Since DBSCAN is a distance-based algorithm, the data should be scaled before being input into the model.
This is different than Isolation Forests, which is NOT distance-based, so no scaling is needed

*Copyright Maven Analytics, LLC

 DBSCAN FOR ANOMALY DETECTION IN PYTHON

You can use sklearn’s DBSCAN() to apply the DBSCAN algorithm in Python

Anomaly
Detection Basics

Isolation Forests

DBSCAN
You can view the cluster assignments using the .labels_ attribute
(-1 values represent noise points)
Comparing
Models

For anomaly detection, the

cluster labels are irrelevant,
but the noise points are the
anomalies we’re looking for!

*Copyright Maven Analytics, LLC

 VISUALIZING DBSCAN ANOMALIES

You can visualize anomalies by using the anomaly labels as the “hue” in a pair plot

Anomaly
Detection Basics

Isolation Forests

DBSCAN

The student that looked like an

Comparing anomaly in two dimensions WAS
Models flagged using DBSCAN
Additionally, the anomalies were
the students that consumed the
most entertainment

*Copyright Maven Analytics, LLC

 ASSIGNMENT: DBSCAN FOR ANOMALY DETECTION

Key Objectives
NEW MESSAGE
March 21, 2024 1. Paste the DBSCAN function from the anomaly
From: Molly Anomaly (Researcher) detection demo notebook
Subject: RE: Identify Tourist Anomalies 2. Apply the function on the tourist rating data set

3. Find the highest silhouette score and note down

Hi again!
the eps and min_samples values
Thanks for the Isolation Forest results earlier - those
anomalies were very interesting. 4. Fit a single DBSCAN model using those eps and
min_sample values
Can you try using another technique on the data set to
confirm our results? 5. Note the anomalies (-1) and visualize them on a
pair plot
This time, please give DBSCAN a try and visualize the results.

Thanks!
Molly

*Copyright Maven Analytics, LLC

 RECAP: ANOMALY DETECTION

Each anomaly detection technique has pros and cons to keep in mind:
Anomaly
Detection Basics

Model Pros Cons In Practice

Isolation Forests ✓ Great for high-dimensional data Popular anomaly detection choice
Isolation Not as good for low-dimensional data
✓ Computationally efficient Typically the first choice when it comes
❌

Forests Not as good for local anomalies

✓ Great for global anomalies to anomaly detection
❌

DBSCAN
Best for local anomaly detection
✓ Handles complex cluster shapes Computationally intensive
DBSCAN Lets you identify local anomalies based
❌

✓ Great for local anomalies Hyperparameter tuning is challenging

on the internal clusters in the data
❌

Comparing
Models

*Copyright Maven Analytics, LLC

 RECAP: CLUSTERING & ANOMALY DETECTION

So far, we’ve covered two unsupervised learning concepts:

Anomaly
Detection Basics

Isolation Forests Clustering Anomaly Detection

Used to identify data points that are Used to identify data points that are
DBSCAN similar to one another different from the rest

Unsupervised learning techniques: Unsupervised learning techniques:

Comparing
Models • K-Means Clustering • Isolation Forests
• Hierarchical Clustering • DBSCAN
• DBSCAN

Although they are different concepts, the common thread is that you’re comparing data points to
determine how similar and different they are from one another.
Unsupervised learning is all about finding relationships in data, and these are two ways to capture that

*Copyright Maven Analytics, LLC

 KEY TAKEAWAYS

Anomaly detection aims to find observations different than the rest

• Common unsupervised techniques for anomaly detection are Isolation Forests and DBSCAN
• There are other ways to detect anomalies, including statistics, data visualization, and supervised learning

For unsupervised learning, follow the anomaly detection workflow

1) Prep your data for anomaly detection (scaling is not required for Isolation Forests, but it is for DBSCAN)
2) Start with Isolation Forests, then visualize, tune, and select a model
3) Try DBSCAN for different results, then visualize, tune and select a model
4) Compare the anomalies from all the models using data visualization, exploratory analysis, and your intuition
5) Select the model that best addresses your business question
6) You can choose to further explore or exclude anomalies detected by one or more models

*Copyright Maven Analytics, LLC

DIMENSIONALITY REDUCTION

*Copyright Maven Analytics, LLC

 DIMENSIONALITY REDUCTION

In this section we’ll introduce the fundamentals of dimensionality reduction and cover
two popular techniques: Principal Component Analysis (PCA) and t-SNE

TOPICS WE’LL COVER: GOALS FOR THIS SECTION:

• Review where dimensionality reduction sits within

Dimensionality Reduction PCA the data science workflow

• Learn how popular dimensionality reduction

t-SNE techniques fundamentally work

• Apply and interpret the results of PCA and t-SNE

models in Python

*Copyright Maven Analytics, LLC

 DIMENSIONALITY REDUCTION BASICS

Dimensionality reduction involves reducing the number of columns (dimensions)

in a data set while losing as little information as possible
Dimensionality
Reduction

The “pc1” and “pc2” columns each hold a

PCA combination of information from the “books”,
“tv_shows” and “video_games” columns

t-SNE
How is this different
from feature selection?
• Feature selection involves
dropping entire columns
• Dimensionality reduction
involves transforming the
original columns
3 dimensions 2 dimensions

*Copyright Maven Analytics, LLC

 DIMENSIONALITY REDUCTION BASICS

Dimensionality reduction involves reducing the number of columns (dimensions)

in a data set while losing as little information as possible
Dimensionality
Reduction Notice how with fewer dimensions
it’s easier to see clusters in the data!
PCA

t-SNE

3 dimensions 2 dimensions

*Copyright Maven Analytics, LLC

 WHY REDUCE DIMENSIONS?

Reducing the numbers of dimensions in a data set helps with these issues:

Dimensionality
Reduction

PCA
EDA Supervised Learning Unsupervised Learning

It’s difficult for humans to Predictive models fit on “wide” Data points spread out with
t-SNE visualize data with more than data don’t perform as well as more dimensions, and their
2 or 3 dimensions clearly those fit on “long” data distances become more similar
(curse of dimensionality)
Solution: Solution:
Solution:
• Using dimensionality • By reducing dimensions
reduction, 4+ columns in feature engineering, • By reducing dimensions,
can be reduced to 2 or 3 the data gets “narrower” distance calculations
become more meaningful

*Copyright Maven Analytics, LLC

 DIMENSIONALITY REDUCTION IN DS WORKFLOW

1 2 3 4 5 6
Scoping a Gathering Cleaning Exploring Modeling Sharing
Dimensionality project data data data data insights
Reduction

PCA PCA / t-SNE PCA SVD

Can be used as a machine learning Can be used for Feature Engineering Can be used as an unsupervised
t-SNE
alternative to Data Visualization when preparing supervised learning learning technique for Recommenders

• Data is typically visualized using • Feature selection or engineering • SVD is a general matrix
two dimensions (x and y-axes) is typically done manually by factorization technique that
removing features or applying encompasses PCA
• By using PCA or t-SNE, high-
transformations to create them
dimensional data can also be • SVD reduces the dimensions of a
visualized in two dimensions • By using PCA, multiple fields can user’s behavior to the most
be mathematically reduced to important so that they can be
fewer fields, which is called better compared with others
feature extraction
We’ll cover SVD in the
Recommenders section

*Copyright Maven Analytics, LLC

 DIMENSIONALITY REDUCTION WORKFLOW

The general dimensionality reduction workflow consists of the following steps:

Dimensionality
Reduction
Data Prep Modeling Tuning Selection
PCA
Get your data ready to be Apply a dimensionality Evaluate & tune the model Pick the best results and
input into an ML model reduction technique using metrics and intuition identify any insights

t-SNE
• Single table, non-null • PCA • Explained variance • Business objective
and numeric data • t-SNE • Interpret the results • Domain expertise
• Feature engineering,
• SVD
selection and scaling

Remember that dimensionality reduction is often a step taken before other modeling steps, so in addition to going
through the workflow above, you may need to follow it up with other supervised or unsupervised learning workflows

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

Principal Component Analysis (PCA) is a dimensionality reduction technique that

finds linear combinations of features that explain the most variation in the data
Dimensionality
Reduction

PCA

t-SNE
y

x x

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

Principal Component Analysis (PCA) is a dimensionality reduction technique that

finds linear combinations of features that explain the most variation in the data
Dimensionality Here’s how it works:
Reduction
1. Create a scatter plot and find the center of the data
PCA 2. Trace a line through the center that captures the most spread, or variation, in the data –
this is the first principal component (PC1)

t-SNE 3. Create another line perpendicular to the first one that captures the most spread, or
variation, in the data– this is the second principal component (PC2)
4. Repeat Step 3 until you have as many principal components as original columns
5. To reduce dimensions, keep a subset of principal components (i.e. PC 1 only)

Example use cases:

• Visualizing clusters in fewer dimensions to spot-check clustering/segmentation results
• Improving a model’s predictive power by reducing redundant or unnecessary dimensions

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 1: Create a scatter plot and find the center of the data

Dimensionality
Reduction

PCA

t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 1: Create a scatter plot and find the center of the data

Dimensionality
Reduction

PCA

t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 2: Trace a line through the center that captures the most spread, or
variation, in the data – this is the first principal component (PC 1)
Dimensionality
Reduction

PCA

t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 2: Trace a line through the center that captures the most spread, or
variation, in the data – this is the first principal component (PC 1)
Dimensionality
Reduction

PCA
Start with a
random line
t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 2: Trace a line through the center that captures the most spread, or
variation, in the data – this is the first principal component (PC 1)
Dimensionality
Reduction

PCA

t-SNE
y

Project each point onto

the line at a 90° angle

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 2: Trace a line through the center that captures the most spread, or
variation, in the data – this is the first principal component (PC 1)
Dimensionality
Reduction

PCA d12

t-SNE
y

Sum the squared distances between the

projected points and the center

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 2: Trace a line through the center that captures the most spread, or
variation, in the data – this is the first principal component (PC 1)
Dimensionality
Reduction

PCA d12 + d22

t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 2: Trace a line through the center that captures the most spread, or
variation, in the data – this is the first principal component (PC 1)
Dimensionality
Reduction

PCA d12 + d22 + d32

t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 2: Trace a line through the center that captures the most spread, or
variation, in the data – this is the first principal component (PC 1)
Dimensionality
Reduction

PCA d12 + d22 + d32 + d42

t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 2: Trace a line through the center that captures the most spread, or
variation, in the data – this is the first principal component (PC 1)
Dimensionality
Reduction

PCA d12 + d22 + d32 + d42 + d52

t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 2: Trace a line through the center that captures the most spread, or
variation, in the data – this is the first principal component (PC 1)
Dimensionality
Reduction

PCA d12 + d22 + d32 + d42 + d52 + d62

t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 2: Trace a line through the center that captures the most spread, or
variation, in the data – this is the first principal component (PC 1)
Dimensionality
Reduction

PCA d12 + d22 + d32 + d42 + d52 + d62 + d72

t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 2: Trace a line through the center that captures the most spread, or
variation, in the data – this is the first principal component (PC 1)
Dimensionality
Reduction

PCA d12 + d22 + d32 + d42 + d52 + d62 + d72 + d82 = 4.8

t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 2: Trace a line through the center that captures the most spread, or
variation, in the data – this is the first principal component (PC 1)
Dimensionality
Reduction

PCA d12 + d22 + d32 + d42 + d52 + d62 + d72 + d82 = 4.8
Find the line that maximizes
the sum of squared distances
t-SNE
y
Why maximize the distance?
• This guarantees that the
principal component line
captures the most variation
in the data
x

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 2: Trace a line through the center that captures the most spread, or
variation, in the data – this is the first principal component (PC 1)
Dimensionality
Reduction

Try a new line

PCA

t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 2: Trace a line through the center that captures the most spread, or
variation, in the data – this is the first principal component (PC 1)
Dimensionality
Reduction

PCA d12 + d22 + d32 + d42 + d52 + d62 + d72 + d82 = 9.4
The distance almost doubled!
t-SNE Note that the points are
y more spread out now

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 2: Trace a line through the center that captures the most spread, or
variation, in the data – this is the first principal component (PC 1)
Dimensionality
Reduction

PCA

t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 2: Trace a line through the center that captures the most spread, or
variation, in the data – this is the first principal component (PC 1)
Dimensionality
Reduction

PCA d12 + d22 + d32 + d42 + d52 + d62 + d72 + d82 = 12.2
This is the maximum sum
of squared distances
t-SNE The points are even more
y spread out now

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 2: Trace a line through the center that captures the most spread, or
variation, in the data – this is the first principal component (PC 1)
Dimensionality
Reduction

PCA

t-SNE
y

The line captures the

widest spread in our data

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 3: Create another line perpendicular to the first one that captures the
most spread, or variation, in the data – this is PC2
Dimensionality
Reduction

PCA

t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 4: Repeat Step 3 until you have as many PCs as original columns

Dimensionality
Reduction

PCA
We can’t draw another perpendicular line
in a 2-dimensional space, so we’re all set!
t-SNE
y

In practice, these steps are completed

using a linear algebra technique called
eigendecomposition

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 5: Transform the data into a new space with the PC 1 line as the x-axis, the
PC2 line as the y-axis, and so on
Dimensionality
Reduction

PCA

t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 5: Transform the data into a new space with the PC 1 line as the x-axis, the
PC2 line as the y-axis, and so on
Dimensionality
Reduction

PCA

t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 6: To reduce dimensions, keep a subset of principal components – for

example, remove PC2 and project the points onto PC1, which is your new x-axis
Dimensionality
Reduction

PCA

t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 6: To reduce dimensions, keep a subset of principal components – for

example, remove PC2 and project the points onto PC1, which is your new x-axis
Dimensionality
Reduction

PCA

t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 6: To reduce dimensions, keep a subset of principal components – for

example, remove PC2 and project the points onto PC1, which is your new x-axis
Dimensionality
Reduction

PCA

t-SNE
y

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

STEP 6: To reduce dimensions, keep a subset of principal components – for

example, remove PC2 and project the points onto PC1, which is your new x-axis
Dimensionality
Reduction

PCA

t-SNE

PC1

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

By using principal component analysis, we’ve reduced the number of columns

(dimensions) in the data while losing as little information as possible
Dimensionality
Reduction

PCA

PC1 captures information about both

t-SNE x and y, but all within 1 dimension
y instead of 2 dimensions

x PC1

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS

By using principal component analysis, we’ve reduced the number of columns

(dimensions) in the data while losing as little information as possible
Dimensionality
Reduction

PCA

t-SNE These clusters are easier to distinguish

y within fewer dimensions

x x

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS IN PYTHON

You can use sklearn’s PCA() to apply the Principal Component Analysis algorithm

Dimensionality
Reduction

PCA

t-SNE
The number of columns you want
to reduce the dataset into

PRO TIP: For visualization, set n_components to 2 or 3; for feature

extraction, use the number of columns in your original data set

*Copyright Maven Analytics, LLC

 PRINCIPAL COMPONENT ANALYSIS IN PYTHON

You can use sklearn’s PCA() to apply the Principal Component Analysis algorithm

Dimensionality
Reduction

You need to center each column around zero

PCA before applying a PCA model by either:
• Subtracting the mean from each value
• Standardizing the columns
t-SNE

PRO TIP: If the columns are on

very different scales, it is
recommended that you scale the
data as well, which is something
that standardization takes care of
Principal components 1 and 2
have been determined!

*Copyright Maven Analytics, LLC

 EXPLAINED VARIANCE RATIO

The explained variance ratio shows how much of the variation from the data is
captured by each principal component
Dimensionality
Reduction • The first component will always capture the most variance, and then it decreases from there
• The sum of the explained variance ratios across all components will be equal to 1
PCA

t-SNE
You can represent 88% of the information
captured in the 3 original columns by only
keeping one principal component

You can represent 97% of the information

captured in the 3 original columns by
PRO TIP: Typically you want your keeping two principal components
components to capture 80%+ of
the variance in your data

*Copyright Maven Analytics, LLC

 ASSIGNMENT: PRINCIPAL COMPONENT ANALYSIS

Key Objectives
NEW MESSAGE
March 25, 2024
1. Read in the student grades data set
From: Tim Menschen (Assistant Principal)
Subject: Student Analysis 2. Drop the first column with student_id
3. Center the data
Hello!
4. Fit a PCA model with 2 components
Our guidance counselor would like to understand the types of
students at our school to recommend appropriate colleges. 5. View and interpret the explained variance ratios
I was a data scientist in my past life, and I remember that PCA
helps visualize many dimensions in two dimensional charts.

Could you apply PCA on the attached student grades data set
and let me know if two dimensions are able to capture a decent
amount of variance in the data?

Thanks!
Tim

student_grades.csv

*Copyright Maven Analytics, LLC

 INTERPRETING PCA

The .components_ attribute lets you interpret the principal components

Dimensionality
Reduction

PCA
Each row represents a principal component, and each
column represents a column from the original data set

t-SNE

PC 1: Higher values = more books (0.93) PC 2: Higher values = more TV shows (0.89)
and some more video games (0.34) and some more video games (0.43)

*Copyright Maven Analytics, LLC

 VISUALIZING PCA

You can use the .transform() method to transform the original observations into
the new dimensional space, which you can then visualize in a scatter plot

Dimensionality
Reduction

PCA

t-SNE

Students in the left cluster read

very few books

Students in the top right corner

love all 3 types of entertainment

*Copyright Maven Analytics, LLC

 ASSIGNMENT: INTERPRETING & VISUALIZING PCA

Key Objectives
NEW MESSAGE
March 26, 2024 1. Interpret the components of the PCA model
From: Tim Menschen (Assistant Principal) 2. Plot the students on a scatter plot with the x-axis
Subject: Student Analysis Visualization as PC1 and the y-axis as PC2

3. Interpret the clusters of students that you see on

Hi again!
the plot and make recommendations for the
Thanks for kicking off the PCA analysis earlier. I’m excited to guidance counselor
hear that the explained variance ratio is high enough so that we
can get a good idea of the students in just two dimensions!

Can you plot the students in the two dimensions, explain what
the axes represent, and what you observe visually in the plot?

I’ll pass that info along to the guidance counselor.

Thanks for your help!

Tim

*Copyright Maven Analytics, LLC

 FEATURE SELECTION VS FEATURE EXTRACTION

Remember that feature selection and feature extraction are both ways to
reduce the number of dimensions or columns in a table

Dimensionality
Reduction Feature Selection Feature Extraction

PCA

t-SNE

For this particular data set, PCA didn’t perform much

better than simple feature selection, since the “books”
and “tv_shows” variables highlight the same clusters

*Copyright Maven Analytics, LLC

 FEATURE SELECTION VS FEATURE EXTRACTION

Remember that feature selection and feature extraction are both ways to
reduce the number of dimensions or columns in a table

Dimensionality
Reduction

PCA

t-SNE

In a data set like this one, reducing the 4 columns

of data using PCA to just 2 lets you convey a lot of
information in a single scatter plot

Raisin Bran is healthy but sugary, and Cheerios

are probably the best cereal you can eat

*Copyright Maven Analytics, LLC

 PCA NEXT STEPS

Once you have fit and interpreted a PCA model, the next steps are:

Dimensionality
Data Visualization
Reduction
• If you’re able to distinguish data points visually, you’re all set!
• If it’s difficult to visually distinguish data points:
PCA
• You can vary the inputs, modify the number of dimensions, etc.
• You can try other dimensionality reduction techniques, such as t-SNE (coming up next!)
t-SNE
Modeling
• Once you’ve identified the principal components, they can used be inputs into a machine
learning model
• You’ll need to apply the same PCA transformations to the test data

While PCA is a good technique for reducing dimensions to potentially increase predictive model performance,
one of the main downsides is that the transformed features are difficult to interpret

*Copyright Maven Analytics, LLC

 T-SNE

t-Distributed Stochastic Neighbor Embedding (t-SNE) is a non-linear

dimensionality reduction technique that emphasizes relationships between points

Dimensionality
Reduction
B R
S
A C Q
Z
PCA D T
U
X

X
t-SNE Q
U

Z
C
D
B
S A
T

t-SNE isn’t concerned with maintaining the absolute distances between the points accurate,
instead it focuses on offering a useful estimate of the relationships between them

*Copyright Maven Analytics, LLC

 T-SNE

t-Distributed Stochastic Neighbor Embedding (t-SNE) is a non-linear

dimensionality reduction technique that emphasizes relationships between points

Dimensionality Here’s how it works:

Reduction

1. Calculate the distance between points in a high-dimensional space

PCA 2. Use the distances to calculate the affinity score (0-1) for each point with the rest; in other words,
how likely it is that they are “neighbors” (high affinity score = high probability of being neighbors)

t-SNE 3. Calculate the average affinity score for each pair of points
4. Randomly place the data points in a low dimensional space
5. Use the affinity scores between pairs to attract (high affinity) or repulse (low affinity) the points until
they have “stabilized” into their final position

Example use cases:

• Visualizing clusters in limited dimensions to spot-check cluster/segmentation outputs

*Copyright Maven Analytics, LLC

 T-SNE

STEP 1: Calculate the distance between points in a high-dimensional space

Dimensionality
Reduction

B
PCA
A C

t-SNE
X
Q
U

T S

*Copyright Maven Analytics, LLC

 T-SNE

STEP 1: Calculate the distance between points in a high-dimensional space

Dimensionality
Reduction

PCA B distanceAB = 1.5

A C

t-SNE
X
Q
U

T S

*Copyright Maven Analytics, LLC

 T-SNE

STEP 1: Calculate the distance between points in a high-dimensional space

Dimensionality
Reduction

PCA B distanceAB = 1.5

A C

D distanceAX = 5.3
t-SNE
X
Q
U

T S

*Copyright Maven Analytics, LLC

 T-SNE

STEP 1: Calculate the distance between points in a high-dimensional space

Dimensionality
Reduction

PCA B distanceAB = 1.5 distanceZQ = 3.2

A C

D distanceAX = 5.3
t-SNE
X
Q
U

T S

*Copyright Maven Analytics, LLC

 T-SNE

STEP 1: Calculate the distance between points in a high-dimensional space

Dimensionality
Reduction

PCA B distanceAB = 1.5 distanceZQ = 3.2

A C

D distanceAX = 5.3 distanceZX = 5.3

t-SNE
X
Q
U
Point X is the same distance
R
from points A and Z
Z

T S

*Copyright Maven Analytics, LLC

 T-SNE

STEP 2: Use the distances to calculate the affinity score (0-1) for each point with
the rest (high affinity score = high probability of being neighbors)
Dimensionality
Reduction

B
PCA
A C

t-SNE
X
Q
U

T S

*Copyright Maven Analytics, LLC

 T-SNE

STEP 2: Use the distances to calculate the affinity score (0-1) for each point with
the rest (high affinity score = high probability of being neighbors)
Dimensionality
Reduction

distanceAB = 1.5 affinityAB = high

B
PCA
C
distanceZQ = 3.2 affinityZQ = high
A
D

t-SNE How does this work?

X
Q
U • Based on the density of points
R around A and the distance
Z between A and B, it’s highly likely
that B is A’s neighbor
T S
• Even though Q is further from Z
than B is from A, since Z is part of a
less dense region there is still a
high chance that Q is Z’s neighbor

*Copyright Maven Analytics, LLC

 T-SNE

STEP 2: Use the distances to calculate the affinity score (0-1) for each point with
the rest (high affinity score = high probability of being neighbors)
Dimensionality
Reduction

distanceAX = 5.3 affinityAX = low

B
PCA
C
distanceZX = 5.3 affinityZX = medium
A
D

t-SNE How does this work?

X
Q
U • Even though X is the same distance
R from A and Z, it’s more likely that X
Z is a neighbor of Z due to the sparse
density around Z
T S

*Copyright Maven Analytics, LLC

 T-SNE

STEP 2: Use the distances to calculate the affinity score (0-1) for each point with
the rest (high affinity score = high probability of being neighbors)
Dimensionality
Reduction

distanceAX = 5.3 affinityAX = low

B
PCA
C
distanceXA = 5.3 affinityXA = medium
A
D

t-SNE How does this work?

X
Q
U • The density of points around X is
R much sparser, so it’s much more
Z likely that A is a neighbor of X than
X being a neighbor of A
T S

*Copyright Maven Analytics, LLC

 T-SNE

STEP 2: Use the distances to calculate the affinity score (0-1) for each point with
the rest (high affinity score = high probability of being neighbors)
Dimensionality
Reduction

PCA B affinityAB = high affinityZQ = high

A C

D affinityAX = low affinityZX = medium

t-SNE
X
affinityXA = medium affinityXZ = medium
Q
U

If you’re curious, the affinity score is

T S calculated by putting the distances
through a Gaussian transformation and
calculating probabilities – but it’s not
something you should be worried about!

*Copyright Maven Analytics, LLC

 T-SNE

STEP 3: Calculate the average affinity score for each pair of points

Dimensionality
Reduction

PCA B
affinityAX = low affinityXA = medium
A C

t-SNE Affinity between A and X: medium-low

X
Q
U

T S

*Copyright Maven Analytics, LLC

 T-SNE

STEP 4: Randomly place the data points in a low dimensional space

Dimensionality
Reduction

PCA D
Q

S Z

t-SNE C

R A

B U
T

*Copyright Maven Analytics, LLC

 T-SNE

STEP 5: Use the affinity scores between pairs to attract (high affinity) or repulse
(low affinity) the points until they have “stabilized” into their final position
Dimensionality
Reduction

PCA D
Q

S Z

t-SNE C

R A

B U
T

*Copyright Maven Analytics, LLC

 T-SNE

STEP 5: Use the affinity scores between pairs to attract (high affinity) or repulse
(low affinity) the points until they have “stabilized” into their final position
Dimensionality
Reduction

PCA D Q
Z

S
t-SNE This attraction and repulsion is done by
C gradient descent on a KL loss function
R
A

X
B
U
T

*Copyright Maven Analytics, LLC

 T-SNE

STEP 5: Use the affinity scores between pairs to attract (high affinity) or repulse
(low affinity) the points until they have “stabilized” into their final position
Dimensionality
Reduction

PCA D Q
Z

S
t-SNE
C
R
A

X
B
U
T

*Copyright Maven Analytics, LLC

 T-SNE

STEP 5: Use the affinity scores between pairs to attract (high affinity) or repulse
(low affinity) the points until they have “stabilized” into their final position
Dimensionality
Reduction

PCA
Q
D Z
S

t-SNE
R

X A
T U
B

*Copyright Maven Analytics, LLC

 T-SNE

STEP 5: Use the affinity scores between pairs to attract (high affinity) or repulse
(low affinity) the points until they have “stabilized” into their final position
Dimensionality
Reduction

PCA
S Q
Z

R
t-SNE D

T
X U
C
A

*Copyright Maven Analytics, LLC

 T-SNE

STEP 5: Use the affinity scores between pairs to attract (high affinity) or repulse
(low affinity) the points until they have “stabilized” into their final position
Dimensionality
Reduction

PCA R
S
Q
Z
T
t-SNE
U
X
D

A B

*Copyright Maven Analytics, LLC

 T-SNE

STEP 5: Use the affinity scores between pairs to attract (high affinity) or repulse
(low affinity) the points until they have “stabilized” into their final position
Dimensionality
Reduction

PCA R
S
Z Q
T

U
t-SNE X

D
C
B
A

*Copyright Maven Analytics, LLC

 T-SNE

STEP 5: Use the affinity scores between pairs to attract (high affinity) or repulse
(low affinity) the points until they have “stabilized” into their final position
Dimensionality
Reduction

PCA S
R

Q
Z
T
U
t-SNE X

C
D
B
A

*Copyright Maven Analytics, LLC

 T-SNE

STEP 5: Use the affinity scores between pairs to attract (high affinity) or repulse
(low affinity) the points until they have “stabilized” into their final position
Dimensionality
Reduction

B
PCA S
R

A C Q
Z
D T
U
t-SNE X

X
Q
U

Z
C
D
B
S A
T

*Copyright Maven Analytics, LLC

 T-SNE IN PYTHON

You can use sklearn’s TSNE() to apply the t-SNE algorithm in Python

Dimensionality
Reduction

PCA

t-SNE
The number of columns you want Setting a random_state value
to reduce the dataset into guarantees the same output
(ideally 2 or 3 for visualization) after each run

PRO TIP: Unlike PCA, you don’t need to center the data for t-SNE since the
algorithm is based on pairwise similarities, but like PCA, scaling is recommended

*Copyright Maven Analytics, LLC

 T-SNE IN PYTHON

You can use sklearn’s TSNE() to apply the t-SNE algorithm in Python

Dimensionality
Reduction

PCA

t-SNE

Using t-SNE, it’s much easier to

identify the two clusters in the data

*Copyright Maven Analytics, LLC

 ASSIGNMENT: T-SNE

Key Objectives
NEW MESSAGE
March 28, 2024 1. Fit a t-SNE model with 2 components
From: Tim Menschen (Assistant Principal) 2. Plot the students on a scatter plot with the x-axis
Subject: RE: Student Analysis Visualization as component 1 and the y-axis as component 2

3. Interpret the data you see on the plot

Hello again!

Thanks for plotting the students using PCA earlier.

Can you do the same using t-SNE?

I’m hoping we can better distinguish them visually.

Thanks,
Tim

*Copyright Maven Analytics, LLC

 PCA VS T-SNE

PCA visualizations are more accurate and interpretable, while t-SNE visualizations
are better at visually separating groups of data into clusters
Dimensionality
Reduction
PCA (Entertainment) t-SNE (Entertainment)

PCA

t-SNE

With t-SNE, the two clusters in the data are clearly displayed

*Copyright Maven Analytics, LLC

 PCA VS T-SNE

PCA visualizations are more accurate and interpretable, while t-SNE visualizations
are better at visually separating groups of data into clusters
Dimensionality
Reduction
PCA (Cereal) t-SNE (Cereal)

PCA

t-SNE

With t-SNE, the large blob of data is shown as more distinct groups

*Copyright Maven Analytics, LLC

 DIMENSIONALITY REDUCTION & CLUSTERING

Visualizations created from dimensionality reduction techniques like PCA and

t-SNE can help identify clusters while fitting clustering algorithms
Dimensionality
Reduction

PCA

t-SNE

PRO TIP: You can use this visualization approach in addition to metrics (inertia plots, silhouette
scores, etc.) and intuition when deciding on the ideal number of clusters when clustering

*Copyright Maven Analytics, LLC

 ASSIGNMENT: T-SNE & K-MEANS CLUSTERING

Key Objectives
NEW MESSAGE
March 29, 2024 1. Fit a K-Means model with 3 clusters
From: Tim Menschen (Assistant Principal) 2. Overlay the 3 clusters onto a t-SNE plot
Subject: RE: Student Analysis Visualization
3. Interpret the cluster centers
Happy Friday!

Final request here, I promise.

Can you highlight the clusters in the t-SNE plot using colors?

I think that will really impress the staff.

Thanks,
Tim

*Copyright Maven Analytics, LLC

 KEY TAKEAWAYS

Dimensionality reduction helps reduce columns without losing information

• Common dimensionality reduction techniques include Principal Component Analysis (PCA) and t-SNE
• PCA is a good first algorithm to try, while t-SNE typically does a better job distinguishing clusters

PCA is great for both data visualization and feature extraction

• Use the explained variance ratio to understand how representative of the original data your components are
• The resulting components of PCA are somewhat interpretable, and can be input into other ML algorithms

t-SNE is ideal for data visualization

• Even though its components are more difficult to interpret, it creates a clear visual separation between clusters
• t-SNE can also help determine the ideal number of clusters for modeling

*Copyright Maven Analytics, LLC

RECOMMENDERS

*Copyright Maven Analytics, LLC

 RECOMMENDERS

In this section we’ll introduce the concept of recommenders and cover two common
ways of making recommendations: content-based filtering and collaborative filtering

TOPICS WE’LL COVER: GOALS FOR THIS SECTION:

• Understand the difference between content-based

Recommenders Basics Content-Based Filtering and collaborative filtering recommenders

• Use cosine similarity to make recommendations by

Collaborative Filtering Next Steps identifying similar items

• Use Singular Value Decomposition (SVD) to make

recommendations by identifying similar users

*Copyright Maven Analytics, LLC

 RECOMMENDERS BASICS

Recommenders, also known as recommender systems or recommendation engines,

are used to suggest personalized items to users based on their preferences
Recommenders
Basics There are two main approaches to making recommendations:
Content-Based
Filtering Content-Based Filtering Collaborative Filtering
Collaborative Recommend items to users based on items Recommend items to users based on the
Filtering with similar characteristics behaviors or preferences of similar users

Popular approach: Cosine similarity Popular approach: SVD

Next Steps
Required input data: Required input data:
• Rows: Items • Rows: Users
• Columns: Item characteristics • Columns: Items

In addition to using cosine similarity and SVD, there are many other approaches to creating recommenders,
including algorithms like regression, classification, clustering, association rule mining, and more

*Copyright Maven Analytics, LLC

 CONTENT-BASED FILTERING

Content-based filtering recommenders are used to suggest personalized items to

users based on items with similar characteristics
Recommenders • In other words, if a user listens to a lot of classical music, recommend more classical music
Basics

Content-Based
Filtering EXAMPLE Recommending fruits based on their nutritional profiles

Collaborative The features are item characteristics

Filtering

Fruit Energy Vitamin C Sugar

Next Steps
What other fruit would you
Each row Banana 89 8.7 12.2
represents recommend to someone who
an item to Lime 30 29.1 1.69 likes mangos?
recommend
Mango 46 4.1 8.39

Peach 60 36.4 13.7

*Copyright Maven Analytics, LLC

 COSINE SIMILARITY

Cosine similarity is a metric used to calculate the similarity between observations

• Values range from -1 (dissimilar) to 1 (similar)
Recommenders
Basics
• If all data points are in the first quadrant, the values will range from 0 (dissimilar) to 1 (similar)

Content-Based
Filtering EXAMPLE Recommending fruits based on their nutritional profiles

Collaborative
Filtering

Peach
Next Steps Banana
cos 𝟔𝟎 = 𝟎. 𝟓
Sugar

Mango
Mangos and limes
are not very similar
Lime
60°

Vitamin C

*Copyright Maven Analytics, LLC

 COSINE SIMILARITY

Cosine similarity is a metric used to calculate the similarity between observations

• Values range from -1 (dissimilar) to 1 (similar)
Recommenders
Basics
• If all data points are in the first quadrant, the values will range from 0 (dissimilar) to 1 (similar)

Content-Based
Filtering EXAMPLE Recommending fruits based on their nutritional profiles

Collaborative
Filtering

Peach
Next Steps Banana
cos 𝟒𝟑 = 𝟎. 𝟕𝟑
Sugar

Mango
Peaches are more similar
to mangos than limes
43° Lime

Vitamin C

*Copyright Maven Analytics, LLC

 COSINE SIMILARITY

Cosine similarity is a metric used to calculate the similarity between observations

• Values range from -1 (dissimilar) to 1 (similar)
Recommenders
Basics
• If all data points are in the first quadrant, the values will range from 0 (dissimilar) to 1 (similar)

Content-Based
Filtering EXAMPLE Recommending fruits based on their nutritional profiles

Collaborative
Filtering

Peach
Next Steps Banana
cos 𝟗 = 𝟎. 𝟗𝟖
Sugar

Mango
Bananas are the most similar
fruit to mangos – we should
Lime recommend them!
9°

Vitamin C

*Copyright Maven Analytics, LLC

 COSINE SIMILARITY

Cosine similarity is a metric used to calculate the similarity between observations

• Values range from -1 (dissimilar) to 1 (similar)
Recommenders
Basics
• If all data points are in the first quadrant, the values will range from 0 (dissimilar) to 1 (similar)

Content-Based
Filtering EXAMPLE Recommending fruits based on their nutritional profiles

Collaborative
Filtering
There are many similarity metrics to
choose from, but cosine similarity is
Peach popular in machine learning because:
Next Steps Banana
• It focuses on direction instead of
magnitude
Sugar

Mango
• It can handle high dimensions
• It works well on sparse data
(data containing many 0 values)
Lime
9°

Vitamin C

*Copyright Maven Analytics, LLC

 COSINE SIMILARITY IN PYTHON

You can use sklearn’s cosine_similarity() function to calculate the cosine

similarity between two items (rows) in a DataFrame
Recommenders
Basics

Content-Based
Filtering

Collaborative
Filtering

Next Steps

Each row / column in the matrix

contains the cosine similarities
between that item and the rest

*Copyright Maven Analytics, LLC

 COSINE SIMILARITY IN PYTHON

You can use sklearn’s cosine_similarity() function to calculate the cosine

similarity between two items (rows) in a DataFrame
Recommenders
Basics

Content-Based
Filtering

Collaborative
Filtering

Next Steps

Bananas are the most similar fruit to mangos!

(if we only use sugar & vitamin C as characteristics)

*Copyright Maven Analytics, LLC

 CONTENT-BASED FILTERING RECOMMENDATIONS

To make content-based recommendations using cosine similarity, simply filter

the cosine similarity matrix to the relevant item and sort the similarity scores
Recommenders
Basics

Content-Based
Filtering

Collaborative
Filtering

If you like mangos, I’d recommend trying bananas

Next Steps

*Copyright Maven Analytics, LLC

 ASSIGNMENT: CONTENT-BASED FILTERING

Key Objectives
NEW MESSAGE
April 1, 2024 1. Read in the data from the first tab of the
From: Becca Mender (Data Scientist) spreadsheet, which contains genre labels
Subject: Movie Recommender Project 2. Remove the non-genre columns in the data

3. Calculate the cosine similarity between Toy

Hi there, I’m creating a website that recommends movies.
Story (1995) and Get Shorty (1995)
I’ve manually labelled 1600+ movies with 18 genres, and
would like to create a proof of concept to see if this data can 4. Calculate the cosine similarity between Toy
actually be effective at recommending movies. Story (1995) and the rest of the movies
Can you help me: 5. Return the 5 movies with the highest similarity
• Find the similarity between Toy Story and Get Shorty
• Find the top 5 movies that are most similar to Toy Story

Thanks!
Becca

Movie_Ratings.xlsx

*Copyright Maven Analytics, LLC

 COLLABORATIVE FILTERING

Collaborative filtering recommenders are used to suggest personalized items to

users based on the behaviors or preferences of similar users
Recommenders • User-based: find users that like the same songs as me and recommend other songs they like
Basics
• Item-based: find songs similar to songs that I like and recommend those songs
Content-Based
Filtering

EXAMPLE Recommending fruits based on personal taste preferences

Collaborative
Filtering The features are items to recommend

Next Steps This is known as a user-item matrix

User Banana Lime Mango
Each row Adam 1.1 4.7 1
represents a What other fruit would you
user Liz 4.2 2.3 3.2 recommend to someone who
Daisy 3.7 1.4 4.9 likes mangos?

Bob 1.2 4.5 1.3

The values represent ratings

*Copyright Maven Analytics, LLC

 USER-ITEM MATRIX IN PYTHON

You may need to pivot your data in Python to structure it as a user-item matrix
Recommenders
Basics

Content-Based
Filtering

Make sure to fill the

Collaborative blank values with the
Filtering mean of the ratings

Next Steps

Notice that the data has changed from each row

representing a rating to each row representing a user

*Copyright Maven Analytics, LLC

 ASSIGNMENT: USER-ITEM MATRIX

Key Objectives
NEW MESSAGE
April 2, 2024 1. Read in the movies, users, and ratings tabs of the
From: Becca Mender (Data Scientist) spreadsheet into three DataFrames
Subject: Collaborative Filtering Data Prep 2. Use .pivot() to restructure the ratings data into a
user-item matrix
Hi again,

Thanks for your help earlier providing recommendations

using the movies data set!

I’d like to try using a collaborative filtering approach as well,

so I’ve collected movie ratings from almost a thousand people
who responded to my survey.

Can you help me restructure the ratings data into a user-item

matrix to get it ready for modeling?

Thanks!

Movie_Ratings.xlsx

*Copyright Maven Analytics, LLC

 SINGULAR VALUE DECOMPOSITION

Singular Value Decomposition (SVD)* is a matrix factorization technique used to

decompose the user-item matrix (X) into three other matrices:
Recommenders • U: Represents the users as rows and latent features as columns
Basics
• Σ: Represents the importance, or magnitude, of each latent feature across the diagonal
Content-Based • VT: Represents the latent features as rows and the items as columns
Filtering

Pineapple
Pineapple
Collaborative
Banana

Banana
Mango

Mango
Lemon

Lemon
Peach

Peach
Filtering

Lime

Lime
f1 f2 f1 f2

Adam Adam f1 f1
Next Steps
Liz
≈ Liz
x
f2
Σ x
f2
VT
Daisy
X Daisy
U
Bob Bob Both SVD and PCA are matrix
factorization techniques, with PCA often
Hannah Hannah used for data visualization and feature
extraction, and Truncated SVD for
recommenders and other applications

*SVD is the true mathematical model, but we’ll technically be working with Truncated SVD, which is a popular application *Copyright Maven Analytics, LLC
 SINGULAR VALUE DECOMPOSITION

Singular Value Decomposition (SVD)* is a matrix factorization technique used to

decompose the user-item matrix (X) into three other matrices:
Recommenders • U: Represents the users as rows and latent features as columns
Basics
• Σ: Represents the importance, or magnitude, of each latent feature across the diagonal
Content-Based • VT: Represents the latent features as rows and the items as columns
Filtering

Pineapple
Pineapple
Collaborative
Banana

Banana
Mango

Mango
Lemon

Lemon
Peach

Peach
Filtering

Lime

Lime
f1 f2 f1 f2

Adam 1.1 4.3 4.7 1.0 2.4 4.9 Adam 7.5 4.0 f1 42 0 f1 0.42 0.37 0.38 0.37 0.43 0.44
Next Steps
Liz 1.2 4.6 4.5 1.3 2.1 4.8
= Liz 7.5 3.9
x
f2 0 17
x
f2 -0.4 0.41 0.47 -0.4 -0.2 0.33

Daisy 4.8 3.7 4.1 2.2 3.3 2.4 Daisy 8.3 0.1
The V T matrix helps understand the combination of
Bob 2.5 2.9 3.7 4.1 1.8 4.1 Bob 7.7 0.9 items that make up each latent feature

Hannah 4.2 4.1 2.3 3.2 5.0 1.7 Hannah 8.3 -1.4
The U matrix is what’s of interest to us, as it
contains the preferences of every user

*SVD is the true mathematical model, but we’ll technically be working with Truncated SVD, which is a popular application *Copyright Maven Analytics, LLC
 SINGULAR VALUE DECOMPOSITION IN PYTHON

You can use sklearn’s TruncatedSVD() function to apply the SVD algorithm
• Use .fit_transform(X) to return the U matrix
Recommenders • Use .singular_values_ to return the importance for each latent feature, or the sigma matrix
Basics
• Use .components_ to return the VT matrix
Content-Based
Filtering

Collaborative
Filtering

Next Steps
You can choose the
number of latent
features to extract

*Copyright Maven Analytics, LLC

 ASSIGNMENT: SINGULAR VALUE DECOMPOSITION

Key Objectives
NEW MESSAGE
April 3, 2024 1. Apply TruncatedSVD to the user-item matrix
From: Becca Mender (Data Scientist) from the last assignment
Subject: Let’s create a recommender! 2. View the user-item matrix and the shape of the
user-item matrix
Hi again,
3. View the U matrix and the shape of the U matrix
Thanks for pulling together the user-item matrix yesterday.
4. View the VT matrix and the shape of the VT
Next, let’s create a recommender! matrix
Can you apply TruncatedSVD to the user-item matrix using 2
components?

Thanks!
Becca

*Copyright Maven Analytics, LLC

 CHOOSING THE NUMBER OF COMPONENTS

You can use the explained variance ratio to choose the number of components

Recommenders
Basics

Content-Based
Filtering

Collaborative
Filtering

Next Steps
The first two components capture
85% of the variance in the data

PRO TIP: You typically want to

capture around 80%+ of the
explained variance with your model

*Copyright Maven Analytics, LLC

ASSIGNMENT: CHOOSING THE NUMBER OF COMPONENTS

Key Objectives
NEW MESSAGE
April 4, 2024 1. Fit a TruncatedSVD model with 500 components
From: Becca Mender (Data Scientist) 2. Plot the cumulative explained variance ratios
Subject: Tuning the recommender
3. Suggest a “good” number of components that
best captures the variance in the data
Hi again,

Using the code from yesterday, can you try applying 4. Fit another TruncatedSVD model with the
TruncatedSVD with 500 components? “good” number of components

1. Fit a model with 500 components, and suggest a good

number of components for this data set
2. Fit a model using your suggested number of components

Thanks!
Becca

*Copyright Maven Analytics, LLC

 COLLABORATIVE FILTERING RECOMMENDATIONS

STEP 1: Select and fit a model using TruncatedSVD() and .fit_transform(X)

Recommenders
Basics
The model captures 85% of
the variance in the data
Content-Based
Filtering

Collaborative
Filtering STEP 2: Use .transform() to get the latent features for a new user

Next Steps

All we know is the user likes

mangos more than lemons

You need to fill in the blank

ratings with the mean (3)

*Copyright Maven Analytics, LLC

 COLLABORATIVE FILTERING RECOMMENDATIONS

STEP 3: Use [Link]() and .components_ to get the model’s recommendations

Recommenders
Basics
How does this work?
• By multiplying the new user’s U
Content-Based matrix with the model’s VT matrix
Filtering
(svd.components_), we get an
approximation of their ratings
Collaborative
Filtering

Next Steps

If they like mangos and don’t like

lemons, we should suggest
bananas and peaches

*Copyright Maven Analytics, LLC

 ASSIGNMENT: COLLABORATIVE FILTERING

Key Objectives
NEW MESSAGE
April 5, 2024 1. Open the notebook and take a look at the new
user DataFrame
From: Becca Mender (Data Scientist)
Subject: Recommend movies 2. Transform the user into the latent space using
.transform()
Happy Friday! 3. Reconstruct the user-item matrix for the user
I have one final request for you. Let’s test out our SVD model
using [Link]()
on a new user.
4. Make 10 movie recommendations for the user
In the attached notebook, I’ve included the movie ratings that
a new user provided.
5. Review them to determine if they make sense

Can you generate 10 movie recommendations for her using 6. Optional: Try playing around with the number of
our final SVD model (with 250 components) and see if they components to see the difference in movie
make sense? recommendations
Thanks for all your help this week!

section08_recommenders_assignments.ipynb

*Copyright Maven Analytics, LLC

 RECOMMENDERS NEXT STEPS

Once you’ve fit an initial content-based filtering or collaborative filtering model,

some potential next steps are to:
Recommenders
Basics
1• Consider a hybrid approach
• In practice, you often combine approaches (content-based, collaborative, popularity,
Content-Based
Filtering
rule-based, etc.) to create a hybrid model and make the best recommender

Collaborative
2• Evaluate the quality of recommendations
Filtering • A common approach is to deploy your model, see how users react to the
recommendations, and then make tweaks to create a better recommender
Next Steps
3• Deal with the difficulties of recommenders
• Tuning recommenders is an on-going process, and includes difficulties like changes to
items, data sparsity, popularity bias, and the “cold start” problem

4• Be creative in your approach

• All the concepts covered in this course can be mixed and matched when it comes to
recommenders (clustering, anomaly detection, dimensionality reduction, etc.)

*Copyright Maven Analytics, LLC

 KEY TAKEAWAYS

You can use content-based and collaborative filtering for recommenders

• Content-based filtering recommenders are based on the characteristics of the items, while collaborative
filtering recommenders are based on user-item interactions and preferences

Cosine similarity is often used to find the distance between items

• The range of cosine similarities falls between -1 and +1, with positive numbers being more similar

SVD is a matrix factorization technique commonly used for recommenders

• Modeling & interpreting the results of Singular Value Decomposition (SVD) in Python is very similar to PCA

Tuning recommenders is a long and difficult process

• In practice, hybrid recommenders that combine multiple recommender techniques are often used
• Models can be evaluated further down the line to note user interactions with their recommendations

*Copyright Maven Analytics, LLC

RECOMMENDER PROJECT

*Copyright Maven Analytics, LLC

 PROJECT: RECOMMENDING RESTAURANTS

THE You’ve just been hired as a Data Scientist for MavenEats, a restaurant review
SITUATION website that suggests restaurants to users based on their past restaurant ratings

You have access to data on each user’s restaurant ratings on the website
THE
Your task is to create a recommender that (1) displays five suggested restaurants on a
ASSIGNMENT user’s homepage and (2) five similar restaurants on each restaurant’s details page

THE 1. Prep the data for modeling

OBJECTIVES 2. Create a recommender using TruncatedSVD
3. Recommend five restaurants based on a user,
and five restaurants based on a restaurant

*Copyright Maven Analytics, LLC

UNSUPERVISED LEARNING REVIEW

*Copyright Maven Analytics, LLC

 UNSUPERVISED LEARNING REVIEW

In this section we’ll review the foundations of unsupervised learning covered in the
course, including techniques, applications, and its place in the data science workflow

TOPICS WE’LL COVER: GOALS FOR THIS SECTION:

• Review the basics of unsupervised learning

Unsupervised Learning Techniques & Applications
• Define a flowchart for selecting the correct
unsupervised learning technique
Data Science Workflow • Revisit the different applications for each
unsupervised learning technique
• Identify where each application of unsupervised
learning fits within the data science workflow

*Copyright Maven Analytics, LLC

 UNSUPERVISED LEARNING FLOWCHART

Unsupervised learning is about finding insights & patterns hidden in the data
• We care about understanding the relationships in unlabeled data, not making predictions

Unsupervised
Learning
Are you trying to predict something?

Techniques &
Applications Yes No

Data Science Use a supervised learning technique Use an unsupervised learning technique
Workflow

Are you trying to group data points

or reduce features?

Group Reduce
Evaluating the results of
unsupervised learning models
is less about metrics and more
about intuition and fulfilling
the business objective

Clustering Dimensionality Reduction

*Copyright Maven Analytics, LLC

 UNSUPERVISED TECHNIQUES & APPLICATIONS

Unsupervised learning techniques can be used for multiple applications:

Unsupervised
Learning Category Unsupervised Learning Techniques Application

Techniques &
K-Means Clustering
Applications
Clustering / Segmentation
Hierarchical Clustering Anomaly Detection
Data Science Recommenders
Workflow Clustering
DBSCAN (Density-Based Clustering)

Isolation Forests Anomaly Detection

Feature Extraction
PCA (Principal Component Analysis)
Data Visualization

t-SNE (t-Distributed Stochastic Neighbor Embedding) Data Visualization

Dimensionality
Reduction SVD (Singular Value Decomposition) Recommenders

*Copyright Maven Analytics, LLC

 UNSUPERVISED LEARNING IN THE DS WORKFLOW

Unsupervised learning can be applied at multiple phases of the workflow:

Unsupervised

1 2 3 4 5 6
Learning

Techniques &
Applications

Data Science
Workflow
Scoping a Gathering Cleaning Exploring Modeling Sharing
Project Data Data Data Data Insights

Anomaly Detection Data Visualization Feature Extraction

Clustering
Anomaly Detection
Recommenders

*Copyright Maven Analytics, LLC

 KEY TAKEAWAYS

Unsupervised learning is used to find patterns & relationships in data

• There are no predictions or labels with unsupervised learning – we are just trying to better understand the
data’s non-obvious structure, organization, and relationships between data points

Unsupervised learning model evaluation focuses on intuition

• While there are evaluation metrics available, the focus for unsupervised learning is on using intuition and
domain expertise to make recommendations for specific business objectives

There are multiple applications for unsupervised learning techniques

• While the two main categories of unsupervised learning techniques fall under clustering and dimensionality
reduction, these techniques can be applied to segmentation, anomaly detection, recommenders, and more

The techniques can be used at multiple steps of the data science workflow
• In addition to using unsupervised learning techniques during the modeling step of the data science workflow, select
techniques can also be used during the data cleaning, exploration, and feature engineering phases

*Copyright Maven Analytics, LLC

FINAL PROJECT

*Copyright Maven Analytics, LLC

 RECAP: THE COURSE PROJECT

THE You’ve just been hired as an Associate Data Scientist for the HR Analytics team at a
SITUATION medium-sized software company that’s trying to increase employee retention

You have access to the company’s employee database, including demographic info,
THE performance history, tenure at the company, attrition, and more
ASSIGNMENT Your task is to use unsupervised learning techniques to define employee
segments and make recommendations to increase retention within each one

1. Prepare the data for unsupervised modeling

THE
2. Segment the employees using clustering
OBJECTIVES
3. Visualize the clusters using dimensionality reduction
4. Explore the employees within each cluster
5. Recommend next steps to increase retention

*Copyright Maven Analytics, LLC