Rohail Qamar
CT-377 DATA MINING Lecturer
rohailqamar@[Link]
WHAT IS MACHINE LEARNING?
The branch of AI that have the ability to automatically learn and improve from
experience without being explicitly programmed.
Machine learning teaches computers to do what comes naturally to humans and
animals: learn from experience.
Machine learning algorithms use computational methods to “learn” information
directly from data without relying on a predetermined equation as a model.
The algorithms adaptively improve their performance as the number of samples
available for learning increases
DATA MINING BY ROHAIL QAMAR 2
MORE DATA MORE QUESTIONS, BETTER ANSWERS
Machine learning algorithms find natural patterns in data that generate insight and
help you make better decisions and predictions.
They are used every day to make critical decisions in medical diagnosis, stock
trading, energy load forecasting, and more.
Media sites rely on machine learning to sift through millions of options to give you
song or movie recommendations.
Retailers use it to gain insight into their customers’ purchasing behavior.
DATA MINING BY ROHAIL QAMAR 3
REAL-WORLD APPLICATIONS
With the rise in big data, machine learning has become particularly important for
solving problems in areas like these
Computational finance, for credit scoring and algorithmic trading
Image processing and computer vision, for face, motion and object detection
Computational biology, for tumor detection, drug discovery, and DNA sequencing
Energy production, for price and load forecasting
Natural language processing
DATA MINING BY ROHAIL QAMAR 4
HOW ML WORKS
DATA MINING BY ROHAIL QAMAR 5
HOW ML WORKS
DATA MINING BY ROHAIL QAMAR 6
DATA MINING BY ROHAIL QAMAR 7
THREE FLAVORS OF MACHINE LEARNING
1. Unsupervised Learning
2. Supervised Learning
3. Reinforcement Learning
DATA MINING BY ROHAIL QAMAR 8
WHAT IS MODEL
1. A simplified representation of a process or a system
2. A statistical representation of a real-world process based on data
3. Complexity and accuracy requirements defined by the application
DATA MINING BY ROHAIL QAMAR 9
MACHINE LEARNING MODEL
DATA MINING BY ROHAIL QAMAR 10
HOW DO WE BUILD ONE?
1. Define the problem
2. Collect the data
3. Configure & fit the model
4. Use it!
DATA MINING BY ROHAIL QAMAR 11
DEFINE THE PROBLEM
Poor problem definition = highway to disaster
Key questions
1. What is the pain point
2. How do we create value by solving this problem? (value proposition)
3. How do we know if we've succeeded or failed?
DATA MINING BY ROHAIL QAMAR 12
CONFIGURE THE MODEL
1. Specify the technical problem
2. Select the model type
3. Select the specific algorithm
DATA MINING BY ROHAIL QAMAR 13
FIT THE MODEL
Fitting = mathematical optimization
Ingredients
1. Optimization algorithm
2. Cost function
3. Data
Common Python syntax
DATA MINING BY ROHAIL QAMAR 14
PARAMETERS AND HYPERPARAMETERS
1. Model Parameters
Learned at the fitting time, fast, automated
2. Hyperparameters
Defined prior to fitting, slow and costly optimization process
DATA MINING BY ROHAIL QAMAR 15
REALITY CHECK
Danger: Overfitting
DATA MINING BY ROHAIL QAMAR 16
MACHINE LEARNING WORKFLOW
DATA MINING BY ROHAIL QAMAR 17
UNSUPERVISED LEARNING
1. Unlabeled data.
2. No labels => No supervision.
3. Only input relationships examined.
Clustering ? Classification!
Classification = Model learns existing groups.
Clustering = Model discovers groups on its own.
DATA MINING BY ROHAIL QAMAR 18
UNSUPERVISED ALGORITHMS
DATA MINING BY ROHAIL QAMAR 19
UNSUPERVISED LEARNING: ANOMALY DETECTION
1. Detecting unusual entities or events.
2. Hard to define what's odd, but possible to define what's normal.
3. Use cases
Credit card fraud detection
Network security monitoring
Heart-rate monitoring
DATA MINING BY ROHAIL QAMAR 20
UNSUPERVISED LEARNING: ANOMALY DETECTION
DATA MINING BY ROHAIL QAMAR 21
UNSUPERVISED LEARNING: ANOMALY DETECTION
DATA MINING BY ROHAIL QAMAR 22
ANOMALY DETECTION ALGORITHMS
DATA MINING BY ROHAIL QAMAR 23
ANOMALY DETECTION ALGORITHMS
DATA MINING BY ROHAIL QAMAR 24
DIMENSIONALITY REDUCTION
"Dimensionality reduction is the process of reducing the number of variables under
consideration by obtaining a set of principal variables.“
Pro’s
Reduce overfitting
Obtain independent features
Lower computational intensity
Enable visualization
Con’s
Compression => Loss of information => Loss of performance
DATA MINING BY ROHAIL QAMAR 25
DIMENSIONALITY REDUCTION ALGORITHMS
DATA MINING BY ROHAIL QAMAR 26
UNSUPERVISED LEARNING
Grouping of elements on the basis of similarities
No testing and training data
No class labels available
Algorithms : k-mean clustering, k-mediod clustering, hierarchical clustering
Techniques of Unsupervised Learning
Partitional clustering AKA hard clustering (no overlapping)
• Eg. apple, mango
Hierarchical clustering AKA soft clustering (overlapping)
• Eg. different varieties of mangoes
DATA MINING BY ROHAIL QAMAR 27
MAJOR CLUSTERING APPROACHES
Partitioning approach:
Construct various partitions and then evaluate them by some criterion, e.g., minimizing the sum of square
errors
Typical methods: k-means, k-medoids, CLARANS
Hierarchical approach:
Create a hierarchical decomposition of the set of data (or objects)
using some criterion
Typical methods: Diana, Agnes, BIRCH, CAMELEON
Density-based approach:
Based on connectivity and density functions
Typical methods: DBSACN, OPTICS, DenClue
Grid-based approach:
based on a multiple-level granularity structure
Typical methods: STING, WaveCluster, CLIQUE
DATA MINING BY ROHAIL QAMAR 28
CLUSTERING
Grouping of objects on the basis of similarities
Intra-distance should be minimum for similarity
Inter-distance should be maximum to differentiate
Reduce large data set into sub-set
It is the part of unsupervised learning
DATA MINING BY ROHAIL QAMAR 29
APPLICATION OF CLUSTERING
Understanding
Group students who succeed and fails in the same exercises
Summarization
Reduce the size of large data sets
DATA MINING BY ROHAIL QAMAR 30
K-MEAN CLUSTERING
DATA MINING BY ROHAIL QAMAR 31
K-MEAN CLUSTERING
DATA MINING BY ROHAIL QAMAR 32
PARTITIONAL CLUSTERING
DATA MINING BY ROHAIL QAMAR 33
PARTITIONAL CLUSTERING
DATA MINING BY ROHAIL QAMAR 34
PARTITIONAL CLUSTERING
DATA MINING BY ROHAIL QAMAR 35
PARTITIONAL CLUSTERING
DATA MINING BY ROHAIL QAMAR 36
PARTITIONAL CLUSTERING
DATA MINING BY ROHAIL QAMAR 37
PARTITIONAL CLUSTERING
DATA MINING BY ROHAIL QAMAR 38
PARTITIONAL CLUSTERING
DATA MINING BY ROHAIL QAMAR 39
TWO DIFFERENT K-MEAN CLUSTERING
DATA MINING BY ROHAIL QAMAR 40
ADVANTAGES OF K-MEAN CLUSTERING
Is efficient
Can be computed in a distributive way
Is easy to apply
DATA MINING BY ROHAIL QAMAR 41
LIMITATIONS OF K-MEAN CLUSTERING
How to determine the best K ?
May give a sub-optimal(not best) solution
K-means has problems when clusters are of differeing
Sizes
Densities
Non-globular shapes
K-mean is sensible to outliers
DATA MINING BY ROHAIL QAMAR 42
LIMITATIONS OF K-MEAN CLUSTERING
Non globular shapes
DATA MINING BY ROHAIL QAMAR 43
THANK YOU
DATA MINING BY ROHAIL QAMAR 44