Analytics, Data Science and AI: Systems for

Decision Support
Eleventh Edition, Global Edition

Chapter 4
Data Mining Process,
Methods, and Algorithms

Copyright © 2021Copyright © 2021 Pearson

Pearson Education Ltd. All Education Ltd. All Rights Reserved.
Rights Reserved.
Learning Objectives
4.1 Define data mining as an enabling technology for
business analytics
4.2 Understand the objectives and benefits of data mining
4.3 Become familiar with the wide range of applications of
data mining
4.4 Learn the standardized data mining processes
4.5 Learn different methods and algorithms of data mining

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Learning Objectives
4.6 Build awareness of the existing data mining software tools
4.7 Understand the privacy issues, pitfalls, and myths of data
mining

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Opening Vignette: Miami-Dade Police Department Is
Using Predictive Analytics to Foresee and Fight Crime
• Predictive analytics in law enforcement
– Policing with less
– New thinking on cold cases
– The big picture starts small
– Success brings credibility
– Just for the facts
– Safer streets for smarter cities

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Opening Vignette: Miami-Dade Police Department Is
Using Predictive Analytics to Foresee and Fight Crime
• What We Can Learn from This Vignette
– The law enforcement agencies and departments are under
tremendous pressure
– The environment within which they perform their duties is
becoming increasingly more challenging
– Applying advanced analytics and data mining tools (i.e.,
knowledge discovery techniques) to these large and rich data
sources provides

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Opening Vignette: Miami-Dade Police Department Is
Using Predictive Analytics to Foresee and Fight Crime
• What We Can Learn from This Vignette
– Data mining is a prime candidate for better understanding and
management of these mission critical tasks with a high level of
accuracy and timeliness
– Illustrates the power of analytics and data mining to create a
holistic view of the world of crime and criminals for better and
faster reaction and management

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Concepts
• Reason for data mining
– More intense competition at the global scale
– Recognition of the value in data source
– Availability of quality data on customers, vendors, transactions,
Web, etc
– Consolidation and integration of data repositories into data
warehouses
– The exponential increase in data processing and storage
capabilities; and decrease in cost
– Movement toward conversion of information resources into
nonphysical form
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Concepts
• Definitions, Characteristics, and Benefits
– A term used to describe discovering or “mining” knowledge from
large amounts of data
– Also known as knowledge extraction, pattern analysis, data
archaeology, information harvesting, pattern searching, and data
dredging
– Is a process that uses statistical, mathematical, and artificial
intelligence techniques
▪ Extract and identify useful information
▪ Subsequent knowledge (or patterns) from large sets of data
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Concepts
• Definitions, Characteristics, and Benefits
– Knowledge can be in the form of business rules, affinities,
correlations, trends, or prediction models
– The nontrivial process of identifying valid, novel, potentially
useful, and ultimately understandable patterns in data stored in
structured databases
▪ Process (comprises of many iterative steps)
▪ Nontrivial (experimentation-type search is involved)
▪ Valid (hold true on new data with sufficient degree)
▪ Novel (previously known to the user within the context of the system)
▪ Potentially useful (discovered patterns should lead to some benefit)
▪ Ultimately understandable (the pattern should make business sense)
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Concepts
• Definitions, Characteristics, and Benefits
– Data mining
▪ Is not a new discipline, but rather a new
definition for the use of many disciplines
▪ Is tightly positioned at the intersection
of many disciplines, including statistics,
artificial intelligence, machine learning,
management science, information
systems (IS), and databases
▪ Strives to make progress in extracting
useful information and knowledge from
Figure 4.1 Data Mining Is a Blend of Multiple Disciplines.
large databases Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Concepts
• Definitions, Characteristics, and Benefits
– Data mining characteristics and objectives
▪ Data are often buried deep within very large databases
▪ The data mining environment is usually a client/server or a Web-based
IS architecture
▪ Sophisticated new tools, advanced visualization tools, help to remove
information
▪ Miner is often end user, empowered by data drills and other query tools
▪ Striking it rich often involves finding an unexpected result and requires
▪ Data mining tools are readily combined with spreadsheets and other
software development tools
▪ Parallel processing for data mining large amounts of data and massive
search efforts Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Concepts
• How Data Mining Works
– Data mining builds models
▪ Using existing and relevant data obtained from within and outside
the organization
– The mathematical representations model
▪ Simple linear relationships/affinities and/or complex and highly
nonlinear relationships
▪ Identify the patterns among the attributes of things (e.g., customers,
events)
▪ Some patterns are explanatory (explaining the interrelationships
and affinities among the attributes)
▪ Others are predictive (foretelling future values of certain attributes
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Concepts
• How Data Mining Works
– DM extract patterns from data
▪ Pattern? A mathematical (numeric and/or symbolic) relationship
among data items
– Four (4) major types of patterns
1. Associations
– Commonly co-occurring groupings of things
2. Predictions
– The nature of future occurrences
3. Clusters
– Natural groupings of things based on their known characteristics
4. Sequential relationships
– Time-ordered events Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Concepts
• How Data Mining Works
– These patterns have been manually extracted
– The increasing volume of data in modern times has created a
need for automatic approaches
– Direct manual data analysis has increasingly been augmented
with indirect
▪ Automatic data processing tools that use sophisticated
methodologies, methods, and algorithms
– Data mining is referred as
▪ Evolution of automated and semiautomated means of processing
large data sets
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Concepts
• How Data Mining Works
– A Taxonomy for Data Mining
▪ Tasks can be classified into
three main categories:
– Prediction, Association and
Clustering
▪ Methods
– Supervised
• Descriptive attributes
• Class attribute
– Unsupervised
• Only descriptive attributes
Figure 4.2 Simple Taxonomy for Data Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Mining Tasks, Methods, and Algorithms.
Data Mining Concepts
• How Data Mining Works
– A Taxonomy for Data Mining categories;
▪ Prediction
– The act of talking about the future or guessing or forecasting
▪ Classification (supervised induction)
– Common of all data mining tasks
– To analyze the historical data stored in a database
– Automatically generate a model that can predict future behavior
– Common classification tools
• Neural networks and decision trees (from machine learning)
• Logistic regression and discriminant analysis (from traditional statistics)
• Emerging tools such as rough sets, support vector machines (SVMs),
and genetic algorithms Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Concepts
• How Data Mining Works
– A Taxonomy for Data Mining categories;
▪ Clustering
– A collection of things
– Selected algorithm goes through the data set, identifying the
commonalities
– Optimization technique
• K-means (from statistics) and self-organizing maps (from machine learning)
▪ Associations (rule learning in data mining)
– A popular and well-researched technique
– Two commonly used derivatives rule mining
• Link analysis (discovered automatically)
• Sequence mining (relationships are examined order of occurrence in over
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
time)
Data Mining Concepts
• How Data Mining Works
– A Taxonomy for Data Mining categories;
▪ Visualization
– Used in conjunction with other data mining techniques to gain a clearer
understanding of underlying relationships
– Visual analytics (new term)
• To combine analytics and visualization in a single environment for easier
and faster knowledge creation
▪ Time-series forecasting
– Data consists of values of the same variable
• Captured and stored over time in regular intervals
– To develop forecasting models to extrapolate the future values
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Concepts
• Data Mining versus Statistics
– Both look for relationships within data
– Statistics is the “foundation of data mining.”
– The main difference
▪ Statistics starts with a well-defined proposition and hypothesis
▪ Data mining starts with a loosely defined discovery statement
– Another difference is the size of data used
▪ Statistics looks for the right size of data (A few hundred to a
thousand data points are large enough)
▪ Data mining looks for data sets that are as “big” as possible
(several million to a few billion data points are considered large)
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Applications
• Customer Relationship Management
– To create one-on-one relationships with customers
– Combined with demographic and socioeconomic attributes
▪ Maximize return on marketing campaigns
▪ Improve customer retention (churn analysis)
▪ Maximize customer value (cross-, up-selling)
▪ Identify and treat most valued customers
• Banking & Other Financial
– Automate the loan application process
– Detecting fraudulent transactions
– Maximize customer value (cross-, up-selling)
– Optimizing cash reserves with forecasting
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Applications
• Retailing and Logistics
– Optimize inventory levels at different locations
– Improve the store layout and sales promotions
– Optimize logistics by predicting seasonal effects
– Minimize losses due to limited shelf life (in SCM, RFID is used)
• Manufacturing and Maintenance
– Predict/prevent machinery failures
– Identify anomalies in production systems to optimize the use
manufacturing capacity
– Discover novel patterns to improve product quality
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Applications
• Brokerage and Securities Trading
– Predict changes on certain bond prices
– Forecast the direction of stock fluctuations
– Assess the effect of events on market movements
– Identify and prevent fraudulent activities in trading
• Insurance
– Forecast claim costs for better business planning
– Determine optimal rate plans
– Optimize marketing to specific customers
– Identify and prevent fraudulent claim activities
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Applications
• Computer hardware and software
– Predict disk drive failures before it happen
– Identify and filter unwanted Web content and e-mail messages
– Detect and prevent computer network security breaches
– Identify potentially unsecure software products
• Government and defense
– Forecast the cost of moving military personnel and equipment
– Predict an adversary’s moves and develop more successful
strategies
– Predict resource consumption for better planning and budgeting
– Identify classes of unique experiences, strategies, and lessons
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Applications
• Travel industry (airlines, hotels/resorts, rental car
companies)
– Predict sales of different services
– Forecast demand locations to better allocate limited
organizational resources
– Identify the most profitable customers and provide them with
personalized services to maintain their repeat business
– Retain valuable employees by identifying and acting on the root

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Applications
• Healthcare
– Identify people without health insurance and the factors
underlying this undesired phenomenon
– Identify novel cost–benefit relationships to develop more effective
strategies
– Forecast the level and the time of demand at different service
locations
– Understand the underlying reasons for customer and employee

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Applications
• Medicine
– Identify novel patterns to improve survivability of patients
– Predict success rates of organ transplantation patients to develop
better organ donor matching policies
– Identify the functions of different genes in the human chromosome
– Discover the relationships between symptoms and illnesses
• Entertainment industry
– Analyze viewer data to decide what programs to show during
prime time and how to maximize returns
– Predict the financial success of movies before they are produced
to make investment decisions and to optimize the returns
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Applications
• Homeland security and law enforcement
– Identify patterns of terrorist behaviors
– Discover crime patterns to help solve criminal
– Predict and eliminate potential biological and chemical attacks
– Identify and stop malicious attacks on critical information
infrastructures (information warfare)
• Sports
– To improve the performance
– Predictive analytics and data mining to optimally utilize their
limited resources
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Process
• A manifestation of the best practices
• A systematic way to conduct D M projects
• Moving from Art to Science for D M project
• Everybody has a different version
• Most common standard processes:
– C R I S P-D M (Cross-Industry Standard Process for Data Mining)
– S E M MA (Sample, Explore, Modify, Model, and Assess)
– K D D (Knowledge Discovery in Databases)
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Process
• CRISP-DM (Cross Industry Standard Process for Data Mining)
– Proposed in 1990s by a European consortium
– Composed of six consecutive steps
▪ Step 1: Business Understanding  Accounts for

▪ Step 2: Data Understanding  ~85% of total
 project time
▪ Step 3: Data Preparation 
▪ Step 4: Model Building
▪ Step 5: Testing and Evaluation
▪ Step 6: Deployment

Figure 4.3 Six-Step CRISP-DM Data Mining Process.

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Process
• Other Data Mining Standardized Processes and Methodologies
– SEMMA (Sample, Explore, Modify, Model and Assess)
▪ Easy to apply exploratory statistical and visualization techniques
▪ Select and transform the most significant
predictive variables
▪ Model the variables to predict outcomes
▪ Confirm a model’s accuracy
▪ The main difference
– CRISP-DM (more comprehensive
approach)
– SEMMA (assumes goals and
objectives)
Figure 4.5 SEMMA Data Mining Process. Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Process
• Other Data Mining Standardized Processes and Methodologies
– KDD (Knowledge Discovery in Databases)
▪ Defined knowledge discovery in databases
as a process of using data mining methods
▪ A comprehensive process that
encompasses data mining
▪ KDD process;
– Data selection,
– Data preprocessing,
– Data transformation,
– Data mining,
– Interpretation/evaluation Figure 4.6 KDD (Knowledge Discovery in Databases) Process..
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Process
• Other Data Mining Standardized Processes and Methodologies
– Which Data Mining Process
is the Best?

Figure 4.7 Ranking of Data Mining Methodologies/Processes.

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• The most popular data mining methods and explains their
representative techniques
– Classification
– Regression
– Clustering
– Association
• DM software tools employ more than one technique or
algorithm

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Classification
– Most frequently used in the real-world problems as a machine-
learning family of techniques
– Learns patterns from past data to place new instances
▪ A set of information
– Traits,
– Variables,
– Features
▪ On characteristics of the previously labeled
– Items,
– Objects, or
– Events Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Classification
– Clustering (another popular data mining method)
▪ Can be used to determine groups (or class memberships) of things
– A significant difference between
▪ Classification learns the function between
– The characteristics of things (i.e., independent variables)
– Their membership (i.e., output variable)
– Through a supervised learning process where both types (input and
output) of variables are presented to the algorithm
▪ Clustering, the membership of the objects is learned through an
unsupervised learning process where only the input variables are
presented to the algorithm
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Classification
– The most common two-step methodology of classification-type
prediction
▪ Model development/training
– A collection of input data, including the actual class labels, is used
▪ Model testing/deployment
– The holdout sample for accuracy assessment and eventually deployed
for actual use where it is to predict classes of new data instances

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Classification
– Assessment Methods for Classification
▪ Predictive accuracy
– Ability to correctly predict the class label of new or previously unseen
data
– Actual class labels of a test data set are matched against the class
labels predicted
– Computed as the accuracy rate
▪ Speed
– The computational costs involved in generating and using the model

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Classification
– Assessment Methods for Classification
▪ Robustness
– Ability to make reasonably accurate predictions, given noisy data or
data with missing and erroneous values
▪ Scalability
– Ability to construct a prediction model efficiently
▪ Interpretability
– The level of understanding and insight provided

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Estimating the True Accuracy of Classification Models
– In classification problems, the primary source for accuracy
estimation is the confusion matrix
– Also known as classification matrix or contingency table
TP + TN
Accuracy =
TP + TN + FP + FN
TP
True Positive Rate =
TP + FN
TN
True Negative Rate =
TN + FP Figure 4.8 Simple Confusion
Matrix for Tabulation of Two-
Class Classification Results.
TP TP
Precision = Recall = Predicted
TP + FP TP + FN Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Estimating the True Accuracy of Classification Models
– If is not binary
▪ Confusion matrix gets bigger
▪ Accuracy metrics become limited to
– Per class accuracy
– Overall classifier accuracy
– A supervised learning algorithm is important
• Can be used to estimate its future prediction accuracy
• Can be used for choosing a classifier from a given set

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Estimating the True Accuracy of Classification Models
– Simple split (or holdout or test sample estimation)
▪ Split the data into 2 mutually exclusive sets:
– Training
• Is used by the inducer
– Testing (holdout)
• The built classifier is tested
▪ Neural Networks mutually,
– Training
– Validation
• Is used during model building to prevent overfitting
– Testing Figure 4.9 Simple Random Data Splitting.

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Estimating the True Accuracy of Classification Models
– k-Fold Cross Validation (rotation estimation)
▪ Data is split into
– k mutual subsets
– k number training/testing experiments are conducted

Figure 4.10 Graphical Depiction of k-Fold Cross-Validation. Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Estimating the True Accuracy of Classification Models
– Additional Classification Assessment Methodologies
▪ Leave-one-out
– Similar to k-fold where k = number of samples
– Time consuming but suitable for small data sets
▪ Bootstrapping
– Random sampling with replacement for training
– The balance is for testing
▪ Jackknifing
– Similar to leave-one-out
– The accuracy is calculated by leaving one sample out at each iteration
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Estimating the True Accuracy of Classification Models
– Additional Classification Assessment Methodologies
▪ Area Under the ROC Curve (AUC)
– Is a graphical assessment technique
• True positive rate is plotted (y-axis)
• false positive rate is plotted (x-axis)
– Determines the accuracy measure of a
classifier
• A value of 1 indicates a perfect classifier
• A has a better classification
performance than B, whereas C is not
any better
Figure 4.11 Sample ROC Curve.
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Estimating the Relative Importance of Predictor Variables
– Data mining methods (i.e., machine-learning algorithms) are good
at capturing complex relationships between input and output
variables (producing very accurate prediction models) but are not
nearly as good at explaining how they do what they do (i.e.,
model transparency)
– To mitigate this deficiency (also called the black-box syndrome),
the machine-learning community have several methods
▪ Predictive modeling, sensitivity analysis refers to an exclusive
experimentation process aimed at discovering the cause-and-effect
relationship between the input variables and output variable
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Estimating the Relative Importance of Predictor Variables
– The commonly used variable importance methods employed in
machine learning and predictive modeling:
1. Developing and observing a well-trained decision tree model to
see the relative discernibility of the input variables
2. Developing and observing a rich and large random forest model
and assessing the variable split statistics
3. Sensitivity analysis based on input value perturbation by which
the input variables are gradually changed/perturbed one at a time
and the relative change in the output is observed
4. Sensitivity analysis based on leave-one-out methodology
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Estimating the Relative Importance of Predictor Variables
– The sensitivity analysis (based on leave-one-out methodology)
relies on the experimental process of systematically
▪ Removing input variables, one at a time, from the input variable set,
developing and testing
a model and observing
the impact of the
absence of this variable
on the predictive
performance of the
machine-learning
model Figure 4.12 Graphical Depiction of
the Sensitivity Analysis Process.
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Estimating the Relative Importance of Predictor Variables
– Often used for support vector machines, decision trees, logistic
regression, and artificial neural networks
– The formalized the algebraic representation of this measurement
process:

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Estimating the Relative Importance of Predictor Variables
– Classification Techniques
▪ Decision tree analysis (Machine-learning technique)
– Arguably the most popular classification technique
▪ Statistical analysis
– Include logistic regression and discriminant analysis
• The assumptions that the relationships between the input and output
variables are linear
• The data is normally distributed
• The variables are not correlated and are independent
▪ Neural networks (most popular ML techniques)
– can be used for classification-type problems
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Estimating the Relative Importance of Predictor Variables
– Classification Techniques
▪ Case-based reasoning
– Uses historical cases to recognize commonalities for new case
▪ Bayesian classifiers
– Uses probability theory to build classification models based on
• The past occurrences that are capable of placing a new instance
▪ Genetic algorithms
– Use of the analogy of natural evolution to build directed search-based
mechanisms
▪ Rough sets
– Takes into account the partial membership of class labels
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Estimating the Relative Importance of Predictor Variables
– Decision Trees
▪ Include many input variables that impact on the classification of
different patterns
– Usually called attributes
▪ Consists of branches and nodes
– Branch represents the outcome of a test to classify a pattern
– Leaf node represents the final class choice for a pattern
▪ Divides a training set until each division consists entirely

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Estimating the Relative Importance of Predictor Variables
– Decision Trees
▪ Each non-leaf node of the tree contains a split point
– One or more attributes and determines how the data are divided
▪ Decision tree algorithms
– Build an initial tree from the training data such that each leaf node is
pure
– Prune the tree to increase its generalization
– The prediction accuracy on test data

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Estimating the Relative Importance of Predictor Variables
– Decision Trees
▪ Employs a divide-and-conquer method
▪ Recursively divides a training set until each division consists of
examples from one class:
A general 1. Create a root node and assign all of the training data to it
algorithm 2. Select the best splitting attribute
(steps) for 3. Add a branch to the root node for each value of the split. Split the data
building a into mutually exclusive subsets along the lines of the specific split
decision 4. Repeat the steps 2 and 3 for each and every leaf node until the
tree stopping criteria is reached

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Estimating the Relative Importance of Predictor Variables
– Decision Trees
▪ The algorithms mainly differ on
1. Splitting criteria
• Which variable, what value, etc.
2. Stopping criteria
• When to stop building the tree
3. Pruning (generalization method)
• Pre-pruning versus post-pruning
▪ Most popular algorithms include
– ID3 (Iterative Dichotomiser 3), C4.5 & C5 (Improve version of ID3)
– CART (classification and regression trees)
– CHAID (chi-squared automatic interaction detector)
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Estimating the Relative Importance of Predictor Variables
– Decision Trees
▪ The Gini index
– Used in economics to measure the diversity of a population
– Can be used to determine the purity of a specific class
– The best split is the one that increases the purity
of the sets resulting from a proposed split
– If a data set S contains examples from n classes
where pj is a relative frequency of class j in S
– If a data set S is split into two subsets,
S1 and S2, with sizes N1 and N2,
respectively, the Gini index of the split
data contains examples from n classes
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Estimating the Relative Importance of Predictor Variables
– Decision Trees
▪ Information gain
– The splitting mechanism used in ID3
• To use a concept called Entropy
• Measures the extent of uncertainty or randomness in a data set
– Assume that there are two classes
• P (positive)
• N (negative)
– Let the set of examples S contain p counts of class P and n counts of
class N. The amount of information needed to decide if an arbitrary
example in S belongs to P or N
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Estimating the Relative Importance of Predictor Variables
– Decision Trees
▪ Information gain
– Assume that using attribute A, the
set S will be partitioned into sets
{S1, S2,…, Sv}. If Si contains pi
examples of P and ni examples of
N, the entropy, or the expected
information needed to classify
objects in all subtrees, Si,
– Then, the information that would be
gained by branching on attribute A
would be;
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Cluster Analysis for Data Mining
– Cluster analysis
▪ An essential data mining method for classifying items, events, or
concepts into common groupings
▪ Underlying techniques have been applied to business
▪ Used extensively for fraud detection (both credit card and e-
commerce fraud)
▪ Market segmentation of customers in contemporary CRM systems

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Cluster Analysis for Data Mining
– Cluster analysis
▪ An exploratory data analysis tool for solving classification problems
▪ The objective
– Sort cases (e.g., people, things, events) into groups
– Weak among members of different clusters

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Cluster Analysis for Data Mining
– Cluster analysis
▪ Clustering results may be used to
– Identify natural groupings of customers
– Identify rules for assigning new cases to classes for
targeting/diagnostic purposes
– Provide characterization, definition, labeling of populations
– Decrease the size and complexity of problems for other data mining
methods
– Identify outliers in a specific domain (e.g., rare-event detection)

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Cluster Analysis for Data Mining
– Determining the optimal number of clusters
▪ Usually require one to specify the number of clusters
▪ Several different heuristic methods
– Look at the percentage of variance as a function of number of clusters
– Set the number of clusters to (n/2)½, where n is the number of data
points
– Use the Akaike information criterion (AIC), which measure of the
goodness of fit
– Use Bayesian information criterion, which is a model-selection
criterion
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Cluster Analysis for Data Mining
– Analysis methods
▪ Statistical methods (including both hierarchical and nonhierarchical),
such as k-means, k-modes, and so on.
▪ Neural networks (adaptive resonance theory [ART], self-organizing
map [SOM])
▪ Fuzzy logic (e.g., fuzzy c-means algorithm)
▪ Genetic algorithms
▪ Generally, works with one of two general method classes
– Divisive (all item start in cluster and broken apart)
– Agglomerative (all items start in individual cluster and are joined
together) Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Cluster Analysis for Data Mining
– k-Means Clustering Algorithm
▪ k: pre-determined number of clusters
▪ Algorithm (Step 0: determine value of k)
Step 1: Randomly generate k random points as initial cluster centers
Step 2: Assign each point to the nearest cluster center
Step 3: Re-compute the new cluster centers
Repetition step: Repeat steps 3 and 4
until some convergence criterion is
met (usually that the assignment of
points to clusters becomes stable)
Figure 4.13 A Graphical Illustration of
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
the Steps in the k-Means Algorithm.
Data Mining Methods
• Association Rule Mining
– Also known as market basket analysis or affinity analysis
– A very popular DM method in business
– Can do to a technologically less-savvy audience
– Finds interesting relationships (affinities) between variables (items
or events)
– Retail business problems, it is commonly called market-basket
analysis

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Association Rule Mining
– Input: the simple point-of-sale transaction data
– Output: Most frequent affinities among items
– A business can take advantage of pattern/knowledge?
▪ Put the items next to each other to make it more convenient
▪ Promote the items as a package
▪ Place items far apart from each other and potentially seeing and
buying other items

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Association Rule Mining
– A representative applications of association rule mining include
▪ In business: cross-marketing, cross-selling, store design, catalog
design, e-commerce site design, optimization of online advertising,
product pricing, and sales/promotion configuration
▪ In medicine: relationships between symptoms and illnesses;
diagnosis and patient characteristics and treatments (to be used in
medical D S S); and genes and their functions (to be used in
genomics projects)

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Association Rule Mining
– A few common areas and uses for association rule mining
▪ Sales transactions
– Combinations of retail products purchased can be used to improve
product placement
▪ Credit card transactions
▪ Banking services
– The sequential patterns of services used
▪ Insurance service products
▪ Telecommunication services
▪ Medical records
– Combinations of conditions indicate increased risk various complications
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Association Rule Mining
– Are all association rules interesting and useful?
A Generic Rule: X  Y [S%, C%]
X, Y: products and/or services
X: Left-hand-side (LH S)
Y: Right-hand-side (R H S)
S: Support: how often X and Y go together
C: Confidence: how often Y go together with the X
Example: {Laptop Computer, Antivirus Software}  {Extended Service Plan}
[30%, 70%]
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Methods
• Association Rule Mining
– The simple formulas for Support, Confidence, and Lift

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Association Rule Mining
– Several algorithms are developed for discovering (identifying)
association rules
▪ Apriori These
algorithms
▪ Eclat only do half
▪ F P-Growth the job

▪ + Derivatives and hybrids of the three

– The algorithms help identify the frequent item sets, which are,
then converted to association rules

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Association Rule Mining
– Apriori Algorithm
▪ Commonly used algorithm to discover association rules
▪ Finds subsets that at least a minimum number of the itemsets
▪ Uses a bottom-up approach
– Frequent subsets are extended one item at a time (known as
candidate generation)
– Groups of candidates at each level are tested against the data for
minimum support
▪ Terminates when no further successful extensions are found

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Methods
• Association Rule Mining
– Apriori Algorithm

Figure 4.14 A Graphical Illustration of the Steps in the k-Means Algorithm.

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Software Tools
• Commercial
– IBM SPSS Modeler (formerly Clementine)
– SAS Enterprise Miner
– Statistica - Dell/Statsoft
– … many more
• Free and/or Open Source
– KNIME
– RapidMiner
– Weka
– R, … Figure 4.15 Popular Data Mining Software Tools (Poll Results).
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Software Tools
• The major products
and their Web sites

Table 4.2 Selected Data Mining Software

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Privacy Issues, Myths, and
Blunders
• Data that is collected, stored, and analyzed
– Often contains information about real people
• Most of these data can be accessed through some third-
party data providers

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Data Mining Privacy Issues, Myths, and
Blunders
• Data Mining Myths and Blunders
Myth Reality
Data mining provides instant, crystal-ball-like Data mining is a multistep process that
predictions. requires deliberate, proactive design
and use.
Data mining is not yet viable for mainstream The current state of the art is ready for
business applications. almost any business type and/or size.
Data mining requires a separate, dedicated Because of the advances in database
database. technology, a dedicated database is
not required.
Only those with advanced degrees can do Newer Web-based tools enable
data mining. managers of all educational levels to
do data mining.
Data mining is only for large firms that have If the data accurately reflect the
lots of customer data. business or its customers, any
company can use data mining.
Table 4.6 Data Mining Myths. Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Privacy Issues, Myths, and
Blunders
• Data Mining Myths and Blunders
– Data Mining Mistakes
1. Selecting the wrong problem for data mining
2. Ignoring what your sponsor thinks data mining is and what it really
can/cannot do
3. Beginning without the end in mind
4. Define the project around a foundation that your data can’t support
5. Leaving insufficient time for data acquisition, selection, and
preparation
6. Looking only at aggregated results and not at individual
records/predictions
7. Being sloppy about keeping track of the data mining procedure and
results Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
Data Mining Privacy Issues, Myths, and
Blunders
• Data Mining Myths and Blunders
– Data Mining Mistakes
8. Using data from the future to predict the future
9. Ignoring suspicious findings and quickly moving on
10. Starting with a high-profile complex project that will make you a
superstar
11. Running data mining algorithms repeatedly and blindly
12. Ignore the subject matter experts
13. Believing everything you are told about the data
14. Assuming that the keepers of the data
15. Measuring the way your sponsor measures them
16. If you build it, they will come
Copyright © 2021 Pearson Education Ltd. All Rights Reserved.
End of Chapter 4

Questions / Comments

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.

Copyright

This work is protected by United States copyright laws and is

provided solely for the use of instructors in teaching their
courses and assessing student learning. Dissemination or sale of
any part of this work (including on the World Wide Web) will
destroy the integrity of the work and is not permitted. The work
and materials from it should never be made available to students
except by instructors using the accompanying text in their
classes. All recipients of this work are expected to abide by these
restrictions and to honor the intended pedagogical purposes and
the needs of other instructors who rely on these materials.

Copyright © 2021 Pearson Education Ltd. All Rights Reserved.