Module 1:

Selecting and Constructing Data

Collection Instruments

1
What is Business Intelligence

Basic Definition :Information that people use to

support their decision making efforts. Data
Mining and Data Analytics/predictive Analytics
falls within this field.
What is Data Mining
Data Mining is the analysis step of the Knowledge
Discovery in Databases (KDD) process, an
interdisciplinary subfield of computer Science, is the
computational process of discovering patterns in large
data sets involving methods at the intersection of
artificial intelligence, machine learning, statistics, and
database management systems.
KDD is the overall process of finding useful
knowledge from data
Example
• Step 1: A bank has data of millions of customers about
Age, Income, Credit score, Loan repayment history
• Step 2: KDD Process
1. Data selection – choose loan-related data
2. Data cleaning – remove errors & missing values
3. Data transformation – normalize income, scores
4. Data mining (analysis step) – �
Find patterns like:
“Customers with credit score > 750 rarely default”
Young professionals with stable income repay loans on
time”
 Basic-Definitions of Data
Mining
• The discovery of new, non-obvious, valuable information from
a large collection of raw data
• Data Mining (DM) is the core of the KDD [Knowledge Discovery in
Databases] process, involving the inferring of algorithms that explore
the data, develop the model and discover previously unknown
patterns.
• The set of activities used to find new, hidden or unexpected
patterns in data
Data Mining -continued
The overall goal of the data mining process is to
extract information from a data set and transform it
into an understandable structure for further use:
(Predictive analytics)
Discovering meaningful new corrections, patterns,
trends.
Example : Forecasting
Emerging Technology

Data mining is one of the “10 emerging technologies that will

change the world” listed by the MIT Technology
Review(magazine publised by Massachusetts Institute of
Technology) (Larose).

There is no doubt why many firms embrace data mining in

their operations. An article in Information System Management
points out that “data mining has become a widely accepted
process for organizations to enhance their organizational
performance and gain a competitive advantage”
Emerging Technology

• “Enhance organizational performance”

Data mining improves how an organization functions by:
Increasing sales through customer behavior analysis
Detecting fraud and risks early
Improving efficiency and productivity

“Gain a competitive advantage”

Competitive advantage means doing better than competitors.
With data mining, organizations can:
Predict customer needs
Offer personalized products and services
Respond quickly to market changes
 Keys to a Successful Data Mining Project
• Credible source of data
Data must come from reliable, accurate, and trustworthy sources Poor or biased data
leads to incorrect patterns and wrong decisions

• Knowledgeable personnel
Skilled professionals are required to:
Understand the business problem
Prepare and clean data
Select and interpret models correctly

• Appropriate algorithms
• The chosen algorithm must match the type of problem and data
• Wrong algorithms produce misleading results even with good data
• � Examples:
• Classification → fraud detection
• Clustering → customer segmentation
• Regression → sales prediction
Primary Tasks of Data Mining
Classification classify a data item into one of
several predefined classes

Regression map a data item to a real-value

prediction variable

Clustering identify a finite set of

categories or clusters to
describe the data
Summarization find a compact description for a
set (or subset) of data

Dependency Modeling describe significant dependencies

between variables or between the
values of a feature

Change and Deviation Discover the most significant

Detection changes
Some of the commonly used data
mining methods are:

• Statistical Data Analysis

• Cluster Analysis
• Decision Trees and Decision Rules
• Association Rules
• Artificial Neural Networks
• Genetic Algorithms
• Fuzzy Sets and Fuzzy Logic
Some of the commonly used data
mining methods are:
1. Statistical Data Analysis
• Uses statistical methods to analyze and summarize
data
• Identifies trends, correlations, and distributions

• � Examples: Mean, variance, regression,

correlation analysis
• � Use: Sales trend analysis, risk assessment
Some of the commonly used data
mining methods are:
2. Cluster Analysis
•Groups similar data objects into clusters
•� Examples: Customer segmentation, market analysis
� Algorithms: K-means, Hierarchical clustering

3. Decision Trees and Decision Rules

Uses a tree-like structure to make decisions

� Examples: Loan approval, medical diagnosis
� Popular algorithms: ID3, C4.5, CART

4. Association Rules
Discovers relationships between variables in large datasets
Often used in market basket analysis

� Example: If a customer buys bread, they also buy butter

� Algorithms: Apriori, FP-Growth
Some of the commonly used data
mining methods are:
5. Artificial Neural Networks (ANN)
Inspired by the human brain

� Examples: Image recognition, speech recognition

� Use: Fraud detection, Demand forecasting in retail and supply chains

6. Genetic Algorithms
Based on natural evolution and selection
Finds optimal or near-optimal solutions
� Examples: Shortest route selection, scheduling problems
� Key operations: Selection, crossover, mutation
7. Fuzzy Sets and Fuzzy Logic
Handles uncertainty and imprecision
Values range between 0 and 1 instead of true/false
� Examples: Washing machines, temperature
control
� Use: Decision support systems
Data Mining Applications
In direct marketing a company saves much
time by marketing to prospects that would have
the highest reply rate. Instead of random
selection on which customers to pick for their
surveys, a company could use direct marketing
from data mining to find the “correct” customers
to ask.
Given (Without Data Mining)

Number of mailers sent: 1,000,000

Cost per mailer: $0.40
� Total mailing cost
1,000,000×0.40=$400,000
Conversion Rate (Without Data Mining)

Conversion rate: 1%
(means only 1 out of every 100 people respond)

� Number of responses

1% of 1,000,000=10,000 customers
With Data Mining (Targeted Marketing)
Step 1: Identify a Smaller, Relevant Group
•Total population = 1,000,000
•Data mining identifies ¼ of the population that is more likely to respond
1/4×1,000,000=250,000 people
Step 2: Mailing Cost
•Cost per mailer = $0.40
•250,000×0.40=$100,000
•✅ Cost reduced from $400,000 to $100,000
Step 3: Higher Conversion Rate
Conversion rate with data mining = 3%
3% of 250,000=7,500 responses
Data Mining Applications
Market Segmentation

Market segmentation is used in data mining to identify common

characteristics and behaviors of customers who purchase a
company’s products or services. It involves grouping customers
based on similarities such as buying patterns, preferences,
demographics, or usage behavior.

Understanding these customer behaviors helps organizations

identify customer needs, design targeted marketing strategies,
and improve products and services, ultimately leading to better
business performance and customer satisfaction.
 Data Mining Applications

Customer Churn
Customer churn predicts which customers will
have a change of heart towards your company
and join another company (competitor).
Although customer churns are negative to
one’s business, it allows the corporation to
seek out the problem they are facing and
create solutions.
Customer Churn
Example: Magazine subscriber
Ideas to keep customer:
Discount, coupons, etc.
Data Mining Applications
Market Basket Analysis

Market Basket Analysis involves

studying customer purchase patterns to
discover relationships between products
that are frequently bought together.

Example: Ralphs Club Card

Cereal and Milk
Prediction based on Data
mining/Predictive analysis
Target(Retail store) analyzes customer shopping patterns to
predict which customers are likely to be pregnant.

Hospitals can predict which patients may need [Link]

helps in Resource planning, Bed management and Improving
patient care

Credit card – can predict which customers may miss their

payment based upon where card is used.
Bars / Alcohol outlets → Higher probability of missed payments
 . Class Identification

• Mathematical taxonomy
• Concept clustering
Data Mining Applications
Class identification, which consists of mathematical
taxonomy and concept clustering. Mathematical
taxonomy focuses on what makes the members of a
certain class similar, as opposed to differentiating one
class from another.

Ralphs may classify customers based on:

Income level
Past purchase behavior
Shopping frequency
This helps identify groups such as:
High-income frequent shoppers
Budget-conscious buyers
Occasional customers
Data Mining Applications
Concept Clustering determines groups (clusters)
according to attribute similarity, such as income, age
group, and purchase history. Using data mining,
companies can discover hidden purchasing patterns.

A retail company finds the following pattern:

Customers who purchase toys for children aged 3–5

years tend to purchase a kid’s bicycle within 6 months
about 90% of the time, especially among high-income
customers.

Business Application : Send targeted mail catalogs or

promotions for kid’s bicycles
Data mining Applications

Deviation analysis, A deviation can be fraud or a

change. In the past, such deviations were difficult to
detect in time to take corrective action. Data mining
tools help identify such deviations .
For example, a higher than normal credit purchase on
a credit card can be a fraud, or a genuine purchase by
the customer. Once a deviation has been discovered
as a fraud, the company takes steps to prevent such
frauds and initiates corrective action
Data Mining in Medical
The most recent and most promising use of data mining has
been the development of data mining
tools for the medical sector. The use of
data mining to extract patterns from medical data provides
near endless opportunities for symptom trend detection,
earlier detection of illness, DNA trend analysis and
improved patient reactions to medicines. These many
advantages allow doctors and hospitals to be more effective
and more efficient.
 Advantages of Data Mining: Medicine

• Earlier detection of illness

• Symptom trends

• Data analysis

• Improved drug reactions

 Disadvantages of Data Mining:
Medicine
• No uniform language - Medical

• Incomplete records

• Privacy
Data mining - Medical

How data mining is actually used to analyze individual

data can become quite complex due to the data. The
goal of the process is to take the medical data which
contain many attributes and determine which ones are
actually relevant to the diagnosis, symptom or result.

Two methods used in medical data mining are

clustering, discussed previously and biclustering.