CST 466 DATA MINING

WHY DATA MINING?

 The Explosive Growth of Data: from terabytes to
petabytes
 Data collection and data availability
 Automated data collection tools, database systems, Web,
computerized society
 Major sources of abundant data
 Business: Web, e-commerce, transactions, stocks, …
 Science: Remote sensing, bioinformatics, scientific simulation,
…
 Society and everyone: news, digital cameras, YouTube
 We are drowning in data, but starving for knowledge!
 “Necessity is the mother of invention”— Data
mining—Automated analysis of massive data sets

12/31/2025 2
WHAT IS (NOT) DATA MINING

12/31/2025
 What is not Data  What is Data Mining
Mining
 Certain names are more
 Look up phone number prevalent in certain US
in phone directory locations (OBrien,
ORurke, OReilly... In
Boston area)

 Query a Web search

engine for information  Group together similar
about ”Amazon” documents returned by
search engine according
to their context (e.g.
Amazon rainforest,
[Link],)
3
DATA MINING: ON WHAT KINDS OF DATA?
 Database-oriented data sets and applications
 Relational database, data warehouse, transactional
database
 Advanced data sets and advanced applications
 Data streams and sensor data
 Time-series data, temporal data, sequence data (incl. bio-
sequences)
 Structure data, graphs, social networks and multi-linked
data
 Object-relational databases
 Heterogeneous databases and legacy databases
 Spatial data and spatiotemporal data
 Multimedia database
 Text databases
 The World-Wide Web
12/31/2025 4
DATA MINING APPLICATIONS
●Classification: Eg: In loan database, to classify an applicant as
a prospective or defaulter, given his various personal and
demographic features along with previous purchase
characteristics.
●Estimation: Predict the attribute of a data instance. Eg:
estimate the percentage of marks of a student, whose previous
marks are already known.
●Prediction: Predictive model predicts a future outcome rather
than the current behaviour. Eg: Predict next week’s closing
price for the Google share price per unit.
●Market basket analysis(association rule mining): Eg: Analyses
hidden rules called association rule in a large transactional
database. pen, pencil → book – whenever pen and pencil are
purchased together, book is also purchased.
DATA MINING APPLICATIONS
●Clustering: Classification into different classes based on some
similarities but the target classes are unknown.
●Business intelligence: Provide historical, current and predictive views of
business operations
●Web Search engines: Specialized server that searches for information on
the web
●Education: Institutions use data mining to take accurate decisions and
predict academic results
●CRM: Customer relationship management: help analyse and process the
data, making it easier for the business to interact with customers and
future clients
●Fraud detection: help to anticipate and quickly detect fraud and take
immediate action to minimize costs.
DATA MINING APPLICATIONS
●Business data analytics: is the practice of using data, statistical
methods, and analytical tools to support business decision-
making. It turns raw data into insights that help organizations
improve performance, reduce risk, and identify opportunities.

●Bioinformatics: helps to extract useful knowledge from massive

datasets gathered in biology, and in other related life sciences
areas such as medicine and neuroscience
●Web mining: techniques with information gathered over the
World Wide Web
●Text mining: equivalent to text analytics, is the process of
deriving high-quality information from text.
●Social network data analysis: helps to study of social networks
and understand their structure and behavior.
DATA MINING AS A PROCESS IN KNOWLEDGE DISCOVERY
DATA MINING STAGES
1. Data cleaning : to remove noise and inconsistent data
Eg: Methods to remove noisy data and find out missing
values etc
2. Data integration: where multiple data sources may be
combined
Eg: Data warehousing, Enterprise information integration
3. Data selection: where data relevant to the analysis task
are retrieved from the database.
Eg: Attribute subset selection, Sampling, Data partitioning
etc
4. Data transformation
where data are transformed or consolidated into forms
appropriate for mining by performing summary or
aggregation operations
Eg: Includes method likes normalization, Smoothing,
Aggregation etc.
DATA MINING STAGES
5. Data mining : an essential process where intelligent
methods are applied in order to extract data patterns.
Eg: Classification, Prediction, Clustering, Association
rule mining etc
6. Pattern evaluation : to identify the truly interesting
patterns representing knowledge based on some
interestingness measures
Eg: Accuracy measures, Error measures, cross
validation etc.
7. Knowledge presentation: where visualization and
knowledge representation techniques are used to
present the mined knowledge to the user.
Eg: Graphs, Pi charts, Classification rules etc.
 Data Mining Concepts Architecture of a typical data mining system.

Architecture of a typical data mining system

CS 402 Data Mining & Warehousing

 Data Mining Concepts Architecture of a typical data mining system.

Architecture of a typical data mining system

Database, data warehouse, World Wide Web, or other information
repository
● Represent different kinds of information repositories where the data
resides.
● Data cleaning and data integration techniques may be performed on
the data

Database or data warehouse server

● The database or data warehouse server is responsible for fetching the
relevant data, based on the user’s data mining request.

CS 402 Data Mining & Warehousing

 Data Mining Concepts Architecture of a typical data mining system.

Architecture of a typical data mining system

Knowledge base
● This is the domain knowledge that is used to guide the search or
evaluate the interestingness of resulting patterns
● Knowledge such as user beliefs, which can be used to assess a
pattern’s interestingness based on its unexpectedness, may also be
included.

Data mining engine

● Consists of a set of functional modules for tasks such as
characterization, association and correlation analysis, classification,
prediction, cluster analysis, outlier analysis, and evolution analysis.

CS 402 Data Mining & Warehousing

 Data Mining Concepts Architecture of a typical data mining system.

Architecture of a typical data mining system

Pattern evaluation module
● This component typically employs interestingness measures
● Interacts with the data mining modules so as to focus the search
toward interesting patterns
● This module may be integrated with the mining module, depending
on the implementation of the data mining method used

CS 402 Data Mining & Warehousing

 Data Mining Concepts Architecture of a typical data mining system.

Architecture of a typical data mining system

User interface
● Module communicates between users and the data mining system
● Allow the user to interact with the system
● Users can specify a data mining query or task, providing information
to help focus the search, and performing exploratory data mining
based on the intermediate data mining results.
● Allows the user to browse database and data warehouse schemas or
data structures, evaluate mined patterns, and visualize the patterns in
different forms.

CS 402 Data Mining & Warehousing

DATA MINING MODELS /FUNCTIONALITIES
Predictive Model
● Makes a prediction about values of data using known results found
from different data
● May be made based on the use of other historical data.
● Classification, Regression, Time Series analysis, Predication are
predictive in nature
16 /
72

Descriptive Model
● Identifies patterns or relationships in data.
● Serves as a way to explore the properties of the data examined, not to
predict new properties.
● Clustering, summarization, association rules, and sequence discovery
are usually viewed as descriptive in nature.
DATA MINING MODELS/FUNCTIONALITIES
DATA MINING MODELS/FUNCTIONALITIES
1. Predictive model
 makes a prediction about values of data using known
results found from different data.
 may be made based on the use of other historical
data.
2. Descriptive model
 identifies patterns or relationships in data.
 serves as a way to explore the properties of the data
examined, not to predict new properties.
 Clustering, summarization, association rules, and
sequence discovery are usually viewed as descriptive
in nature.
DATA MINING MODELS/FUNCTIONALITIES
1.1 Classification
 maps data into predefined groups or classes.
 It is often referred to as supervised learning because
the classes are determined before examining the
data.
 Classification algorithms require that the classes be
defined based on data attribute values. They often
describe these classes by looking at the
characteristics of data already known to belong to the
classes
 Eg: Naïve Bayes Classifier, Support vector machine,
Decision tree based algorithms
DATA MINING MODELS/FUNCTIONALITIES
 An airport security screening station is used to
determine: if passengers are potential terrorists
or criminals.
 To do this, the face of each passenger is scanned
and its basic pattern (distance between eyes, size
and shape of mouth, shape of head, etc.) is
identified.
This pattern is compared to entries in a database
to see if it matches any patterns that are
associated with known offenders.
DATA MINING MODELS/FUNCTIONALITIES
1.2 Regression
 Regression is used to map a data item to a real
valued prediction variable.
 regression involves the learning of the function that
does this mapping.
 Regression assumes that the target data fit into some
known type of function (e.g., linear, logistic, etc.) and
then determines the best function of this type that
models the given data.
 Some type of error analysis is used to determine
which function is "best."
DATA MINING MODELS/FUNCTIONALITIES
 A college professor wishes to reach a certain level
of savings before her retirement.
 Periodically, she predicts what her retirement
savings will be based on its current value and
several past values.
 She uses a simple linear regression formula to
predict this value by fitting past behaviour to a
linear function and then using this function to
predict the values at points in the future.
 Based on these values, she then alters her
investment portfolio.
DATA MINING MODELS/FUNCTIONALITIES
1.3 Time Series Analysis
 The value of an attribute is examined as it varies
over time.
 The values usually are obtained as evenly spaced
time points (daily, weekly, hourly, etc.).
 A time series plot , is used to visualize the time
series.
 There are three basic functions performed in time
series analysis:
 distance measures are used to determine the similarity
between different time series.
 the structure of the line is examined to determine its

behaviour.
 use the historical time series plot to predict future values.
DATA MINING MODELS/FUNCTIONALITIES
 Mr. Smith is trying to determine whether to
purchase stock from Companies X, Y, or Z. For a
period of one month he charts the daily stock
price for each company. Figure 1.3 shows the
time series plot that Mr. Smith has generated.
Using this and similar information available
from his stockbroker, Mr. Smith decides to
purchase stock X because it is less volatile while
overall showing a slightly larger relative amount
of growth than either of the other stocks. As a
matter of fact, the stocks for Y and Z have a
similar behaviour. The behaviour of Y between
days 6 and 20 is identical to that for Z between
days 13 and 27.
 DATA MINING MODELS/FUNCTIONALITIES
__ o __ X
---x--- Y
Time Series plot __ __ Z
DATA MINING MODELS/FUNCTIONALITIES
1.4 Prediction
 Many real-world data mining applications can be seen as
predicting future data states based on past and current data.
 Prediction can be viewed as a type of classification.
 Prediction is predicting a future state rather than a current
state.
 Prediction applications include flooding, speech recognition,
machine learning, and pattern recognition.
DATA MINING MODELS/FUNCTIONALITIES
 Predicting flooding is a difficult problem. One
approach uses monitors placed at various ; points
in the river. These monitors collect data relevant
to flood prediction: water level, ' rain amount,
time, humidity, and so on. Then the water level
at a potential flooding point in the river can be
predicted based on the data collected by the
sensors upriver from this point. The prediction
must be made with respect to the time the data
were collected.
DATA MINING MODELS/FUNCTIONALITIES
2.1 Clustering
 similar to classification except that the groups
are not predefined, but rather defined by the data
alone.
 unsupervised learning or segmentation.

 It can be thought of as partitioning or

segmenting the data into groups that might or
might not be disjointed.
 The clustering is usually accomplished by
determining the similarity among the data on
predefined attributes.
 The most similar data are grouped into clusters.
DATA MINING MODELS/FUNCTIONALITIES
 A certain national department store chain
creates special catalogs targeted to various
demographic groups based on attributes such as
income, location, and physical characteristics of
potential customers (age, height, weight, etc.). To
determine the target mailings of the various
catalogs and to assist in the creation of new,
more specific catalogs, the company performs a
clustering of potential customers based on the
determined attribute values. The results of the
clustering exercise are then used by management
to create special catalogs and distribute them to
the correct target population based on the cluster
for that catalog.
DATA MINING MODELS/FUNCTIONALITIES
2.2 Summarization
 Summarization maps data into subsets with
associated simple descriptions.
 also called characterization or generalization.
 It extracts or derives representative information
about the database.
 This may be accomplished by actually retrieving
portions of the data.
 summary type information (such as the mean of
some numeric attribute) can be derived from the
data.
 The summarization characterizes the contents of the
database
DATA MINING MODELS/FUNCTIONALITIES
 One of the many criteria used to compare
universities by the U.S. News & World Report is
the average SAT or ACT score [GM99]. This is a
summarization used to estimate the type and
intellectual level of the student body.
DATA MINING MODELS/FUNCTIONALITIES
2.3 Association Rules
 Link analysis, alternatively referred to as affinity
analysis or association, refers to the data mining
task of uncovering relationships among data.
 An association rule is a model that identifies
specific types of data associations.
 These associations are often used in the retail
sales community to identify items that are
frequently purchased together.
 Associations are also used in many other
applications such as predicting the failure of
telecommunication switches.
DATA MINING MODELS/FUNCTIONALITIES
 A grocery store retailer is trying to decide
whether to put bread on sale. To help determine
the impact of this decision, the retailer generates
association rules that show what other products
are frequently purchased with bread. He finds
that 60% of the time that bread is sold so are
pretzels and that 70% of the time jelly is also
sold. Based on these facts, he tries to capitalize
on the association between bread, pretzels, and
jelly by placing some pretzels and jelly at the end
of the aisle where the bread is placed. In
addition, he decides not to place either of these
items on sale at the same time.
DATA MINING MODELS/FUNCTIONALITIES
2.4 Sequence Discovery
 Sequential analysis or sequence discovery is used
to determine sequential patterns in data.
 These patterns are based on a time sequence of
actions.
 These patterns are similar to associations in that
data (or events) are found to be related, but the
relationship is based on time.
 In sequence discovery the items are purchased
over time in some order.
 For example, most people who purchase CD
players may be found to purchase CDs within one
week.
DATA MINING MODELS/FUNCTIONALITIES
 The Webmaster at the XYZ Corp. periodically
analyzes the Web log data to determine how
users of the XYZ's Web pages access them. He is
interested in determining what sequences of
pages are frequently accessed. He determines
that 70 percent of the users of page A follow one
of the following patterns of behavior: (A, B, C) or
(A, D, B, C) or (A, E, B, C). He then determines to
add a link directly from page A to page C.
DATA MINING ISSUES

 Major issues in the data mining portioned into five groups.

1. Mining methodology
2. User interaction.
3. Efficiency and Scalability
4. Diversity of database types
5. Data mining and Society
DATA MINING ISSUES CONTD….

I. Mining methodology:
 * Mining various and new kinds of knowledge in
databases: Data mining covers a wide spectrum of
data analysis and knowledge discovery tasks through
classification, clustering, regression, association rule
mining etc. These may use the same data base in
different ways and require to develop different data
mining techniques.
 * Mining knowledge in multidimensional space:
Mining knowledge in multidimensional space is
challenging one, multi dimensional data cube space
can substantially enhance the power and flexibility of
data mining.
DATA MINING ISSUES CONTD….

I. Mining methodology:
 *Inter disciplinary effort: The power of the data
mining can be substantially enhanced by integrating
new methods from multiple discipline.
 *Boosting the power of discovery in a networked
environment.
 *Handling uncertainty, noise or incompleteness of
data.
 *Pattern evaluation and patterns or constraint
guided mining.
DATA MINING ISSUES CONTD…
 II. User interaction
• Interactive mining: It should allow users to dynamically change
the focus of search, refine mining request based on returned
results and to drill, dice and pivot the data and knowledge space
interactively.
• Incorporation of background knowledge: Background
knowledge, constraints, rules, and other information regarding the
domain under study should be incorporated into the knowledge
discovery process.
• Ad hoc data mining and data mining query language: high
level data mining query language or high level flexible user
interface should be designed.
• Presentation and visualization of data mining results: Proper
presentation of data mining results is very important so that
discovered knowledge can be easily understood.
DATA MINING ISSUES CONTD…
 III. Efficiency and scalability
• Efficiency and scalability of data mining algorithm. Running time
of data mining algorithm must be predictable, short and acceptable by
application. Efficiency, scalability, optimization and ability to execute
in real time are key criteria that drive the development of many new
data mining algorithms.
• Parallel distributed and incremental data mining
algorithms: The huge size of many data sets, the wide
distribution of data and computational complexity motivate
the development of parallel, distributed and incremental data
mining algorithm.
DATA MINING ISSUES CONTD…
 IV. Diversity of database types
• Handling complex types of data: Diverse application generate a wide
spectrum of new data types, from structured to semi structured and
unstructured data, from stable to dynamic data streams, from simple
data objects to temporal data, biological sequence, sensor data, spatial
data, hypertext data, multimedia data, software program code, web
data and social network data. To handle wide variety of data, the
construction of effective data mining tool is needed which remain a
challenge.
• Mining dynamic, networked and global data repositories: The
discovery of knowledge from different sources of structured, semi
structured or unstructured yet interconnected data with diverse data
semantics poses great challenges to data mining.
DATA MINING ISSUES CONTD…
 V. Data mining and society
• Social impacts of data mining: The improper disclosure or use of
data and the potential violation of individual privacy and data
protection rights are areas of concern that needs to be addressed.
• Privacy preserving data mining: Data mining will help scientific
discovery, business management. Economic recovery and security
protection. However, it poses the risk of disclosing an individual’s
personal information. The philosophy is to observe the data sensitivity
and preserve people privacy while performing successful data mining.
• Invisible data mining: More and more systems should have data
mining functionalities within so the people can perform data mining or
use mining results simply by clicking mouse, without any knowledge
of data mining algorithm.
DATA WAREHOUSING
●A data warehouse is a repository of information
collected from multiple sources, stored under a
unified schema, and that usually resides at a
single site.
Data warehouse refers to a database that is
maintained separately from an organization’s
operational databases.
A data warehouse is a subject-oriented,
integrated, time-variant, and non-volatile
collection of data in support of management’s
decision making process
DATA WAREHOUSE
 Subject-oriented:
 A data warehouse is organized around major subjects, such
as customer, supplier, product, and sales.
 A data warehouse focuses on the modelling and analysis of
data for decision makers(not on day to day transaction).
 Provide a simple and concise view around particular subject
issues by excluding data that are not useful in the decision
support process.
 Integrated:
 A data warehouse is usually constructed by integrating
multiple heterogeneous sources, such as relational
databases, flat files, and on-line transaction records.
DATA WAREHOUSE
 Time-variant:
 Data are stored to provide information from a historical
perspective
 Every key structure in the data warehouse contains, either

implicitly or explicitly, an element of time.

 Non-volatile:
 A data warehouse is always a physically separate store of
data, transformed from the application data found in the
operational environment.
 Due to this separation, a data warehouse does not require
transaction processing, recovery, and concurrency control
mechanisms.
 It usually requires only two operations in data accessing:
initial loading of data and access of data.
DATA WAREHOUSING
 Data warehousing is the process of constructing
and using data warehouses.
 The construction of a data warehouse requires data
cleaning, data integration, and data consolidation.
 The utilization of a data warehouse often necessitates
a collection of decision support technologies. This
allows “knowledge workers” (e.g., managers,
analysts, and executives) to use the warehouse to
quickly and conveniently obtain an overview of the
data, and to make sound decisions based on
information in the warehouse.
DATA WAREHOUSING
 Data warehousing is very useful from the point of
view of heterogeneous database integration.
 The traditional database approach to heterogeneous
database integration was a ‘query- driven’ approach
 data warehousing employs an update-driven
approach in which information from multiple,
heterogeneous sources is integrated in advance and
stored in a warehouse for direct querying and
analysis
 OPERATIONAL DATABASE SYSTEMS
VS
DATAWARE HOUSES
 Operational Database systems
 Main task is to perform on-line transaction and query
processing. These systems are called on-line transaction
processing (OLTP) systems.
 They cover most of the day-to-day operations of an
organization, such as purchasing, inventory,
manufacturing, banking, payroll, registration, and
accounting.
 Data Warehouse
 serve users or knowledge workers in the role of data
analysis and decision making.
 Such systems can organize and present data in various
formats in order to accommodate the diverse needs of the
different users. These systems are known as on-line
analytical processing (OLAP) systems.
OLTP VS OLAP
 Users and system orientation:
 OLTP system is customer-oriented and is used for transaction
and query processing by clerks, clients, and information
technology professionals.
 OLAP system is market-oriented and is used for data analysis
by knowledge workers, including managers, executives, and
analysts.
 Data contents:
 OLTP system manages current data
 OLAP system manages large amounts of historical data,
provides facilities for summarization and aggregation, and
stores and manages information at different levels of
granularity.
 Database design:
 An OLTP system usually adopts an entity-relationship (ER)
data model and an application-oriented database design.
 An OLAP system typically adopts either a star or snowflake
model and a subjectoriented database design.
OLTP VS OLAP
 View:
 An OLTP system focuses mainly on the current data within
an enterprise or department, without referring to historical
data or data in different organizations.
 An OLAP system often spans multiple versions of a
database schema, due to the evolutionary process of an
organization.
 OLAP systems also deal with information that originates
from different organizations.
 OLAP data are stored on multiple storage media.
 Access patterns:
 The access patterns of an OLTP system consist mainly of
short, atomic transactions. Such a system requires
concurrency control and recovery mechanisms.
 Accesses to OLAP systems are mostly read-only operations
although many could be complex queries.
NEED FOR DATA WAREHOUSING
 To promote the high performance of both online transaction
processing and online analytical processing
 Data warehouse queries are often complex. They involve
the computation of large groups of data at summarized
levels, and may require the use of special data
organization, access, and implementation methods based
on multidimensional views. Processing OLAP queries in
operational databases would substantially degrade the
performance of operational tasks.
 An operational database supports the concurrent
processing of multiple transactions. Concurrency control
techniques are required in OLTP. But such measures will
degrade the performance of OLAP.
 Structures, contents, and uses of the data in these two
systems are different.
OLTP VS DATA WAREHOUSE
 OLTP systems
 designed to maximize the transaction processing
capacity.
 commonly used in clerical data processing tasks,
structured repetitive tasks, read update a few records.
 isolation, recovery and integrity are critical.

 Data warehouse
 holds data that is historical, detailed, and summarized
to various levels and rarely subject to change.
 designed to support relatively low numbers of
transactions that are unpredictable in nature and
require answers to queries that are ad
hoc, unstructured, and heuristic.
APPLICATIONS OF DWH
 Banking Industry
 In the banking industry, concentration is given to risk
management and policy reversal as well analyzing
consumer data, market trends, government regulations
and reports, and more importantly financial decision
making.
 Certain banking sectors utilize them for market research,
performance analysis of each product, interchange and
exchange rates, and to develop marketing programs.
 Analysis of card holder’s transactions, spending patterns
and merchant classification, all of which provide the bank
with an opportunity to introduce special offers and
lucrative deals based on cardholder activity.
 Finance Industry
 revolve around evaluation and trends of customer expenses
which aids in maximizing the profits earned by their
clients.
APPLICATIONS OF DWH
 Consumer Goods Industry
 They are used for prediction of consumer trends, inventory
management, market and advertising research.
 In-depth analysis of sales and production is also carried out.
 Government and Education
 The federal government utilizes the warehouses for research
in compliance, whereas the state government uses it for
services related to human resources like recruitment, and
accounting like payroll management.
 The government uses data warehouses to maintain and
analyze tax records, health policy records and their respective
providers.
 Criminal law database is connected to the state’s data
warehouse. Criminal activity is predicted from the patterns
and trends, results of the analysis of historical data associated
with past criminals.
 Universities use warehouses for extracting of information used
for the proposal of research grants, understanding their
student demographics, and human resource management.
APPLICATIONS OF DWH
 Healthcare
 All of their financial, clinical, and employee records
are fed to warehouses as it helps them to strategize
and predict outcomes, track and analyze their service
feedback, generate patient reports, share data with
tie-in insurance companies, medical aid services, etc.
 Hospitality Industry
 A major proportion of this industry is dominated by
hotel and restaurant services, car rental services, and
holiday home services.
 They utilize warehouse services to design and
evaluate their advertising and promotion campaigns
where they target customers based on their feedback
and travel patterns.
APPLICATIONS OF DWH
 Insurance
 The warehouses are primarily used to analyze data patterns
and customer trends, apart from maintaining records of
already existing participants.
 The design of tailor-made customer offers and promotions is
also possible through warehouses.
 Manufacturing and Distribution Industry
 A manufacturing organization has to take several make-or-
buy decisions which can influence the future of the sector,
which is why they utilize high-end OLAP tools as a part of
data warehouses to predict market changes, analyze current
business trends, detect warning conditions, view marketing
developments, and ultimately take better decisions.
 They also use them for product shipment records, records of
product portfolios, identify profitable product lines, analyze
previous data and customer feedback to evaluate the weaker
product lines and eliminate them.
 For the distributions, the supply chain management of
products operates through data warehouses.
APPLICATIONS OF DWH
 The Retailers
 Retailers serve as middlemen between producers and
consumers.
 They use warehouses to track items, their
advertising promotions, and the consumers buying
trends.
 They also analyze sales to determine fast selling and
slow selling product lines and determine their shelf
space through a process of elimination.
 Services Sector
 Data warehouses find themselves to be of use in the
service sector for maintenance of financial records,
revenue patterns, customer profiling, resource
management, and human resources.
APPLICATIONS OF DWH
 Telephone Industry
 The telephone industry operates over both offline and
online data burdening them with a lot of historical data
which has to be consolidated and integrated.
 Analysis of fixed assets, analysis of customer’s calling
patterns for sales representatives to push advertising
campaigns, and tracking of customer queries, all require
the facilities of a data warehouse.
 Transportation Industry
 In the transportation industry, data warehouses record
customer data enabling traders to experiment with target
marketing where the marketing campaigns are designed
by keeping customer requirements in mind.
 To analyze customer feedback, performance, manage crews
on board as well as analyze customer financial reports for
pricing strategies.
 Data Mining Applications

A Three-Tier Data Warehouse Architecture

61 /

72
 A THREE-TIER DATA WAREHOUSE
ARCHITECTURE
Bottom tier: Warehouse database server
● Consists of a warehouse database server that is almost always a
relational database system.
● Back-end tools and utilities are used to feed data into the bottom tier
from operational databases or other external sources
● These tools and utilities perform data extraction, cleaning, and
transformation, load and refresh functions to update the data
warehouse
● These tools support to merge similar data from different sources into
a unified format.

CS 402 Data Mining & Warehousing

A THREE-TIER DATA WAREHOUSE
ARCHITECTURE III
Bottom tier: Warehouse database server
● The data are extracted using application program interfaces known as
gateways.
● A gateway is supported by the underlying DBMS and allows client
programs to generate SQL code to be executed at a server.(ODBC,
63 /

72

JDBC, OLEDB)
● This tier also contains a metadata repository, which stores
information about the data warehouse and its contents.

CS 402 Data Mining & Warehousing

A THREE-TIER DATA WAREHOUSE
ARCHITECTURE IV
Middle Tier: OLAP server
● Consists OLAP server that is typically implemented using either a
relational OLAP (ROLAP) model, a multidimensional OLAP
(MOLAP) model
● ROLAP: an extended relational DBMS that maps operations on
multidimensional data to standard relational operations;
64 /
72

● MOLAP: a special-purpose server that directly implements

multidimensional data and operations.
● For a user, this application tier presents an abstracted view of the
database.
● This layer also acts as a mediator between the end-user and the
database.

CS 402 Data Mining & Warehousing

A THREE-TIER DATA WAREHOUSE ARCHITECTURE V

Top Tier: Front-end client layer

● Front-end client layer, which contains query and reporting tools,
analysis tools, and/or data mining tools (e.g., trend analysis,
prediction, and so on).
● Consists of the tools and API that you connect and get data out from
65 /
72

the data warehouse.

CS 402 Data Mining & Warehousing

MULTIDIMENSIONAL DATA MODEL
● Data warehouses and OLAP tools are based on a multidimensional
data model.
● This model views data in the form of a data cube
● The data cube is a metaphor for multidimensional data storage.
● The actual physical storage of such data may differ from this logical
representation.
● A data cube allows data to be modeled and viewed in multiple
dimensions.
● It is defined by dimensions and facts.
● Dimensions are the perspectives or entities with respect to which an
organization wants to keep records
● Dimension table: A table associated with each dimension, which
further describes the dimension.

CS 402 Data Mining & Warehousing

MULTIDIMENSIONAL DATA MODEL
● Dimension tables can be specified by users or experts, or
automatically generated and adjusted based on data distributions
● A multidimensional data model is typically organized around a central
theme, which is is represented by a fact table.
● Facts are numerical measures.

Example: AllElectronics
67 /

72

CS 402 Data Mining & Warehousing

MULTIDIMENSIONAL DATA MODEL

● Sale’s data warehouse

● Keep records of the store’s sales with respect to the dimensions time,
item, branch, and location.
● These dimensions allow the store to keep track of things like monthly
sales of items and the branches and locations
68 /

72

● A dimension table for item may contain the attributes item name,
brand, and type.
● Facts: dollars sold (sales amount in dollars), units sold (number of
units sold), and amount budgeted
● The fact table contains the names of the facts, or measures, as well
as keys to each of the related dimension tables.

CS 402 Data Mining & Warehousing

MULTIDIMENSIONAL DATA MODEL
2-D data Cube for AllElectronics sale’s data warehouse
● Represents 2 dimensions(item, time) and location is fixed as
Vancouver.
● ie., We look at the AllElectronics sales data for items(item) sold per
quarter(time) in the city of Vancouver.
69 /

72

● A simple 2-D data cube that is, in fact, a table or spreadsheet for
sales data from AllElectronics.
● In this 2-D representation, the sales for Vancouver are shown with
respect to the time dimension (organized in quarters) and the item
dimension (organized according to the types of items sold).
● The fact or measure displayed is dollars sold (in thousands).

CS 402 Data Mining & Warehousing

MULTIDIMENSIONAL DATA MODEL V

Figure: 2-D data Cube for AllElectronics sale’s data warehouse

CS 402 Data Mining & Warehousing

 Data Mining Applications

3-D data Cube for AllElectronics sale’s data warehouse

MULTIDIMENSIONAL
● Represents DATA
3 dimensions(item, time, MODEL VI
location)
● ie.,View the data according to time and item, as well as location for
the cities Chicago, New York, Toronto, and Vancouver.
71 /

72

● 3-D data are represented as a series of 2-D tables

● The measure displayed is dollars sold (in thousands).

CS 402 Data Mining & Warehousing

 Data Mining Applications

Multidimensional Data Model VII

72 /

72

CS 402 Data Mining & Warehousing

 Data Mining Applications

MULTIDIMENSIONAL DATA MODEL VIII

73 /

72

Figure: 3-D data Cube for AllElectronics sale’s data warehouse

CS 402 Data Mining & Warehousing

4-D DATA CUBE FOR ALLData
ELECTRONICS SALE’S DATA
Mining Applications

WAREHOUSE

● Represents 4 dimensions(item, time, location,supplier)

● ie., View our sales data with an additional fourth dimension
● We can think of a 4-D cube as being a series of 3-D cube.
● If we continue in this way, we may display any n −D data as a series
of (n −1) −D “cubes.”

74

72

CS 402 Data Mining & Warehousing 52 / 72

SCHEMAS FOR MULTIDIMENSIONAL DATABASES I

● The most popular data model for a data warehouse is a

multidimensional model
● Multidimensional models are logical representation of data.
● The physical storage of such models can exist in the form of a star
schema, a snowflake schema, or a fact constellation schema

CS 402 Data Mining & Warehousing 52 / 72

SCHEMAS FOR MULTIDIMENSIONAL DATABASES

Star schema
● The most common modeling paradigm
● The data warehouse contains:
● A large central table (fact table) containing the bulk of the data, with
no redundancy 76 /

● A set of smaller attendant tables (dimension tables), one for each

72

dimension
● The schema graph resembles a starburst, with the dimension tables
displayed in a radial pattern around the central fact table.
● Each dimension is represented by only one table, and each table
contains a set of attributes.

CS 402 Data Mining & Warehousing

SCHEMAS FOR MULTIDIMENSIONAL DATABASES

Star schema: Example AllElectronics sales

CS 402 Data Mining & Warehousing

 Data Mining Applications

● Sales are considered along four dimensions, namely, time, item,

branch, and location.
● Schema contains: Central fact table for sales that contains keys to
each of the four dimensions along with two measures: dollars sold and
SCHEMAS
units sold FOR MULTIDIMENSIONAL DATABASES IV
● To minimize the size of the fact table, dimension identifiers (such as
time key and item key) are system-generated identifiers.
● The location dimension table contains the attribute set location key,
street, city, province or state, country.
● Example for redundancy
● “Vancouver” and “Victoria” are both cities in the Canadian province of
British Columbia.
● Entries will be like (..., Vancouver, British Columbia, Canada) and (...,
Victoria, British Columbia, Canada). Canada)

CS 402 Data Mining & Warehousing

 Data Mining Applications

SCHEMAS FOR MULTIDIMENSIONAL DATABASES

Figure: Star schema of a data warehouse for sales.

CS 402 Data Mining & Warehousing

SCHEMAS FOR MULTIDIMENSIONAL DATABASES
Snowflake schema
● The snowflake schema is a variant of the star schema model, where
some dimension tables are normalized
● Further splitting the data into additional tables.
● The resulting schema graph forms a shape similar to a snowflake.
● Major difference between the snowflake and star schema models
● Dimension tables of the snowflake model may be kept in normalized
form to reduce redundancies.
● Snowflake structure can reduce the effectiveness of browsing, since
more joins will be needed to execute a query.
● Snowflake schema reduces redundancy.
● It is not as popular as the star schema.

CS 402 Data Mining & Warehousing

SCHEMAS FOR MULTIDIMENSIONAL DATABASES
Snowflake schema: Example AllElectronics sales
● The sales fact table is identical to that of the star schema
● Difference between the two schemas is in the definition of dimension
tables.
● The single dimension table for item in the star schema is normalized
81 /

72

in the snowflake schema, resulting in new item and supplier tables.

● Example: In item dimesion table item(item key, item name, brand,
type, and supplier key)
● where supplier key is linked to the supplier dimension table
supplier(supplier key, supplier type)

CS 402 Data Mining & Warehousing

SCHEMAS FOR MULTIDIMENSIONAL DATABASES

Figure: Snowflake schema of a data warehouse for sales.

CS 402 Data Mining & Warehousing

SCHEMAS FOR MULTIDIMENSIONAL DATABASES
Fact constellation Schema
● Multiple fact tables share dimension tables
● This kind of schema can be viewed as a collection of stars, and hence
is called a galaxy schema or a fact constellation.

Fact constellation Schema: Example AllElectronics sales

● This schema specifies two fact tables, sales and shipping.
● shipping table has five dimensions, or keys: item key, time key,
shipper key, from location, and to location, and two measures: dollars
cost and units shipped.
● Dimensions tables for time, item, and location are shared between
both the sales and shipping fact tables.

CS 402 Data Mining & Warehousing

 Data Mining Applications

SCHEMAS FOR MULTIDIMENSIONAL DATABASES X

Figure: Fact constellation schema of a data warehouse for sales and shipping.

CS 402 Data Mining & Warehousing

 Data Mining Applications

TYPICAL OLAP OPERATIONS

• Since OLAP servers are based on

multidimensional view of data, we will
discuss OLAP operations in
multidimensional data.

85 /

There exists different types of OLAP operations:

72

● Roll Up
● Drill Down
● Slice & Dice
● Pivot(Rotate)

CS 402 Data Mining & Warehousing

86

72
87

72
TYPICAL OLAP
Data Mining Applications

OPERATIONS II
Question 1
● Suppose that a data warehouse consists of the three dimensions time,
doctor, and patient, and the two measures count and charge, where
charge is the fee that a doctor charges a patient for a visit.
(a) Draw a schema diagram for the above data warehouse using one of the
schema classes.
(b) Starting with the base cuboid [day, doctor, patient], what specific
OLAP operations should be performed in order to list the total fee
88 /

collected by each doctor in 2010?

72

CS 402 Data Mining & Warehousing

TYPICAL OLAP
Data Mining Applications

OPERATIONS III
(a) Draw a schema diagram for the above data warehouse using one of
the schema classes.

CS 402 Data Mining & Warehousing

TYPICAL OLAP
Data Mining Applications

OPERATIONS IV
(b) Starting with the base cuboid [day, doctor, patient], what specific
OLAP operations should be performed in order to list the total fee
collected by each doctor in 2010?
● Roll-up on time from day to year.
● Slice for time = 2010.
● Roll-up on patient from individual patient to all.
90 /

72

CS 402 Data Mining & Warehousing

TYPICAL OLAP
Data Mining Applications

OPERATIONS V
Question 2
● Suppose that a data warehouse for Big-University consists of the
following four dimensions: student, course, semester, and instructor,
and two measures count and avg. grade. When at the lowest
conceptual level (e.g., for a given student, course, semester, and
instructor combination), the avg grade measure stores the actual
course grade of the student. At higher conceptual levels, avg grade
stores the average grade for the given combination.
91 /

(a) Draw a schema diagram for the above data warehouse using one of the
72

schema classes.
(b) Starting with the base cuboid [student, course, semester, instructor],
what specific OLAP operations should one perform in order to list the
average grade of CS courses for each Big University student.

CS 402 Data Mining & Warehousing

TYPICAL OLAP OPERATIONS
(a) Draw a schema diagram for the above data warehouse using one of
the schema classes.

University

Figure: Snowflake Schema

CS 402 Data Mining & Warehousing
TYPICAL OLAP OPERATIONS VII
(b) Starting with the base cuboid [student, course, semester, instructor],
what specific OLAP operations should one perform in order to list the
average grade of CS courses for each Big University student.
● Roll-up on course from course id to department.
● Roll-up on semester from semester id to all.
● Slice for Department=“CS” .

CS 402 Data Mining & Warehousing

 Data Mining Applications

TYPICAL OLAP OPERATIONS

Question 3
● Suppose that a data warehouse consists of the four dimensions, date,
spectator, location, and game, and the two measures, count and
charge, where charge is the fare that a spectator pays when watching
a game on a given date. Spectators may be students, adults, or
94 /

seniors, with each category having its own charge rate.

72

(a) Draw a schema diagram for the above data warehouse using one of the
schema classes.
(b) Starting with the base cuboid [date, spectator, location, game], what
specific OLAP operations should one perform in order to list the total
charge paid by student spectators at GM Place in 2010?

CS 402 Data Mining & Warehousing

 Data Mining Applications

(a) Draw a schema diagram for the above data warehouse using one of
the schema classes.

TYPICAL OLAP OPERATIONS

Game
IX
95 /

72

CS 402 Data Mining & Warehousing

 Data Mining Applications

TYPICAL OLAP OPERATIONS X

(b) Starting with the base cuboid [date, spectator, location, game], what
specific OLAP operations should one perform in order to list the total
charge paid by student spectators at GM Place in 2010?
● Roll-up on date from date id to year.
● Roll-up on game from game id to all.
● Roll-up on location from location id to location name.
● Roll-up on spectator from spectator id to status.
● Dice with status=“students”, location name=“GM Place”, and year
= 2010.

CS 402 Data Mining & Warehousing

Data Mining Applications

CS 402 Data Mining & Warehousing