DATA WAREHOUSING AND DATA MINING
1 4/23/2019 1:36:14 PM
DATA WAREHOUSING
Data warehousing is combining data from
multiple sources into one comprehensive and
easily manipulated database.
The primary aim for data warehousing is to
provide businesses with analytics results from
data mining, OLAP, Scorecarding and
reporting.
2 4/23/2019 1:36:14 PM
Introduction, Definitions, and
Terminology
W. H Inmon characterized a data warehouse
as:
– “A subject-oriented, integrated, nonvolatile,
time-variant collection of data in support of
management’s decisions.”
3 4/23/2019 1:36:14 PM
NEED FOR DATA WAREHOUSING
Information is now considered as a key for all
the works.
Those who gather, analyze, understand, and
act upon information are winners.
Information have no limits, it is very hard to
collect information from various sources, so
we need an data warehouse from where we
can get all the information.
4 4/23/2019 1:36:14 PM
Applications that data warehouse
supports are:
– OLAP (Online Analytical Processing) is a term used to
describe the analysis of complex data from the data
warehouse.
– DSS (Decision Support Systems) also known as EIS
(Executive Information Systems) supports
organization’s leading decision makers for making
complex and important decisions.
– Data Mining is used for knowledge discovery, the
process of searching data for unanticipated new
knowledge.
5 4/23/2019 1:36:14 PM
Conceptual Structure of Data
Warehouse
Data Warehouse processing involves
– Cleaning and reformatting of data
– OLAP
– Data Mining
6 4/23/2019 1:36:14 PM
Conceptual Structure of Data
Warehouse
Back Flushing
Data Warehouse
OLAP
Data
Cleaning Reformatting DSSI
Databases EIS
Metadata
Data
Mining
Other Data Inputs Updates/New Data
7 4/23/2019 1:36:14 PM
DATA WAREHOUSING INCLUDES:-
Retrieving data
Analyzing data
Extracting data
Loading data
Transforming data
Managing data
8 4/23/2019 1:36:14 PM
DATA WAREHOUSE ARCHITECTURE
Data warehousing is designed to provide an
architecture that will make cooperate data
accessible and useful to users.
There is no right or wrong architecture.
The worthiness of the architecture can be
judge by its use, and concept behind it .
Data Warehouses can be architected in
many different ways, depending on the
specific needs of a business.
9 4/23/2019 1:36:14 PM
Typical Data Warehousing Environment
10 4/23/2019 1:36:14 PM
An operational data store (ODS) is basically a
database that is used for being an temporary
storage area for a datawarehouse.
Its primary purpose is for handling data which
are progressively in use.
Operational data store contains data which are
constantly updated through the course of the
business operations.
11 4/23/2019 1:36:14 PM
ETL (Extract, Transform, Load) is used to copy
data from:-
ODS to data warehouse staging area.
Data warehouse staging area to data warehouse
.
Data warehouse to data mart .
ETL extracts data, transforms values of
inconsistent data, cleanses "bad" data, filters
data and loads data into a target database.
12 4/23/2019 1:36:14 PM
The Data Warehouse Staging Area is
temporary location where data from source
systems is copied.
It increases the speed of data warehouse
architecture.
It is very essential since data is increasing
day by day.
13 4/23/2019 1:36:14 PM
The purpose of the Data Warehouse is to integrate
corporate data.
The amount of data in the Data Warehouse is
massive. Data is stored at a very deep level of
detail.
This allows data to be grouped in unimaginable
ways.
Data Warehouses does not contain all the data in
the organization ,It's purpose is to provide base that
are needed by the organization for strategic and
tactical decision making.
14 4/23/2019 1:36:14 PM
ETL extract data from the Data Warehouse and
send to one or more Data Marts for use of users.
Data marts are represented as shortcut to a data
warehouse ,to save time.
It is just an partition of data present in data
warehouse.
Each Data Mart can contain different
combinations of tables, columns and rows from
the Enterprise Data Warehouse.
15 4/23/2019 1:36:14 PM
REASONS FOR CREATING AN DATA MART
Easy access to frequently needed data.
Creates collective view by a group of users.
Improves user response time.
Ease of creation.
Lower cost than implementing a full Data
warehouse
16 4/23/2019 1:36:14 PM
DATA MINING
The non-trivial extraction of implicit,
previously unknown, and potentially useful
information from large databases.
– Extremely large datasets
– Useful knowledge that can improve
processes
– Cannot be done manually
17 4/23/2019 1:36:14 PM
Where Has it Come From ?
18 4/23/2019 1:36:14 PM
Motivation
Databases today are huge:
– More than 1,000,000 entities/records/rows
– From 10 to 10,000 fields/attributes/variables
– Giga-bytes and tera-bytes
Databases a growing at an unprecendented rate
The corporate world is a cut-throat world
– Decisions must be made rapidly
– Decisions must be made with maximum knowledge
19 4/23/2019 1:36:14 PM
How does data mining work?
Extract, transform, and load transaction data onto
the data warehouse system.
Store and manage the data in a multidimensional
database system.
Provide data access to business analysts and
information technology professionals.
Analyze the data by application software.
Present the data in a useful format, such as a graph
or table
20 4/23/2019 1:36:14 PM
DATA MINING MEASURES
Accuracy
Clarity
Dirty Data
Scalability
Speed
Validation
21 4/23/2019 1:36:14 PM
Typical Applications of Data Mining
22 4/23/2019 1:36:14 PM
ADVANTAGES OF DATA MINING
Engineering and Technology
Medical Science
Business
Combating Terrorism
Games
Research and Development
23 4/23/2019 1:36:14 PM
Engineering and Technology
In Electrical Power Engineering
- used for condition monitoring of high
voltage electrical equipment
- vibration monitoring and analysis of
transformer on-load tap-changers
Education
- to concentrate their knowledge
24 4/23/2019 1:36:14 PM
Medical Science
Data mining has been widely used in area of
bioinformatics , genetics
DNA sequences and variability in disease
susceptibility which is very important to help
improve the diagnosis, prevention and
treatment of the diseases
25 4/23/2019 1:36:14 PM
BUSINESS
In Customer Relationship Management
applications
It Translate data from customer to merchant
Accurately
Distribute Business Processes
Powerful Tool For Marketing
26 4/23/2019 1:36:14 PM
Combating terrorism
Concept used by Interpol against terrorists
for searching their records by Multistate Anti-
Terrorism Information Exchange
In the Secure Flight program , Computer
Assisted Passenger Pre screening System ,
Semantic Enhancement
27 4/23/2019 1:36:14 PM
Games
for certain combinatorial games, also called
table bases (e.g. for 3x3-chess)
It includes extraction of human-usable
strategies
Berlekamp in dots-and-boxes and Joh Nunn
in chess endgames are notable examples
28 4/23/2019 1:36:14 PM
Research And Development
Helps to Develop the search algorithms
It offers huge libraries of graphing and
visualisation softwares
The users can easily create the models
optimally
29 4/23/2019 1:36:14 PM
List of the top eight data-mining
software vendors
Angoss Software
Infor CRM Epiphany
Portrait Software
SAS
G-Stat
SPSS
ThinkAnalytics
Unica
Viscovery
30 4/23/2019 1:36:14 PM
THANK YOU
31 4/23/2019 1:36:14 PM