Module 2: Data Warehouse and

OLAP Technology for Data Mining

Data Warehouse, Multidimensional Data Model, Data Warehouse

Architecture, Data Warehouse Implementation, Further Development of Data
Cube Technology, From Data Warehousing to Data Mining Data Cube
Computation and Data Generalization: Efficient Methods for Data Cube
Computation, Further Development of Data Cube and OLAP Technology,
Attribute-Oriented Induction. OLAP (Online Analytical Processing)
Data Warehouse
• A data warehouse is a centralized repository designed for data
analysis and reporting, collecting historical and current data from
multiple sources into a single, structured, and unified location.
• It serves as a core component of business intelligence, enabling
organizations to perform in-depth analytics, generate insights, and
make informed decisions by storing integrated, time-variant, and non-
volatile data.
Data Warehousing
• Data warehousing is the process of collecting, integrating,
storing, and managing data from multiple sources in a central
repository.
• It enables organizations to organize large volumes of historical
data for efficient querying, analysis, and reporting.
• The main goal of data warehousing is to support decision-
making by providing clean, consistent, and timely access to
data.
• It ensures fast data retrieval even when working with massive
datasets.
Data Warehouse Architecture
• [Link]
%20data%20warehouse%20is%20a,and%20thousands%20of%20users
%20concurrently.
Data warehouse
• A data warehouse is a central repository of information that
can be analyzed to make more informed decisions.
• Data flows into a data warehouse from transactional systems,
relational databases, and other sources, typically on a regular
cadence.
• Business analysts, data engineers, data scientists, and decision
makers access the data through business intelligence (BI)
tools, SQL clients, and other analytics applications.
Data warehouse…
• Data and analytics have become indispensable to businesses
to stay competitive. Business users rely on reports,
dashboards, and analytics tools to extract insights from their
data, monitor business performance, and support decision
making.
• Data warehouses power these reports, dashboards, and
analytics tools by storing data efficiently to minimize the
input and output (I/O) of data and deliver query results
quickly to hundreds and thousands of users concurrently.
How is a data warehouse
architected?
• A data warehouse architecture is made up of tiers.
• The top tier is the front-end client that presents results
through reporting, analysis, and data mining tools.
• The middle tier consists of the analytics engine that is used
to access and analyze the data.
• The bottom tier of the architecture is the database server,
where data is loaded and stored.
Data warehouse architecture
How is a data warehouse
architected?
• Data is stored in two different types of ways:
1) Data that is accessed frequently is stored in very fast storage (like
SSD drives) and
2) Data that is infrequently accessed is stored in a cheap object store,
like Amazon S3.
• The data warehouse will automatically make sure that frequently
accessed data is moved into the “fast” storage so query speed is
optimized.
Data warehouse architecture
[Link]
Modern ELT process
How does a data warehouse
work?
• A data warehouse may contain multiple databases.
• Within each database, data is organized into tables and columns.
• Within each column, you can define a description of the data, such as
integer, data field, or string.
• Tables can be organized inside of schemas, which you can think of as
folders.
• When data is ingested, it is stored in various tables described by the
schema.
• Query tools use the schema to determine which data tables to access
and analyze.
[Link]

Need for Data Warehousing

• Handling Large Data Volumes: Traditional databases store limited

data (MBs to GBs), while data warehouses are built to handle huge
datasets (up to TBs), making it easier to store and analyze long-term
historical data.
• Enhanced Analytics: Databases handle transactions; data warehouses
are optimized for complex analysis and historical insights.
• Centralized Data Storage: A data warehouse combines data from
multiple sources, giving a single, unified view for better decision-
making.
Need for Data Warehousing…
• Trend Analysis: By storing historical data, a data warehouse allows
businesses to analyze trends over time, enabling them to make
strategic decisions based on past performance and predict future
outcomes.
• Business Intelligence Support: Data warehouses work with BI tools to
give quick access to insights, helping in data-driven decisions and
improving efficiency.
Components of Data Warehouse
• Data Sources: These are the various operational systems,
databases, and external data feeds that provide raw data to be
stored in the warehouse.
• ETL (Extract, Transform, Load) Process: The ETL process is
responsible for extracting data from different sources,
transforming it into a suitable format, and loading it into the
data warehouse.
• Data Warehouse Database: This is the central repository
where cleaned and transformed data is stored. It is typically
organized in a multidimensional format for efficient querying
and reporting.
Components of Data
Warehouse…
• Metadata: Metadata describes the structure, source,
and usage of data within the warehouse, making it
easier for users and systems to understand and work
with the data.
• Data Marts: These are smaller, more focused data
repositories derived from the data warehouse,
designed to meet the needs of specific business
departments or functions.
Components of Data
Warehouse…
• OLAP (Online Analytical Processing) Tools: OLAP tools
allow users to analyze data in multiple dimensions,
providing deeper insights and supporting complex
analytical queries.
• End-User Access Tools: These are reporting and
analysis tools, such as dashboards or Business
Intelligence (BI) tools, that enable business users to
query the data warehouse and generate reports.
Characteristics of Data Warehousing
• Centralized Data Storage: Combines data from various sources into one
place for a complete view.
• Query & Analysis: Supports fast and flexible data analysis for better
decision-making.
• Data Transformation: Cleans and formats data for consistency and
quality.
• Data Mining: Finds hidden patterns to discover insights and predict
trends.
• Data Security: Protects data with encryption, access control, and
backups.
Types of Data Warehouses
• Enterprise Data Warehouse (EDW): A centralized warehouse
that stores data from across the organization for analysis and
reporting.
• Operational Data Store (ODS): Stores real-time operational
data used for day-to-day operations, not for deep analytics.
• Data Mart: A subset of a data warehouse, focusing on a
specific business area or department.
• Cloud Data Warehouse: A data warehouse hosted in the cloud,
offering scalability and flexibility.
Types of Data Warehouses…
• Big Data Warehouse: Designed to store vast amounts of
unstructured and structured data for big data analysis.
• Virtual Data Warehouse: Provides access to data from
multiple sources without physically storing it.
• Hybrid Data Warehouse: Combines on-premises and cloud-
based storage to offer flexibility.
• Real-time Data Warehouse: Designed to handle real-time
data streaming and analysis for immediate insights.
 Database Data Warehouse

A common Database is based on operational or

transactional processing. Each operation is an indivisible A data Warehouse is based on analytical processing.
transaction.

A Data Warehouse maintains historical data over time.

Generally, a Database stores current and up-to-date data Historical data is the data kept over years and can used for
which is used for daily operations. trend analysis, make future predictions and decision
support.

A Data Warehouse is integrated generally at the

organization level, by combining data from different
A database is generally application specific.
databases.
Example: A database stores related data, such as the
Example: A data warehouse integrates the data from one or
student details in a school.
more databases , so that analysis can be done to get
results , such as the best performing school in a city.

Constructing a Database is not so expensive. Constructing a Data Warehouse can be expensive.

Multidimensional Data Model
MultiDimensional Data Model
[Link]

• A Multidimensional Data Model is defined

as a model that allows data to be organized
and viewed in multiple dimensions, such as
product, time and location
Features of Multi-dimensional
data model
• It allows users to ask analytical questions associated with multiple
dimensions which help us know market or business trends.
• OLAP (online analytical processing) and data warehousing uses multi
dimensional databases.
• It represents data in the form of data cubes. Data cubes allow to model
and view the data from many dimensions and perspectives.
• It is defined by dimensions and facts and is represented by a fact table.
Facts are numerical measures and fact tables contain measures of the
related dimensional tables or names of the facts.
Working on a Multidimensional
Data Model
Stage 1: Assembling data from the client
Stage 2: Grouping different segments of the system
Stage 3: Noticing the different proportions
Stage 4: Preparing the actual-time factors and their respective
qualities
Stage 5: Finding the actuality of factors which are listed
previously and their qualities
Stage 6: Building the Schema to place the data, with respect to
the information collected from the steps above
Working on a Multidimensional
Data Model
Stage 1: Assembling data from the client
• In first stage, a Multi Dimensional Data Model collects correct data from
the client. Mostly, software professionals provide simplicity to the client
about the range of data which can be gained with the selected
technology and collect the complete data in detail.

Stage 2: Grouping different segments of the system

• In the second stage, the Multi Dimensional Data Model recognizes and
classifies all the data to the respective section they belong to and also
builds it problem-free to apply step by step.
 Working on a Multidimensional Data Model...

Stage 3: Noticing the different proportions :

• In the third stage, it is the basis on which the design of the system is
based. In this stage, the main factors are recognized according to the
user's point of view. These factors are also known as "Dimensions".

Stage 4: Preparing the actual-time factors and their respective

qualities :
• In the fourth stage, the factors which are recognized in the previous
step are used further for identifying the related qualities. These
qualities are also known as "attributes" in the database.
Working on a Multidimensional Data Model...

Stage 5: Finding the actuality of factors which are listed previously

and their qualities :
• In the fifth stage, A Multi Dimensional Data Model separates and
differentiates the actuality from the factors which are collected by it.
These actually play a significant role in the arrangement of a Multi
Dimensional Data Model.

Stage 6: Building the Schema to place the data, with respect to the
information collected from the steps above :
• In the sixth stage, on the basis of the data which was collected
previously, a Schema is built.
Let us take the example of the data of a factory
which sells products per quarter in Bangalore. The
data is represented in the table given below :