UNIT II

Data mining ,Primitives,Languages and System Architecture: Data mining-

Primitives-Data Mining Query Language,Architecture of Data mining
Systems,Concept Description,Characterization and Comparison:Concept
Description,Data Generalization and Summarization,Analytical
characterization,Mining Class Comparison-Statistical Measures.

[Link] are the factors to be considered while selecting the sample in

statistics?

What population parameters we want to estimate

Cost of sampling (importance of information)

How much is already known

Spread (variability) of the population

2. Name some specific application oriented databases.

Spatial databases,,Time-series databases, and Text databases and multimedia

databases

3. Define concept description.

Concept description is a data mining task that generates concise and summarative
descriptions of data collections or classes . The goal of concept description is to
provide an overall picture of a class of data or distinguish it from a set of comparative
classes . It is used to identify patterns and extract useful information from large sets of
data

4. What is data characterization?

Data characterization is a technique used in data mining to transform raw data into
useful information . It involves generating condensed representations of the

Department of Computer Science,SAASC,Trichy-5 1

information content hidden within data. Data characterization can be used to
explore the data and identify patterns. It is used in machine learning algorithms and
data mining .

5. What is concept hierarchy?

It reduce the data by collecting and replacing low level concepts (such as
numeric values for the attribute age) by higher level concepts (such as young,
middle-aged, or senior).

6. What is the use of Ad hoc query tool?

An ad hoc query tool is a software tool that allows users to perform ad hoc
queries on a database or data warehouse . Ad hoc queries are user-defined searches
that are used to gain insight into a given data set without requiring any predefined
dashboards, drill paths, and coding Ad hoc query tools are used to explore the data
and identify patterns . They are used in machine learning algorithms and data
mining. Some examples of ad hoc query tools include Qlikview, Spotfire, Crystal
Reports.

7. Define DBMiner

DBMiner is a data mining system that allows users to mine multiple-level

knowledge in large relational databases . It provides a wide range of data mining
functions, including generalization, characterization, association, classification,
and prediction . DBMiner is used to mine three typical kinds of knowledge:
associations, classifications, and clusters.. It can be used to explore the data and
identify patterns. DBMiner is used in machine learning algorithms and data
mining.

Department of Computer Science,SAASC,Trichy-5 2

8. List out the tasks of DBMiner.

The following are the tasks that can be performed using DBMiner

Mining associations from a data cube.

Mining classifications from a data cube.

Mining clusters from a data cube

9. What is task relevant data and write the syntax.

Task-relevant data is a subset of data that is relevant to a specific data mining

task. It is used to extract knowledge from a data cube created by the OLAP
Manager .The syntax for specifying task-relevant data using the Data Mining
Query Language (DMQL) is as follows
use database database_name or use data warehouse data_warehouse_name in
relevance to att_or_dim_list from relation(s)/cube(s) [where condition] order
by order_list group by grouping_list
10. Define schema

A database schema is a logical representation of data that shows how the data in
a database should be stored logically . It defines the structure of a database and
includes information such as table names, fields, data types, and the relationships
between these entities . The schema does not represent the data type of the
attributes

11. What is dimensionality reduction?

Dimensionality reduction is a technique used to reduce the number of features

in a dataset while retaining as much of the important information as possible. In
other words, it is a process of transforming high-dimensional data into a lower-
dimensional space that still preserves the essence of the original data.
Department of Computer Science,SAASC,Trichy-5 3
12. What are the types of concept hierarchies?

[Link] Hierarchy [Link] grouping Hierarchy [Link]-derived Hierarchy

[Link] based Hierarchy

13. List any two important issues that have to be addressed during data
integration.

Data integration is the process of combining data from multiple sources into a
single, unified view.

Schema integration: Schema integration is the process of combining the schemas

of multiple data sources into a single, unified schema . This can be challenging
because different data sources may use different naming conventions, data types,
and structures. Schema integration involves identifying the common attributes and
resolving any inconsistencies or conflicts that may exist between the different
sources

Data quality: Data quality refers to the accuracy, completeness, and consistency of
the data . When integrating data from multiple sources, it is important to ensure
that the data is of high quality. This can involve cleaning and transforming the
data, as well as resolving any inconsistencies or conflicts that may exist between
the different sources . Poor data quality can lead to inaccurate results and poor
decision making.

14. List the strategies for data reduction.

[Link] Sampling [Link] reduction [Link] compression [Link]

discretization [Link] selection.

15. List any two primitives for specifying a data mining task.

[Link] of task relevant data to be mined [Link] of knowledge to be mined

Department of Computer Science,SAASC,Trichy-5 4
16. What is data generalization?

Data generalization is a technique used in data mining to summarize data by

replacing relatively low-level values with higher-level concepts . It is a form of
descriptive data mining that can help to identify patterns and trends in the data.
There are two basic approaches to data generalization

1. Data cube approach [Link]-oriented induction

17. Show how the concept hierarchies are useful in data mining?

Concept hierarchies are a useful tool in data mining that can help to organize and
simplify large and complex datasets . In data mining, a concept hierarchy is a tree-like
structure that arranges more specific concepts (called “child” concepts) under broader
concepts (called “parent” concepts) .

Concept hierarchies are useful in data mining for several reasons:

[Link] Visualization [Link] Cleaning and preprocessing [Link] Integration

18. What is the main goal of statistics?

The main goal of statistics in data mining is to identify patterns and trends in
large datasets. Statistical methods are used to analyze, explore, and present data in
order to identify useful patterns and relationships

19. Name some advanced database systems.

[Link] oriented databases [Link] databases [Link] databases

[Link] databases [Link] databases

Department of Computer Science,SAASC,Trichy-5 5

20. What is data summarization?

Data summarization is the process of presenting a summary or report of

generated data in a comprehensible and informative manner. The goal of data
summarization is to convey trends and patterns from the dataset in a simplified
manner
Part B
1 .Explain data mining task primitives in detail.
Data mining task primitives refer to the basic operations or building blocks used in
the process of extracting valuable patterns and knowledge from large datasets.
There are several key primitives commonly used in data mining.

Classification

Description: Assigning predefined categories or labels to data instances based on

their characteristics.

Example: Predicting whether an email is spam or not based on features like sender,
subject, and content.

Regression

Description: Estimating a continuous numeric value based on input variables.

Example: Predicting house prices based on features like location, size, and number
of bedrooms.

Clustering

Description: Grouping similar data instances into clusters based on their inherent
similarities.

Department of Computer Science,SAASC,Trichy-5 6

Example: Grouping customers based on purchasing behavior to identify market
segments.

Association Rule Mining

Description: Discovering relationships or associations among variables in large

datasets.

Example: Finding associations like "customers who buy product A also tend to buy
product B."

Anomaly Detection

Description: Identifying data instances that deviate significantly from the expected
patterns or norms.

Example: Detecting fraudulent transactions in a credit card dataset based on

unusual spending patterns.

Sequential Pattern Mining

Description: Discovering patterns in sequences or time-series data.

Example: Identifying frequent sequences of events in a user's clickstream data on a

website.

[Link] an example for DMQL queries by taking all data mining primitives
and explain in detail.
The Data Mining Query Language (DMQL) was proposed by Han, Fu, Wang, et
al. for the DBMiner data mining system. The Data Mining Query Language is
actually based on the Structured Query Language (SQL). Data Mining Query
Languages can be designed to support ad hoc and interactive data mining. This
DMQL provides commands for specifying primitives. The DMQL can work with
Department of Computer Science,SAASC,Trichy-5 7
databases and data warehouses as well. DMQL can be used to define data mining
tasks.

Syntax for Task-Relevant Data Specification

use database database_name

or

use data warehouse data_warehouse_name

in relevance to att_or_dim_list
from relation(s)/cube(s) [where condition]
order by order_list
group by grouping_list

DMQL (Data Mining Query Language) is a query language designed for data
mining tasks, allowing users to express complex data mining operations.

Example : Customer Segmentation

Imagine we have a dataset of customer information for an e-commerce platform.

The dataset includes features such as customer age, purchase history, location, and
the amount spent. Our goal is to perform several data mining tasks using DMQL.

Classification

Query: Classify customers into "High Value" and "Low Value" based on the
amount spent.

SELECT CustomerID, Age, Location,

CLASSIFY(AmountSpent, 'High Value', 'Low Value') AS CustomerSegment

FROM CustomerData;
Department of Computer Science,SAASC,Trichy-5 8
Regression

Query: Predict the expected amount spent by a customer based on their age and
purchase history.

SELECT CustomerID, Age, PurchaseHistory,

PREDICT(AmountSpent) AS PredictedAmountSpent

FROM CustomerData;

Clustering

Query: Group customers into clusters based on age and purchase behavior.

SELECT CustomerID, Age, PurchaseHistory,

CLUSTER(2) AS CustomerCluster

FROM CustomerData;

Association Rule Mining

Query: Find associations between products frequently purchased together.

SELECT Products,

ASSOCIATION_RULES(MIN_SUPPORT=0.1, MIN_CONFIDENCE=0.5)
AS Rules

FROM TransactionData;

Anomaly Detection

Query: Identify customers with unusually high purchase amounts.

Department of Computer Science,SAASC,Trichy-5 9

 SELECT CustomerID, AmountSpent,

DETECT_ANOMALIES(AmountSpent) AS AnomalyScore

FROM CustomerData;

Sequential Pattern Mining

Query: Discover sequences of events in the customer journey.

SELECT CustomerID, EventSequence,

SEQUENTIAL_PATTERN_MINING(MIN_SUPPORT=0.2) AS Patterns

FROM CustomerJourneyData;

[Link] the significance of analytical characterization. How it is performed?

Analytical characterization plays a crucial role in data mining by helping us

identify irrelevant or weakly relevant attributes.

Significance of Analytical Characterization

o Attribute Relevance Analysis: Analytical characterization focuses on assessing

the relevance of attributes in a dataset. It helps us determine which attributes
contribute significantly to class characterization or comparison.
o Efficient Data Preparation: By identifying irrelevant attributes, we can exclude
them during data preparation. This streamlines the mining process and improves
efficiency.
o Handling Complex Objects: Analytical characterization addresses limitations in
handling complex objects within OLAP (Online Analytical Processing) tools.
o Explicit Generalization: Since automated generalization is lacking, users
explicitly specify which attributes are relevant and need inclusion in class
characterization.

Department of Computer Science,SAASC,Trichy-5 10

Steps in Analytical Characterization

Data Collection: Gather data for the target class and its contrasting class.

Preliminary Relevance Analysis

Select a set of dimensions and attributes.

Apply a relevance measure (e.g., Gini index, χ² contingency table statistics, Gain
ratio) to assess attribute relevance.

Obtain the candidate relation for the mining task using Attribute Oriented
Induction (AOI).

Remove Irrelevant Attributes:

Conduct relevance analysis to eliminate weakly relevant or irrelevant attributes.

Generate Concepts:

Perform Attribute Oriented Induction (AOI), an algorithm for data summarization.

Address potential over-generalization issues.

Quantitative Relevance Measures:

Use measures to quantify the classifying power of attributes within the dataset.

Example
Suppose we have a dataset containing information about customers, including
attributes like age, income, and purchase history.
We want to characterize customer segments based on their likelihood to make
high-value purchases.
Analytical characterization helps us identify which attributes (e.g., income,
purchase frequency) are most relevant for this task.
By excluding less relevant attributes (e.g., hair color), we create a more focused
dataset for mining.
Department of Computer Science,SAASC,Trichy-5 11
 4. Explain about data generalization based on characterization in detail.

What is Data Generalization?

Data generalization aims to abstract a large set of task-relevant data from a low
conceptual level to higher ones.

It involves transforming detailed, low-level data into more generalized and concise
representations.

The goal is to capture essential features of objects within a target class while
simplifying the data.

Example

Online Retail Sales Analysis

Suppose we have a dataset containing information about online retail sales. Our
goal is to understand customer purchasing behavior and segment customers based
on their spending patterns. Here’s how data generalization and characterization can
be applied:
Data Collection

o Gather data on customer transactions, including attributes such as:

 Customer ID
 Product categories (e.g., electronics, clothing, home goods)
 Purchase amounts
 Time of purchase

Characterization

o We want to create concise summaries for different customer segments.

Department of Computer Science,SAASC,Trichy-5 12

 Relevant questions

 Which product categories are popular among high-spending customers?

 What time of day do most purchases occur?
 Are there specific customer groups (e.g., frequent buyers, occasional shoppers)?

Data Generalization Steps

a. Attribute Selection

Choose relevant attributes for characterization (e.g., product categories, purchase

amounts).

Exclude irrelevant attributes (e.g., customer name, address).

b. Attribute Hierarchies

Create hierarchies for categorical attributes (e.g., product categories).

Example hierarchy for the “Product Category” attribute

 Electronics
 Laptops
 Smartphones
 Accessories
 Clothing
 Men’s
 Women’s
 Home Goods
 Furniture
 Appliances

Department of Computer Science,SAASC,Trichy-5 13

 Data Summarization

o Apply aggregation functions (e.g., sum, average) to numeric attributes (e.g.,

purchase amounts).
o Generate characteristic rules:
 “High-spending customers prefer electronics, especially smartphones.”
 “Most purchases occur during lunchtime (12:00 PM to 2:00 PM).”

Segmentation

Use the summarized data to create customer segments

 High spenders
 Occasional buyers
 Night owls (late-night shoppers)

Benefits

o Marketing Insights: Identify which product categories drive revenue.

o Operational Efficiency: Optimize inventory management based on popular items

5) Write short notes on 1) Measuring the central tendency 2) Measuring the

dispersion of data

Measuring Central Tendency

Central tendency refers central value around which data points tend to cluster. It
provides a single value that represents the entire dataset.

Common measures of central tendency include

1. Mean: The arithmetic average of all data points.

Department of Computer Science,SAASC,Trichy-5 14

2. Median: The middle value of a sorted dataset.

3. Mode:The most frequently occurring value in the dataset.

Example

Consider a dataset of exam scores: {85, 90, 92, 88, 75}

Mean = (85 + 90 + 92 + 88 + 75) / 5 = 86

Median = 88

Mode = No mode as each value occurs only once.

Measuring Dispersion

Dispersion refers to the spread or variability of data points around the central
tendency. It quantifies how far apart the data points are from each other.

Common measures of dispersion include

1. Range: The difference between the maximum and minimum values in the
dataset.

2. Variance: The average of the squared differences from the mean.

3. Standard Deviation:The square root of the variance, indicating the average

deviation from the mean.

Example

Consider a dataset of exam scores: {85, 90, 92, 88, 75}

Range = 92 (max) - 75 (min) = 17

Department of Computer Science,SAASC,Trichy-5 15

Variance = [(85-86)^2 + (90-86)^2 + (92-86)^2 + (88-86)^2 + (75-86)^2] / 5 ≈
35.2

Standard Deviation ≈ √35.2 ≈ 5.94

6) Compare the differences between concept description in large databases

and OLAP.

Concept Description in Large Databases

Large Databases

- In large databases, the focus is primarily on storing and managing vast amounts
of structured data efficiently.

- Concept description involves understanding the structure, relationships, and

characteristics of the data stored in the database.

- It aims to define the attributes, data types, constraints, and relationships within
the database schema.

Example

Consider a large database for a retail company. The concept description would
involve defining tables for products, customers, orders, transactions, etc. Each
table would have attributes such as product ID, name, price, customer ID, order
date, quantity, etc., along with their data types and relationships (e.g., a customer
can place multiple orders).

OLAP (Online Analytical Processing)

OLAP is designed for multidimensional analysis of large datasets, enabling

complex and interactive analysis for decision-making.

Department of Computer Science,SAASC,Trichy-5 16

Concept description in OLAP involves defining dimensions, measures, hierarchies,
and cubes to facilitate analytical querying and reporting.

Example

In a retail OLAP system, the concept description would involve defining

dimensions such as time (year, quarter, month), products (category, brand),
geography (region, country), and measures such as sales revenue, quantity sold,
profit margin, etc. These dimensions and measures are organized into
multidimensional cubes, allowing users to slice, dice, drill-down, and pivot the
data for analysis (e.g., analyzing sales performance by product category over
different time periods).

7) Explain in graph displays of basic statistical class description.

In data mining, graphical displays play a crucial role in visually representing the
basic statistical class descriptions of a dataset. These displays help in
understanding the distribution, central tendency, dispersion, and relationships
among variables.

1. Histograms

Description:Histograms show the frequency distribution of a single numerical

variable by dividing it into intervals (bins) and counting the number of data points
in each bin.

Example : In a dataset of student exam scores, a histogram would display the

frequency distribution of scores across different score intervals (e.g., 0-20, 21-40,
etc.).

Department of Computer Science,SAASC,Trichy-5 17

2. Box Plots (Box-and-Whisker Plots)

Description: Box plots display the distribution of numerical data based on

quartiles, highlighting the median, interquartile range, and potential outliers.

Example: In the same dataset of student exam scores, a box plot would show the
median score, the spread of scores between quartiles, and any outliers.

3. Scatter Plots

Description: Scatter plots visualize the relationship between two numerical

variables by plotting data points on a Cartesian plane.

Example: In a dataset containing both exam scores and study hours, a scatter plot
would show how exam scores vary with different levels of study hours.

4. Bar Charts

Description: Bar charts represent categorical data by displaying bars of different

heights corresponding to the frequency or proportion of each category.

Example: In a dataset categorizing students by their grade levels (e.g., freshman,

sophomore, junior, senior), a bar chart would display the number of students in
each grade level.

5. Pie Charts

Description: Pie charts display the proportion of different categories within a

dataset as slices of a circular pie.

Example: If a dataset contains the distribution of favorite sports among students, a

pie chart would visually represent the percentage of students who prefer each sport.

Department of Computer Science,SAASC,Trichy-5 18

Example

Consider a dataset of student performance in a class, containing variables like

exam scores, study hours, and grade levels. To visually describe the basic statistics
of this dataset, we can create histograms for exam scores and study hours, box
plots for exam scores by grade levels, scatter plots to explore the relationship
between study hours and exam scores, bar charts for grade level distribution, and
pie charts for the distribution of favorite subjects among students.

8) Explain the discretization and concept hierarchy generation for numeric

data and categorical data.

Discretization

Discretization involves converting continuous numeric data into discrete intervals

or categories. It simplifies data analysis and reduces complexity by grouping
similar numeric values together.

Discretization for Numeric Data

1. Equal Width Discretization

Divides the range of numeric values into equal-width intervals.

Example: Age values (0-100) can be discretized into intervals like [0-20], [21-
40], [41-60], [61-80], [81-100].

2. Equal Frequency Discretization

Divides the data into intervals containing approximately equal numbers of data
points.

Department of Computer Science,SAASC,Trichy-5 19

 Example: Salary values can be discretized into income groups where each group
contains roughly the same number of individuals.

3. Custom Discretization

Involves defining specific intervals based on domain knowledge or business

requirements.

Example: Temperature ranges can be discretized into "cold," "mild," "warm,"

and "hot" categories.

Concept Hierarchy Generation

Concept Hierarchy Generation involves organizing categorical data into a

hierarchical structure based on their relationships or levels of abstraction. It
facilitates efficient data analysis and exploration by capturing inherent
relationships among categorical variables.

Concept Hierarchy Generation for Categorical Data

1. Simple Hierarchy

Represents a straightforward hierarchy with one level of abstraction.

Example: A simple hierarchy for educational qualification can include categories

like "High School," "Bachelor's Degree," "Master's Degree," and "Doctorate."

2. Composite Hierarchy

Consists of multiple levels of abstraction, capturing complex relationships among

categories.

Department of Computer Science,SAASC,Trichy-5 20

 Example: A composite hierarchy for geographical regions can include levels
such as "Continent" (e.g., Asia, Europe), "Country" (e.g., China, Germany),
"State/Province" (e.g., California, Bavaria), and "City" (e.g., Beijing, Munich).

Example

Consider a dataset containing information about individuals, including age,

income, education level, and occupation.

[Link] for Numeric Data

Age can be discretized into intervals such as [0-20], [21-40], [41-60], [61-80],
[81-100].

Income can be discretized into income groups like "Low," "Medium," and
"High."

2. Concept Hierarchy Generation for Categorical Data

Education level can have a hierarchy with categories like "High School,"
"Bachelor's Degree," "Master's Degree," and "Doctorate."

Occupation can have a hierarchy with categories like "Blue-collar Worker,"

"White-collar Worker," "Professional," and "Managerial."

[Link] about data summarization in detail

Data summarization is the process of reducing large datasets to smaller, more

manageable summaries that retain the essential information. The goal of data
summarization is to provide a concise representation of the data that is easy to
understand and analyze. Data summarization can be used to identify patterns,
trends, and relationships in the data, and to make predictions about future events

Department of Computer Science,SAASC,Trichy-5 21

There are several techniques for data summarization, including:

1. Sampling: Sampling is a technique for selecting a subset of the data that

represents the entire dataset. Sampling can be used to reduce the size of the
dataset while retaining the essential information. For example, if you have a
dataset of 100,000 records, you can use sampling to select a random subset
of 10,000 records that represents the entire dataset.
2. Aggregation: Aggregation is a technique for combining multiple values into
a single value. Aggregation can be used to summarize the data by calculating
the average, sum, or count of the values. For example, if you have a dataset
of sales transactions, you can use aggregation to calculate the total sales for
each product.
3. Dimensionality reduction: Dimensionality reduction is a technique for
reducing the number of variables in the dataset. Dimensionality reduction
can be used to simplify the data by removing redundant or irrelevant
variables. For example, if you have a dataset of customer information, you
can use dimensionality reduction to remove variables that are not relevant to
the analysis.
4. Clustering: Clustering is a technique for grouping similar data points
together. Clustering can be used to identify patterns in the data by grouping
similar data points together. For example, if you have a dataset of customer
purchases, you can use clustering to group customers who have similar
purchasing patterns.

Example

Suppose you have a dataset of 10,000 customer transactions. You want to

summarize the data to identify patterns in the purchasing behavior of the

Department of Computer Science,SAASC,Trichy-5 22

customers. You can use data summarization techniques to reduce the size of the
dataset and to identify the essential information. The following table shows the
results of the data summarization:

Technique Result

Sampling 1,000

Aggregation Total sales: $1,000,000

Dimensionality Removed variables: age,

reduction gender

Clustering 5 clusters

From the table, we can see that the dataset has been summarized using four
different techniques. Sampling was used to select a random subset of 1,000
transactions that represents the entire dataset. Aggregation was used to calculate
the total sales for all the transactions. Dimensionality reduction was used to
remove the age and gender variables from the dataset. Clustering was used to
group the transactions into five clusters based on the purchasing behavior of the
customers.

Department of Computer Science,SAASC,Trichy-5 23

10) Explain the tools available for ETL

ETL (Extract, Transform, Load) is a crucial process in data warehousing and

analytics, involving extracting data from various sources, transforming it into a
suitable format, and loading it into a target destination, such as a data warehouse.
Several tools are available to facilitate the ETL process efficiently. Here are some
of the commonly used ones

1. Apache Spark

Description: Apache Spark is a powerful open-source distributed computing

framework that offers ETL capabilities through its Spark SQL and DataFrame API.

Features:It provides scalable and high-performance ETL processing, supports

various data sources, and offers advanced transformations and analytics
functionalities.

Use Cases Ideal for processing large-scale datasets and real-time streaming data.

2. Apache Hadoop

Description: Apache Hadoop is an open-source framework that facilitates

distributed storage and processing of large datasets across clusters of computers.

Features: Hadoop ecosystem components like Apache Hive, Apache Pig, and
Apache Sqoop offer ETL capabilities for processing and transforming data stored
in Hadoop Distributed File System (HDFS).

Use Cases Suitable for batch processing of big data and handling complex ETL
workflows.

Department of Computer Science,SAASC,Trichy-5 24

3. Informatica PowerCenter

Description:Informatica PowerCenter is an enterprise-grade ETL tool that provides

a scalable and robust platform for data integration and management.

Features: It offers extensive connectivity options, advanced transformation

capabilities, metadata management, and scheduling features.

Use Cases: Ideal for large enterprises with complex data integration requirements
and a need for comprehensive ETL solutions.

4. Microsoft SQL Server Integration Services (SSIS)

Description: SSIS is a component of Microsoft SQL Server that enables the

creation and management of ETL workflows.

Features:It provides a visual development environment, extensive connectivity

options, built-in transformations, and scheduling capabilities.

Use Cases:Well-suited for organizations using Microsoft SQL Server as their

database platform and requiring seamless ETL integration within the Microsoft
ecosystem.

5. Amazon Web Services (AWS) Glue

Description: AWS Glue is a fully managed ETL service provided by Amazon Web
Services, designed for cloud-based data integration and preparation.

Features: It offers automatic schema discovery, job scheduling, serverless

architecture, and integration with other AWS services like S3, Redshift, and RDS.

Use Cases: Ideal for organizations leveraging AWS cloud infrastructure and
looking for a serverless, scalable ETL solution.

Department of Computer Science,SAASC,Trichy-5 25

 Part C

[Link] the architecture of a typical data mining system

A typical data mining system consists of several components that work together to
extract useful information from large datasets:

1. Data source: The data source is the actual source of data, which can be a
database, data warehouse, text files, or other documents. The data source
must be cleaned, integrated, and selected before passing it to the data mining
engine.
2. Data mining engine: The data mining engine is the core component of the
data mining system. It contains several modules for operating data mining
tasks, including association, characterization, classification, clustering,
prediction, time-series analysis, etc.
3. Data warehouse server: The data warehouse server consists of the original
data that is ready to be processed. It is responsible for retrieving the relevant
data based on data mining as per user request.
4. Pattern evaluation module: The pattern evaluation module is primarily
responsible for the measure of investigation of the pattern by using a
Department of Computer Science,SAASC,Trichy-5 26
 threshold value. It collaborates with the data mining engine to focus the
search on exciting patterns. This segment commonly employs stake
measures that cooperate with the data mining modules to focus the search
towards fascinating patterns. It might utilize a stake threshold to filter out
discovered patterns.
5. Graphical user interface: The graphical user interface is the component
that allows users to interact with the data mining system. It provides an easy-
to-use interface for users to specify data mining tasks, visualize data mining
results, and interact with the system.
6. Knowledge base: The knowledge base is the repository of knowledge that is
generated by the data mining system. It contains the discovered patterns,
models, and rules that are generated by the data mining engine.

[Link] about data mining query language.

Data Mining Query Language (DMQL) is a language that is used to perform

data mining tasks. It was proposed by Han, Fu, Wang, et al. for the DBMiner data
mining system. DMQL is based on Structured Query Language (SQL) and
provides commands for specifying primitives. It can be used to define data mining
tasks and work with databases and data warehouses

DMQL is not a standard language but a derived language that stands as a general
query language to perform data mining techniques . It is executed in DB miner
systems for collecting data from several layers of databases . DMQL is designed
based on SQL, which is a relational query language . For the given data mining
task, the corresponding datasets must be defined in the form of a data mining
request . The data miner can use the database query to retrieve the suitable datasets
before the process of data mining . The pattern evaluation module collaborates

Department of Computer Science,SAASC,Trichy-5 27

with the data mining engine to focus the search on exciting patterns .The
knowledge base is the repository of knowledge that is generated by the data mining
system

Example

SELECT * FROM customers WHERE age > 25 AND income > 50000;

This command retrieves all the customers from the database whose age is greater
than 25 and income is greater than 50000

DMQL provides commands for specifying primitives such as selection, projection,

join, aggregation, and classification . These primitives can be used to define data
mining tasks and work with databases and data warehouses . DMQL can also be
used to specify the kind of patterns to be mined, define the constraints on the
desired patterns, and post-process extracted patterns

3. Discuss the following 4 concept hierarchy types a) Schema b) Set-grouping

c) Operation derived d) rule based

In data mining, a concept hierarchy is an organization of data into a tree-

like structure, where each level of the hierarchy represents a concept that is more
general than the level below it 1. The concept of hierarchy is used to organize and
classify data in a way that makes it more understandable and easier to analyze
There are several types of concept hierarchies, including:

1. Schema Hierarchy: Schema Hierarchy is a type of concept hierarchy that is

used to organize the schema of a database in a logical and meaningful way,
grouping similar objects together. A schema hierarchy can be used to
organize different types of data, such as tables, attributes, and relationships,

Department of Computer Science,SAASC,Trichy-5 28

 in a logical and meaningful way. This can be useful in data warehousing,
where data from multiple sources needs to be integrated into a single
database . For example, a schema hierarchy can be used to organize the data
of a company’s employees based on their departments, job titles, and
locations.
2. Set-Grouping Hierarchy: Set-Grouping Hierarchy is a type of concept
hierarchy that is based on set theory, where each set in the hierarchy is
defined in terms of its membership in other sets. Set-grouping hierarchy can
be used for data cleaning, data pre-processing, and data integration. This
type of hierarchy can be used to identify and remove outliers, noise, or
inconsistencies from the data and to integrate data from multiple sources

For example, a set-grouping hierarchy can be used to group customers based

on their purchase history, demographics, and preferences.

3. Operation-Derived Hierarchy: Operation-Derived Hierarchy is a type of

concept hierarchy that is derived from the operations performed on the data.
This type of hierarchy is used to organize data based on the operations that
are performed on it, such as aggregation, summarization, or filtering. This
type of hierarchy can be useful in data analysis, where the data needs to be
organized based on the operations that are performed on it . For example, an
operation-derived hierarchy can be used to group sales data based on the
time of day, day of the week, or month of the year.
4. Rule-Based Hierarchy: Rule-Based Hierarchy is a type of concept
hierarchy that is based on the rules that are used to classify the data. This
type of hierarchy is used to organize data based on the rules that are used to
classify it, such as decision trees or association rules. This type of hierarchy
can be useful in data mining, where the data needs to be organized based on
Department of Computer Science,SAASC,Trichy-5 29
 the rules that are used to classify it .. For example, a rule-based hierarchy
can be used to group customers based on their purchasing behavior, such as
frequent buyers, occasional buyers, and one-time buyers.

4. Discuss the integration of a data mining system with a data warehouse

Data mining is the process of discovering patterns in large datasets. Data

warehousing is the process of collecting, storing, and managing data from
different sources to provide a unified view of the data. The integration of a data
mining system with a data warehouse can help organizations to extract useful
information from large datasets and make better decisions .

The integration of a data mining system with a data warehouse involves several
steps, including:

1. Data selection: The first step is to select the relevant data from the data
warehouse. This involves identifying the data that is required for the data
mining task and retrieving it from the data warehouse .
2. Data preprocessing: The second step is to preprocess the data to prepare it
for data mining. This involves cleaning the data, transforming it into a
suitable format, and selecting the appropriate attributes
3. Data mining: The third step is to apply data mining algorithms to the
preprocessed data to extract useful information. This involves using
techniques such as clustering, classification, association rule mining, and
anomaly detection to identify patterns in the data
4. Pattern evaluation: The fourth step is to evaluate the patterns that are
discovered by the data mining algorithms. This involves using measures
such as accuracy, precision, recall, and F1-score to evaluate the quality of
the patterns
Department of Computer Science,SAASC,Trichy-5 30
 5. Knowledge representation: The fifth step is to represent the discovered
patterns in a suitable format. This involves using techniques such as decision
trees, rules, and graphs to represent the patterns .
6. Deployment: The final step is to deploy the discovered patterns in the data
warehouse. This involves integrating the patterns into the data warehouse so
that they can be used for decision making

The integration of a data mining system with a data warehouse can provide several
benefits, including

1. Improved decision making: The integration of a data mining system with a

data warehouse can help organizations to extract useful information from
large datasets and make better decisions .
2. Increased efficiency: The integration of a data mining system with a data
warehouse can help organizations to automate the process of extracting
useful information from large datasets, which can increase efficiency and
reduce costs
3. Better data quality: The integration of a data mining system with a data
warehouse can help organizations to improve the quality of their data by
identifying and removing errors, inconsistencies, and redundancies .

5. Analyze the various descriptive statistical measures for data mining

Descriptive statistics is a branch of statistics that deals with the summary and
analysis of data. It provides a way to describe the characteristics of a dataset in
a meaningful way. There are several descriptive statistical measures that are
commonly used in data mining, including:

Department of Computer Science,SAASC,Trichy-5 31

 1. Measures of central tendency: Measures of central tendency describe the
center of the data set. The most common measures of central tendency are the
mean, median, and mode. The mean is the average of all the values in the
dataset. The median is the middle value in the dataset when the values are
arranged in order. The mode is the value that appears most frequently in the
dataset.

2. Measures of variability: Measures of variability describe the spread of the

data set. The most common measures of variability are the range, variance, and
standard deviation. The range is the difference between the largest and smallest
values in the dataset. The variance is the average of the squared differences
from the mean. The standard deviation is the square root of the variance.

3. Frequency distribution: Frequency distribution is a way to summarize the

distribution of values in a dataset. It shows how many times each value appears
in the dataset. Frequency distribution can be represented using a histogram, bar
chart, or frequency table.

Example

Suppose you have a dataset that contains the ages of 100 people. You want to
describe the characteristics of this dataset using descriptive statistical measures.
The following table shows the results of the analysis:

Measure Value
Mean 35.2
Median 34.5
Mode 32

Department of Computer Science,SAASC,Trichy-5 32

 Range 56
Variance 122.7
[Link] 11.1

From the table, we can see that the mean age of the people in the dataset is 35.2
years. The median age is 34.5 years, which means that half of the people in the
dataset are younger than 34.5 years and half are older. The mode age is 32
years, which means that this is the most common age in the dataset. The range
of ages in the dataset is 56 years, which means that the oldest person in the
dataset is 56 years older than the youngest person. The variance of the ages is
122.7, which means that the ages are spread out over a wide range. The
standard deviation of the ages is 11.1, which means that the ages are relatively
close to the mean.

Department of Computer Science,SAASC,Trichy-5 33