CoSc4152- Elective II- Int.

to Data mining and Warehousing

CHAPTER 3: DATA MINING

3.1. Introduction
Data mining refers to extracting or mining" knowledge from large amounts of data.
There are many other terms related to data mining, such as knowledge mining,
knowledge extraction, data/pattern analysis, data archaeology, and data dredging. Many
people treat data mining as a synonym for another popularly used term, Knowledge
Discovery in Databases", or KDD.
The major reason that data mining has attracted a great deal of attention in
information industry in recent years is due to the wide availability of huge amounts of
data and the imminent need for turning such data into useful information and
knowledge. The information and knowledge gained can be used for applications
ranging from business management, production control, and market analysis, to
engineering design and science exploration.

3.2. Evolution of Database Technology

3.3. Steps in KDD (Knowledge discovery in Database)

Knowledge discovery as a process is depicted in following figure and consists of an
iterative sequence of the following steps:

Wachemo University Chapter – III Page 1 of 9

 CoSc4152- Elective II- Int. to Data mining and Warehousing

1. Data Cleaning: to remove noise or irrelevant data

2. Data Integration: where multiple data sources may be combined
3. Data Selection: where data relevant to the analysis task are retrieved from the
database
4. Data Transformation: where data are transformed or consolidated into
forms appropriate for mining by performing summary or aggregation
operations
5. Data Mining:an essential process where intelligent methods are applied
in order to extract data patterns
6. Pattern Evaluation: to identify the truly interesting patterns representing
knowledge based on some interestingness measures.
7. Knowledge Presentation: where visualization and knowledge
representation techniques are used to present the mined knowledge to the user.

Wachemo University Chapter – III Page 2 of 9

 CoSc4152- Elective II- Int. to Data mining and Warehousing

3.4. Data mining functionalities

Data mining functionalities are used to specify the kind of patterns to be found in
data mining tasks. In general, data mining tasks can be classified into two categories:
1. Descriptive
2. predictive

Predictive mining tasks perform inference on the current data in order to make
predictions.

Classification:
 It predicts categorical class labels
 It classifies data (constructs a model) based on the training set and the values (class
labels) in a classifying attribute and uses it in classifying new data
 Typical Applications
o credit approval
o target marketing

Wachemo University Chapter – III Page 3 of 9

 CoSc4152- Elective II- Int. to Data mining and Warehousing

o medical diagnosis
o treatment effectiveness analysis
Classification can be defined as the process of finding a model (or function) that
describes and distinguishes data classes or concepts, for the purpose of being able to use
the model to predict the class of objects whose class label is unknown. The derived
model is based on the analysis of a set of training data (i.e., data objects whose class
label is known).

Prediction:
Find some missing or unavailable data values rather than class labels referred to as
prediction. Although prediction may refer to both data value prediction and class label
prediction, it is usually confined to data value prediction and thus is distinct from
classification. Prediction also encompasses the identification of distribution trends based
on the available data.

Descriptive mining tasks characterize the general properties of the data in the database.

Clustering analysis
Clustering analyzes data objects without consulting a known class label. The objects
are clustered or grouped based on the principle of maximizing the intraclass
similarity and minimizing the interclass similarity. Each cluster that is formed can be
viewed as a class of objects.

Association:
It is the discovery of association rules showing attribute-value conditions that occur

association rules like major (X, “computing science””) ⇒ owns (X, “personal
frequently together in a given set of data. For example, a data mining system may find

computer”) [support = 12%, confidence = 98%] where X is a variable representing a

student. The rule indicates that of the students under study, 12% (support) major in
computing science and own a personal computer .There is a 98% probability
(confidence, or certainty) that a student in this group owns a personal computer.

3.5. Five primitives for specifying a data mining task

1. Task -relevant data: This primitive specifies the data upon which mining is to
be performed. It involves specifying the database and tables or data warehouse
containing the relevant data, conditions for selecting the relevant data, the
relevant attributes or dimensions for exploration, and instructions regarding the
ordering or grouping of the data retrieved.
2. Knowledge type to be mined: This primitive specifies the specific data
mining function to be performed, such as characterization, discrimination,
association, classification, clustering, or evolution analysis. As well, the user
can be more specific and provide pattern templates that all discovered patterns
must match. These templates or meta patterns (also called meta rules or
meta queries), can be used to guide the discovery process.
3. Background knowledge: This primitive allows users to specify knowledge they
have about the domain to be mined. Such knowledge can be used to guide
the knowledge discovery process and evaluate the patterns that are found.
Of the several kinds of background knowledge, this chapter focuses on
concept hierarchies.

Wachemo University Chapter – III Page 4 of 9

 CoSc4152- Elective II- Int. to Data mining and Warehousing

4. Pattern interestingness measure: This primitive allows users to specify

functions that are used to separate uninteresting patterns from knowledge and
may be used to guide the mining process, as well as to evaluate the discovered
patterns. This allows the user to confine the number of uninteresting patterns
returned by the process, as a data mining process may generate a large number of
patterns. Interestingness measures can be specified for such pattern
characteristics as simplicity, certainty, utility and novelty.
5. Visualization of discovered patterns: This primitive refers to the form in
which discovered patterns are to be displayed. In order for data mining to be
effective in conveying knowledge to users, data mining systems should be able
to display the discovered patterns in multiple forms such as rules, tables,
cross tabs (cross-tabulations), pie or bar charts, decision trees, cubes or other
visual representations.

3.6. Data pre-processing

Data pre-processing describes any type of processing performed on raw data to prepare it
for another processing procedure. Commonly used as a preliminary data mining
practice, data pre-processing transforms the data into a format that will be more easily
and effectively processed for the purpose of the user.

Why Data Pre-processing?

Data in the real world is dirty. It can be in incomplete, noisy and inconsistent from.
These data needs to be pre-processed in order to help improve the quality of the data, and
quality of the mining results.
 If no quality data, then no quality mining results. The quality decision is always
based on the quality data.
 If there is much irrelevant and redundant information present or noisy and
unreliable data, then knowledge discovery during the training phase is more difficult

Incomplete data: lacking attribute values, lacking certain attributes of interest, or

containing only aggregate data. e.g., occupation= “ ”.
 Incomplete data may come from
o “Not applicable” data value when collected
o Different considerations between the time when the data was collected
and when it is analysed.
o Human/hardware/software problems

Noisy data: containing errors or outliers data. e.g., Salary= “ - 10”

 Noisy data (incorrect values) may come from
o Faulty data collection by instruments
o Human or computer error at data entry
o Errors in data transmission

 Inconsistent data: containing discrepancies in codes or names. e.g., Age=“42”

Birthday=“03/07/1997”
 Inconsistent data may come from
 Different data sources
 Functional dependency violation (e.g., modify some linked data)

Major Tasks in Data Pre-processing

Wachemo University Chapter – III Page 5 of 9

 CoSc4152- Elective II- Int. to Data mining and Warehousing

Major Tasks in Data Pre-processing

 Data cleaning
o Fill in missing values, smooth noisy data, identify or remove outliers, and
resolve inconsistencies
 Data integration
o Integration of multiple databases, data cubes, or files
 Data transformation
o Normalization and aggregation
 Data reduction
o Obtains reduced representation in volume but produces the same or
similar analytical results
 Data discretization
o Part of data reduction but with particular importance, especially for
numerical data
Data cleaning:
Data cleaning routines attempt to fill in missing values, smooth out noise while
identifying outliers, and correct inconsistencies in the data.

The various methods for handling the problem of missing values in data tuples include:

(a) Ignoring the tuple : This is usually done when the class label is missing
(assuming the mining task involves classification or description). This method is not
very effective unless the tuple contains several attributes with missing values. It is
especially poor when the percentage of missing values per attribute varies considerably.
(b) Manually filling in the missing value: In general, this approach is time-consuming
and may not be a reasonable task for large data sets with many missing values, especially
when the value to be filled in is not easily determined.
(c) Using a global constant to fill in the missing value: Replace all missing attribute
values by the same constant, such as a label like “Unknown,” or −∞. If missing values

Wachemo University Chapter – III Page 6 of 9

 CoSc4152- Elective II- Int. to Data mining and Warehousing

are replaced by, say, “Unknown,” then the mining program may mistakenly think
that they form an interesting concept, since they all have a value in common —
that of “Unknown.” Hence, although this method is s imple, it is not recommended.
(d) Using the attribute mean for quantitative (numeric) values or attribute
mode for categorical (nominal) values, for all samples belonging to the same
class as the given tuple: For example, if classifying customers according to
credit risk, replace the missing value with the average income value for customers
in the same credit risk category as that of the given tuple.
(e) Using the most probable value to fill in the missing value : This may be
determined with regression, inference-based tools using Bayesian formalism, or decision
tree induction. For example, using the other customer attributes in your data set, you
may construct a decision tree to predict the missing values for income.

Noisy data:

Noise is a random error or variance in a measured variable. Data smoothing tech

is used for removing such noisy data.

Several Data smoothing techniques:

1 Binning methods: Binning methods smooth a sorted data value by consulting
the neighbourhood", or values around it. The sorted values are distributed into a
number of 'buckets', or bins. Because binning methods consult the neighborhood of
values, they perform local smoothing.
In this technique,
1. The data for first sorted
2. Then the sorted list partitioned into equi-depth of bins.
3. Then one can smooth by bin means, smooth by bin median, smooth by bin boundaries,
etc.
a. Smoothing by bin means : Each value in the bin is replaced by the mean value of
the bin.
b. Smoothing by bin medians : Each value in the bin is replaced by the bin median.
c. Smoothing by boundaries : The min and max values of a bin are identified
as the bin boundaries. Each bin value is replaced by the closest boundary value.
Example: Binning Methods for Data Smoothing
o Sorted data for price (in dollars): 4, 8, 9, 15, 21, 21, 24, 25, 26, 28, 29, 34
o Partition into (equi-depth) bins(equi depth of 3 since each bin contains three values):
- Bin 1 : 4, 8, 9, 15
- Bin 2 : 21, 21, 24, 25
- Bin 3 : 26, 28, 29, 34
o Smoothing by bin means:
- Bin 1 : 9, 9, 9, 9
- Bin 2 : 23, 23, 23, 23
- Bin 3 : 29, 29, 29, 29
o Smoothing by bin boundaries:
- Bin 1 : 4, 4, 4, 15
- Bin 2 : 21, 21, 25, 25
- Bin 3 : 26, 26, 26, 34

In smoothing by bin means, each value in a bin is replaced by the mean value of the bin.

Wachemo University Chapter – III Page 7 of 9

 CoSc4152- Elective II- Int. to Data mining and Warehousing

For example, the mean of the values 4, 8, and 15 in Bin 1 is 9. Therefore, each original
value in this bin is replaced by the value 9. Similarly, smoothing by bin medians can be
employed, in which each bin value is replaced by the bin median. In smoothing by bin
boundaries, the minimum and maximum values in a given bin are identified as the
bin boundaries. Each bin value is then replaced by the closest boundary value.

Suppose that the data for analysis include the attribute age. The age values for the data
tuples are (in increasing order): 13, 15, 16, 16, 19, 20, 20, 21, 22, 22, 25, 25, 25, 25, 30,
33, 33, 35, 35, 35, 35, 36, 40, 45, 46, 52, 70.
(a) Use smoothing by bin means to smooth the above data, using a bin depth of 3.
Illustrate your steps. Comment on the effect of this technique for the given data.

The following steps are required to smooth the above data using smoothing by bin means
with a bin depth of 3.
• Step 1: Sort the data. (This step is not required here as the data are already sorted.)

• Step 2: Partition the data into equidepth bins of depth 3.

Bin 1: 13, 15, 16 Bin 2: 16, 19, 20 Bin 3: 20, 21, 22
Bin 4: 22, 25, 25 Bin 5: 25, 25, 30 Bin 6: 33, 33, 35
Bin 7: 35, 35, 35 Bin 8: 36, 40, 45 Bin 9: 46, 52, 70

• Step 3: Calculate the arithmetic mean of each bin.

• Step 4: Replace each of the values in each bin by the arithmetic mean calculated for the
bin.
Bin 1: 14, 14, 14 Bin 2: 18, 18, 18 Bin 3: 21, 21, 21
Bin 4: 24, 24, 24 Bin 5: 26, 26, 26 Bin 6: 33, 33, 33
Bin 7: 35, 35, 35 Bin 8: 40, 40, 40 Bin 9: 56, 56, 56
2 Clustering: Outliers in the data may be detected by clustering, where similar values
are organized into groups, or „clusters‟. Values that fall outside of the set of clusters
may be considered outliers.

3 Regression : smooth by fitting the data into regression functions.

 Linear regression involves finding the best of line to fit two variables, so that
one variable can be used to predict the other.

Wachemo University Chapter – III Page 8 of 9

 CoSc4152- Elective II- Int. to Data mining and Warehousing

 Multiple linear regression is an extension of linear regression, where more than

two variables are involved and the data are fit to a multidimensional surface.

Using regression to find a mathematical equation to fit the data helps smooth out the
noise.

Field overloading: is a kind of source of errors that typically occurs when

developers compress new attribute definitions into unused portions of already defined
attributes.

Unique rule is a rule says that each value of the given attribute must be different from
all other values of that attribute

Consecutive rule is a rule says that there can be no missing values between the lowest
and highest values of the attribute and that all values must also be unique.

Null rule specifies the use of blanks, question marks, special characters or other strings
that may indicate the null condition and how such values should be handled.

Wachemo University Chapter – III Page 9 of 9