‫‪Data Mining‬‬

‫تعدين البيانات‬
‫محاضرة رقم (‪)3‬‬
 Getting to Know Your Data
• Data Objects and Attribute Types

• Basic Statistical Descriptions of Data

• Measuring Data Similarity and Dissimilarity

2
 Getting to Know Your Data
• Data Objects and Attribute Types

• Basic Statistical Descriptions of Data

• Measuring Data Similarity and Dissimilarity

3
 Data-Related Issues
for Successful Data Mining
Type of Data:
– Data sets differ in a number of ways.
– Type of data determines which techniques can be used to analyze the data.
Quality of Data:
– Data is often far from perfect.
– Improving data quality improves the quality of the resulting analysis.
Preprocessing Steps to Make Data More Suitable for Data Mining:
– Raw data must be processed in order to make it suitable for analysis.
• Improve data quality,
• Modify data so that it better fits a specified data mining technique.
Analyzing Data in Terms of its Relationships:
– find relationships among data objects and then perform remaining analysis
using these
relationships rather than data objects themselves.
– There are many similarity or distance measures, and the proper choice
depends on the type of data and application. 4
 What is Data?

• Data sets are made up of data objects.

• A data object represents an entity.

– Also called sample, example, instance, data point, object, tuple.

• Data objects are described by attributes.

• An attribute is a property or characteristic of a data object.

– Examples: eye color of a person, temperature, etc.

– Attribute is also known as variable, field, characteristic, or feature

• A collection of attributes describe an object.

• Attribute values are numbers or symbols assigned to an attribute.

5
A Data Object

• database rows ➔ data objects

• database columns ➔ attributes

6
 Attributes
• Attribute (or dimensions, features, variables): a data field, representing a
characteristic or feature of a data object.
– E.g., customer _ID, name, address
• Attribute values are numbers or symbols assigned to an attribute

• Distinction between attributes and attribute values

– Same attribute can be mapped to different attribute values
• Example: height can be measured in feet or meters
– Different attributes can be mapped to the same set of values
• Example: Attribute values for ID and age are integers
• But properties of attribute values can be different; ID has no limit but age has a
maximum and minimum value

7
 Attribute Types
Four main types of attributes
Nominal: Categorical (Qualitative)
– categories, states, or “names of things”
• Hair color, marital status, occupation, ID numbers, zip codes
– An important nominal attribute: Binary
• Nominal attribute with only 2 states (0 and 1)
Ordinal: Categorical (Qualitative)
– Values have a meaningful order (ranking) but magnitude between successive
values is not known.
• Size = {small, medium, large}, grades, army rankings
Interval: Numeric (Quantitative)
– Measured on a scale of equal-sized units
– Values have order:
• temperature in C˚ or F˚, calendar dates
– No true zero-point: ratios are not meaningful
Ratio: Numeric (Quantitative)
– Inherent zero-point: ratios are meaningful
• temperature in Kelvin, length, counts, monetary quantities
8
 Attribute Types
Four main types of attributes: Nominal Attributes
▪ The values of a nominal attribute are symbols or names of things.

o Each value represents some kind of category, code, or state.

▪ Nominal attributes are also referred to as categorical attributes.

▪ The values of nominal attributes do not have any meaningful order.

o Example: The attribute marital_status can take on the values single, married,
divorced, and widowed.

▪ Because nominal attribute values do not have any meaningful order about them and
they are not quantitative.

o It makes no sense to find the mean (average) value or median (middle) value for
such an attribute.

o However, we can find the attribute’s most commonly occurring value (mode).
9
 Attribute Types
Four main types of attributes: Nominal Attributes
• A binary attribute is a special nominal attribute with only two states: 0 or 1.
• A binary attribute is symmetric if both of its states are equally valuable and carry the
same weight.
– Example: the attribute gender having the states male and female.

• A binary attribute is asymmetric if the outcomes of the states are not equally
important.
– Example: Positive and negative outcomes of a medical test for HIV.
– By convention, we code the most important outcome, which is usually the rarest one, by 1
(e.g., HIV positive) and the other by 0 (e.g., HIV negative).

10
 Attribute Types
Four main types of attributes: Ordinal Attributes
• An ordinal attribute is an attribute with possible values that have a meaningful order

or ranking among them, but the magnitude between successive values is not known.

• Example: An ordinal attribute drink_size corresponds to the size of drinks available at a

fast-food restaurant.

– This attribute has three possible values: small, medium, and large.

– The values have a meaningful sequence (which corresponds to increasing drink size);

however, we cannot tell from the values how much bigger, say, a medium is than a large.

• The central tendency of an ordinal attribute can be represented by its mode and its

median (middle value in an ordered sequence), but the mean cannot be defined.
11
 Attribute Types
Four main types of attributes: Interval Attributes
• Interval attributes are measured on a scale of equal-size units.
– We can compare and quantify the difference between values of interval attributes.

• Example: A temperature attribute is an interval attribute.

– We can quantify the difference between values. For example, a temperature of 20oC is five
degrees higher than a temperature of 15oC.

– Temperatures in Celsius do not have a true zero-point, that is, 0oC does not indicate “no
temperature.”

– Although we can compute the difference between temperature values, we cannot talk of
one temperature value as being a multiple of another.
• Without a true zero, we cannot say, for instance, that 10oC is twice as warm as 5oC . That is, we

cannot speak of the values in terms of ratios.

• The central tendency of an interval attribute can be represented by its mode, its
median (middle value in an ordered sequence), and its mean.
12
 Attribute Types
Four main types of attributes: Ratio Attributes

• A ratio attribute is a numeric attribute with an inherent zero-point.

• Example: A number_of_words attribute is a ratio attribute.

– If a measurement is ratio-scaled, we can speak of a value as being a multiple (or ratio) of
another value.
• The central tendency of a ratio attribute can be represented by its mode, its median
(middle value in an ordered sequence), and its mean.

13
 Properties of Attribute Values
• The type of an attribute depends on which of the following properties it possesses:

– Distinctness: = 
– Order: < >
– Addition: + -
– Multiplication: * /

• Nominal attribute: distinctness

• Ordinal attribute: distinctness & order
• Interval attribute: distinctness, order & addition
• Ratio attribute: all 4 properties

14
 Properties of Attribute Values
Attribute Description Examples
Type
Nominal The values of a nominal attribute are just zip codes, employee ID
different names, numbers, eye color, sex:
i.e., nominal attributes provide only enough {male, female}
information to distinguish one object from
another. (=, )
Ordinal The values of an ordinal attribute provide hardness of minerals, {good,
enough information to order objects. (<, >) better, best}, grades, street
numbers
Interval For interval attributes, the differences calendar dates, temperature
between values are meaningful, in Celsius or Fahrenheit
i.e., a unit of measurement exists. (+, - )
Ratio For ratio variables, both differences and ratios temperature in Kelvin,
are meaningful. (*, /) monetary quantities, counts,
age, mass, length,

15
 Attribute Types
Categorical (Qualitative) and Numeric (Quantitative)
• Nominal and Ordinal attributes are collectively referred to as categorical or
qualitative attributes.

– qualitative attributes, such as employee ID, lack most of the properties of numbers.

– Even if they are represented by numbers, i.e. , integers, they should be treated more
like symbols .

– Mean of values does not have any meaning.

• Interval and Ratio are collectively referred to as quantitative or numeric attributes.

– Quantitative attributes are represented by numbers and have most of the properties of
numbers .

– Note that quantitative attributes can be integer-valued or continuous.

– Numeric operations such as mean, standard deviation are meaningful

16
 Discrete vs. Continuous Attributes
▪ Discrete Attribute
– Has only a finite or countably infinite set of values
• zip codes, profession, or the set of words in a collection of documents
– Sometimes, represented as integer variables
– Note: Binary attributes are a special case of discrete attributes
– Binary attributes where only non-zero values are important are called
asymmetric binary attributes.

▪ Continuous Attribute
– Has real numbers as attribute values
• temperature, height, or weight
– Practically, real values can only be measured and represented using a finite number
of digits
– Continuous attributes are typically represented as floating-point variables
17
 Types of data sets
• Record • Ordered
– Relational records – Video data: sequence of images
– Data matrix, e.g., numerical matrix, – Temporal data: time-series
crosstabs – Sequential Data: transaction
– Document data: text documents: sequences
term-frequency vector – Genetic sequence data
– Transaction data • Spatial, image and multimedia:
• Graph and network – Spatial data: maps
– World Wide Web – Image data:
– Social or information networks – Video data:
– Molecular Structures

18
 Record Data
• Data that consists of a collection of records, each of which consists of a fixed set of
attributes

19
 Data Matrix
• If data objects have the same fixed set of numeric attributes, then the data objects can
be thought of as points in a multi-dimensional space, where each dimension
represents a distinct attribute
• Such data set can be represented by an m by n matrix, where there are m rows, one for
each object, and n columns, one for each attribute.
• A data matrix is a variation of record data, but because it consists of numeric
attributes, standard matrix operation can be applied to transform and manipulate the
data.

Projection Projection Distance Load Thickness

of x Load of y load

10.23 5.27 15.22 2.7 1.2

12.65 6.25 16.22 2.2 1.1

20
 Document (Text) Data
• Each document becomes a term vector,
– each term is a component (attribute) of the vector,
– the value of each component is the number of times the corresponding term occurs in the
document
• Convert text documents to record data by counting word frequencies (document-term
matrix).

21
 Transaction Data

• Transaction data is a special type of record data, where:

– each record (transaction) involves a set of items.

– Example: The set of products purchased by a customer constitute a transaction,

while the individual products that were purchased are the items.

22
 Transaction Data
Convert to Record Data

Requires less space

Asymmetric attributes
Requires more space

• In real-world data, the table would contain hundreds or thousands of columns,

depending on the number of items to be considered.
• The number of items bought in a transaction, say 5, is very small in comparison to
the number of columns
• Most values in this matrix are “0”. Such a matrix is called sparse matrix.
23
‫‪Data Mining‬‬

‫تعدين البيانات‬
‫محاضرة رقم (‪)4‬‬

‫‪2-24‬‬
 Getting to Know Your Data
• Data Objects and Attribute Types

• Basic Statistical Descriptions of Data

• Measuring Data Similarity and Dissimilarity

25
 Basic Statistical Descriptions of Data

• Basic statistical descriptions can be used to identify properties of the data and
highlight which data values should be treated as noise or outliers.
• For data preprocessing tasks, we want to learn about data characteristics regarding
both central tendency and dispersion of the data.

• Measures of central tendency include mean, median, mode, and midrange.

• Measures of data dispersion include quartiles, interquartile range (IQR), and

variance.
• These descriptive statistics are of great help in understanding the distribution of the
data.

26
 Measuring Central Tendency: Mean
• The most common and most effective numerical measure of the “center” of a set of
data is the arithmetic mean.
n
1
Arithmetic Mean: x =
n
i=1
xi

• Sometimes, each value xi in a set may be associated with a weight wi.

– The weights reflect the significance and importance attached to their respective values.

wx i i
Weighted Arithmetic Mean: x = i=1
n
w i
i=1

27
 Measuring Central Tendency: Mean

• Although the mean is the single most useful quantity for describing a data set, it is not
always the best way of measuring the center of the data.

– A major problem with the mean is its sensitivity to extreme (outlier) values.

– Even a small number of extreme values can corrupt the mean.

• To offset the effect caused by a small number of extreme values, we can instead use
the trimmed mean,

• Trimmed mean can be obtained after chopping off values at the high and low
extremes.

28
 Measuring Central Tendency: Median

• Another measure of the center of data is the median.

• Suppose that a given data set of N distinct values is sorted in numerical order.

– If N is odd, the median is the middle value of the ordered set;

– If N is even, the median is the average of the middle two values.

• In probability and statistics, the median generally applies to numeric data; however,
we may extend the concept to ordinal data.
– Suppose that a given data set of N values for an attribute X is sorted in increasing order.

– If N is odd, then the median is the middle value of the ordered set.

– If N is even, then the median may not be not unique.

• In this case, the median is the two middlemost values and any value in between.
29
 Measuring Central Tendency: Mode
• Another measure of central tendency is the mode.

• The mode for a set of data is the value that occurs most frequently in the set.
– It is possible for the greatest frequency to correspond to several different values, which
results in more than one mode.
– Data sets with one, two, or three modes: called unimodal, bimodal, and trimodal.
– At the other extreme, if each data value occurs only once, then there is no mode.

• Central Tendency Measures for Numerical Attributes: Mean, Median, Mode

• Central Tendency Measures for Categorical Attributes: Mode (Median?)

– Central Tendency Measures for Nominal Attributes: Mode
– Central Tendency Measures for Ordinal Attributes: Mode, Median

30
 Measuring Central Tendency -
Mean, Median, Mode
Median, mean and mode of symmetric, positively and negatively skewed data

symmetric data positively skewed data negatively skewed data

31
 Measuring Central Tendency: Example
What are central tendency measures (mean, median, mode)for the following attributes?
attr1 = {2,4,4,6,8,24}

attr2 = {2,4,7,10,12}

attr3 = {xs,s,s,s,m,m,l}

32
 Measuring Central Tendency: Example
What are central tendency measures (mean, median, mode)for the following attributes?
attr1 = {2,4,4,6,8,24}
mean = (2+4+4+6+8+24)/6 = 8 average of all values
median = (4+6)/2 = 5 avg. of two middle values
mode = 4 most frequent item
attr2 = {2,4,7,10,12}
mean = (2+4+7+10+12)/5 = 7 average of all values
median = 7 middle value
mode = any of them (no mode) all of them has same freq.
attr3 = {xs,s,s,s,m,m,l}
mean is meaningless for categorical attributes.
median = s middle value
mode = s most frequent item

33
 Measuring Dispersion of Data
• The degree to which numerical data tend to spread is called the dispersion, or
variance of the data.

The most common measures of data dispersion:

• Range: Difference between the largest and smallest values.
• Interquartile Range (IQR): range of middle 50%
– quartiles: Q1 (25th percentile), Q3 (75th percentile) IQR=Q3-Q1
– five number summary: Minimum, Q1, Median, Q3, Maximum
• Variance and Standard Deviation: (sample: s, population: σ)
– variance of N observations:
𝑛 𝑛 where  is the mean
2
1 2 2
1 value of the observations
𝜎 = ∑(𝑥 𝑖− 𝜇) 𝑠 = ∑(𝑥 𝑖− 𝜇)2
𝑛 𝑛−1
1 1

– standard deviation σ (s) is the square root of variance σ2 ( s2)

34
Measuring Dispersion of Data
 Measuring Dispersion of Data: Quantiles
• Suppose that set of observations for numeric attribute X is sorted in increasing order.

• Quantiles are points taken at regular intervals of a data distribution, dividing it into
essentially equal size consecutive sets.
– The kth q-quantile - k is an integer such that 0<k<q. There are q-1 q-quantiles.

• The 2-quantile is the data point dividing the lower and upper halves of the data
distribution. It corresponds to the median.

• The 4-quantiles are the three data points that split the data distribution into four
equal parts; each part represents one-fourth of the data distribution; they are
more commonly referred to as quartiles.

• The 100-quantiles are more commonly referred to as percentiles; they divide

the data distribution into 100 equal-sized consecutive sets.
36
Measuring Dispersion of Data: Quartiles
Measuring Dispersion of Data: Interquartile Range

38
Measuring Dispersion of Data: five-number summary

39
Measuring Dispersion of Data: five-number summary

40
Measuring Dispersion of Data: five-number summary

41
Measuring Dispersion of Data: five-number summary

42
Measuring Dispersion of Data: five-number summary

43
 Getting to Know Your Data
• Data Objects and Attribute Types

• Basic Statistical Descriptions of Data

• Measuring Data Similarity and Dissimilarity

44
 Similarity and Dissimilarity
Similarity
• The similarity between two objects is a numerical measure of the degree to which the
two objects are alike.
• Similarities are higher for pairs of objects that are more alike.
• Similarities are usually non-negative and are often between 0 (no similarity) and 1
(complete similarity).
Dissimilarity
• The dissimilarity between two objects is a numerical measure of the degree to which
the two objects are different.
• Dissimilarities are lower for more similar pairs of objects.
• The term distance is used as a synonym for dissimilarity, although the distance is
often used to refer to a special class of dissimilarities.
• Dissimilarities sometimes fall in the interval [0,1], but it is also common for them to
range from 0 to ∞.
Proximity refers to a similarity or dissimilarity
45
 Similarity/Dissimilarity for Simple Attributes
• The proximity of objects with a number of attributes is typically defined by
combining the proximities of individual attributes.
• Consider objects described by one nominal attribute.
– What would it mean for two such objects to be similar?

• p and q are the attribute values for two data objects

46
 Cosine Similarity
• A document can be represented by thousands of attributes, each recording the
frequency of a particular word or phrase in the document.

• A similarity measure for documents needs to ignores 0–0 matches like the Jaccard
measure, but also must be able to handle non-binary vectors.
• Cosine similarity is one of the most common measure of document similarity.

where x and y are two document vectors

• where • indicates vector dot product and ||x|| is the length of

vector x.

47
 Cosine Similarity
• Cosine similarity really is a measure of the (cosine of the) angle between x and y.
– If the cosine similarity is 1, the angle between x and y is 0o, and x and y are same
– If the cosine similarity is 0, then the angle between x and y is 90o, and they do not share
any terms.

• Cosine similarity can be written as

– Dividing x and y by their lengths normalizes them to have a length of 1.

– This means that cosine similarity does not take the magnitude of the two data objects into
account when computing similarity.
• Euclidean distance might be a better choice when magnitude is important.
48
 Cosine Similarity : Example
x = (3, 2, 0, 5, 0, 0, 0, 2, 0, 0)
y = (1, 0, 0, 0, 0, 0, 0, 1, 0, 2)

cos(x,y) = (x.y) / (||x|| ||y||) = 5 / (6.48*2.24) = 0.34

49
 Measuring Dispersion of Data: HW

Consider following two attribute values:

attr1: {2,3,4,5,6,7,8,9}

attr2: {1,5,9,10,11,12,18,30}

1) Give interquartile ranges of attribute values?

2) Are there any outliers in these datasets?

50
THE END..