MODULE 4

UNDERSTANDING RELATIONSHIPS

INTRODUCTION:
A critical step in understanding any dataset is examining the relationships between variables. For instance,
we may want to know whether interest rates are related to inflation, or whether education level is
associated with income. It is important to recognize that the presence of an association does not
necessarily mean that one variable causes the other. These relationships can be explored through summary
tables, data visualizations, and statistical calculations that indicate the strength and reliability of the
association. The following sections describe several methods for examining relationships between pairs of
variables, including visual techniques, summary tables, and calculated metrics. The choice of method
depends on how the variables are categorized and the scale on which they are measured. Data
visualizations play a particularly important role, as they leverage the human visual system’s ability to
detect complex patterns and trends within graphical representations of data.

VISUALIZING RELATIONSHIPS BETWEEN VARIABLES:

1. Scatterplots: Scatterplots are commonly used to determine whether a relationship exists between
two continuous variables measured on ratio or interval scales. In a scatterplot, one variable is placed
on the x-axis and the other on the y-axis, and each point on the chart represents a single observation.
The position of each point reflects the values of the two variables for that observation. As shown in
Figure 4.1, a scatterplot can display hundreds of observations on a single graph, making it an
effective tool for visually assessing patterns, trends, and potential associations between variables.

FIGURE 4.1: Example of a scatterplot where each point corresponds to an observation.

Scatterplots help reveal the type of relationship that may exist between two variables by showing
how their values change together. A positive relationship is observed when higher values of the first
variable correspond to higher values of the second variable, and lower values of the first variable
correspond to lower values of the second; in such cases, the points on the graph trend upward from
left to right. In contrast, a negative relationship occurs when higher values of the first variable are
associated with lower values of the second, and lower values of the first variable align with higher
values of the second, resulting in a downward trend from left to right. For instance, the scatterplot in
Figure 4.2 illustrates a positive relationship between petal length (cm) and sepal length (cm), as the
data points rise steadily across the chart.

FIGURE 4.2: A scatterplot showing a positive relationship.

The nature of the relationship between two variables whether it is linear or nonlinear is also an
important aspect to consider. A linear relationship occurs when changes in the second variable are
proportional to changes in the first, resulting in a straight-line pattern. In contrast, a nonlinear
relationship appears as a curve, indicating that the change in the second variable is not proportional
as the first variable changes. In Figure 4.2, the relationship between sepal length(cm) and petal
length(cm) is primarily linear, with petal length increasing proportionally as sepal length increases.
A scatterplot can also reveal observations that do not follow this pattern, such as the point marked X
in Figure 4.3. These unusual points are known as outliers, as they fall outside the general trend of
the data.
 FIGURE 4.3: Observation (marked as X) that does not follow the relationship
Scatterplots can also reveal negative relationships between variables.

FIGURE 4.4: Scatterplot showing a negative nonlinear relationship

EX: In Figure 4.4, we can see that when weight increases, the MPG value goes down, showing a
negative relationship. Sometimes, relationships between variables are not straightforward and may
include both upward and downward trends. In this example, the points form a curved pattern, meaning
the relationship is nonlinear MPG decreases as weight increases, but it does not decrease at a steady or
proportional rate.

Scatterplots can also show the lack of any relationship.

FIGURE 4.5: Scatterplot showing no relationships

In Figure 4.5 the points scattered throughout the graph indicates that there is no obvious relationship
between Alcohol and Nonflavonoid phenols in this data set.
2. Summary Tables and Charts:
A simple summary table is a useful tool for understanding the relationship between two variables when at
least one of them is discrete. It helps condense large amounts of information into an easy-to-interpret
format. For example, a national retail company that has collected product-level sales data from all its
stores may use a summary table to evaluate overall performance. By calculating and presenting the
average sales for each store, the company can quickly compare results, identify trends, and communicate
performance insights across locations.

In a summary table, a single categorical variable or a continuous variable that has been converted into
categories is used to group the observations, with each row representing one group. These tables
typically display the count or percentage of observations belonging to each category. Additional columns
may be included to summarize a second variable, but because each row represents a group rather than an
individual observation, the values in these columns must also be summary statistics. Common
descriptive statistics used in such tables include the mean, median, mode, sum, minimum, maximum,
variance, and standard deviation, all of which provide meaningful summaries of the grouped data.
 FIGURE 4.6: Example of a summary table

In Figure 4.6, the relationship between the variables class and petal width (cm) from Fisher (1936) is
illustrated. The class variable is a nominal categorical variable with three possible values“Iris-
setosa,” “Iris-versicolor,” and “Iris-virginica” which appear in the first column of the table. Each
class contains 50 observations, and each row summarizes the petal width measurements for its
corresponding group of 50 flowers. The table presents summary statistics for petal width, including
the minimum, maximum, mean, median, and standard deviation. From these summaries, it is clear
that “Iris-setosa” has the smallest petal width, with a mean of 0.2 cm. “Iris-versicolor” has a higher
mean petal width of 1.33 cm, while “Iris-virginica” shows the greatest petal width, with a mean of
2.03 cm.

It can also be useful to present this information graphically. For example, in Figure 4.7, a bar chart
is used where the x-axis displays the three nominal class categories and the y-axis shows the mean
petal width (cm) for each class. This visualization makes it easy to compare the relative mean values
across the three classes at a glance.

FIGURE 4.7: Example of a summary graph

More detailed information about the frequency distribution of the three groups can be examined using
box plots, as shown in Figure 4.8. In this figure, the x-axis represents the three flower classes, while the
y-axis displays the actual petal width values for each individual observation. The box plot shows how the
observations in each class are distributed in comparison with the others. For example, the 50 observations
in the “Iris-setosa” class have petal width values that do not overlap with either of the other two classes.
In contrast, the distributions of “Iris-versicolor” and “Iris-virginica” show some overlap at their extreme
values, indicating partial similarity in their petal width ranges.

FIGURE 4.8: Example of a multiple box plot graph.

Summary tables can also be used to explore the relationship between ordinal variables and another
variable. In the example shown in Figure 4.9, the observations are grouped into three ordered categories.
Because these categories have a natural order, it becomes possible to observe how the mean weight
changes across the MPG categories. From the table, it is evident that as the MPG category increases, the
mean weight consistently decreases, clearly illustrating the relationship between these two variables.

FIGURE 4.9: Example of summary table where the categorical variable

The same information can also be displayed using a histogram and a series of box plots. When the
categories are arranged in order along the x-axis and the data are plotted accordingly, the trend becomes
even easier to observe. As shown in Figure 4.10, this graphical arrangement provides a clearer visual
representation of how the values change across the ordered categories.
FIGURE 4.10: Graph of summary data for an ordinal variable against a continuous (weight).

In many cases, a binary variable is used to represent data that has only two possible outcomes,
typically coded as 0 and 1. For example, 0 may indicate that a patient has a particular infection,
while 1 indicates that the patient does not. To illustrate, consider a new blood test being evaluated
for its ability to predict whether a patient has a specific type of infection. The test

produces values ranging from −5.0 to +1.0, with higher values suggesting the presence of the
infection and lower values suggesting its absence. Figure 4.11 summarizes these results by patient
class: 23 patients do not have the infection (infection = 0), and 32 patients do have it (infection = 1).
The mean blood test value for infected patients is 3.05, while the mean value for non-infected
patients is −4.13. Although there is a noticeable difference between the two groups, the overlap in
the test results makes interpretation challenging, since some patients' values fall within similar
ranges regardless of infection status.
 FIGURE 4.11: Summary table and corresponding box plot chart to summarizing the results of a trial
for a new blood test to predict an infection.

Another way to analyze this data is to use the blood test results as the grouping variable and then
compute descriptive statistics for the binary infection variable. Because the blood test values are
continuous, the first step is to divide them into intervals. In this example, the range boundaries were
set at −5, −4, −3, −2, −1, 0, +1, and +2, and the groups were formed as shown in Figure 4.12. For
each group, the mean value of the binary variable infection is calculated and displayed in the
summary table, and also plotted in a histogram. Since the infection variable is coded as 0 and 1, the
mean for each group must lie between 0 and 1: a mean of 0 indicates that all patients in the group
tested negative, while a mean of 1 indicates that all tested positive. This representation makes the
relationship between the two variables clearer. For lower blood test values (from −5 to −2), there are
very few infection cases, meaning the test correctly predicts most of these as negative. Likewise, for
blood test values from −1 to +2, most cases are positive, and the test correctly identifies them as
such. However, in the range −2 to −1, the mean infection value is 0.55, meaning 55% are predicted
positive and 45% negative. In this range, the test performs poorly, and it would be reasonable to
conclude that blood test results within this interval should not be trusted.

FIGURE 4.12: Summary table and histogram using continuous data that has been binned to
generate the group summarized using the binary variable.
3. Cross-Classification Tables: Cross-classification tables, also known as contingency tables, are
useful for examining the relationship between two categorical variables—or continuous variables
that have been converted into categories. While one of the variables is often dichotomous,
contingency tables can accommodate variables with more than two categories.

Figure 4.13 presents an example of such a table for the variables Test Results and Infection Class
across a group of patients. Infection Class identifies whether a patient has the infection and consists
of two values: “Infection negative” and “Infection positive.” Test Results indicates whether the
blood test outcome was “Blood test negative” or “Blood test positive.” The dataset contains 55
observations, shown in the bottom-right cell of the table. The rightmost column labelled “Totals”
shows the total number of patients in each Test Results category 27 with negative blood test results
and 28 with positive results. The bottom row labeled “Totals” shows the total number of patients in
each Infection Class 23 infection-negative and 32 infection-positive. The inner cells of the table
present the counts of patients for each pair of categories: 17 patients had both a negative blood test
and no infection, 10 had a negative blood test but were infected, 6 had a positive blood test but no
infection, and 22 had both a positive blood test and an infection. This table provides a clear view of
the relationship between the two variables. Although the new blood test does not perfectly identify
infection status, it correctly classified 39 patients (22 + 17) and incorrectly classified 16 patients (10
+ 6).

FIGURE 4.13 Contingency table showing the relationship between two dichotomous variables.

Contingency tables are also useful for examining relationships between categorical variables
whether nominal or ordinal when the variables contain more than two possible values. In Figure
4.14, the dataset is summarized using two such variables: gender and age-group. The gender
variable is dichotomous, with the two categories “female” and “male” shown across the top of
the table on the x-axis. The age-group variable has been divided into nine ordered categories
such as 10–19, 20–29, 30–39, and so on. For each category of each variable, the table displays a
total count. For instance, there are 21,790 observations classified as “male,” and 1,657
observations where the age of the individual falls between 10 and 19 years. The overall number
of observations represented in the table is provided in the bottom-right corner, which in this case
is 32,561.

FIGURE 4.14 Contingency table illustrating the number of females and males in each age-
group.
4.1 CALCULATING METRICS ABOUT RELATIONSHIPS

There are several ways to measure the strength of the relationship between two variables, and the
appropriate metric depends largely on the types of variables involved for example, whether the
comparison is between two categorical variables, two continuous variables, or a mix of both.
Different types of data require different methods to capture how strongly the variables are associated.
The following section outlines commonly used techniques for quantifying the strength of these
relationships and explains how they apply to various types of variables.

1. Correlation Coefficients: For pairs of variables measured on an interval or ratio scale, the
correlation coefficient (r) is commonly used to quantify the strength of their linear relationship.
This coefficient ranges from −1.0 to +1.0 and indicates how closely the data points align with an
optimal straight line drawn through a scatterplot. A positive value of r signifies a positive
correlation between the two variables, while a negative value indicates a negative correlation.
When r is close to 0, it suggests that there is little to no linear relationship between the variables.

For example, the two scatterplots shown in Figure 4.15 illustrate different values for r. The
first graph illustrates a strong positive correlation because the points lie relatively close to an
imaginary line sloping upward from left to right through the center of the points; the second
graph illustrates a weaker correlation.

Figure 4.15 Scatterplots illustrate values for the correlation coefficient (r). The
formula used to calculate r is shown here:

where x and y are variables, xi are the individual values of x, yi are the individual values
of y, is the mean of the x variable, is the mean of the y variable, sx and sy are the
standard deviations of the variables x and y, respectively, and n is the number of observations.
To illustrate the calculation, two variables (x and y) are used and shown
in Table 4.1. The scatterplot of the two variables indicates a positive correlation between them,
as shown in Figure 4.16 The specific value of r is calculated using Table 4.2.
Table 4.1: Table of Data with Values for the x and y Variables

x y

92 6.3

145 7.8

30 3.0

70 5.5

75 6.5

105 5.5

110 6.5

108 8.0

45 4.0

x y

50 5.0
 160 7.5

155 9.0

180 8.6

190 10.0

63 4.2

85 4.9

130 6

132 7

TABLE 4.2: Table Showing the Calculation of the Correlation Coefficient

xi yi

92 6.3 −14.94 −0.11 1.64

145 7.8 38.06 1.39 52.90

30 3 −76.94 −3.41 262.37

70 5.5 −36.94 −0.91 33.62

75 6.5 −31.94 0.09 −2.87

105 5.5 −1.94 −0.91 1.77

110 6.5 3.06 0.09 0.28

108 8 1.06 1.59 1.69

45 4 −61.94 −2.41 149.28

 50 5 −56.94 −1.41 80.04

160 7.5 53.06 1.09 58.07

155 9 48.06 2.59 124.68

180 8.6 73.06 2.19 160.00

190 10 83.06 3.59 298.19

63 4.2 −43.94 −2.21 97.11

85 4.9 −21.94 −1.51 33.13

130 6 23.06 −0.41 −9.45

132 7 25.06 0.59 14.79

Sum = 1,357.06
106.94
6.41
sx = sy = 1.86
47.28

FIGURE 4.16: Scatterplot to illustrate the relationship between the x and y variables.

2. Kendall Tau:
Kendall Tau is another method for measuring the association between two variables and is based on
comparing the rankings of observations rather than their actual numerical values. To compute this
 measure, the observations for each variable are ordered, and their values are replaced with ranks
from 1 to n, where n represents the total number of observations. The Kendall Tau statistic is then
calculated using the number of concordant and discordant pairs of observations that either maintain
the same order or differ in order across the two variables. Table 4.3 demonstrates this approach
using two variables, Variable X and Variable Y, each containing ranked values for a set of 10
observations labeled A through J.

TABLE 4.3: Data Table of Rankings for Two Variables

Observation Variable X Variable Y

name

A 1 2

B 2 4

C 3 1

D 4 3

E 5 6

F 6 5

G 7 7

H 8 8

I 9 10

J 10 9

A pair of observations is concordant if the two variables change in the same direction, i.e., the
observation with the higher value of Variable X also has the higher value of Variable Y (or
both are lower together); mathematically, the differences (X₂ − X₁) and (Y₂ − Y₁) have the
same sign (both positive or both negative). For example, for points A(1,2) and B(2,4), the
differences are (2−1)=+1 and (4−2)=+2 (both positive), so A and B are concordant; reversing
the order gives (−1) and (−2), still the same sign, yielding the same result. Conversely, a pair
is discordant if the variables move in opposite directions, meaning the differences have
opposite signs. For instance, for points B(2,4) and C(3,1), the differences are (3−2)=+1 and
(1−4)=−3 (one positive, one negative), so B and C are discordant. Thus, concordance occurs
when the relative ordering of the two variables agrees, while discordance occurs when the
orderings disagree.

In Table 4.4, the observations (A through J) are first arranged in increasing order of Variable
X. To count concordant and discordant pairs systematically without double-counting or
missing any pair, each observation is compared only with the observations that follow it in
this ordered list. For example, observation A (the smallest X) is paired with each of the
remaining nine observations (B through J); among these, eight pairs (A–B, A–D, A–E, A–F,
A–G, A–H, A–I, A–J) are concordant and one pair (A–C) is discordant. Observation B is then
compared only with C through J, observation C with D through J, and so on, until the last
observation J has no further pairs. The concordant and discordant counts for each observation
are recorded in the last two columns of the table, and finally, all individual concordant counts
are summed to give the total number of concordant pairs (C) and all discordant counts are
summed to give the total number of discordant pairs (D) across the entire data set, ensuring
every unique pair of observations is considered exactly once.

TABLE 4.4: Calculation of Concordant and Discordant Pairs

Observation Concordant Discordant

Name Variable X Variable Y Pairs Pairs

A 1 2 8 1

B 2 4 6 2

C 3 1 7 0

D 4 3 6 0

E 5 6 4 1

F 6 5 4 0
 G 7 7 2 1

H 8 8 2 0

I 9 10 0 1

J 10 9 0 0

Sum: 39 6

A final Kendall Tau measure is computed based on these computed sums (the total number of
concordant and discordant pairs obtained from the table). Kendall Tau measures associations
between variables with 1 indicating a perfect ranking (complete agreement between the two
variables' orderings) and −1 a perfect disagreement of the rankings (completely reversed
orderings). A zero value assigned when the ranks are tied indicates a lack of association, or
in other words, that the two variables are independent. The simplest form of the Kendall Tau
calculation is referred to as Tau A and has the formula:

where nc and nd are the number of concordant and discordant pairs, respectively, and
n is the number of observations. In this example, the Tau A (τA) would be:

In most practical situations, there are ties based on either variable. In these situations, the
formula Tau B is often used. It considers the ties in the first variable (tx) and the ties in
the second variable (ty) and is computed using the following formulas:
In most software applications the Kendall Tau B function is used.

t-Tests Comparing Two Groups: In the context of describing data, we previously used a
one-sample t-test to determine whether the observed mean of a variable differs significantly
from a specific hypothesized value, allowing us to conclude that such a difference is unlikely
to be due to normal sampling variation or chance alone. This same principle of statistical
significance testing can now be extended when we want to compare the means of two distinct
groups or subsets within our data. Instead of comparing one mean to a fixed number, we
examine whether the difference between the means of the two groups is large enough to be
considered meaningful, or whether it could reasonably occur just by chance.

Although the basic idea remains the same as the earlier single-sample case, the comparison of
two groups requires a different approach and formula. The two-sample t-test not only
considers the size of the difference between the two group means but also takes into account
the amount of variation (spread or deviation) within each group. The test assumes that the
observations in the two groups are independent, approximately normally distributed, and that
the population variances are equal or very similar. By combining information about both the
mean difference and the within-group variability, the formula enables us to assess whether
the two groups truly differ in their underlying population means or whether the observed
difference is likely due to random variation alone.

where is the mean value of the first group, is the mean value of the second
group, and n1 and n2 are the number of observations in the first and second group
respectively, and sp is an estimate of the standard deviation (pooled estimate). The sp is
calculated using the following formula:

where n1 and n2 are again the number of observations in group 1 and group 2 respectively,
and s2 , s2 are the calculated variances for group 1 and group 2. This formula follows a t-
distribution, with the number of degrees of freedom (df) calculated as
where it cannot be assumed that the variances across the two groups are equal, another
formula is used:

where are the average values for the two groups (1 and 2), s2 and s21 are 2
the calculated variances for the two groups, and n1 and n2 are the number of
observations in the two groups.

Again, it follows a t-distribution and the number of degrees of freedom (df) is calculated
using the following formula:

The calculated t-value will be positive when the mean of Group 1 (x̄ ₁) is larger than the mean
of Group 2 (x̄ ₂), and negative when the mean of Group 2 is larger than the mean of Group 1.
Just as we did earlier when describing data, this t-value is used in a formal hypothesis test:

 Null hypothesis (H₀): The population means of the two groups are equal (μ₁ = μ₂).
 Alternative hypothesis (H₁): The population means of the two groups are not equal
(μ₁ ≠ μ₂) a two-tailed test.

The decision to accept or reject the null hypothesis follows the same process as before:
compare the absolute value of the computed t with the critical t-value from the t-distribution
table (using the calculated degrees of freedom), or compute a p-value. A small p-value
(typically < 0.05) or a |t| larger than the critical value leads to rejection of H₀, meaning the
difference between the two group means is statistically significant. The entire procedure
obtaining the t-statistic, determining significance, and interpreting the p-value is carried out
in exactly the same way as we learned earlier for the one-sample case.
3. ANOVA: One-way analysis of variance (ANOVA) is a statistical method used to compare
the means of three or more independent groups at the same time. It tests whether there is
evidence that at least one group has a different population mean. The completely
randomized one-way ANOVA is suitable when the groups are independent, the data in each
group are approximately normally distributed, and the population variances are equal.

Example: An online computer retailer has four call centers of similar size, and we want to
determine if the average number of daily calls processed differs significantly among them
(i.e., whether some centers are performing better or worse than others). The daily call data are
shown in Table 4.5. One-way ANOVA will be applied to these data to test the null hypothesis
that all four call centers have the same mean daily calls against the alternative that at least
one center differs.

TABLE 4.5: Calls Processed by Different Call Centers

Call Center A Call Center B Call Center C Call Center D

136 124 142 149

145 131 145 157

139 128 139 154

132 130 145 155

141 129 143 151

143 135 141 156

138 132 138

139 146

As with other hypothesis tests, it is necessary to state a null and alternative hypothesis.
Generally, the hypothesis statement will take the standard form:
 H0: The means are equal.
  Ha: The means are not equal.

To determine whether a difference exists between the means or whether the difference is
due to random variation, we must perform a hypothesis test. This test will look at both
the variation within the groups and the variation between the groups. The test performs
the following steps:

1. Calculate group means and variance.

2. Determine the within-group variation.

3. Determine the between-group variation.

4. Determine the F-statistic, which is based on the between-group and within group
ratio.
5. Test the significance of the F-statistic.

The following sections describe these steps in detail:

Calculate group means and variances

In Table 4.6., for each call center a count along with the mean and variance has been
calculated. In addition, the total number of groups (k = 4) and the total number of
observations (N = 29) is listed. An average of all values (=140.8) is calculated by
summing all values and dividing it by the number of observations:

TABLE 4.6 : Calculating Means and Variances

Call Call Call Call Groups

Center A Center B Center C Center D (k = 4)

136 124 142 149

145 131 145 157

139 128 139 154

 132 130 145 155

141 129 143 151

143 135 141 156

138 132 138

139 146

Count 8 7 8 6 Total count N = 29

Mean 139.1 129.9 142.4 153.7

Variance 16.4 11.8 8.6 9.5

Determine the within-group variation

The variation within groups is defined as the within-group variance or mean square
within (MSW). To calculate this value, we use a weighted sum of the variance for the
individual groups. The weights are based on the number of observations in each group.
This sum is divided by the number of degrees of freedom calculated by subtracting the
number of groups (k) from the total number of observations (N):

In this example
Determine the between-group variation

Next, the between-group variation or mean square between (MSB) is calculated. The
MSB is the variance between the group means. It is calculated using a weighted
sum of the squared difference between the group mean ( )and the average of all

observations ( ). This sum is divided by the number of degrees of freedom. This is

calculated by subtracting one from the number of groups (k). The following formula is
used to calculate the MSB:

where ni is the number for each group and is the average for each group. In

this example,

Determine the F- statistic

The F-statistic is the ratio of the MSB and the MSW:

In this example:

Test the significance of the F-statistic

Before we can test the significance of this value, we must determine the degrees of
freedom (df) for the two mean squares (within and between). The degrees of freedom for
the MSW (dfwithin) is calculated using the following formula:

where N is the total number of observations in all groups and k is the number of groups.
The degrees of freedom for the MSB (dfbetween) is calculated using the following formula:

where k is the number of groups.

In this example,

We have already computed the F-statistic as 53.39. This value shows that the average
variation between the four call-center groups is much larger than the average variation within
the groups that is due to random error. To determine whether this difference is statistically
significant, we compare the calculated F-statistic with the critical F-value obtained from the
F-table (refer to the Further Readings section for the table). The critical value is found using
the significance level α (here α = 0.05), the degrees of freedom between groups (v₁ or
df_between), and the degrees of freedom within groups (v₂ or df_within). For these data, the
critical F-value at α = 0.05 is 3.01. Because the calculated F-statistic of 53.39 is greater than
the critical value of 3.01, we reject the null hypothesis and conclude that the means of the
different call centers are not all equal.

6. Chi- Square:

The chi-square test for indepedence is a hypothesis test for use with variables measured on a
nominal or ordinal scale. It allows an analysis of whether there is a relationship between two
categorical variables. As with other hypothesis tests, it is necessary to state a null and
alternative hypothesis. Generally, these hypothesis statements are as follows:
 H0: There is no relationship.
  Ha: There is a relationship.

Using Table 4.7 we will look at whether a relationship exists between where a consumer
lives (represented by a zip code) and the brand of washing powder they buy (brand X,
brand Y, and brand Z). The ``r'' and ``c'' refer to the row (r) and column (c) in a
contingency table.

TABLE 4.7: Calculation of Chi-Square

k Category Observed (O) Expected (E) (O − E)2/E

1 r = Brand X, c = 43221 5,521 4,923 72.6

2 r = Brand Y, c = 43221 4,597 4,913 20.3

3 r = Brand Z, c = 43221 4,642 4,925 16.3

4 r = Brand X, c = 43026 4,522 4,764 12.3

5 r = Brand Y, c = 43026 4,716 4,754 0.3

6 r = Brand Z, c = 43026 5,047 4,766 16.6

7 r = Brand X, c = 43212 4,424 4,780 26.5

8 r = Brand Y, c = 43212 5,124 4,770 26.3

9 r = Brand Z, c = 43212 4,784 4,782 0.0008

k Category Observed (O) Expected (E) (O − E)2/E

Sum = 191.2

The Chi-Square test compares the observed frequencies with the expected frequencies.
The expected frequencies are calculated using the following formula:
where Er,c is the expected frequency for a particular cell in a contingency table, r is the row
count, c is the column count and n is the total number of observations in the sample.
For example, to calculate the expected frequency for the table cell where the washing powder is
brand X and the zip code is 43221 would be

The Chi-Square test (χ2) is computed with the following equation:

where k is the number of all categories, Oi is the observed cell frequency and Ei is the
expected cell frequency. The test is usually performed when all observed cell frequencies are
greater than [Link] 4.7 shows the computed χ2 for this example
There is a critical value at which the null hypothesis is rejected (χ 2 ) and this value is
c
found using a standard Chi-Square table (see Further Reading Section). The value is
dependent on the degrees of freedom (df ), which is calculated:

For example, the number of degrees of freedom for the above example is (3 − 1) × (3 − 1)
which equals 4. Looking up the critical value for df = 4 and α = 0.05, the critical value is
9.488. Since 9.488 is less than the calculated chi-square value of 191.2, we reject the null
hypothesis and state that there is a relationship between zip codes and brands of washing
powder.

TABLE 4.8: Retail Transaction Data Set

Product Product Sale Profit

Customer Store Category Description Price ($) ($)

B. March New York, NY Laptop DR2984 950 190

B. March New York, NY Printer FW288 350 105

B. March New York, NY Scanner BW9338 400 100

J. Bain New York, NY Scanner BW9443 500 125

T. Goss Washington, DC Printer FW199 200 60

T. Goss Washington, DC Scanner BW39339 550 140

L. Nye New York, NY Desktop LR21 600 60

L. Nye New York, NY Printer FW299 300 90

S. Cann Washington, DC Desktop LR21 600 60

E. Sims Washington, DC Laptop DR2983 700 140

Customer Store Category Description Price ($) ($)

 P. Judd New York, NY Desktop LR22 700 70

P. Judd New York, NY Scanner FJ3999 200 50

G. Hinton Washington, DC Laptop DR2983 700 140

G. Hinton Washington, DC Desktop LR21 600 60

G. Hinton Washington, DC Printer FW288 350 105

G. Hinton Washington, DC Scanner BW9443 500 125

H. Fu New York, NY Desktop ZX88 450 45

H. Taylor New York, NY Scanner BW9338 400 100

TABLE 4.8 shows a series of retail transactions monitored by the main office of a computer
store.

IDENTIFYING AND UNDERSTANDING GROUPS:

Decomposing a dataset into smaller, meaningful subsets is a valuable approach to better
understand the overall patterns in the data. These subsets often represent natural types or
categories of observations, such as different customer profiles in a shop based on
demographic characteristics (age, income, location, etc.).

By identifying such subgroups, we can reveal hidden [Link] example, discovering

that certain groups of consumers consistently purchase specific combinations of products.
This grouping process also enables the extraction of practical decision rules from historical
data. For instance, analyzing past medical records may show which treatment combinations
yield the best outcomes for patients with particular symptoms; these discovered rules can
then guide optimal treatment choices for future patients.

Additionally, grouping helps highlight observations that do not fit well into any major cluster
these outliers stand out from the main patterns and should be carefully examined, as they may
represent data errors, rare events, or genuinely unusual but important cases.
Identifying meaningful groups within a dataset is not only a frequent and valuable outcome of
data analysis projects, but also plays an important supporting role in other data-mining tasks,
particularly in building predictive models (as discussed in Chapter 6). The process of
discovering and interpreting these groups gives the analyst a deeper understanding of the
data’s structure, which directly improves the quality of subsequent modeling efforts.

Moreover, clustering often reveals specific subsets of the data where relationships are clearer
and stronger than in the full dataset. Building separate, tailored models for these
homogeneous subgroups can result in simpler, more accurate, and more interpretable
predictions than a single model applied to the entire dataset. For example, when predicting
house prices, houses in certain neighborhoods or locations may show a much stronger and
simpler relationship with variables like size or age; creating dedicated models for each such
subgroup typically yields better forecasting performance than one overall model.

Clustering:
Clustering is an unsupervised machine learning technique that groups similar data points
together into clusters based on their characteristics, without using any labeled data. The
objective is to ensure that data points within the same cluster are more similar to each other
than to those in different clusters, enabling the discovery of natural groupings and hidden
patterns in complex datasets.
 Goal: Discover the natural grouping or structure in unlabeled data without
predefined categories.
 How: Data points are assigned to clusters based on similarity or distance measures.
 Similarity Measures: Can include Euclidean distance, cosine similarity or other
metrics depending on data type and clustering method.
 Output: Each group is assigned a cluster ID, representing shared characteristics within
the cluster.

For example, if we have customer purchase data, clustering can group customers with
similar shopping habits. These clusters can then be used for targeted marketing,
personalized recommendations or customer segmentation.
Types of Clustering
Let's see the types of clustering,
1. Hard Clustering: In hard clustering, each data point strictly belongs to exactly one
cluster, no overlap is allowed. This approach assigns a clear membership, making it easier
to interpret and use for definitive segmentation tasks.
 Example: If clustering customer data into 2 segments, each customer belongs fully to
either Cluster 1 or Cluster 2 without partial memberships.
 Use cases: Market segmentation, customer grouping, document clustering.
 Limitations: Cannot represent ambiguity or overlap between groups; boundaries are
crisp.
Let's see an example to see the difference between the hard and soft clustering using a
distribution,

2. Soft Clustering: Soft clustering assigns each data point a probability or degree of
membership to multiple clusters simultaneously, allowing data points to partially belong to
several groups.
 Example: A data point may have a 70% membership in Cluster 1 and 30% in Cluster 2,
reflecting uncertainty or overlap in group characteristics.
 Use cases: Situations with overlapping class boundaries, fuzzy categories like customer
personas or medical diagnosis.
 Benefits: Captures ambiguity in data, models gradual transitions between clusters.
Types of Clustering Methods
Clustering methods can be classified on the basis of how they form clusters,
1. Centroid-based Clustering (Partitioning Methods)
Centroid-based clustering organizes data points around central prototypes called centroids,
where each cluster is represented by the mean (or medoid) of its members. The number of
clusters is specified in advance and the algorithm allocates points to the nearest centroid,
making this technique efficient for spherical and similarly sized clusters but sensitive to
outliers and initialization.
Algorithms:
 K-means: Iteratively assigns points to nearest centroid and recalculates centroids to
minimize intra-cluster variance.
 K-medoids: Similar to K-means but uses actual data points (medoids) as centers, robust
to outliers.
Pros:
 Fast and scalable for large datasets.
 Simple to implement and interpret.
Cons:
 Requires pre-knowledge of kk.
 Sensitive to initialization and outliers.
 Not suitable for non-spherical clusters.
2. Density-based Clustering (Model-based Methods)
Density-based clustering defines clusters as contiguous regions of high data density
separated by areas of lower density. This approach can identify clusters of arbitrary shapes,
handles noise well and does not require predefining the number of clusters, though its
effectiveness depends on chosen density parameters.
Algorithms:
 DBSCAN (Density-Based Spatial Clustering of Applications with Noise): Groups
points with sufficient neighbors; labels sparse points as noise.
 OPTICS (Ordering Points To Identify Clustering Structure): Extends DBSCAN to
handle varying densities.
Pros:
 Handles clusters of varying shapes and sizes.
 Does not require cluster count upfront.
 Effective in noisy datasets.
Cons:
 Difficult to choose parameters like epsilon and min points.
 Less effective for varying density clusters (except OPTICS).
3. Connectivity-based Clustering (Hierarchical Clustering)
Connectivity-based (or hierarchical) clustering builds nested groupings of data by
evaluating how data points are connected to their neighbors. It creates a dendrogram a tree-
like structure that reflects relationships at various granularity levels and does not require
specifying cluster numbers in advance, but can be computationally intensive.
Approaches:
 Agglomerative (Bottom-up): Start with each point as a cluster; iteratively merge closest
clusters.
 Divisive (Top-down): Start with one cluster; iteratively split into smaller clusters.
Pros:
 Provides a full hierarchy, easy to visualize.
 No need to specify number of clusters upfront.
Cons:
 Computationally intensive for large datasets.
 Merging/splitting decisions are irreversible.
4. Distribution-based Clustering
Distribution-based clustering assumes data is generated from a mixture of probability
distributions, such as Gaussian distributions and assigns points to clusters based on
statistical likelihood. This method supports clusters with flexible shapes and overlaps, but
usually requires specifying the number of distributions.
Algorithm:
 Gaussian Mixture Model (GMM): Fits data as a weighted mixture of Gaussian
distributions; assigns data points based on likelihood.
Pros:
 Flexible cluster shapes.
 Provides probabilistic memberships.
 Suitable for overlapping clusters.
Cons:
 Requires specifying number of components.
 Computationally more expensive.
 Sensitive to initialization.
5. Fuzzy Clustering
Fuzzy clustering extends traditional methods by allowing each data point to belong to
multiple clusters with varying degrees of membership. This approach captures ambiguity
and soft boundaries in data and is particularly useful when the clusters overlap or
boundaries are not clear-cut.
Algorithm:
 Fuzzy C-Means: Similar to K-means but with fuzzy memberships updated iteratively.
Pros:
 Models data ambiguity explicitly.
 Useful for complex or imprecise data.
Cons:
 Choosing fuzziness parameter can be tricky.
 Computational overhead compared to hard clustering.
Use Cases
 Customer Segmentation: Grouping customers based on behavior or demographics for
targeted marketing and personalized services.
 Anomaly Detection: Identifying outliers or fraudulent activities in finance, network
security and sensor data.
 Image Segmentation: Dividing images into meaningful parts for object detection,
medical diagnostics or computer vision tasks.
 Recommendation Systems: Clustering user preferences to recommend movies,
products or content tailored to different groups.
 Market Basket Analysis: Discovering products frequently bought together to
optimize store layouts and promotions.

K-means:
K-Means Clustering groups similar data points into clusters without needing labeled
data. It is used to uncover hidden patterns when the goal is to organize data based on
similarity.
 Helps identify natural groupings in unlabeled datasets
 Works by grouping points based on distance to cluster centers
 Commonly used in customer segmentation, image compression, and pattern
discovery
 Useful when you need structure from raw, unorganized data
Working of K-Means Clustering
Suppose we are given a data set of items with certain features and values for these
features like a vector. The task is to categorize those items into groups. To achieve this
we will use the K-means algorithm. "k" represents the number of groups or clusters we
want to classify our items into.
The algorithm will categorize the items into "kk" groups or clusters of similarity. To
calculate that similarity we will use the Euclidean distance as a measurement. The
algorithm works as follows:
1. Initialization: We begin by randomly selecting k cluster centroids.
2. Assignment Step: Each data point is assigned to the nearest centroid, forming
clusters.
 3. Update Step: After the assignment, we recalculate the centroid of each cluster by
averaging the points within it.
4. Repeat: This process repeats until the centroids no longer change or the maximum
number of iterations is reached.
The goal is to partition the dataset into k clusters such that data points within each
cluster are more similar to each other than to those in other clusters.

Why Use K-Means Clustering?

K-Means is popular in a wide variety of applications due to its simplicity, efficiency
and effectiveness. Here’s why it is widely used:
1. Data Segmentation: One of the most common uses of K-Means is segmenting data
into distinct groups. For example, businesses use K-Means to group customers
based on behavior, such as purchasing patterns or website interaction.
2. Image Compression: K-Means can be used to reduce the complexity of images by
grouping similar pixels into clusters, effectively compressing the image. This is
useful for image storage and processing.
3. Anomaly Detection: K-Means can be applied to detect anomalies or outliers by
identifying data points that do not belong to any of the clusters.
4. Document Clustering: In natural language processing (NLP), K-Means is used to
group similar documents or articles together. It’s often used in applications like
recommendation systems or news categorization.
5. Organizing Large Datasets: When dealing with large datasets, K-Means can help
in organizing the data into smaller, more manageable chunks based on similarities,
improving the efficiency of data analysis.
Challenges with K-Means Clustering
K-Means algorithm has the following limitations:
 Choosing the Right Number of Clusters (kk): One of the biggest challenges is
deciding how many clusters to use.
 Sensitive to Initial Centroids: The final clusters can vary depending on the initial
random placement of centroids.
 Non-Spherical Clusters: K-Means assumes that the clusters are spherical and equally
sized. This can be a problem when the actual clusters in the data are of different shapes
or densities.
 Outliers: K-Means is sensitive to outliers, which can distort the centroid and,
ultimately, the clusters.

Implementation of K-Means Clustering

We will be using blobs datasets and show how clusters are made
using Python programming language.
Step 1: Importing the necessary libraries
We will be importing the following libraries.
 Numpy: for numerical operations (e.g., distance calculation).
 Matplotlib: for plotting data and results.
 Scikit learn: to create a synthetic dataset using make_blobs

Association Rule


 Association rules are a fundamental concept used to find relationships, correlations or
patterns within large sets of data items. They describe how often itemsets occur together in
transactions and express implications of the form:
X→YX→Y
Where XX and YY are disjoint sets of items. This rule suggests that when items in XX appear,
items in YY tend to appear as well. Association rules originated from market basket analysis
and help retailers and analysts understand customer behavior by discovering item
associations in transaction data. For example, a rule stating
{Bread,Butter}→{Milk}
indicates that customers who buy bread and butter also tend to buy milk.
Key Components
 Antecedent (X): The "if" part representing one or more items found in transactions.
 Consequent (Y): The "then" part, representing the items likely to be purchased when
antecedent items appear.
Rules are evaluated based on metrics that quantify their strength and usefulness:

Rule Evaluation Metrics

1. Support: Fraction of transactions containing the itemsets in both X and Y.

Support measures how frequently the combination appears in the data.

2. Confidence: Probability that transactions with X also include Y.

Confidence measures the reliability of the inference.

3. Lift: The ratio of observed support to that expected if X and Y were independent.

 Lift > 1 implies a positive association — items occur together more than expected.
 Lift = 1 implies independence.
 Lift < 1 implies a negative association.

Apriori Algorithm:
Apriori Algorithm is a basic method used in data analysis to find groups of items that often
appear together in large sets of data. It helps to discover useful patterns or rules about how
items are related which is particularly valuable in market basket analysis.

How the Apriori Algorithm Works?

The Apriori Algorithm operates through a systematic process that involves several key
steps:
1. Identifying Frequent Item-Sets
 The Apriori algorithm starts by looking through all the data to count how many times
each single item appears. These single items are called 1-Item-Sets.
 Next it uses a rule called minimum support this is a number that tells us how often an
item or group of items needs to appear to be important. If an item appears often enough
meaning its count is above this minimum support it is called a frequent Item-Set.
2. Creating Possible Item Group
 After finding the single items that appear often enough (frequent 1-item groups) the
algorithm combines them to create pairs of items (2-item groups). Then it checks which
pairs are frequent by seeing if they appear enough times in the data.
 This process keeps going step by step making groups of 3 items, then 4 items and so on.
The algorithm stops when it can’t find any bigger groups that happen often enough.
3. Removing Infrequent Item Groups
 The Apriori algorithm uses a helpful rule to save time. This rule says: if a group of
items does not appear often enough then any larger group that incl2 udes these items
will also not appear often.
 Because of this, the algorithm does not check those larger groups. This way it avoids
wasting time looking at groups that won’t be important make the whole process faster.
4. Generating Association Rules
 The algorithm makes rules to show how items are related.
 It checks these rules using support, confidence and lift to find the strongest ones.
Key Metrics of Apriori Algorithm
 Support: This metric measures how frequently an item appears in the dataset relative to
the total number of transactions. A higher support indicates a more significant presence
of the Item-Set in the dataset. Support tells us how often a particular item or
combination of items appears in all the transactions like Bread is bought in 20% of all
transactions.
 Confidence: Confidence assesses the likelihood that an item Y is purchased when item
X is purchased. It provides insight into the strength of the association between two
items. Confidence tells us how often items go together i.e If bread is bought, butter is
bought 75% of the time.
 Lift: Lift evaluates how much more likely two items are to be purchased together
compared to being purchased independently. A lift greater than 1 suggests a strong
positive association. Lift shows how strong the connection is between items. Like Bread
and butter are much more likely to be bought together than by chance.

Applications of Association Rules:

Association rule learning is a type of unsupervised learning methods that tests for the
dependence of one data element on another data element and create appropriately so that it
can be more effective. It tries to discover some interesting relations or relations among the
variables of the dataset. It depends on several rules to find interesting relations between
variables in the database.
The association rule learning is the important technique of machine learning, and it is
employed in Market Basket analysis, Web usage mining, continuous production, etc. In
market basket analysis, it is an adequate used by several big retailers to find the relations
among items.
Association rules were originally transformed from point-of-sale data that represent what
products are purchased together. Although its roots are in linking point-of-sale transactions,
association rules can be used external the retail market to find relationships among types of
“baskets.”
There are various applications of Association Rule which are as follows –
1. Items purchased on a credit card, such as rental cars and hotel rooms, support insight into
the following product that customer are likely to buy.
2. Optional services purchased by tele-connection users (call waiting, call forwarding, DSL,
speed call, etc.) support decide how to bundle these functions to maximize revenue
3. Banking services used by retail users (money industry accounts, CDs, investment services,
car loans, etc.) recognize users likely to needed other services.
4. Unusual group of insurance claims can be an expression of fraud and can spark higher
investigation.
5. Medical patient histories can supports expressions of likely complications based on
definite set of treatments.
Association rules falls to live up to expectations. For instance, they are not the best method
for producing cross-selling models in market such as retail banking, because the rules end up
describing previous marketing promotions. Also, in retail banking, users frequently start with
a checking account and then a saving account. Differentiation among products does not occur
until users have higher products.
In Apriori Algorithm, this algorithm needed frequent datasets to create association rules. It is
created to work on databases that includes transactions. This algorithm needed a breadth-first
search and hash tree to compute the itemset effectively.
It is generally used for market basket analysis and support to understand the products that can
be purchased. It is used in the healthcare space to discover drug reactions for patients.
In Eclat algorithm, it represents Equivalence Class Transformation. This algorithm needed a
depth-first search method to discover frequent itemsets in a transaction database. It
implements quicker implementation than Apriori Algorithm.