Global Academy of Technology, Bengaluru

Computer Science and Engineering

Course: Computational Data Science
Module 3: Basics of Statistics
3.1 Introduction to Statistics
• Statistic as a numerical measurement describing a characteristic of a sample of a population.
• Statistics is a branch of mathematics that deals with the collection, review, and analysis of
data. It is known for drawing the conclusions of data with the use of quantified models.
• Statistics is defined as the science of collecting, organizing, presenting, analyzing and
interpreting numerical data for the purpose of assisting in making a more effective decision.
• A. L. Bowley defined that “Statistics may be called the science of counting”. He also said
that “Statistics may rightly be called the science of average”. According to Boddington
“Statistics is the science of estimates and probabilities”.
• According to Selligman “Statistics is the science which deals with the methods of collecting,
classifying, tabulation, comparing and interpreting numerical data collected to throw some
light on any sphere of enquiry”.
• Croxton and Cowden defined that “statistics as the collection, tabulation, presentation,
analysis and interpretation of numerical data”.

3.2 Terminologies in Statistics

1. Mean – The average value of a data set. This can be calculated by summing up all of the
individual values in the data set and dividing the total by the number of data values (n) in the
set.
2. Median - The middle value in a sorted (i.e. low to high) data set. If there is an even number
of values, then it is the average of the two middle values.
3. Range – The difference between the highest and lowest values of a data set.
4. Variation - A measure of how widely data values are spread out from the center of a data set.
5. Variance - A measure of how far the values in a data set are from the mean, on the average.
To complete the calculation, it is necessary to know whether the data set is from a
population or a sample.
6. Standard Deviation – A measure of how far data values are spread around the mean of a data
set. It is computed as the square root of the variance. Therefore, to complete the calculation,
it is necessary to know whether the data set is from a population or a sample.
7. z-score - A measure of how many standard deviations a specific value (x) in the data set is
from the mean of the data set. The z-score is positive if the data value is greater than the
mean and negative if it is less than the mean. This term can also be referred to as “the
standard score.”
8. Statistical Population: It is the collection of all possible observations of a specified
characteristic of interest (possessing certain common property) and being under study. An
example is all of the students in AAU 3101 course in this term.
9. Sample: It is a subset of the population, selected using some sampling technique in such a
way that they represent the population.
10. Sampling: The process or method of sample selection from the population.
 11. Sample size: The number of elements or observation to be included in the sample.
12. Census: Complete enumeration or observation of the elements of the population. Or it is the
collection of data from every element in a population
13. Parameter: Characteristic or measure obtained from a population.
14. Statistic: Characteristic or measure obtained from a sample.
15. Variable: It is an item of interest that can take on many different numerical values.

3.3 Measure of Center, Variance and Relative Standing

Measures of center are how we define the middle, or center, of a dataset.
A measure of center is a value in the "middle" of a dataset.
The arithmetic mean of a dataset is found by adding up all of the values and then dividing it by the
number of data values.
The median is the number found in the middle of the dataset when it is sorted in order, as shown:
[Link]([11, 15, 17, 14]) == 14.5
[Link]([11, 15, 17, 14, 31]) == 15

Measures of variation
Measures of center are used to quantify the middle of the data, but now we will explore ways of
measuring how "spread out" the data we collect is. This is a useful way to identify if our data has
many outliers lurking inside.
Now, let's take a look at the most commonly used measure of variation, the standard deviation.
It's basically a way to see how spread out the data is. There is a general formula to calculate the
standard deviation, which is as follows:

The coefficient of variation is defined as the ratio of the data's standard deviation to its mean.

Measures of relative standing

We can combine both the measures of centers and variations to create measures of relative
standings. Measures of variation measure where particular data values are positioned, relative to the
entire dataset. Let's begin by learning a very important value in statistics, the z-score.
The z-score is a way of telling us how far away a single data value is from the mean. The z-score of
a x data value is as follows
3.4 Empirical rule
The Empirical rule states that we can expect a certain amount of data to live between sets of
standard deviations. Specifically, the Empirical rule states for data that is distributed normally:
• about 68% of the data fall within 1 standard deviation
• about 95% of the data fall within 2 standard deviations
• about 99.7% of the data fall within 3 standard deviations
3.5 Categories in Statistics
• There are the two broad ways of classifying statistics, one is the on the basis of function and
another is the on the basis of distribution of data.
• There are three types of statistics on the basis of subject matter/ function.
• There are two types of statistics on the basis of distribution of data
List the two main ways of collection of data. Highlight the pros and cons of each
of the two approaches.
Obtaining data
There are two main ways of collecting data for our analysis: observational and experimentation.
Both these ways have their pros and cons, of course. They each produce different types of behavior
and, therefore, warrant different types of analysis.
1. Observational
We might obtain data through observational means, which consists of measuring specific
characteristics but not attempting to modify the subjects being studied. For example, you have a
tracking software on your website that observes users' behavior on the website, such as length of
time spent on certain pages and the rate of clicking on ads, all the while not affecting the user's
experience, then that would be an observational study. This is one of the most common ways to get
data because it's just plain easy. All you have to do is observe and collect data. Observational
studies are also limited in the types of data you may collect. This is because the observer (you) is
not in control of the environment. You may only watch and collect natural behavior. If you are
looking to induce a certain type of behavior, an observational study would not be useful.
2. Experimental
An experiment consists of a treatment and the observation of its effect on the subjects. Subjects in
an experiment are called experimental units. This is usually how most scientific labs collect data.
They will put people into two or more groups (usually just two) and call them the control and the
experimental group. The control group is exposed to a certain environment and then observed. The
experimental group is then exposed to a different environment and then observed. The experimenter
then aggregates data from both the groups and makes a decision about which environment was
more favorable (favorable is a quality that the experimenter gets to decide).
Discuss about the following types of sampling:
(i) Probability Sampling (ii) Random Sampling (iii) Unequal Probability Sampling
Probability sampling
Probability sampling is a way of sampling from a population, in which every person has a known
probability of being chosen but that number might be a different probability than another user. The
simplest (and probably the most common) probability sampling method is random sampling.
Random sampling is a method of choosing a sample of observations from a population to make
assumptions about the population. It is also called probability sampling. The counterpart of this
sampling is Non-probability sampling or Non-random sampling. The primary types of this sampling
are simple random sampling, stratified sampling, cluster sampling, and multistage sampling. In the
sampling methods, samples which are not arbitrary are typically called convenience samples.
Unequal probability sampling
If the selection probabilities are unequal, the sample mean is not unbiased for the population mean
and the sample total is not unbiased for the population total.
Example: If larger firms are sampled with higher probability, the sample mean for job openings will
be biased upward.
Define correlation coefficient, with respect to statistics. Describe the salient
features of correlation coefficient.
Correlation coefficients are a quantitative measure that describe the strength of
association/relationship between two variables.
Features of correlation coefficient are as follows:
 It will lie between -1 and 1
 The greater the absolute value (closer to -1 or 1), the stronger the relationship between the
variables: The strongest correlation is a -1 or a 1 ° The weakest correlation is a 0
 A positive correlation means that as one variable increases, the other one tends to increase as
well
 A negative correlation means that as one variable increases, the other one tends to decrease

Describe the overview of the following Graphical Displays of Basic Statistical

Descriptions, with an example each:
(i) Boxplot (ii) Bar Chart (iii) Histogram (iv) Quantile Plot (v) Q-Q Plot (vi)
Scatter Plot

Boxplots are a popular way of visualizing a distribution and a boxplot incorporates five-number
summary:
– The ends of the box are at the quartiles Q1 and Q3, so that the box length is the
interquartile range, IQR.
– The median is marked by a line within the box. (median of values in IQR)
– Two lines outside the box extend to the smallest and largest observations (outliers are excluded).
Outliers are marked separately.
• If there are no outliers, lower extreme line is the smallest observation (Minimum) and upper
extreme line is the largest observation (Maximum).
Bar Chart
Bar graphs are the pictorial representation of grouped data in vertical or horizontal rectangular bars,
where the length of bars is proportional to the measure of data. The chart’s horizontal axis
represents categorical data, whereas the chart’s vertical axis defines discrete data.

Histogram
A histogram chart displays the frequency of discrete and continuous data in a dataset using
connected rectangular bars. Here, the number of observations that fall into a predefined class
interval represented by a rectangular bar.

Quantile Plot
Displays all of the data (allowing the user to assess both the overall behaviour and unusual
occurrences)
• Plots quantile information
– For a data xi data sorted in increasing order, fi indicates that approximately 100 fi% of the data
are below or equal to the value xi
Quantile-Quantile (Q-Q) Plot
• Graphs the quantiles of one univariate distribution against the corresponding quantiles of another
• View: Is there is a shift in going from one distribution to another?
• Example shows unit price of items sold at Branch 1 vs. Branch 2 for each quantile. Unit prices of
items sold at Branch 1 tend to be lower than those at Branch 2.

Scatter Plot
• A scatter plot is one of the most effective graphical methods for determining if there appears to be
a relationship, pattern, or trend between two numeric attributes.
– To construct a scatter plot, each pair of values is treated as a pair of coordinates in an algebraic
sense and plotted as points in the plane.
• The scatter plot is a useful method for providing a first look at bivariate data to see clusters of
points and outliers, or to explore the possibility of correlation relationships.
• Two attributes, X, and Y, are correlated if one attribute implies the other.
• Correlations can be positive, negative, or null (uncorrelated).
Differentiate the attribute of ‘Similarity’ and ‘Dissimilarity’ between two
objects. Also, list the properties to be satisfied by ‘Similarity’ and Dissimilarity’
respectively.
Similarity
• The similarity between two objects is a numerical measure of the degree to which the two objects
are alike.
• Similarities are higher for pairs of objects that are more alike.
• Similarities are usually non-negative and are often between 0 (no similarity) and 1
(complete similarity).
Dissimilarity
• The dissimilarity between two objects is a numerical measure of the degree to which the two
objects are different.
• Dissimilarities are lower for more similar pairs of objects.
• The term distance is used as a synonym for dissimilarity, although the distance is often used to
refer to a special class of dissimilarities.
• Dissimilarities sometimes fall in the interval [0,1], but it is also common for them to range from 0
to ∞.

Properties of Distances
• Distances, such as the Euclidean distance, have some well-known properties.
• If distance d(x, y) between x and y, then the following properties hold.
1. Positivity
a) d(x, y) ≥ 0 for all x and y,
b) d(x, y) = 0 only if x = y.
2. Symmetry
d(x, y) = d(y, x) for all x and y.
3. Triangle Inequality
d(x, z) ≤ d(x, y) + d(y, z) for all points x, y, and z.

Properties of Similarities
• For similarities, triangle inequality typically does not hold, but symmetry and positivity typically
do.
• If s(x, y) is the similarity between points x and y, then the typical properties of similarities are:
1. Positivity
s(x, y) = 1 only if x = y. (0 ≤ s(x,y) ≤ 1)
2. Symmetry
s(x, y) = s(y, x) for all x and y.

Describe the different types of Distance on Numeric Data, with mathematical

formula representing it.
r = 1: Manhattan distance.
– A common example of this is the Hamming distance, which is just the number of bits that are
different between two binary vectors
• r = 2: Euclidean distance (L2 norm)
• r =∞: Supremum distance.
– This is the maximum difference between any component of the vectors

Describe the concept of Cosine Similarity.

List the drawbacks associated with correlation. List the issues associated with
Proximity Calculation
Problems on Mean, Median and Mode

1. Compute the mean, median and mode for the following DAM STATISTICS:

Solution:
We will let n represent the number of data points in the distribution. Then
Mean = sum of all data points/n
(The mean is also known as the "average" or the "arithmetic average.")
Median = "middle" data point (or average of two middle data points) when the data
points are arranged in numerical order.
If n represents the number of data points in a distribution, then:
the position of the "middle value" is
n+1
2
Mode = the value that occurs most often (if there is such a value).
2. Find the mean, median and mode for the following collection of responses to the question:
"How many parking tickets have you received this semester?"
1, 1, 0,1, 2, 2, 0, 0, 0, 3, 3,0, 3, 3, 0,2, 2, 2, 1, 1,4, 1, 1,0,3, 0, 0, 0, 1, 1, 2, 2, 2, 2,1, 1, 1, 1, 4,
4, 4,1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2,1, 1, 1, 1, 1, 3,3,0, 3, 3, 1, 1, 1, 1,0, 0, 1, 1, 1, 1, 3, 3, 3, 2,
3, 3, 1, 1, 1,2, 2, 2,4, 5, 5, 4, 4, 1, 1, 1, 4,1, 1, 1,3, 3, 5,3, 3, 3, 2,3, 3, 0, 0, 0, 0, 3, 3, 3, 3, 3,
3, 0, 2, 2, 2, 2, 1, 1, 1,3, 1, 0, 0, 0,1, 1, 3,1, 1, 1, 2, 2, 2, 4, 2, 2, 2, 1, 1, 1, 1,0, 0, 2, 2, 3, 3,2,
2, 3,2, 0, 0, 1, 1,3, 3, 3, 1, 1, 1, 1, 1,2, 2, 2, 2, 1, 1, 1, 1, 0,1, 1, 1, 3,1, 1, 1, 2, 2, 2, 1, 1, 1,2,
 1, 1, 1,3, 3,5, 3, 3, 1, 1, 1, 3, 3, 3, 3, 1, 1, 1,4, 1, 1, 4, 4, 4, 4, 4, 4,1, 1, 1,2, 2,5, 5, 2, 3, 3, 4,
4,3,2, 2, 2, 1,5, 1,2, 2, 1, 1, 1, 2, 2, 2, 2, 2,1, 1, 0,1, 1, 1,3, 3, 3, 3, 3.

3. The frequency table below represents the distribution of scores on a ten-point quiz. Compute
the mean, median, and mode for this distribution.

Quiz Scores

4. The frequency table below represents the distribution, according to age, of students in a
certain class.

1. Find the mean, median, and mode for this distribution.

2. True or false: 18 students were 31 years old.

5. The relative frequency table below shows the distribution of scores on a quiz in the course
Quantum Electrodynamics For Liberal Arts. Find the mean, median and mode.

6. A number of people invested $1000 each in the Gomer Family of Mutual Funds. The
frequency table below shows the current values of those investments. Compute the mean,
median and mode.

Problems based on z-score

1. The mean growth of the thickness of trees in a forest is found to be .5 cm/year with a
standard deviation of .1 cm/year. What is the z-score corresponding to 1 cm/year?
 2. A particular leg bone for dinosaur fossils has a mean length of 5 feet with a standard
deviation of 3 inches. What is the z-score that corresponds to a length of 62 inches?

Problems on Empirical Rule

1. The lifespans of lizards in a particular zoo are normally distributed. The average lizard
lives 3.1 years; the standard deviation is 0.6 years. Use the empirical rule (68−95−99.7%) to
estimate the probability of a lizard living longer than 5 years.
2. A normally distributed data set has µ = 10 and σ = 2.5, what is the probability of randomly
selecting a value greater than 17.5 from the set?
3. A normally distributed data set has µ = .05 and σ = .01, what is the probability of randomly
choosing a value between .05 and .07 from the set?
4. A normally distributed data set has µ = 514 and an unknown standard deviation, what is the
probability that a randomly selected value will be less than 514?

Problems on Similarity and Dissimilarity

1. For the given collection of points, compute:
(i) Manhattan Distance Matrix
(ii) Euclidean Distance Matrix
(iii) Supremum Distance matrix
Point x y
P1 1 2
P2 2 3
P3 3 2
P4 2 1

2. Compute Simple Matching Coefficient (SMC), Jaccard Coefficient (J), Hamming Distance
for the following pair of binary vectors:
p=0101110001
q=1100110111

3. Compute Cosine Similarity for the following pair of vectors:

X = (2, 5, 1, 0, 0, 3, 6, 1, 0, 4)
Y = (3, 3, 2, 1, 0, 3, 2, 5, 5, 2)
Also compute Extended Jaccard Coefficient/ Tanimoto coefficient for the same data.

4. Midterm exam scores for a small advanced neuroanatomy class are provided below.
Scores represent percent of items marked correct on the exam.
87,99,75,87,94,75,35,88,87,93 Compute:
(i) Range (ii) Interquartile Range (iii) Variance (iv) Standard deviation

Q1. Distance matrices — Given the points A(0,0), B(3,4), C(1,3), D(4,1), compute the
Manhattan, Euclidean, and Supremum distance matrices.

Q2. Binary vectors (SMC, Jaccard, Hamming) — For the following pair:

r=1001011010
s=1101001100
Compute SMC, Jaccard Coefficient, and Hamming Distance.

Q3. Cosine & Tanimoto — For:

A = (1, 0, 3, 2, 0, 5, 1, 0, 2, 1)
B = (2, 1, 0, 3, 1, 4, 0, 2, 1, 0)

Compute Cosine Similarity and the Tanimoto Coefficient.

Q4. Mixed challenge — You have three documents represented as term-frequency vectors:

D1 = (3, 2, 0, 5, 0, 1)
D2 = (1, 0, 4, 2, 3, 0)
D3 = (2, 1, 1, 0, 2, 3)

Compute pairwise Cosine Similarities and identify which two documents are most similar.

Q5. Minkowski distance — For P(2,4) and Q(5,8), compute the Minkowski distance for r =
1 (Manhattan), r = 2 (Euclidean), and r = 3. Observe how the distance changes as r increases.