Numbers for Data
PERCENTILES:
Example: P37 , the idea of the 37th percentile is to be a number
with at most 37% of the data below and 63% above. We must
say at most since there could be ties in the data. If there are no
ties, then there should be 37% below and 63% above.
There are many ways to calculate percentiles. There is no
universally agreed upon method. These different methods can
give different answers! They might not even quite meet the idea
in the above paragraph. The differences/problems, if any,
become very small with large sample sizes. The following
method is popular.
Method: Rank the data from smallest to biggest. Next figure out
how far to count in the data to get the percentile. The rank or
𝑛𝑝
how far to count is given by 𝑟 = 100 + .5. Note the +.5 is
needed because halfway through say 20 pieces of data is not
piece of data #10, it is piece of data #10.5. If 𝑟 is not an integer
we have to go a fraction of the way between two pieces of data.
For example, if 𝑟 = 8.6 then you want to go 60% of the way
from piece of data #8 to piece of data #9. If 𝑟 were bigger than
the number of pieces of data or smaller than 1, we would just
take the percentile to be the largest or smallest piece of data.
Example: Consider the data set 2, 3, 5. What is the 50th
percentile? _____ Note that we should be able to do this
intuitively and by using the formula above.
Example: Consider the data set 2, 3, 5, 9. What is the 50th
percentile? ______ For practice let’s also find the 8th percentile
and the 81st percentile.
Some common percentiles might include deciles, 𝑃10 , 𝑃20 , 𝑃30 ,
etc. But we will only use the quartiles, 𝑄1 = 𝑃20 , 𝑄2 = 𝑃50 =
the median, 𝑄3 = 𝑃75 . The median will always be a piece of data
or halfway between two pieces of data. The quartiles will
always be a piece of data, or halfway between two pieces of
data, or a quarter or three quarters of the way between two
pieces of data. Another method for quartiles, that may give a
different answer is to take the median of all the data ranked
below or above the mean.
Mean/Variance/Standard Deviation
We are interested in measuring the middle of a data set and how
much a data set varies. There are lots of ways of doing this, but
here we give by far the most popular ways, especially
concerning this course. Details of these will be discussed as the
course continues. For now, let’s just say that the properties of
the mean tend to be intuitive while the properties of the variance
and standard deviation are not as much, but we will try to make
the basic properties make as much sense as possible. Formulas
are given. We want to get across is that the mean is measuring
the middle, and the variance and standard deviation are
measuring how the data varies. Later we will try to explain how
the variance and standard deviation are really measuring how
the data are spread out.
The middle….
POPULATION MEAN
= = xP( x) = AREA _ under _ xp( x)
x
N
SAMPLE MEAN
_
x=
x
n
How data varies….
POPULATION VARIANCE/STANDARD DEVIATION
( x ) 2
x − 2
=
|x−| 2
N
2
= =
N N
| x − |2 P( x) = AREA _ under _ | x − |2 p( x)
= 2
SAMPLE VARIANCE/STANDARD DEVIATION
( x ) 2
| x − x | 2 x2
−
n
s 2
= =
n −1 n −1
s = s2
Which three things above are the numbers we really would like
to know? Which three things above are the numbers we can
always calculate?
If these sample numbers were like the population numbers, it
seems that there should be an n on the bottom instead of n-1!
Why???
Turns out we need the n-1 instead of n so that s is nearly an
unbiased estimator of (with an n on the bottom it
underestimates). Also consider an example in which we have a
sample of size 1, say the data is 22. From this you can find the
sample mean (22) and it is an unbiased estimator (perhaps not
very reliable) of the population mean. But from 1 piece of data,
you have absolutely no idea about how the data is spread out in
the population. So, it makes sense that s should not be able to be
calculated (can’t divide by 0) since you have no idea about what
you really care about, namely .
The sample mean and standard deviation are better for
symmetric data without skewness or outliers. If these problems
occur, the 5 number summary (min, Q1, median, Q3, max) is
usually better.
Example: Suppose we want to find out how much on average
each person is worth financially in this class. This may be OK,
but if Bill Gates walks in then the average would be over a
billion dollars, a somewhat meaningless number, in this case the
median is better. The median wouldn’t change much (you
would count ½ of a piece of data further)
Both the mean and the median represent the middle, but they can
be quite different. Many data such as house prices and incomes
are skewed to the right. In these cases the mean is higher than
the median. If the NBA players were to go on strike the players
would probably want the _______ reported and the owners
would probably want the _______ reported.
Box plot: a graph of the 5-number summary, a box is made with
Q1, the median and Q3 and then there are whiskers to the min
and the max
You might want to check out the following websites that lets
you play with the mean and the median.
[Link]
gizmo?resourceId=169
[Link]
1.5 IQR rule (for checking for outliers) : an outlier is any piece
of data more than 1.5 IQR’s above Q3 or below Q1. IQR = Q3
– Q1
Roughly the interval from Q1 to Q3 tells you what? It tells
where the middle half of the data is.
Suppose you don’t have raw data, but data already grouped, how
would you estimate the sample mean and standard deviation?
Assume each data was in the middle of its group.
Example: Estimate the sample mean and standard deviation for
the sample data summarized by:
Data range How many?
0 x 10 4
10 x 20 3
20 x 30 1
30 x 40 2
What if you are finding the average age for a sample of people
and you get 48.2 years. Because of the way people report ages
_____ is a better estimate for the population mean because on
average each person will be about ______ older than they report.
You can usually see if your sample mean seems reasonable, but
what about the sample standard deviation. Well, there is a very
rough estimate for s: H − L where H and L are the highest and
4
lowest numbers. This is good when the data is close to normal
and has around 20 pieces of data. Why? With normal data 95%
is within +/- 2 standard deviations and 95% of around 20 pieces
of data is basically all of it. This estimate can be really bad if
the data is not close to normal or if there are a lot of pieces of
data. This estimate is only to see if your calculation of s might
seem reasonable.
Four important properties of the mean and variance we will use
right away.
X +c = X + c X2 +c = X2
cX = c X cX
2
= c 2 X2
The first three are hopefully intuitive, at least with some help.
They will all be discussed later and hopefully the fourth will
seem at least reasonable. But for now, what do they say?
The first says if we add (or subtract) the same number to each
piece of data what will happen to the mean. It says the mean
will have that number also added (or subtracted). As an
example, what would happen to the mean if you subtracted 7
from each piece of data?
The second says if we add (or subtract) the same number to each
piece of data what will happen to the variance. It says the
variance will remain unchanged, after all moving all the data
will not change how it is spread out! As an example, what
would happen to the variance if you subtracted 7 from each
piece of data?
The third says if we multiply (or divide) the same number to
each piece of data what will happen to the mean. It says the
mean will also multiply (or divide) by this same number. As an
example, what would happen to the mean if we divided all the
data by 3?
The last says if we multiply (or divide) the same number to each
piece of data what will happen to the variance. It says the
variance will also multiply (or divide) by this same number. As
an example, what would happen to the variance if we divided all
the data by 3?