Hypothesis testing

Determining Sample Size:

Generalizing from Sample to Population

[Link]

• Basis of statistics: Law of Large Numbers

– As the sample size increases, the sample measure converges to the

population “true” measure

– How large should the sample be?

 How Large Should the Sample Be?

Margin of error

• The amount of random sampling error in a survey's results

• Margin of error occurs whenever a population is incompletely sampled

• The larger the margin of error, the less faith one should have that the
reported results are close to the "true" figures

• Greater sample size → lower margin of error but more expensive (For your
group project, collect data from 100-200 participants)
95% confidence level

If your data follows a normal distribution where mean = 0, SD = 1, when you have a
sample and got a mean as 0, you can be 95% confident that the true mean of the
population lies between -1.96 and 1.96.
→ 95% CI = [-1.96, 1.96]

What about mean = 2.3, SD = 0.8 in your sample? What is the role of the sample size to
determine the confidential interval?
 95% confidence level: for means

• Margin of error = 1.96SE

• 95% CI = Mean ± 1.96SE Sample SD
• Greater n leads to smaller SE
• Greater sample size → lower margin of error but more expensive
• From SD you calculate SE, from SE you calculate Margin of Error.
95% confidence level: for means

You need to calculate “sample mean ± 1.96SE” to calculate 95% confidential interval

A simple random sample of 100 men’s clothing stores was chosen from the population of
such stores in a city. The average annual sales of men’s suits in this sample was 1,278
units, and the standard deviation of sales was 399 units. Construct a 95% confidence
interval for the average annual sales of men’s suits in the population of stores.

Sample size (N) = 100, sample mean = 1,278, sample SD = 399

→ Based on these statistics in the sample, we estimate the true mean in the population

SE = 39.9 units
Mean ± 1.96SE = 1,278 ± (1.96)(39.9) = 1,278 ± 78.204 (margin of error)

From the sample data, we can be 95% confident that the average annual sales of men’s
suits, across all men’s clothing stores in the population, are between 1,200 and 1,356.
 95% confidence level: for proportions

Estimate of the standard error (SE) of the sample proportion

Sample SE

You need to calculate “sample proportion ± 1.96SE” to calculate 95% confidential interval

A simple random sample of 100 grocery stores was chosen from all grocery stores in a city.
Only 64 of the 100 stores carried potted plants; that is the proportion of stores in the
sample that carried potted plants was 0.64 (64%). Construct a 95 percent confidence
interval for the population proportion of stores carrying potted plants.

N = 100, p = 0.64 (64%) → You can calculate SE = 0.048

Then, p ± 1.96SE = 0.64 ± (1.96)(0.048) = about 0.64 ± 0.09

We can be 95% confident that between 55% and 73% of all grocery stores in the city carry
potted plants.
 How Large Should the Sample Be? Example
For proportion of 50% with 95% confidence level
C:\Documents and Settings\ourminsk\Local Settings\Temporary Internet Files\Content.IE5\4XJL1R32\MCj04242240000[1].wmf

N MOE (Margin of Error)

10 +/-30%
30 +/-18%
50 +/-14%
100 +/-10%
250 +/-6%
500 +/-4.4%
1000 +/-3%
If your sample size was 100 and 50 people (50%) preferred Coke then, you are 95%
confident that the true proportion lies between 50% +/-10% (95% confidential interval:
between 40% and 60%).

If your sample size was 1000 and 500 people (50%) preferred Coke, then you are
95% confident that the true proportion lies between 50% +/-3%
(95% confidential interval: between 47% and 53%).
Greater sample size → lower margin of error but more expensive
How Large Should the Sample Be?
C:\Documents and Settings\ourminsk\Local Settings\Temporary Internet Files\Content.IE5\4XJL1R32\MCj04242240000[1].wmf

For proportion of 50% with 95% confidence level

N MOE
10 +/-30%
30 +/-18%
50 +/-14%
100 +/-10%
250 +/-6%
500 +/-4.4%
1000 +/-3%

Work backwards from a margin of error table or calculator.

E.g.) “I can tolerate max10% MOE then I need at least 100.”

Confidence intervals and margin of error

[Link]

If you want to know more about statistics I also recommend to watch the following link too.
[Link]
 Statistical Hypothesis Testing

Let’s say we found…

• Liking: Brand A = 4.5 vs. Brand B = 6.5
• Choice: Brand A = 38% vs. Brand B = 72%
→ Do people like Brand B more?
→ Are they (means or proportions) significantly different?

Step A: Null and alternative hypotheses

Step B: Test statistics
Step C: p value and conclusion (Data interpretation)
 Statistical Hypothesis Testing

Step A: Null and alternative hypotheses

Null hypothesis (Ho) : Nothing is going on (e.g., liking of Brand A = Brand B)

Alternative hypothesis (H1): Something is going on (e.g., liking of Brand A ≠ Brand B)

Alternative hypothesis is usually what researchers want to show so they test whether
they can reject the null hypothesis. If they can reject the null hypothesis they
conclude their data support their hypothesis (H1).
Statistical Hypothesis Testing

Step A: Null and alternative hypotheses

Example 1)
In my sample, men like iPhone more than women. Is it true in the population, too?
[two-tailed test]
Ho: Men = Women
H1: Men ≠ Women (this is what you want to support)
[one-tailed test]
Ho: Men <= Women
H1: Men > Women (this is what you want to support)

Example 2)
In my sample, people like brand A more than brand B. Is it true in the population, too?
[two-tailed test]
Ho: Brand A = Brand B, H1: Brand A ≠ Brand B
[one-tailed test]
Ho: Brand A <= Brand B, H1: Brand A > Brand B
 Statistical Hypothesis Testing

How do we know whether we can reject the null hypothesis and support H1?
We minimize the error that we reject Ho when Ho is true (Type 1 error).

→ Type 1 error is considered to be more serious. (e.g. Ho = not guilty, H1 = guilty)

→ Hypothesis testing is focused on minimizing type 1 error (α = probability to make
type 1 error)
→ If Ho is rejected, H1 is supported
 Statistical Hypothesis Testing

Step B: Test statistics

Calculate test statistics (SPSS will do this step for you)

Z statistics, t statistics, F statistics, χ2 etc.

Step C: p value and conclusion (Data interpretation)

What is p value?
“If the null hypothesis is true, what is the probability of observing the current data?”
“When you reject Ho (and support H1), the probability that you might be wrong (type 1
error).”
→ The smaller the p is, the smaller the chance that Ho is true.
→ If the p-value is small, we can reject Ho and support H1.
→ But how small is small?
→ We set to what extent we can tolerate the type 1 error before the analysis.
 Statistical Hypothesis Testing

When you reject Ho (and support H1), you might be wrong (type 1 error).
How much error can you tolerate?

α = the criteria you set (0.05 or 0.01),

Error that I will tolerate
“I am okay with mistakenly rejecting Ho as long as the probability that I am wrong
is less α (5% or 1%)”

p = the statistics SPSS will give you

If I reject Ho and support H1, the probability that I might be wrong is p.

If you set α = 0.05, that means “I will tolerate 5% error that I reject Ho when it is true.”
If you get p = 0.03 when you set α = 0.05, that means “The probability of having the
current data when Ho is true (p = 0.03) is smaller than the error that I can tolerate (α =
0.05).” → So I reject Ho and support H1. There is something going on (H1).
 Statistical Hypothesis Testing

I think people like brand A (M = 5.6) more than brand B (M = 3.5).

[two tailed test]
H0: Brand A = Brand B
H1: Brand A ≠ Brand B
When we reject H0 and support H1, we say,
“The observed difference (5.6 – 3.5) is statistically significant (very low chance to be
coincidence). Thus, I can say that people like brand A more than brand B (in the
population).”

p = If I reject Ho and support H1, the probability that I might be wrong is p (The
probability of having the current data when Ho is true)
Do the following for your group project

You cannot support H1

p value > 0.10 → The observed difference is “not significant.”

You can support H1

0.05 < p value ≤ 0.10 → The observed difference is “marginally significant.”

p value ≤ 0.05 → The observed difference is “significant.”

 Statistical Hypothesis Testing: Example

Do people like Brand B more than Brand A?

Average liking scores from my sample: Brand A = 4.5 vs. Brand B = 6.5
Step A. Null and alternative hypotheses
Ho: Brand A = Brand B, H1: Brand A ≠ Brand B (two-tailed test, preferred)
OR
Ho: Brand A >= Brand B, H1: Brand A < Brand B (one-tailed test)

Set the criteria: I can tolerate type 1 error up to 5% (α = 0.05).

SPSS will give you p: If the null hypothesis is true, what is the probability of observing
the current data (p)? → Probability that I might be wrong when I reject Ho and support
H1

Step B. Test statistics

Calculate t statistics, let’s say t = 3.7, p = .02 when α = 0.05
 Statistical Hypothesis Testing: Example
Liking: Brand A = 4.5 vs. Brand B = 6.5
Ho: Brand A = Brand B, H1: Brand A ≠ Brand B

Step C. p value and conclusion (Data interpretation)

We set α = 0.05, and SPSS gave us p = .02
I can tolerate up to 5%, and SPSS said I can make 2% error.
The probability of observing the current data when Ho is true is 0.02 (2%).
→ We can reject the H0.
→ The observed difference is therefore “significant.”
→“Liking for Brand A and B are significantly different. People like Brand B more.”

If we set α = 0.05, and SPSS gave us p = .25

p > .10 so we cannot reject the H0 (this does not mean we accept H0)
→ “Liking for Brand A and B are not significantly different.”
 Statistical Hypothesis Testing: Example
Liking: Brand A = 4.5 vs. Brand B = 6.5
Ho: Brand A = Brand B, H1: Brand A ≠ Brand B

Step C. p value and conclusion (Data interpretation)

If we set α = 0.05, and SPSS gave us p = .06
0.05 < p < 0.1, so we can reject the H0 and support H1, but the difference is only
marginally significant.
→ “The difference in liking for Brand A and B is marginally significant. People like
Brand B more.”
Recap

Step A: Null and alternative hypotheses

Null hypothesis (Ho) : Nothing is going on.
Alternative hypothesis (H1): Something is going on. → Usually no direction for two-tailed
analyses

Step B: Test statistics

Calculate test statistics
Z statistics, t statistics, F statistics etc.

Step C: p value and conclusion

When p > 0.10 → Cannot reject Ho (Don’t say you accept Ho), thus cannot support H1,
no significant difference.
When 0.05 < p ≤ 0.1→ Can reject Ho and support H1 but only marginally significant
(e.g., Brand A is more liked than Brand B)
When p ≤ 0.05 → Can reject Ho and support H1 and add conclusion (e.g., Brand A is
more liked than Brand B).
Marketing Research Process

Problem formulation

Research design

Sample design

Data collection instrument

Analysis and interpretation

Research reporting
 Steps in Data Analysis

1) Preliminary steps
✓ Data Validation
✓ Editing (taking care of outliers, double counting, missing data)
✓ Coding (numbering respondents, deciding code levels)

2) Exploratory stage
✓ Plot the data (e.g., pie chart)
✓ Simple tabulation, summary statistics (e.g., means, SD, SE)

3) Analysis
✓ Determine an appropriate analysis depending on the research question
1) Preliminary steps

✓ Data Validation

✓ Editing (taking care of outliers, double counting, missing data)

✓ Coding (numbering respondents, deciding code levels)

 Data Validation and Editing

• Need to check whether the survey/experiment was conducted as you intended

• Any respondents who refused to participate? Why?

• Were respondents whom you wanted? (e.g., college students)
• Did the experimenter follow the correct procedure?
• Any missing data (e.g. non responses)?
• Did the experimenter debrief participants?
1) Preliminary steps

✓ Data Validation

✓ Editing (taking care of outliers, double counting, missing data)

✓ Coding (numbering respondents, deciding code levels)

 Data Coding

• Transformation of data into a form understandable by computer software.

• Need to keep which analysis you will use in mind.

(1) Have you ever purchased “Brand A” this month?

_____ (1) Yes _____ (2) No
(2) Which product did you purchase this month? Check all that apply.
_____ (1) Mobile phone _____ (2) Shoes _____ (3) Books _____ (4) Jewelry _____ (5) Others
(3) To what extent are you satisfied with “Brand A”?
Not at all satisfied 1------2------3------4------5------6-----7 Very much satisfied
(4) If our new product were available today, how likely would you be to purchase it ?
Not at all likely 1------2------3------4------5------6-----7 Extremely likely
(5) If our new product were available today, how likely would you be to recommend it to others?
Not at all likely 1------2------3------4------5------6-----7 Extremely likely
(6) How important is convenience when choosing this type of service?
Not at all important 1------2------3------4------5------6-----7 Extremely important
(7) What is your gender? (1) Male (2) Female
(8) What is your age? ____________
Data Coding
Q1: Purchased or not, Q2: Product category, Q3: Satisfaction, Q4: Purchase intention,
Q5: Recommendation intention, Q6: Importance of convenience, Q7: Gender, Q8: age

Excel file

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8
2 5 7 4 7 3 2 18
1 5 6 5 6 6 1 19
2 1 4 7 5 5 2 21
1 4 5 2 3 2 1 20
2 2 6 4 5 7 1 18
2 4 7 6 5 7 2 19
2 3 4 7 7 3 1 20
1 2 3 3 3 4 2 20
2 1 7 4 5 5 1 21
1 4 4 4 7 4 2 22
1 3 2 1 7 5 2 23
2 3 2 7 6 7 1 19
2 2 4 7 3 7 2 18
2 2 4 4 4 3 1 21
Data Coding with SPSS

Data view: You see your data (numbers)

Variable view: You see variable names

1) Import data (e.g., excel files) – Check any

formatting problems
2) Copy & paste
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8
2 5 7 4 7 3 2 18
1 5 6 5 6 6 1 19
2 1 4 7 5 5 2 21
1 4 5 2 3 2 1 20
2 2 6 4 5 7 1 18
2 4 7 6 5 7 2 19
2 3 4 7 7 3 1 20
1 2 3 3 3 4 2 20
2 1 7 4 5 5 1 21 Copy & Paste
1 4 4 4 7 4 2 22 From Excel to SPSS file
1 3 2 1 7 5 2 23
2 3 2 7 6 7 1 19
2 2 4 7 3 7 2 18
2 2 4 4 4 3 1 21
Variable Names
2) Exploratory stage

✓ Plot the data

✓ Simple tabulation, summary statistics

The Frequency Command

The frequency command determine…

• Usually for nominal variables
• Quartiles and percentiles
• Measures of dispersion (range, standard deviation, variance, minimum
and maximum)
• Histograms

Analyze → Descriptive Statistics

→ Frequencies
The Frequency Command
The Frequency Command

• To understand the data (e.g., distribution,

weird coding?)
• Not about hypothesis testing
 The Descriptive Command

The descriptive command determine…

• Usually for continuous variables
• Measures of central tendency (mean)
• Measures of dispersion (range, standard deviation, variance, minimum
and maximum)
• Measures of kurtosis and skewness

Analyze → Descriptive Statistics

→ Descriptives
The Descriptive Command
The Descriptive Command

Skewness refers to distortion or asymmetry in a symmetrical bell curve, or normal

distribution, in a set of data. If the curve is shifted to the left or to the right, it is said to
be skewed.
The Descriptive Command

Kurtosis is a measure of whether the data are heavy-tailed or light-tailed relative to a

normal distribution.
3) Analysis

✓ Determine an appropriate analysis depending on the research

question
 Data exclusion

In order to exclude some data points, select “data” and “select cases.” Then, check “if
condition is satisfied.” Then, put any condition you want to impose.
 Reliability test

• Before you conduct hypothesis testing, you can check the reliability of your scales.

• Asking more than 1 question for the same concept (e.g. liking) can increase reliability.

• Example of an attitude measure (attitude toward a charity foundation)

Are all three items measuring the same thing?

 Reliability test

What do you think of this brand (1 = not at all, 7 = very much)?

1) Good
2) Positive
3) Like
 Reliability test

• Cronbach’s Alpha (Cronbach’s α) > 0.80 → reliable

• Between 0.60-0.80 is also not too bad.
 Reliability test

If the scale is reliable you can combine them by calculating the average score.