Measures of Central Tendency (Statistics for Psychology - chp 2.

Arthur Aron,
Elliot Coups

Concept of Probability
1. Definition and Core Idea Probability is a fundamental concept for understanding
uncertainty and forms the backbone of inferential statistics12.
•
Classical Probability: This concept applies when all possible outcomes of an
experiment are equally likely. If there are 'N' equally likely possibilities, and 'n' of
these represent a "favorable outcome" or "success," then the probability of that
success is the ratio n/N1.
•
Expected Relative Frequency: More generally, the probability of an event can be
defined as the expected relative frequency of a particular outcome3. This means
that if an experiment were repeated many times, the relative frequency (number of
times the outcome occurs divided by the total number of repetitions) would
approach the true probability of that outcome.
2. Events, Outcomes, and Sample Space
•
Sample Space (S): This is the set of all possible outcomes of a random
experiment1. For example, if three shots are taken at a target, the sample space
consists of all 8 possible combinations of hits (1) and misses (0), e.g., (0,0,0) for
three misses1.
•
Outcome: An individual result from a random experiment.
•
Event (A): An event is defined as any subset of the sample space1. For example,
"missing the target three times" (M = (0,0,0)) or "hitting the target once and
missing twice" (N = {(1,0,0), (0,1,0), (0,0,1)}) are specific events within the
sample space of target shots1.
•
Calculating Event Probabilities: For a discrete sample space, the probability of an
event P(A) is the sum of the probabilities of the individual, mutually exclusive
outcomes that constitute the event1.
3. Random Variables and Their Types In many statistical applications, interest lies
in specific numerical aspects of experimental outcomes rather than every detail. A
random variable is a real-valued function defined over a sample space with an
associated probability measure1. It assigns a numerical value to each outcome of a
random experiment1.
•
Discrete Variables: These variables can only take on specific, distinct values, with
no possible values in between them (e.g., number of verbal fillers, number of
dentist visits)4. Nominal variables, while categorical, are also considered discrete
for some statistical purposes4.
•
Continuous Variables: These variables can theoretically take on an infinite number
of values between any two given values (e.g., age, height, weight, time)4.
Probability for continuous variables is often discussed in terms of probability
densities56.
•
The probability measure assigned to a discrete sample space implicitly provides the
probabilities for a random variable to take on values within its range1.
4. Probability Rules These rules govern how probabilities of different events are
combined:
•
Addition Rule (Addition Theorem of Probability): For two or more mutually
exclusive outcomes (meaning only one can occur), the total probability is the sum
of their individual probabilities1.
•
Multiplication Rule (Multiplication Theorem): For two or more independent
outcomes (meaning the occurrence of one does not affect the occurrence of the
other), the probability of both occurring is the product of their individual
probabilities1. This applies to successive or joint outcomes1.
•
Conditional Probability: This refers to the probability of an event occurring given
that another event has already occurred1.
5. Probability Distributions Probability distributions describe the probabilities of
all possible values that a random variable can take7.
•
Parameters: These are quantities that are constant for particular distributions but
can vary across different members of families of distributions (e.g., mean ($\mu$)
and variance ($\sigma^2$))7.
•
Types of Distributions: The sources refer to various specific probability
distributions:
◦
Normal Distribution: A symmetric, bell-shaped continuous distribution where
specific percentages of scores fall within certain standard deviations from the mean
(e.g., 50%, 34%, 14% rules of thumb)5.... Exact percentages can be found using Z-
scores and normal curve tables89. Many variables in nature and research
approximate a normal curve9.
◦
Binomial Distribution: A discrete probability distribution often approximated by
the normal distribution under certain conditions (e.g., for large 'n')10.
◦
Chi-Square Distribution: Used in hypothesis testing (e.g., chi-square tests for
goodness of fit or independence). Its shape depends on degrees of freedom11.
Related to F and t distributions12.
◦
t-Distribution: Used in hypothesis testing when population variance is unknown. It
differs from the normal curve, and its shape depends on degrees of freedom1213.
◦
F-Distribution: Used in analysis of variance (ANOVA) as a comparison
distribution. Its mean is determined by degrees of freedom1214.
◦
Hypergeometric, Poisson, Multinomial, Multivariate Hypergeometric
Distributions: These are also prominent in statistical theory and applications7.
•
Simulating Distributions: Methods exist for generating values of arbitrarily
distributed random variables, often involving techniques like Monte Carlo methods
or rejection schemes15....
6. Role in Statistical Inference Probability is a core ingredient for inferential
statistics, which involves drawing broader conclusions about theoretical principles
or procedures from studies conducted with smaller groups of participants
(samples)1.... The ability to make such inferences relies on understanding the
likelihood of observed results occurring by chance19.
•
Sampling Distributions: A key concept is the sampling distribution, which
describes the distribution of sample statistics (e.g., sample means) that would occur
if sampling were repeated infinitely from a population20. Understanding these
distributions and their probabilities is crucial for determining how likely a
particular sample result is, given a hypothesized population20.
•
Hypothesis Testing: Probability is central to hypothesis testing, where researchers
determine the likelihood (p-value) of obtaining observed results if a null hypothesis
(e.g., no difference or no effect) were true21.... This process is inherently based on
probabilities to minimize decision errors24.

Non probability distribution

Certainly. Regarding "non probability distribution," it's important to clarify that

this specific term is not a standard statistical concept directly defined or
extensively discussed as a category of distributions in the provided sources. The
sources primarily focus on probability distributions (such as the normal, binomial,
chi-square, t, and F distributions) and situations where data do not conform to
assumptions of these standard distributions.
However, based on the content of the sources, the phrase "non probability
distribution" could implicitly refer to:
1.
Distributions of observed/sample data (empirical distributions) that may not
perfectly fit a theoretical probability distribution.
2.
"Non-normal" population distributions, which deviate from the often-assumed
normal probability distribution.
3.
The concept of "distribution-free" or "nonparametric" statistical methods, which do
not rely on assumptions about the specific shape of the population's probability
distribution.
Let's explore these related concepts based on the information in the sources:
1. Distributions of Observed Data (Empirical Distributions)
The sources distinguish between theoretical probability distributions and the actual
distributions of collected data.
•
Observed Frequencies: When researchers collect data, they record "observed
frequencies" for categories or scores1.... These observed frequencies form an
empirical distribution, which can be visualized using frequency tables, histograms,
or frequency polygons4....
•
Comparison to Expected Frequencies: In statistical tests like the Chi-Square test,
observed frequencies are compared to "expected frequencies" (what would be
expected if a null hypothesis were true)1.... This comparison helps determine if the
observed data deviate significantly from a hypothesized distribution.
•
Properties of Observed Distributions: Observed distributions can have various
shapes, including symmetrical, skewed (positively or negatively), unimodal,
bimodal, multimodal, rectangular, or kurtotic6.... While a bell-shaped "normal
curve" is common in nature and many variables in psychology approximate it1415,
real-world data distributions are often "a bit messy" and not always perfectly
distributed as they would be if generated by random number generators16.
2. Non-Normal Population Distributions and Data Transformations
The sources acknowledge that not all population distributions are normal, which is
a significant "controversy" in statistics1718.
•
Deviation from Normality: Chapter 14 of one source is dedicated to "Strategies
When Population Distributions Are Not Normal"19. If a sample comes from a
population with a "particular nonnormal distribution," researchers might generalize
conclusions only to populations with that same nonnormal distribution20.
•
Impact on Parametric Tests: Many standard statistical procedures (like t-tests and
ANOVA) are based on mathematics that assume samples are drawn from normally
distributed populations2021. When this assumption is violated, it can affect the
accuracy of the test results, particularly concerning Type I and Type II errors22.
•
Data Transformations: One common approach to handle non-normal data is
through "data transformations"16.... This involves applying a mathematical
operation (e.g., square root, logarithm, inverse) to each score to change the
distribution's shape, making it more normal-like2526.
◦
Advantages/Disadvantages: Transformations can make the data more suitable for
parametric tests, but they can also change the interpretation of the results, as the
transformed scores no longer represent the original units of measurement2728.
Their use seems to "wax and wane in popularity"23.
3. Distribution-Free (Nonparametric) Statistical Methods
When population distributions are not normal or cannot be transformed effectively,
"nonparametric statistics" or "distribution-free" tests are often used29....
•
Assumption-Free: These tests are described as "assumption-free" or relying on
"less restrictive assumptions about populations"3.... Specifically, they do not
assume that the population from which the sample(s) was drawn is normally
distributed21.
•
Use of Ranks: A common approach in nonparametric tests is to convert raw scores
into ranks and then perform analyses on these ranks ("rank-order tests")19....
Examples include the Mann-Whitney U test (an alternative to the independent
samples t-test) and the Wilcoxon signed-ranks test (an alternative to the dependent
means t-test)3237.
•
Nature of Null Hypothesis: For nonparametric tests, the null hypothesis is typically
that "the two population distributions are different in some way" or that "the
groups come from identical populations"31.... Rejection of the null hypothesis in a
nonparametric test is often a less specific statement than with a parametric test like
the t-test, as it doesn't necessarily refer to population means directly but to
differences in distributions34.
•
Power: Nonparametric tests can sometimes have lower power compared to their
parametric counterparts if the assumptions of the parametric test are met2240.
However, if the parametric assumptions are severely violated (e.g., highly skewed
data or outliers), nonparametric tests can be more robust and sometimes even more
powerful21.
•
Computer-Intensive Methods: Modern approaches like "resampling" and
"bootstrapping" are also considered "computer-intensive" and "assumption-free" or
"nonparametric." They involve simulating sampling distributions from the
observed data itself, rather than assuming a theoretical distribution24.... These
methods are gaining popularity and may "soon dominate statistics"32.
In summary, while the sources do not use the exact term "non probability
distribution" as a formal classification, they extensively cover situations where
data or populations do not follow assumed probability distributions (especially the
normal distribution) and provide methods (data transformations, nonparametric
tests, resampling) to address these "non-ideal" distributional characteristics.

The next topic we can discuss is Hypothesis Testing, a core concept in inferential
statistics that is extensively covered across the provided sources.
What is Hypothesis Testing?
Hypothesis testing is a systematic procedure for deciding whether the results of a
research study, which examines a sample, support a hypothesis that applies to a
broader population. It is a fundamental procedure in psychology and related fields.
The core logic of hypothesis testing involves evaluating the probability of
obtaining your research results if the opposite of what you are predicting were true.
This often involves a "double negative logic," where researchers aim to reject a
"null hypothesis" to support their "research hypothesis".
The Five Steps of Hypothesis Testing
A standard framework for conducting hypothesis tests involves five key steps:
1.
Restate the Question as a Research Hypothesis and a Null Hypothesis About the
Populations:
◦
The research hypothesis (also called the alternative hypothesis, symbolized as HA
or H1) is the prediction the researcher wants to test. It states that the populations
being compared are different in some way, or that there is an effect. Researchers
are primarily interested in supporting this hypothesis.
◦
The null hypothesis (symbolized as H0) is the opposite of the research hypothesis.
It states that there is no difference between the populations, or that there is no
effect (the difference is "null"). The goal of the statistical test is typically to
determine if there is enough evidence to reject this null hypothesis. Critics argue
that the null hypothesis (of exactly no effect) is "extremely unlikely to be true" in
the real world.
2.
Determine the Characteristics of the Comparison Distribution: This step involves
identifying the theoretical probability distribution that represents what would be
expected if the null hypothesis were true. Common comparison distributions
include the Z-distribution, t-distribution, F-distribution, and Chi-square
distribution, depending on the type of data and research question.
3.
Determine the Significance Cutoff (Critical Value): This is the point on the
comparison distribution at which the null hypothesis will be rejected. This cutoff is
determined by the chosen significance level (alpha, symbolized as $\alpha$),
typically .05 (5%) or .01 (1%). The significance level represents the probability of
making a Type I error.
4.
Determine Your Sample's Score on the Comparison Distribution: This involves
calculating a test statistic (e.g., Z score, t score, F ratio, Chi-square value) based on
your sample data, and placing this score on the comparison distribution.
5.
Decide Whether to Reject the Null Hypothesis: If the sample's test statistic falls
beyond the significance cutoff (i.e., in the "rejection region"), the null hypothesis is
rejected, and the result is considered statistically significant. This means that the
probability of obtaining such a result if the null hypothesis were true is very low
(e.g., p < .05). If the test statistic does not reach the cutoff, the null hypothesis is
retained, and the results are inconclusive, not proving the null hypothesis to be
true.
One-Tailed versus Two-Tailed Tests
The choice between a one-tailed and a two-tailed test depends on the nature of the
research hypothesis:
•
One-Tailed Test (Directional Hypothesis): Used when the researcher is interested
in a difference in only one specific direction (e.g., babies walk earlier, not later).
The significance cutoff is placed entirely in one tail of the comparison distribution.
If the predicted direction is correct, one-tailed tests can have more statistical
power.
•
Two-Tailed Test (Nondirectional Hypothesis): Used when the researcher is
interested in any difference from the null hypothesis, regardless of direction (e.g.,
communication workshops could increase or decrease verbal fillers). The
significance level is split between both tails of the comparison distribution.
The choice between these tests should be made before data collection, based on the
logical rationale of the research question. Misusing this choice (e.g., changing it
after seeing the data) can inflate the actual Type I error rate. Due to potential
misuse, some journal editors are hesitant about one-tailed tests.
Decision Errors in Hypothesis Testing
Even when following proper procedures, there's always a possibility of making an
incorrect decision:
•
Type I Error ($\alpha$ Error): Occurs when a researcher rejects a true null
hypothesis. The probability of making a Type I error is equal to the significance
level ($\alpha$) chosen by the researcher.
•
Type II Error ($\beta$ Error): Occurs when a researcher fails to reject a false null
hypothesis.
Statistical Power
Statistical power is the probability of correctly rejecting a false null hypothesis. It
is symbolized as 1 - $\beta$. A more powerful experiment has a greater chance of
detecting a real effect if one exists.
Several factors influence statistical power:
•
Effect Size: A larger predicted difference between population means generally
leads to higher power.
•
Sample Size: Larger sample sizes increase power.
•
Variability: Less variability in the population (smaller standard deviation)
increases power.
•
Significance Level ($\alpha$): A less stringent significance level (e.g., .05 instead
of .01) increases power, but also increases the risk of a Type I error.
•
Number of Tails: Using a one-tailed test (when appropriate) can increase power
compared to a two-tailed test.
Power is often considered when planning research studies, in grant proposals, and
in discussing the meaning of non-significant results.
Controversies in Hypothesis Testing
The sources highlight several ongoing debates and "controversies" surrounding
hypothesis testing:
•
"Banning Significance Tests": There has been a "lively debate" and even calls to
"ban" significance tests, with critics arguing they are often misused (e.g., equating
non-significance with the null hypothesis being true).
•
"Marginal Significance": The practice of labeling results with p-values between .05
and .10 as "marginally significant" is controversial. Critics call it a "fudge term"
that goes against the "all or none decision" nature of hypothesis testing, while
others argue for flexibility given the arbitrary nature of the .05 cutoff.
•
Confidence Intervals vs. Significance Tests: Some statisticians advocate for
replacing significance tests with confidence intervals, arguing that confidence
intervals provide more comprehensive information, focusing attention on the
estimation of effect sizes and their precision rather than just a dichotomous
decision. While their use is growing, they have not yet replaced significance
testing as the dominant approach.
•
Nonrandom Samples: Many standard statistical procedures assume random
samples from a population, but much psychological research uses nonrandom
samples (e.g., college students or volunteers). This raises concerns about the
generalizability of findings.
•
Bayesian Analysis Revival: There is a "dramatic revival of interest" in Bayesian
analysis, an alternative approach to hypothesis testing. Unlike traditional methods
that assess the probability of results given the null hypothesis, Bayesian methods
can directly test the plausibility of the null hypothesis given the observed data.
While gaining attention, its widespread adoption as the standard method is
considered unlikely in the very near future.
This comprehensive overview of hypothesis testing provides a foundation for
understanding more specific statistical tests and concepts.

Central Tendency

Central tendency refers to the middle or typical value of a group of scores (a

distribution). It aims to provide a single number that describes the center of the
data. There are three main measures of central tendency:
1.
Mean: The most commonly used measure, the mean (or average) is calculated by
summing all the scores and dividing by the number of scores. Understanding the
mean is crucial for later statistical concepts. It is the point about which the sum of
squares of the deviation scores is a minimum.
2.
Mode: The mode is the most frequently occurring score in a distribution. A
distribution can have one mode (unimodal), two modes (bimodal), or multiple
modes (multimodal).
3.
Median: The median is the middle score when all scores in a distribution are
arranged from lowest to highest. If there's an even number of scores, the median is
the average of the two middle scores.
When to Use Each Measure: The choice of which measure of central tendency to
use depends on the nature of the data and the shape of its distribution.
•
Mean: The mean is sensitive to all scores in the distribution. However, it can be
unrepresentative of the main body of scores if the distribution is severely skewed
or contains a few extreme scores (outliers).
•
Median: The median is most likely to be used when a few extreme scores would
make the mean unrepresentative. For example, in a study on the number of sexual
partners desired, the mean showed a large difference between men (64.3 partners)
and women (2.8 partners). However, the median and mode for both groups were
just one partner, revealing that a small group of men desiring many partners
heavily skewed the mean, while most people desired only one.
•
Mode: The mode represents the most common score.
Controversy: The Tyranny of the Mean There has been "rebellion" against the
reliance on statistics, particularly the mean, in behavioral and social science
research. B. F. Skinner, a prominent advocate of behaviorism, was notably
opposed to statistics, stating he would rather see a psychology graduate student
take a course in physical chemistry, poetry, music, or art than statistics. Critics
argue that focusing solely on the mean can misrepresent the reality of a
distribution, especially when extreme scores are present. This perspective
advocates for the in-depth study of individuals, an approach prominent in
qualitative research methods developed in cultural anthropology, where the
researcher's mind is the primary tool for finding important relationships.
Variability
Variability refers to the spread or dispersion of scores in a distribution. It indicates
how much the scores differ from each other. Measures of variability include:
1.
Variance: The variance is a measure of the average squared deviation of scores
from the mean.
2.
Standard Deviation: The standard deviation is the square root of the variance,
providing a measure of variability in the original units of measurement.
While the provided excerpts for the "Central Tendency and Variability" chapter
(Chapter 2 of Aron et al.) discuss the concept of variability and its importance,
they do not provide the calculation formulas within these specific snippets.
However, standard deviation is frequently used alongside the mean in research
articles. Less variability in a population (a smaller standard deviation) generally
increases statistical power [203, 83b].
Central Tendency and Variability in Research Articles
Measures of central tendency and variability, such as means and standard
deviations, are commonly reported in research articles. You may also encounter
frequency tables and graphs to display the order in a group of numbers. However,
Z scores, normal curves, and probability are less often mentioned directly in
research articles, except in discussions of statistical significance or when a variable
does not follow a normal curve.
These concepts are fundamental ingredients for inferential statistics, which allow
researchers to draw conclusions about a larger population based on a smaller
sample.

Inferential Statistics: Drawing General Conclusions

Inferential statistics are methods that allow researchers to make conclusions about
a large group of individuals, known as a population, based on data collected from a
much smaller study group, called a sample1.... These conclusions extend beyond
the specific individuals studied to the larger theoretical principle or procedure
being investigated1.
Chapter 3 of your source introduces four key ingredients for inferential statistics: Z
scores, the normal curve, samples and populations, and probability1....
1. Z Scores
Z scores provide a way to standardize individual scores from different
distributions, allowing for comparison45. While Z scores are "extremely important
as steps in advanced statistical procedures," they are rarely reported directly in
research articles, except in articles specifically about methods or statistics6. The
concept of Z scores is also used in the formula for the correlation coefficient7.
A Z test is a type of hypothesis-testing procedure that uses a distribution of
means8.... Although it's a crucial stepping-stone for learning more advanced
statistical methods, the Z test itself is rarely used in real-world research practice11.
To perform a Z test, you use the mean and standard deviation of the distribution of
means to convert a raw score into a Z score12.
2. The Normal Curve
The normal curve, also known as the normal distribution, is a specific bell-shaped,
symmetrical frequency distribution that is common in nature and crucial for many
statistical concepts13.... It was discovered by Abraham de Moivre15. The normal
curve is used to figure percentages of scores and to convert between Z scores and
raw scores18.... Probabilities are often found using a normal curve table9....
In research articles, the normal curve is rarely mentioned directly, unless a
researcher is describing the pattern of scores on a variable or specifically
highlighting situations where scores do not follow a normal curve (a topic further
explored in Chapter 14)624.
Controversy: There is some debate about whether normal distributions are truly
typical for the populations of scores studied in psychology2425.
3. Sample and Population
Understanding the distinction between a sample and a population is fundamental to
inferential statistics:
•
A population is the entire group of individuals that a research study's findings are
meant to generalize to13.
•
A sample is the smaller, specific group of individuals who actually participate in
the research13.
Population parameters are statistical values (like the mean, variance, and standard
deviation) that describe an entire population. These parameters are typically
unknown and must be estimated from data collected from a sample26. For
example, in a study, you don't taste all the beans in a pot, you just taste a spoonful
(the sample) to infer about the whole pot (the population)26.
Controversy: Using Nonrandom Samples: Most psychology research relies on
nonrandom samples, often using readily available individuals like college students
or volunteers27. This practice raises concerns among some psychologists about the
validity of generalizing findings, suggesting that conclusions should be limited to
populations with similar characteristics to the sample, or that different statistical
approaches are needed24.... This issue is revisited in Chapter 1427.
4. Probability
Probability is a key concept in inferential statistics, particularly in hypothesis
testing. In research articles, probability is usually not discussed directly, except in
connection with statistical significance2429. The 'p' in statements like "p < .05"
found in research results refers to probability2930.
Probability rules are mathematical procedures for calculating probabilities in more
complex scenarios (e.g., the chance of multiple outcomes). While these rules are
part of the mathematical foundation of statistics, they are rarely used directly in the
analysis of psychology research results3132. Conditional probabilities, which
involve the probability of an event occurring given that another event has occurred,
are also an advanced topic within probability3233. In hypothesis testing,
researchers calculate the probability of obtaining their research results if the null
hypothesis were true (P(D|H0)), rather than the probability of the null hypothesis
being true given the results (P(H0|D))33....
Controversy: Bayesian Methods: There is a significant and growing interest in
Bayesian analysis as an alternative approach to hypothesis testing36.... Named
after Thomas Bayes, this approach uses a different framework for determining
probabilities. While mathematically sound, its applications in psychological
research are "hotly debated"37. Despite recent excitement, it is currently
considered unlikely to become the standard method in the near future39.
Presentation in Research Articles
As noted, Z scores, normal curves, and probability are generally not directly
mentioned in research articles, except in specialized methods discussions624.
However, researchers may briefly refer to the normal curve when describing the
distribution of a variable or when a variable's distribution is not normal6.
Descriptions of sampling procedures, especially in surveys, and discussions about
the representativeness of a sample might be included24. The most common
appearance of probability is in relation to statistical significance, indicated by 'p'
values2930.

Introduction to Hypothesis Testing

Hypothesis testing is a systematic procedure that allows researchers to decide
whether the results of a research study, based on a sample, support a prediction (a
hypothesis) that applies to a larger population1.... It's a core theme throughout
much of psychological and related fields' research1.
The fundamental idea of hypothesis testing involves a "double negative logic"4.
Researchers typically predict an effect (e.g., a new treatment will improve scores).
However, to test this, they consider the null hypothesis (the opposite of their
prediction, stating no effect or no difference) and determine the probability of
obtaining their research results if that null hypothesis were true4.... If it's highly
unlikely to get the observed results under the null hypothesis, then the null
hypothesis is rejected, allowing the researcher to accept their original prediction5.
Key Components of Hypothesis Testing:
1.
Research Hypothesis (H1 or HA): This is the prediction the researcher wants to
test. It proposes an effect or a difference between populations1.... It's sometimes
called the alternative hypothesis because it represents an alternative to the null
hypothesis17....
2.
Null Hypothesis (H0): This is the statement of no effect, no difference, or no
relationship between populations2.... For example, if a study predicts a treatment
will increase scores, the null hypothesis would state that the treatment has no effect
on scores, meaning the population mean with treatment is the same as without
treatment617. The term "null hypothesis" arises from this assumption of "no
difference"23. Researchers generally hope to reject the null hypothesis3.
3.
The Five Steps of Hypothesis Testing:
◦
Step ❶: Restate the question as a research hypothesis and a null
hypothesis about the populations. This involves defining the
specific populations being compared (e.g., people who receive an
experimental treatment vs. people in the general population)2....
◦
Step ❷: Determine the characteristics of the comparison
distribution. This involves identifying the appropriate theoretical
distribution (e.g., Z distribution, t distribution, F distribution, chi-square
distribution) against which the sample's results will be compared14....
◦
Step ❸: Determine the cutoff sample score(s) on the comparison
distribution at which the null hypothesis should be rejected. This
involves setting a significance level (alpha, α), typically .05 or .01,
which is the probability threshold for considering a result
"statistically significant"18....
◦
Step ❹: Determine your sample’s score on the comparison
distribution. This involves calculating a test statistic (e.g., a Z
score, t score, F ratio, or chi-square value) from the sample data26....
◦
Step ❺: Decide whether to reject the null hypothesis. If the
sample's test statistic falls beyond the cutoff score(s) (i.e., its
probability under the null hypothesis is less than the significance
level), the null hypothesis is rejected. Otherwise, it is retained26....
One-Tailed versus Two-Tailed Tests
The choice of research hypothesis determines whether a test is one-tailed or two-
tailed:
•
One-tailed (directional) test: Used when the research hypothesis predicts a specific
direction of effect (e.g., scores will increase or decrease)4.... The entire rejection
region is located in one tail of the comparison distribution1441.
•
Two-tailed (nondirectional) test: Used when the research hypothesis predicts a
difference but does not specify the direction (e.g., scores will be different)4.... The
rejection region is split between both tails of the distribution1441.
The decision to use a one- or two-tailed test should always be made before
collecting the data and should be based on the logic of the research question14....
Observing the data first and then choosing the test can lead to inflated Type I error
rates43.
Interpreting Results
•
Statistically Significant Result: If the null hypothesis is rejected, the result is
considered statistically significant4.... This means that the observed result is
unlikely to have occurred by chance if the null hypothesis were true7.... However,
statistical significance does not necessarily mean the result is practically
important4850.
•
Non-significant Result: If the null hypothesis cannot be rejected, the results are
considered inconclusive4.... It is a common mistake to conclude that a non-
significant result means the null hypothesis is true (i.e., there is no difference or
effect)7.... Instead, it simply means there wasn't enough evidence in the study to
reject the idea of no difference5253.
Controversies in Hypothesis Testing
Despite its widespread use, hypothesis testing has been a subject of considerable
debate in psychology and other fields54...:
•
"Tyranny of the Mean" and Opposition to Statistics: Critics like B. F. Skinner have
historically opposed the heavy reliance on statistics, arguing for more in-depth
study of individuals and qualitative methods60....
•
Misinterpretation and Misuse: A major criticism is that researchers often
misinterpret the meaning of statistical significance, particularly by equating a non-
significant result with "no effect"5164.
•
Arbitrary Significance Levels: The .05 and .01 significance levels are arbitrary
conventions, leading to a "dichotomous decision-making" problem where a result
just above .05 is treated vastly differently from one just below it57.... "Surely, God
loves the .06 nearly as much as the .05"5765.
•
The "Truth" of the Null Hypothesis: Some argue that in most real-world scenarios,
the null hypothesis (e.g., an exact zero difference between population means) is
almost always false, making the test of "is it exactly zero?" somewhat
uninformative68....
•
Nonrandom Samples: Many psychological studies use nonrandom samples (e.g.,
college students), which makes it difficult to generalize findings to broader
populations, even if statistically significant results are found71.... This violates
assumptions of many statistical procedures7274.
•
Bayesian Analysis: There is a "dramatic revival of interest" in Bayesian analysis as
an alternative approach to hypothesis testing, offering a different framework for
determining probabilities54.... While mathematically sound, its application in
psychology is "hotly debated" and currently not expected to become the standard
method in the immediate future5175.
Reporting in Research Articles
In research articles, hypothesis testing results are usually reported using specific
statistical methods (e.g., t, F, chi-square)80.... Researchers typically indicate
whether the result was statistically significant and provide the relevant test statistic
symbol (e.g., t, F, χ²) and the significance level (e.g., p < .05 or p < .01)8081.
Direct mention of Z scores, normal curves, or probability (except in relation to
statistical significance) is rare, unless specifically discussing methodological issues
or the distribution of a variable that does not follow a normal curve74....

Introduction to t-Tests
While Z-tests are valuable for introducing the core logic of hypothesis testing, they
require knowing the population's standard deviation (and thus its variance)1. In
real-world research, this information is almost never available1.... This is where t-
tests come in. T-tests are a family of hypothesis-testing procedures used when the
population variance is unknown, and it must be estimated from the sample data25.
The t-test is sometimes referred to as "Student's t" because its fundamental
principles were developed by William S. Gosset, who published his work
anonymously under the pseudonym "Student"26.
The Core Logic of t-Tests
The general five steps of hypothesis testing remain the same for t-tests, but with
key modifications to account for the unknown population variance:
1.
Restate the question as a research hypothesis and a null hypothesis about the
populations57.
2.
Determine the characteristics of the comparison distribution: Instead of a normal
(Z) distribution, the comparison distribution for t-tests is a t distribution. Its mean
is the population mean (as per the null hypothesis), but its variance is estimated
from the sample data5. The shape of this distribution depends on the degrees of
freedom (df)5.
3.
Determine the cutoff sample score(s) on the comparison distribution at which the
null hypothesis should be rejected: This involves using a t table to find critical t
values based on the degrees of freedom and the chosen significance level5.
4.
Determine your sample’s score on the comparison distribution: This involves
calculating a t score from your sample data8.
5.
Decide whether to reject the null hypothesis: If the calculated t score falls beyond
the cutoff, the null hypothesis is rejected8.
Types of t-Tests
There are three primary types of t-tests, each suited for different research designs:
1.
The t-Test for a Single Sample:
◦
Purpose: Compares the mean of a single sample to a known population mean, but
when the population's variance is unknown1....
◦
Degrees of Freedom (df): For a single sample t-test, df = N - 1, where N is the
number of individuals in the sample59. This accounts for the fact that one degree
of freedom is lost when estimating the population variance from the sample
mean10.
◦
Example: Determining if students in a particular dormitory study more hours than
students in general at a college, when the general college's studying variance is
unknown311.
2.
The t-Test for Dependent Means (Paired-Samples t-Test or Repeated Measures
Design):
◦
Purpose: Used when you have two scores for each person in your sample, and the
population variance is unknown112. This is often used for "before and after"
studies or studies comparing matched pairs1213.
◦
Key Concept: Difference Scores: The test is conducted by converting the two
scores for each person into a single "difference score" (e.g., after score minus
before score). The analysis then effectively becomes a single-sample t-test on these
difference scores, comparing their mean to a hypothesized population mean of 0
(representing no change or no difference)13....
◦
Power: Repeated measures designs generally have high statistical power because
they reduce the impact of individual differences (variation among participants) by
focusing on the variation within each participant (the difference scores)616.
◦
Controversies:
▪
Order Effects: A potential problem in repeated measures designs is that exposure
to the first treatment condition might influence performance in the second, biasing
the comparison. Researchers try to counter this by randomizing the order of
conditions1718.
▪
Nonrandom Assignment: If the order of treatment conditions is not random, or if a
comparison group is missing, interpreting the outcome becomes difficult19.
3.
The t-Test for Independent Means:
◦
Purpose: Compares the means of two entirely separate groups of people (e.g., an
experimental group versus a control group, or men versus women)1....
◦
Comparison Distribution: The comparison distribution is a "distribution of
differences between means"20.
◦
Pooled Variance Estimate: Since the population variance is unknown and we have
two samples, a "pooled estimate of the population variance" is calculated by
combining the variance information from both samples22. This combined estimate
is then used to determine the standard deviation of the distribution of differences
between means22....
◦
Controversies:
▪
Too Many t-Tests: Conducting multiple independent t-tests when comparing more
than two groups can inflate the Type I error rate (the chance of incorrectly
rejecting a true null hypothesis). This issue is often addressed by using more
advanced techniques like Analysis of Variance (ANOVA) and post hoc
comparisons25....
One-Tailed vs. Two-Tailed t-Tests
As with Z-tests, the choice between a one-tailed (directional) and a two-tailed
(nondirectional) t-test depends on the research hypothesis21....
•
One-tailed tests are used when the researcher predicts a specific direction of
difference (e.g., Group A will score higher than Group B)21....
•
Two-tailed tests are used when the researcher predicts a difference but not its
direction (e.g., Group A will score differently from Group B)21.... The decision
about which type of test to use should always be made before data collection,
based on the study's rationale, to avoid biasing the results and increasing the actual
Type I error rate36.... Most journal editors "look unkindly" at one-tailed tests due
to historical misuses37.
Effect Size and Power for t-Tests
•
Effect Size: Beyond just determining statistical significance (whether a result is
unlikely to be due to chance), researchers are increasingly encouraged to report
effect size, which measures the magnitude or practical importance of the observed
effect39.... For t-tests, a common measure of effect size is Cohen's d43....
•
Power: Statistical power is the probability of correctly rejecting a false null
hypothesis47.... Factors influencing the power of a t-test include:
◦
Predicted difference between means: Larger predicted differences lead to higher
power5152.
◦
Population standard deviation (variability): Smaller variability leads to higher
power5153.
◦
Sample size: Larger sample sizes lead to higher power51....
◦
Significance level (alpha): A less stringent alpha (e.g., .05 instead of .01) leads to
higher power, but also increases the risk of a Type I error5153.
◦
One-tailed vs. two-tailed test: One-tailed tests generally have more power than
two-tailed tests for a given effect size, if the predicted direction is correct51....
◦
Power analysis is often a major topic in research planning, such as grant and thesis
proposals60.
Reporting t-Tests in Research Articles
In research articles, the results of t-tests are typically reported concisely. This
usually includes:
•
Whether the result was statistically significant.
•
The t statistic symbol (t).
•
The degrees of freedom in parentheses.
•
The p-value (significance level), often expressed as p < .05 or p < .016162.
•
Means and standard deviations of the sample scores are also usually provided61.
While t-tests are a staple in psychological research, the broader field of statistical
methods continues to evolve, with ongoing discussions about the overemphasis on
significance testing and the growing interest in alternatives like confidence
intervals and Bayesian analysis36.... Nevertheless, understanding t-tests is
fundamental for interpreting much of the existing research literature76.
•
Introduction to Analysis of Variance (ANOVA)
While t-tests are used to compare the means of one or two groups, ANOVA is a
hypothesis-testing procedure specifically designed for comparing the means of
three or more groups3.... It is often used when a researcher wants to examine if an
independent variable with multiple levels (e.g., different types of treatments,
different age groups) has a statistically significant effect on a dependent
variable23. Understanding the logic of hypothesis testing and t-tests, particularly
regarding estimated population variance and the distribution of means, is crucial
before delving into ANOVA2.
The Core Logic of ANOVA
The fundamental goal of ANOVA is to determine if the observed differences
among the group means are larger than what would be expected by chance,
assuming the groups come from identical populations6. The process largely
follows the five steps of hypothesis testing you're already familiar with:
1.
Restate the question as a research hypothesis and a null hypothesis about the
populations:
◦
The null hypothesis (H0) for ANOVA states that all the population means are
equal (e.g., μ1 = μ2 = μ3)7....
◦
The research hypothesis (HA) states that the population means are not all equal; at
least one group mean is different from the others7....
2.
Determine the characteristics of the comparison distribution: For ANOVA, the
comparison distribution is the F distribution1112. The F distribution is determined
by two types of degrees of freedom (df)1113:
◦
Between-groups degrees of freedom (dfBetween): This is calculated as the number
of groups minus 1 (NGroups - 1)11.
◦
Within-groups degrees of freedom (dfWithin): This is the sum of the degrees of
freedom for each individual group (number in group minus 1)1113.
3.
Determine the cutoff sample score(s) on the comparison distribution at which the
null hypothesis should be rejected: This involves looking up the critical F-value in
an F table based on the degrees of freedom and the chosen significance level (e.g.,
p < .05)1114.
4.
Determine your sample’s score on the comparison distribution: This involves
calculating an F-ratio from your sample data. The F-ratio is essentially a ratio of
variance between groups to variance within groups11....
5.
Decide whether to reject the null hypothesis: If the calculated F-ratio exceeds the
critical F-value, the null hypothesis is rejected, suggesting that there are
statistically significant differences among the group means7.
Types of ANOVA
ANOVA encompasses several designs to suit different research questions:
1.
One-Way Analysis of Variance:
◦
Purpose: Used when you have one nominal (grouping) independent variable with
three or more levels and one numeric dependent variable3.... It tests for an overall
difference across all group means8.
◦
Example: Comparing the effectiveness of three different teaching methods on
student test scores7.
2.
Factorial Analysis of Variance:
◦
Purpose: Used when a study involves two or more independent variables (factors)
simultaneously1.... This design allows researchers to examine not only the effects
of each independent variable individually (main effects) but also how the
independent variables combine to influence the dependent variable (interaction
effects)16.... An interaction effect means that the effect of one factor depends on
the level of another factor1820.
◦
Example: Investigating the effects of both alcohol consumption and the
expectation of alcohol on aggression levels921.
3.
Repeated Measures Analysis of Variance:
◦
Purpose: Used when the same participants are measured multiple times under
different conditions or at different time points5.... This design is particularly
powerful because it reduces the impact of individual differences by using each
participant as their own control28....
◦
Example: Measuring participants' performance on a task before, during, and after
an intervention2431.
◦
Controversies: Potential issues include "order effects," where exposure to one
condition might influence performance in subsequent conditions32.... Randomizing
the order of conditions or including comparison groups can help mitigate these
problems32....
4.
Analysis of Covariance (ANCOVA):
◦
Purpose: Extends ANOVA by allowing researchers to statistically control for the
influence of one or more continuous "nuisance" variables (called covariates) that
might confound the results36.... It adjusts the dependent variable means based on
the covariate's effect3839.
◦
Example: Comparing test scores between different teaching methods while
controlling for students' pre-existing academic ability38.
5.
Multivariate Analysis of Variance (MANOVA) and Multivariate Analysis of
Covariance (MANCOVA):
◦
Purpose: These are advanced procedures used when a study has multiple dependent
variables that are measured simultaneously3637. MANOVA tests for differences
among group means on a combination of dependent variables, while MANCOVA
adds the control of covariates3637.
Addressing "Too Many t-Tests"
A significant advantage of ANOVA over conducting multiple t-tests is that it
controls the Type I error rate (the probability of incorrectly rejecting a true null
hypothesis)40.... If you were to run many t-tests on the same data, the chance of
finding a "significant" result purely by chance increases4142.
•
Post Hoc Comparisons: If an overall ANOVA is statistically significant (meaning
there's a difference somewhere among the group means), researchers often perform
post hoc comparisons (also called "after the fact" comparisons) to identify which
specific group means differ from each other10.... These procedures (e.g., Tukey's
HSD test45...) are designed to maintain the overall Type I error rate when making
multiple comparisons43. However, conducting too many post hoc tests can be
problematic and resemble "fishing expeditions" for significant results4849.
•
Planned Comparisons: When researchers have specific theoretical predictions
about which groups will differ before collecting data, they can use planned
comparisons (or "a priori" comparisons)50.... These tests are generally more
powerful than post hoc tests for detecting specific differences because they are
more focused5051.
Effect Size and Power for ANOVA
Similar to t-tests, reporting effect size in ANOVA is crucial, as it indicates the
magnitude or practical importance of the findings, going beyond mere statistical
significance53.... Common effect size measures for ANOVA include eta-squared
(η²) and omega-squared (ω²)4259.
Statistical power, the probability of correctly rejecting a false null hypothesis, is
also an important consideration in ANOVA designs15.... Power analysis helps
researchers plan studies to have a reasonable chance of detecting a true effect,
considering factors like predicted effect size, sample size, and significance
level60....
Reporting ANOVA in Research Articles
In research articles, ANOVA results are typically reported concisely, including the
F statistic, its degrees of freedom (for both numerator and denominator), and the p-
value (e.g., F(df1, df2) = [Link], p < .05)74.... Means and standard deviations for
each group are also usually provided74.
ANOVA is part of a broader statistical framework known as the General Linear
Model (GLM), which provides a unifying foundation for many statistical methods
in psychology, including t-tests, correlation, and regression22.... This connection
highlights how these seemingly distinct statistical tools are interconnected and
variations of a common underlying mathematical model78....

Introduction to Correlation and Regression

Correlation is a statistical technique used to measure the extent to which two
variables are associated or "go together in a systematic way" [ACA Chapter 11,
294]. It quantifies the strength and direction of a linear relationship between two
variables [ACA Chapter 11]. Regression, building on correlation, allows for the
prediction of one variable's values from the values of another variable. Both
techniques are closely linked, as the ability to predict one variable from another is
directly related to whether they are correlated.
Correlation: Measuring Association
The primary measure of correlation is the correlation coefficient, often denoted as
Pearson's r [ACA Chapter 11]. This coefficient indicates both the strength and
direction of a linear relationship between two variables. For example,
understanding how the formula for the correlation coefficient is calculated can help
in comprehending the logic behind it. The sampling distribution of r can be
approximated as normally distributed around the true population correlation, with
its standard error depending on the sample size.
Correlation vs. Causation
A crucial point in interpreting correlation is that it does not imply causation. Even
if two variables are strongly correlated, it doesn't mean that one causes the other.
There could be a "third factor" influencing both variables, or the direction of
causality might be ambiguous or non-existent. For instance, while it's known that
the future cannot cause the past, other complex causal relationships, possibly
involving unmeasured third variables, may exist. Inferring causality is
fundamentally an issue of experimental design, such as through random
assignment to groups, rather than solely a statistical one. This is a long-standing
debate in statistics and psychology.
Regression: Making Predictions
Regression analysis focuses on prediction, allowing researchers to forecast the
value of a dependent variable based on one or more independent (predictor)
variables.
Linear Prediction Rule
The simplest form of regression involves a linear prediction rule, often expressed
as a regression equation (e.g., Y' = a + bX). This rule uses a regression coefficient
(the slope, b) to indicate how much the dependent variable is predicted to change
for each unit change in the predictor variable. The "sum of squared errors" is a key
concept here, representing the unexplained variance in the dependent variable after
fitting the regression model. The "residual variance" or "error variance" is an
unbiased estimate of the population variance not accounted for by the model.
Multiple Regression
When a study involves "two or more independent variables simultaneously,"
multiple regression is used to predict a dependent variable from multiple predictor
variables. This method allows for a more comprehensive understanding of how
several factors contribute to an outcome. Other specialized regression techniques
exist, such as logistic regression, which is used when the dependent variable is
categorical (e.g., predicting an outcome like success or failure).
Relationship to the General Linear Model (GLM)
Correlation and regression are integral parts of the General Linear Model (GLM), a
unifying statistical framework that encompasses many statistical procedures used
in psychology, including t-tests and Analysis of Variance (ANOVA). This means
that seemingly distinct statistical tests can often be derived from or performed
using the principles of multiple regression. For example, a one-way ANOVA
comparing two groups would yield the same statistical significance result as a t-test
for independent means. The GLM, as a mathematical statement of relationships
among variables, provides a powerful foundation for statistical analysis.
Important Considerations and Controversies
•
Statistical vs. Clinical Prediction: A historical controversy in psychology concerns
the "superiority of statistical prediction over more intuitive, humanistic, or clinical
approaches". Research has shown that even simple statistical formulas can often be
more accurate in predictions than the "trained intuitions" of experts.
•
Assumptions and Diagnostics: In regression analysis, it's crucial to examine the
"residuals" (the differences between observed and predicted values) to check for
assumptions like normality. Issues like "multicollinearity" (high correlation among
predictor variables) can complicate multiple regression results. Trends in residuals
may indicate the influence of unmeasured variables.
•
Effect Size: As with other inferential tests, reporting effect size is vital in
correlation and regression to convey the practical importance of the findings, not
just their statistical significance.
•
Model Specification: There are ongoing debates about how to specify models and
calculate sums of squares (e.g., Type II vs. Type III sums of squares), particularly
when dealing with unequal sample sizes or complex designs.
While understanding the calculations is important for conceptual grasp, modern
statistical software greatly simplifies the process, allowing researchers to focus
more on the theoretical implications and assumptions.
Please ellaborate on multiple regression analysis.

Multiple regression analysis is a powerful statistical technique used to predict a

criterion variable (also called the dependent variable) based on the scores of two or
more predictor variables (also known as independent variables) simultaneously [i,
iv, v, vi]. It extends the concept of bivariate prediction, which uses only a single
predictor variable [i, v].
Purpose and Core Logic
The main objective of multiple regression is to establish relationships that allow
for the prediction of one or more variables from others [ii]. While it is used for
prediction, a common application in behavioral science is to understand the
complex relationships between variables and the role each predictor plays [v].
The fundamental goal is to find a linear prediction rule that minimizes the sum of
the squared differences between the predicted scores and the actual scores on the
criterion variable [i, iv, v]. This is known as the least squares error principle [i, iv].
The general form of a multiple regression linear prediction rule with k predictor
variables is: Y' = a + (b1)(X1) + (b2)(X2) + ... + (bk)(Xk) [i, iv] Where:
•
Y' is the predicted score on the criterion variable [i, iv].
•
a is the regression constant (also called the intercept), representing the predicted
value of Y when all predictor variables are zero [i, iv].
•
b1, b2, ..., bk are the regression coefficients for each respective predictor variable
(X1, X2, ..., Xk) [i, iv, v]. Each 'b' value indicates how much the criterion variable
is predicted to change for each unit change in that specific predictor variable,
holding all other predictor variables constant [v].
Key Concepts and Measures
1.
Regression Coefficients (b and β):
◦
Unstandardized Regression Coefficients (b): These are the b values in the
regression equation and reflect the change in the criterion variable (Y) for a one-
unit increase in the predictor (X), in the original units of measurement [v].
◦
Standardized Regression Coefficients (β): Often denoted as beta (β), these
coefficients are used when predictor and criterion variables are expressed as Z
scores [i, v]. A standardized regression coefficient shows how much of a standard
deviation the predicted value of the criterion variable changes when the predictor
variable changes by one standard deviation [i]. In bivariate prediction, the
standardized regression coefficient is the same as the correlation coefficient (r) [i].
However, in multiple regression, β for each predictor is generally not the same as
its bivariate r with the criterion [i, v]. This is because β reflects the unique
contribution of that predictor, excluding any overlap with other predictors [i, v].
◦
Controversy in Interpretation: There is debate on whether to present
unstandardized (b) or standardized (β) coefficients, or both [i]. Standardized
coefficients can be less consistent across different samples due to their dependence
on the range and variance of scores, while unstandardized coefficients are not [i].
When comparing the relative importance of multiple predictors within a regression,
standardized β values are generally preferred, as different measurement scales for b
can be misleading [i]. Some experts recommend considering both the bivariate
correlations (r) and the standardized regression coefficients (β) to understand a
predictor's overall versus unique association with the criterion [i].
2.
Multiple Correlation Coefficient (R) and R-squared (R²):
◦
Multiple Correlation Coefficient (R): This is the overall degree of association
between the criterion variable and all the predictor variables taken together [i, v]. R
is defined as the correlation between the criterion (Y) and the "best linear
combination" of the predictors [v]. Unlike the bivariate correlation coefficient (r),
R is always positive [v].
◦
R-squared (R²): The squared multiple correlation (R²) represents the proportionate
reduction in error or the proportion of variance accounted for in the criterion
variable by all the predictor variables combined [i, v]. For example, an R of .40
means an R² of .16, indicating that the predictors together account for 16% of the
variation in the criterion variable [i]. R² is also a measure of effect size for multiple
regression [i]. Cohen's conventions for R² are: .02 (small effect), .13 (medium
effect), and .26 (large effect) [i].
Assumptions and Limitations
Multiple regression, like other statistical procedures, relies on certain assumptions
and has limitations that can affect the accuracy and interpretability of results:
1.
Assumptions:
◦
Linearity: The relationship between the criterion variable and the predictor
variables (or combinations thereof) should be linear [i, v].
◦
Independence of Errors: Error scores should follow a normal distribution [i].
◦
Homoscedasticity: There should be an equal distribution of each variable at each
point of the other variable, meaning the variance of the residuals (errors) is
constant across all levels of the predictor variables [v].
2.
Limitations:
◦
Causality: As with simple correlation, multiple regression does not imply causation
[i, iv]. Even if a strong predictive relationship is found, it does not mean that the
predictor variables cause changes in the criterion variable [i, iv]. Experimental
design, particularly random assignment, is crucial for inferring causality [i].
◦
Restriction in Range: If the sample studied includes only a limited range of
possible values for a variable, the regression coefficients (and correlations) may be
smaller than they should be to reflect the true association in the broader population
[i, iv].
◦
Curvilinearity: The procedures are inaccurate if the true relationship between
variables is curvilinear rather than linear [i, iv].
◦
Unreliable Measures: If the measures used for variables are unreliable, the
regression coefficients can be attenuated (smaller than they should be) [i, iv].
◦
Outliers: Extreme scores or "outliers" can disproportionately influence regression
results [i, iv]. Researchers often check for outliers using scatter diagrams and may
perform analyses with and without them to assess their impact [i, iv].
◦
Multicollinearity: This occurs when two or more predictor variables are highly
correlated with each other [ii, v]. Multicollinearity can lead to:
▪
Increased standard errors of regression coefficients, making it harder to determine
if individual predictors are statistically significant [v].
▪
Instability of regression coefficients, meaning their values might change
considerably if minor changes are made to the model or data [ii, v].
▪
Difficulty in separating the unique effects of highly correlated independent
variables on the dependent variable, as their effects become "confounded" [ii, v].
▪
Identified by examining the correlation matrix of predictor variables [ii] or
statistics like Variance Inflation Factor (VIF) and Tolerance (the reciprocal of VIF)
[v]. A low tolerance value indicates high multicollinearity [v].
◦
Residual Analysis: Examining the residuals (differences between observed and
predicted values) is crucial for checking model assumptions and identifying
potential problems like non-linearity or outliers [ii, v]. Residual plots should show
a random pattern [ii].
Relationship to the General Linear Model (GLM)
Multiple regression is considered the most general and flexible technique within
the General Linear Model (GLM), a unifying framework for many statistical
procedures in psychology [i, ii, vi]. The GLM, which describes an individual's
score on a variable as a sum of a constant, weighted influences of other variables,
and error, is nearly identical to the multiple regression formula [i].
Many other commonly used statistical tests, including t-tests, Analysis of Variance
(ANOVA), and bivariate correlation/prediction, can be understood as special cases
of multiple regression [i, ii, vi]. This means they can be derived from, or performed
using, the principles of multiple regression [i, vi]. For example, a one-way
ANOVA comparing multiple groups can be framed as a multiple regression
problem using nominal coding (dummy variables) for the group membership [vi].
Advanced Applications and Interpretations
•
Hypothesis Testing in Multiple Regression: You can test the overall significance of
the multiple correlation (R), individual regression coefficients (to see if they
contribute significantly above and beyond other predictors), and the regression
constant [i].
•
Multiple Criterion Variables: While this chapter primarily focuses on a single
criterion variable, procedures like Multivariate Analysis of Variance (MANOVA)
and Multivariate Analysis of Covariance (MANCOVA) are extensions for
situations with more than one dependent variable [i, ii].
•
Multilevel Modeling (Hierarchical Linear Modeling - HLM): This advanced form
of regression is used when individuals in a study are grouped in a way that might
affect the pattern of scores (e.g., students within classrooms) [i, ii]. It allows for
analyzing lower-level variables (within groups) and upper-level variables (group-
level characteristics) simultaneously [i, ii].
•
Logistic Regression: Used when the dependent variable is categorical, such as
predicting a binary outcome (e.g., success/failure, presence/absence of a condition)
[i, v].
•
Mediating and Moderating Relationships: Multiple regression is fundamental for
analyzing mediating relationships (where one variable explains the relationship
between two others) and moderating relationships (where the relationship between
two variables changes depending on a third variable) [v].
•
Model Selection: Various methods exist for constructing an "optimal" regression
equation from a large set of variables, such as "all subsets regression," "forward
selection," "backward elimination," and "stepwise regression" [v].
•
Cross-Validation: It is crucial to cross-validate regression equations on
independent data sets to assess their generalizability, as equations derived from one
sample may not perform as well on another [v].
In research articles, multiple regression results are frequently reported in tables,
showing regression coefficients (b and/or β), correlations (r), and the overall R or
R² [i].

The next topic, following our discussion of multiple regression analysis, is **The
General Linear Model (GLM)**.
The General Linear Model is a fundamental idea in statistics that serves as the
unifying framework for almost all statistical methods used in psychology. Many
common statistical procedures, including multiple regression, *t*-tests, and
analysis of variance (ANOVA), are considered special cases of the GLM [i, 201,
204, 219]. Understanding the GLM can provide a "big picture" view of what
you've learned so far and help you make sense of more advanced statistical
procedures encountered in research articles.

The GLM mathematically describes an individual's score on a variable as a sum of

a constant, weighted influences of other variables, and an error term, which is
nearly identical to the multiple regression formula [i, 219, 220]. While the GLM
itself is not highly controversial as a mathematical statement, its role as the
foundation of major statistical techniques is increasingly being recognized among
researchers. This framework also allows for a deeper exploration of causality and
its implications in research.

The next topic we will cover is Discriminant Analysis.

Discriminant analysis is a statistical technique used for distinguishing between two

or more groups based on a set of variables.

However, the sources indicate that common practice in statistics has largely shifted
away from discriminant analysis in favor of other methods, particularly logistic
regression. This preference for logistic regression stems from several limitations
associated with discriminant analysis:

● Probability Range: Discriminant analysis can sometimes produce

probabilities of success that fall outside the logically possible range of 0 and
1.
● Assumptions: It relies on certain restrictive normality assumptions
regarding the independent variables, which are frequently not realistic in
practical research settings.

In contrast, logistic regression is favored because it produces probabilities within

the appropriate 0 to 1 range and does not impose such restrictive normality
assumptions on the independent variables, which can be either categorical or
continuous.

**Non-Parametric Statistical Methods**

Non-parametric statistical methods are a class of hypothesis-testing procedures that
make **fewer or less restrictive assumptions** about the population distributions
from which data are drawn, compared to their parametric counterparts. They are
also often referred to as "distribution-free tests" because they do not assume a
specific shape for the population distribution (like a normal curve). While
"nonparametric" and "distribution-free" are often used interchangeably, the term
"assumption-freer tests" is sometimes preferred to emphasize that they are not
entirely free of assumptions, just less restrictive.

**When to Use Non-Parametric Methods**

Non-parametric tests are particularly useful in situations where the assumptions

required for parametric tests (such as *t*-tests, ANOVA, and standard
correlation/regression significance tests) are clearly violated. These violations
often include:
* **Non-normal population distributions:** When sample data suggest that the
underlying populations are far from normally distributed, perhaps being highly
skewed or having many outliers.
* **Unequal variances:** When the populations being compared are suspected to
have vastly different variances.
* **Small sample sizes:** With small samples, the central limit theorem (which
suggests that the sampling distribution of the mean approaches normality with
larger samples) may not apply, making non-parametric tests more appropriate if
the population is not normal.
* **Rank-order data:** When the original scores in a study are ranks, or when
the exact numeric values of the measure are questionable and only their order is
reliable (ordinal scale data).

**Advantages of Non-Parametric Methods**

1. **Fewer Assumptions:** Their primary advantage is that they do not require

strong assumptions about the shape or parameters (like the mean or variance) of
the underlying population distributions.
2. **Robustness to Outliers:** Many non-parametric tests work by transforming
raw scores into ranks. This process minimizes the distorting effect of extreme
scores (outliers) that can disproportionately influence parametric tests by inflating
variance.
3. **Increased Power (when assumptions are violated):** When the assumptions
for parametric tests are not met, non-parametric methods can be more powerful,
meaning they are more likely to detect a significant effect if one truly exists.
4. **Direct Logic:** Some non-parametric tests, especially computer-intensive
methods like randomization tests, have a direct logic that bypasses the
complexities of estimating population distributions and distributions of means.
5. **Applicability to Ordinal Data:** They are particularly well-suited for data
that are inherently rank-ordered.
6. **Sensitivity to Medians:** Many non-parametric tests are more sensitive to
differences in medians rather than means, which can be advantageous if medians
are a more appropriate measure of central tendency for the data.

**Disadvantages of Non-Parametric Methods**

1. **Lower Power (when assumptions are met):** If the assumptions for

parametric tests *are* fully met, parametric tests are generally as good as or more
powerful than non-parametric alternatives.
2. **Less Specific Null Hypothesis:** The null hypothesis in non-parametric tests
is often less specific, typically stating that distributions are not different in any
respect, rather than focusing on a specific parameter like the mean. This means a
significant result could stem from differences in central tendency, variability, or
shape, which might be less interpretable.
3. **Less Familiarity:** Many researchers are less familiar with non-parametric
tests, and they may not have been developed for all complex research designs.
4. **Ties in Ranks:** When there are many tied scores (identical values) in the
data, the accuracy of some rank-order tests can be distorted.
5. **Information Loss:** Transforming continuous data into ranks can sometimes
lead to a loss of information, which might distort the underlying meaning of the
scores.

**Types of Non-Parametric Tests and Related Concepts**

The following are some common non-parametric tests that serve as alternatives to
parametric procedures:

* **Mann-Whitney U Test (or Wilcoxon Rank-Sum Test):** This test is the non-
parametric equivalent of the *t*-test for independent means, used when comparing
two independent groups. The Wilcoxon rank-sum test is mathematically equivalent
and based on the same logic, differing only in computational details when done by
hand.
* **Wilcoxon Signed-Ranks Test:** This is the non-parametric alternative to the
*t*-test for dependent means (or paired-samples *t*-test), used for comparing two
related samples (e.g., before and after measurements on the same individuals).
* **Kruskal-Wallis H Test:** This test is the rank-order version of the one-way
analysis of variance (ANOVA), used for comparing three or more independent
groups.
* **Friedman's Rank Test:** This is the non-parametric analogue for repeated
measures analysis of variance, used for comparing three or more related samples.
* **Sign Test:** This is another non-parametric alternative to the *t*-test for
dependent means. It's a very simple test that counts the number of positive and
negative difference scores and determines if this count is significantly different
from what would be expected by chance.
* **Spearman's Rho (*r*s):** This is a type of correlation coefficient used for
rank-ordered data. It is computed by changing the scores into ranks (separately for
each variable) and then applying the usual Pearson correlation coefficient formula.
It can be used when assumptions for Pearson correlation are not met or for
curvilinear associations.

**Relationship with Parametric Tests and Data Transformations**

Interestingly, two statisticians (Conover & Iman, 1981) demonstrated that you can
often get approximately the same results as specialized rank-order tests by simply
transforming your data into ranks and then using the usual parametric *t*-test or
analysis of variance procedures. While not as accurate as the dedicated rank-order
tests, this "shortcut" is often quite close to the true result, especially with larger
samples, and can be simpler to implement with statistical software.
Another strategy when parametric assumptions are violated is **data
transformation**. This involves applying a mathematical procedure (e.g., square
root, logarithm, inverse) to each score to make the distribution closer to normal or
to achieve equal variances across groups. Once transformed, the ordinary
parametric hypothesis-testing procedures can be used. However, transformations
do not always work for all groups and can sometimes distort the original meaning
of the scores.

In summary, non-parametric methods provide flexible and robust alternatives when

the stringent assumptions of parametric tests cannot be met, ensuring more
accurate conclusions in diverse research scenarios.

Factor analysis is a statistical procedure used to identify underlying patterns or

"clumps" among a large number of measured variables. It helps researchers
determine which variables tend to be highly correlated with each other, while
showing less correlation with other variables outside their group. This technique is
considered an elaboration of correlation and prediction and is frequently found in
psychology research articles.

Here's a detailed explanation of factor analysis:

* **Purpose and Core Concept**

* When a researcher has measured a group of individuals on many different
variables (e.g., numerous attitude questions on a survey), factor analysis helps
reveal which variables "clump together".
* Each such "clump" or group of variables is called a **factor**. The aim is to
simplify complex data by reducing the number of variables to a smaller set of
underlying factors that explain the observed correlations.

* **Key Terminology**
* **Factor Loading**: This term describes the correlation of an individual
variable with a particular factor. Variables typically show high loadings on only
one factor. Factor loadings range from -1 (perfect negative correlation) to +1
(perfect positive correlation), with 0 indicating no relationship. In psychological
research, a variable is generally considered to contribute meaningfully to a factor if
its factor loading is at least .30 (or below -.30).
* **Variance Explained**: Factor analysis also provides information on how
much of the total variation among the variables is accounted for by each identified
factor.

* **How it Works**
* The analysis is typically performed by computer using complex formulas that
start with the correlations between all the measured variables.
* These calculations result in a set of factor loadings for each variable across
the identified factors.
* Researchers can choose from several different approaches to factor analysis,
and these methods may yield slightly different results.

* **Interpretation and Subjectivity**

* The most subjective aspect of factor analysis lies in the researcher's decision
of how to **name** each factor. The chosen name should accurately reflect the
common underlying concept or theme shared by the variables that have high
loadings on that factor. When reading research articles, it's important to critically
assess whether the factor names chosen by the authors effectively describe the
grouped variables.

* **Examples in Research**
* **Power Sources Study (Koslowsky et al., 2001)**: In a study of 232 nurses,
researchers conducted a factor analysis of 11 power sources. The analysis yielded a
"two-factor solution," with the first factor explaining 41.9% of the variance and the
second explaining 15.5%. For instance, "Impersonal Coercion" had a high loading
(.81) on the first factor, while "Expertise" had a high loading (.78) on the second.
Some items, like "Legitimate Position," showed moderate loadings on both factors,
indicating they were not as clearly part of one group. The researchers then named
these factors based on the variables that clustered within them.
* **PTSD Symptoms Study (Fawzi et al., 1997)**: This study applied factor
analysis to 16 PTSD symptom ratings from Vietnamese refugees, resulting in four
factors. Three of these factors corresponded to the typical DSM-IV dimensions of
arousal, avoidance, and reexperiencing. However, the analysis notably separated
"avoidance" into two distinct factors, suggesting an additional and somewhat
separate aspect of avoidance not previously considered in diagnostic criteria.

* **Distinction from Factorial Analysis of Variance**

* It is important to note that **Factor Analysis** (a technique for grouping
variables) is distinct from **Factorial Analysis of Variance (ANOVA)**. While
Factorial ANOVA uses the term "factor" to refer to independent (or treatment)
variables in an experimental design, it is a method for comparing group means and
testing interactions between these independent variables, rather than identifying
underlying dimensions within a set of measures. Factor analysis, on the other hand,
is a technique for data reduction and revealing latent constructs from observed
correlations.

Based on the provided sources, I can provide information on **path analysis**.

However, the sources do not contain specific information about **cluster
analysis**.

Here is what the sources say about path analysis:

**Path Analysis**

Path analysis is a method of **causal modeling**, an advanced statistical

procedure used when you have measured people on a number of variables [Source
152, 153]. It is considered an elaboration of what you learned about correlation and
prediction [Source 140].

* **Purpose and Core Concept**:

* The goal of causal modeling, including path analysis, is to **test whether the
pattern of correlations among variables in a sample fits with some specific theory
about which variables are causing which** [Source 152, 155, 179]. This helps
establish relationships that enable prediction [Source 366].
* Path analysis is an "older (but still common) method" of causal modeling
[Source 153].

* **How it Works**:
 * In path analysis, you create a **diagram with arrows connecting the
variables** [Source 153, 154, 160]. Each arrow, or "path," represents a causal
effect that the researcher predicts between variables [Source 153, 154, 160].
* Based on the correlations of these variables in a sample and the predicted
path diagram, you can calculate **path coefficients** for each path [Source 153,
154, 160].
* A path coefficient is similar to a **standardized regression coefficient
(beta)** in multiple regression [Source 153, 154, 179]. It tells you how much of a
standard deviation change in the variable at the end of the arrow is produced by a
one standard deviation change in the predictor variable at the start of the arrow,
assuming the path diagram correctly describes the causal relationship [Source 153,
160].
* Path coefficients are also similar to **partial correlations** because they are
calculated to "partial out" (or control for) the influence of any other variables that
have arrows leading to the same criterion (effect) variable [Source 153, 154, 209].
* After computing the path coefficients, you check whether they are all in the
predicted direction and are statistically significant [Source 160]. Researchers often
only include significant paths in their diagrams [Source 156].

* **Relationship to Other Statistical Techniques**:

* Path analysis is described as an elaboration of correlation and regression
[Source 140, 153].
* It uses standardized regression coefficients (betas) and is related to partial
correlation [Source 153, 154, 209].

* **Specific Application: Mediational Analysis**:

* **Mediational analysis** is a widely used contemporary application of
"ordinary path analysis" [Source 153, 156, 158].
* Its purpose is to examine whether a presumed causal relationship between
two variables (X, the cause, and Y, the effect) is due to a particular intervening
variable, known as a **mediator variable (M)** [Source 158, 161, 179]. The
mediator variable explains the causal relationship [Source 158, 179].
* The typical steps for a mediational analysis are:
1. Show that variable X significantly predicts variable Y [Source 161].
2. Show that X significantly predicts variable M [Source 161].
 3. Show that M predicts Y in a multiple regression where X is also included
as a predictor [Source 161].
4. Show that, when M is included alongside X as a predictor of Y, X no
longer significantly predicts Y (for full mediation) or predicts it less strongly (for
partial mediation) [Source 161].
* Identifying these underlying causes is considered extremely important in
scientific research [Source 158].

* **Limitations**:
* Causal modeling methods, like path analysis, rely on correlations, and thus
are subject to the same cautions as other correlation-based techniques (from
Chapters 11 and 12) [Source 159].
* The most important caution is that **correlation does not demonstrate the
direction of causality** [Source 159].
* These techniques primarily take **linear relationships** directly into account
[Source 159].
* Results can be distorted (typically towards smaller path coefficients) if there
is a **restriction in range** in the data [Source 159].

* **Examples in Research Articles**:

* A study by Williamson, Shaffer, and The Family Relationships in Late Life
Project (2001) used path analysis to predict potentially harmful caregiver behavior,
showing significant pathways in a diagram [Source 156, 157].
* MacKinnon-Lewis and colleagues (1997) used causal modeling to examine
predictors of social acceptance in boys, testing several possible models [Source
185, 194].
* DeGarmo and Forgatch (1997) also used structural equation modeling (a type
of path analysis) to examine relationships among variables in a study of social
support for divorced mothers [Source 194, 195].
* Aron and colleagues (2000, Study 1) tested a mediational relationship
between shared participation in novel-arousing activities, relationship boredom,
and relationship quality in married couples [Source 187, 205].

**Regarding Cluster Analysis**

The provided sources do not offer a definition or explanation of "cluster analysis"
as a statistical topic. The term "clumps" is used in the context of factor analysis to
describe variables that are highly correlated with each other [Source 1]. While
factor analysis aims to identify underlying patterns or "clumps" among a large
number of measured variables by grouping variables together [Source 1, 155], this
is distinct from "cluster analysis," which typically groups *cases* or
*observations* rather than variables. The sources do not elaborate on the latter.