Week 03: Regression & Correlation
The Super-Simple, Picture-Filled Guide to Scatterplots, Correlation, and Best-Fit Lines
Welcome to Week 03 Statistics!
In Week 02, we looked at one variable at a time (like how tall babies are). In Week 03, we explore two variables together! We
want to see if changing one thing (like powerboat registrations) causes or connects to changes in another thing (like manatee
deaths). Think of this as discovering picture patterns when connecting the dots!
1. Scatterplots & Quantitative Relationships
When we want to see if two quantitative variables (numbers like height, weight, boats, or temperature) are related,
our very first step is to draw a scatterplot. A scatterplot displays every individual point as a dot on a grid.
Variable Type Which Axis? Simple Definition Everyday Example
Explanatory Horizontal Axis The input or cause that explains Number of powerboats registered
Variable (x) (x) changes. in Florida.
Response Number of manatees killed by
Variable (y) Vertical Axis (y) The output or outcome being measured. boats.
The 4 Questions to Ask About Any Scatterplot:
• 1. Direction (Association): Is it Positive (as x goes UP, y goes UP), Negative (as x goes UP, y goes DOWN), or
None?
• 2. Form (Shape): Is the average pattern a Linear straight line or a Curved bend?
• 3. Strength: How closely do the dots cluster around the average line? (Little spread = Strong, Lots of scatter =
Weak).
• 4. Unusual Points (Outliers): Are there stray dots sitting outside the general 'envelope' of the trend?
Monash University — SCI1020 Page 1 Week 03 Study Notes
Figure 1: Scatterplot of Florida Powerboat Registrations vs. Manatee Deaths (1977–2006). Shows a strong, positive linear
association.
2. The Correlation Coefficient (r)
Looking at dots is great, but statisticians love exact numbers! The correlation coefficient (r) is a single number that
measures the strength and direction of a linear relationship between two quantitative variables.
The Correlation Formula (In Plain English)
Formula: r = (1 / (n - 1)) × Σ [ ( (x_i - mean_x) / s_x ) × ( (y_i - mean_y) / s_y ) ]
Baby Translation: For each point, we convert x and y into z-scores (how many standard deviations above or below average
they are), multiply them together, add them up, and average them! If x and y are usually both positive or both negative together, r
becomes positive!
Property What It Means in Baby Terms Why It Matters
Always between -1 and r = +1 is a perfect upward line; r = -1 is a perfect Tells you both strength and direction
+1 downward line; r = 0 is complete chaos. instantly.
r has no units! If you measure boats in thousands You can compare correlations across
Unitless Number or single boats, r is identical. completely different studies!
It measures mutual association, not
Symmetric (x and y) Swapping x and y gives the exact same r value. who causes whom.
r ONLY measures straight-line trends. It gets
Linear Sensitivity Only fooled by curves! Always draw a scatterplot first!
Monash University — SCI1020 Page 1 Week 03 Study Notes
Figure 2: Scatterplots across the correlation spectrum from r = +1.0 (perfect positive) down to r = -1.0 (perfect negative).
Caution #1: Anscombe's Quartet (Always Plot Your Data!)
In 1973, statistician Francis Anscombe created four famous datasets. All 4 datasets have the exact same mean of x
(9.0), mean of y (7.5), standard deviation, linear regression line (y = 3 + 0.5x), and correlation coefficient (r =
0.816)! However, when you plot them, they look completely different! One is straight linear, one is a smooth curve,
one has a single outlier, and one is a vertical line with one extreme point.
Monash University — SCI1020 Page 1 Week 03 Study Notes
Figure 3: Anscombe's Quartet — Four vastly different scatterplots that share the exact same r = 0.816 and regression line!
3. The Linear Regression Line
Once we confirm a scatterplot has a linear pattern, we want to draw the line of best fit (called the regression line).
This line summarizes the relationship and allows us to make predictions for new values of x!
The Linear Regression Equation
Equation: ŷ = a + bx (or ŷ = b0 + b1 x)
• ŷ (y-hat): The predicted value of y for a given x.
• b (Slope): How much ŷ changes when x increases by 1 unit. Formula: b = r × (s_y / s_x)
• a (Intercept): The predicted value of y when x = 0. Formula: a = mean_y - b × mean_x
Residuals & The Method of Least Squares:
No regression line goes perfectly through every point. For each data point, the vertical distance between the actual
observed point (y) and the line's predicted point (ŷ) is called a residual (e):
Residual (e) = Observed y - Predicted ŷ = y - ŷ
The Method of Least Squares finds the unique line that makes the sum of squared residuals (SSR = Σ e²) as small
as humanly possible!
Monash University — SCI1020 Page 1 Week 03 Study Notes
Figure 4: Anatomy of Linear Regression showing observed points, predicted line, intercept, slope, and vertical residual lines.
4. R-squared (R²) & Model Evaluation
How well does our straight line actually explain the real world? We measure this using R-squared (R²), also known
as the coefficient of determination.
What does R² tell us?
Formula: R² = (r)²
Definition: R² is the percentage of variation in response variable y that is explained by the linear regression model with x.
Example: In our Florida manatee data, r = 0.941. So R² = (0.941)² = 0.885 (or 88.5%). This means 88.5% of the year-to-year
variation in manatee deaths is explained by the number of powerboats! The remaining 11.5% is due to natural randomness,
water temperature, disease, etc.
Residual Plots: Checking if a Straight Line is Appropriate
A high R² does NOT guarantee a straight line is correct! To verify if a linear model is appropriate, we must look at a
residual plot (plotting x on the horizontal axis and residual e on the vertical axis):
• Good Residual Plot: Random scatter around zero with no clear shape or pattern.
• Bad Residual Plot: A curved (U-shaped) pattern or expanding funnel shape. This signals a non-linear relationship!
Monash University — SCI1020 Page 1 Week 03 Study Notes
Figure 5: Residual Plot comparison — Random scatter (left) confirms a linear model; a curved U-shape pattern (right) rejects a
linear model.
Major Cautions in Regression:
Caution #1: Beware of Extrapolation!
Extrapolation means predicting y for x-values far outside the range of your original dataset. For example, using our Florida boat
model (which measured 400 to 1000 thousand boats) to predict manatee deaths for 5,000 thousand boats is dangerous because
the trend may curve, flatten out, or change completely!
Caution #2: Correlation DOES NOT Equal Causation (Lurking Variables)!
A strong correlation between x and y does NOT mean x causes y! Often, a hidden lurking variable (z) influences both x and y
simultaneously.
Classic Example: There is a super strong correlation (r = 0.98) between the number of cell phones and the number of prison
inmates! Does buying a cell phone land you in jail? No! The lurking variable is Time / Population Growth over the decades,
which increased both numbers independently!
Monash University — SCI1020 Page 1 Week 03 Study Notes
Figure 6: Correlation vs. Causation — Cell Phone Subscribers vs. Prison Inmates driven by the lurking variable of general
population growth over time.
Summary Cheat Sheet: Week 03 Formula & Concept Map
Concept / Symbol Formula / Value Range Key Definition in 5 Words
Explanatory Variable (x) Horizontal Axis The input / predictor variable
Response Variable (y) Vertical Axis The outcome being measured
Correlation (r) -1.0 ≤ r ≤ +1.0 Strength & direction of line
Regression Line ŷ = a + bx Line minimizing sum of squared errors
Slope (b) b = r × (s_y / s_x) Change in ŷ per +1 x
Intercept (a) a = mean_y - b × mean_x Value of ŷ when x = 0
Residual (e) e=y-ŷ Observed value minus predicted value
R-squared (R²) R² = r² (0% to 100%) % variation in y explained
Residual Plot Test Random scatter around 0 Confirms if linear model fits
Monash University — SCI1020 Page 1 Week 03 Study Notes