What is Analytics?
• The science of using data to build models that lead to
better decisions that add value to individuals, to companies, to
institutions
• Descriptive analytics: identify patterns in the data
• Summary statistics
• Hypothesis Test
• Visualizations
• Clustering
• Predictive analytics: predict different outcomes
• Linear Regression
• Logistic Regression, CART, Random Forests
• Prescriptive analytics: give advice on actions to take
2
Bordeaux wine
▪ Large differences in price and quality between years, although wine is produced in a
similar way
▪ Meant to be aged, so hard to tell if wine will be good when it is on the market
▪ Expert tasters predict which ones will be good
▪ Can analytics be used to come up with a different system for judging wine?
3
Predicting the quality of wine
▪ March 1990 - Orley Ashenfelter, a Princeton
economics professor, claims he can predict wine
quality without tasting the wine 4
Building a model
▪ Ashenfelter used a method called linear regression
– Predicts an outcome variable, or dependent variable
– Predicts using a set of independent variables
5
Building a model
▪ Dependent variable:
– Typical price in 1990-1991 wine auctions (approximates
quality)
– Conduct logarithmic transformation
❑ A better linear fit
▪ Independent variables:
– Age of wine (in 1990)
❑ Older wines are more expensive
– Weather
❑ Average Growing Season Temperature (AGST)
❑ Harvest Rain
❑ Winter Rain
– Population of France
6
The wine data (1952 - 1978)
7
The wine data (1952 - 1978)
Quick Question: What is the relationship between harvest rain, average
growing season temperature, and wine prices?
8
Baseline model (?)
9
Baseline model (Take the mean)
10
One-Variable Linear Regression
11
Simple Regression Model
The population model of y with one predictor variable x is:
𝑦 = 𝛽0 + 𝛽1 𝑥 + ε
▪ y is the dependent variable (DV)
▪ x is the independent variable (IV)
▪ Regression Function
▪ E[y|x] = 𝛽0 + 𝛽1 𝑥 is the mean of y given x
▪ 𝛽0 is the y-intercept (value of E[Y|0] when x=0)
▪ 𝛽1 is the slope for x, which is the change in E[Y|x] for a unit increase in x
▪ Random errors (not required)
Random samples are i.i.d. or
▪ Random errors are a random sample from 𝑁 0, 𝜎𝜀 independent and identically
▪ Each observation has a random error distributed random variables
▪ The output does not show these, but it does estimate
▪ The random errors and IV (X) are uncorrelated
▪ These assumptions are important for effective business analytics
12
Estimated Regression Function
▪ Estimates the regression model with n observations (xi,yi) for i = 1, …, n
▪ The estimated or predicted value of y given x is:
𝑦ො = 𝑏0 + 𝑏1 𝑥
▪ 𝑏0 is the sample estimate of the population intercept 𝛽0
▪ 𝑏1 is the sample estimate of the population slope 𝛽1
𝑏0 and 𝑏1 are sample statistics
ത
(similar to 𝑋)
and have sampling distributions
13
One-Variable Linear Regression
Interpretation: A one-unit increase in x (e.g., 1 year) is on average
associated with a b1 unit change in y (e.g., dollar).
Not necessarily
Causality!
14
Data and Predicted Values
▪ What is the observed y when x = 1?
▪ What is the predicted y when x = 1?
▪ What is the observed y when x = 4?
▪ What is the predicted y when x = 4?
15
Data and Predicted Values
▪ What is the observed y when x = 1?
y=6
▪ What is the predicted y when x = 1?
𝑦=1+(2)(1)
ො =3
▪ What is the observed y when x = 4?
y=4
▪ What is the predicted y when x = 4?
𝑦=1+(2)(4)
ො =9
16
Estimated Model and Residuals
▪ Residuals are the difference between the observed values of 𝑦 and predicted values of 𝑦ො
– 𝑟 = y - 𝑦ො
– Each observation has one observed y, one predicted 𝑦,
ො and one residual r.
▪ The residuals are errors between the observed and predicted values.
y3
𝑦ො4
𝑟3 = 𝑦3 − 𝑦ො3
y1
𝑦ො2 𝑟4 = 𝑦4 − 𝑦ො4
𝑦ො3
𝑟1 = 𝑦1 − 𝑦ො1
y4
𝑦ො1 y2
𝑟2 = 𝑦2 − 𝑦ො2
17
Computing Residuals
r3
r4
r1 r2
▪ What is the residual r2 at x = 2?
▪ What is the residual r3 at x = 3?
18
Computing Residuals
r3
r4
r1 r2
▪ What is the residual r2 at x = 2?
𝑟2 = 𝑦2 − 𝑦ො2 = 3 − 1 + 2 ∗ 2 = 3 − 5 = −2
▪ What is the residual r3 at x = 3?
𝑟3 = 𝑦3 − 𝑦ො3 = 11 − 1 + 2 ∗ 3 = 11 − 7 = 4
19
Boston Dataset: Prediction and Prediction Interval
A prediction interval (PI) is a range that contains the actual value of Y for a new
individual observation with a stated confidence level (e.g., 95%).
20
Ordinary Least Squares Criterion or (OLS)
The least squares line finds the estimates 𝑏0 and 𝑏1 of the coefficients to
minimize the sums-of-squares error for a sample {(xi,yi)} with n observations:
𝑆𝑆𝐸 = σ𝑛𝑖=1 𝑦𝑖 − 𝑦ො𝑖 2
𝑦ො𝑖 = 𝑏0 + 𝑏1 𝑥𝑖 for 𝑖 = 1, … , 𝑛
Why squared?
The sum of residuals could be zero.
𝑛 Less variant when n bigger
2
𝑆𝑆𝐸(𝑏0 , 𝑏1 ) = 𝑦𝑖 − 𝑏0 − 𝑏1 𝑥𝑖
σ𝑛𝑖=1 𝑥𝑖 − 𝑥ҧ 𝑦𝑖 − 𝑦ത unbiased
𝑖=1 𝑏1 =
𝜕𝑆𝑆𝐸(𝑏0 , 𝑏1 ) σ𝑛𝑖=1 𝑥𝑖 − 𝑥ҧ 2
= 0 𝑏0 = 𝑦ത − 𝑏1 𝑥ҧ
𝜕𝑏0
𝜕𝑆𝑆𝐸(𝑏0 , 𝑏1 ) 𝑥:ҧ sample average of independent variable
= 0
𝜕𝑏1 ത sample average of dependent variable
𝑦:
Do not need to calculate or memorize.
21
Prediction and Confidence Interval
If we repeatedly drew new samples and rebuilt the interval
the same way, about 95% of those intervals would contain
the true mean response
22
Estimate a linear model (OneH Variable
: AGST Coefficient)= 0 versus H : AGST
0 A
Coefficient ≠ 0
Estimated Standard Errors t-score = (Estimated Coefficient – 0)/(Standard Error)
intercept and for estimated
slope coefficients
Two-Tail Test: p-value = 2*P(T<-|t-score|)
Coefficient of Determination: R-Squared
23
Estimate a linear model (One Variable )
• Estimated model for price:
ො -3.4178 + 0.6351*AGST
𝑦=
• The predicted LogPrice increases by
0.6351 for every 1 degree increase in
average growing season temperature.
• If AGST = 15, then 𝑦=ො ?
• If AGST = 18, then 𝑦=
ො ?
• If AGST = 20, then 𝑦=
ො ?
24
One-Variable Linear Regression
ො -3.4178 + 0.6351*AGST
𝑦=
25
Estimate a linear model (One Variable )
Why my estimates are random variables?
Estimated Standard Errors
intercept and for estimated - The observations are regarded as a
random sample from a large population.
slope coefficients - Because the sample is randomly sampled,
the estimation is also random.
σ𝑛𝑖=1 𝑥𝑖 − 𝑥ҧ 𝑦𝑖 − 𝑦ത
𝑏1 =
σ𝑛𝑖=1 𝑥𝑖 − 𝑥ҧ 2
𝑏0 = 𝑦ത − 𝑏1 𝑥ҧ
𝑥:ҧ sample average of independent variable
ത sample average of dependent variable
𝑦:
𝑏0 and 𝑏1 are random variables that
follows t-distribution.
26
T-Tests for the Coefficients: H0: j = 0 versus HA: j ≠ 0
Two-Tail Test for the Slope
(Very important. Can you predict Y from X?)
H0: 1 = 0 versus HA: 1 ≠ 0
• t-score = (coefficient – 0)/([Link])
• t-score = (0.6351-0)/0.1509 = 4.208
• p_value = 2*P(T < -|4.208|)
=2*[Link](-4.208, 23, 1) < .001
• df = n-1-#IV = df Error under Sum of squares
df = 23
27
How well the model fits data
▪ The simplest commonly used measure of fit is R2 (the
coefficient of determination): R2 = 1 − SSE/SST
– SSE = σn i=1 yi − y ො i 2 : sum of squared errors
❑ Variation of Y that cannot be explained by the regression
– SST = σn i=1 yi − yത 2 : total sum of squares
❑ Total amount of variation of Y around its mean
❑ “Error” generated by a baseline model without any inputs
– Decomposition of variation of Y:
𝑛
❑ σ ത 2 = σ𝑛𝑖=1 𝑦𝑖 − 𝑦ො𝑖 2 + σ𝑛𝑖=1 𝑦ො𝑖 − 𝑦ത
𝑖=1 𝑦𝑖 − 𝑦
2
Total variation Unexplained Explained variation
(SST) variation (SSE)
R2 =(SST-SSE)/SST =Proportion of the variance in DV is
explained by the regression model.
28
Coefficient of Determination: R-Squared
• R-Squared is a measure of fit
• Bigger R-Squared indicates better fit all
else being equal
• 43.5% of the variation of prices is
explained by the simple regression on
AGST
• 0 < R-Squared < 1
29
Use each variable on its own
▪ R2 =0.44 using Average growing season temperature (Variable
Significant, 0.001)
▪ R2 =0.32 using Harvest rain (Variable Significant, 0.01)
▪ R2 =0.22 using France Population (Variable Significant, 0.05)
▪ R2 =0.20 using Age (Variable Significant, 0.05)
▪ R2 =0.02 using Winter rain (Not Significant)
▪ Multivariate linear regression allows us to use more than one
variable to potentially improve our predictive ability.
Questions: Does high R-squared imply causality?
Does low R-squared imply that iv is not important?
30
Next Class
▪ Read R installation instructions on Moodle
▪ Install R and R studio following instructions
▪ (Recommended) Bring your laptop to class on Friday
31