Unit -3

Forecasting Numeric Data –

Regression Methods
• Mathematical relationships help us to
understand many aspects of everyday life. For
example, body weight is a function of one's
calorie intake, income is often related to
education and job experience, and poll
numbers help us estimate a presidential
candidate's odds of being re-elected.
• When such relationships are expressed with
exact numbers, we gain additional clarity. For
example, an additional 250 kilocalories
consumed daily may result in nearly a
kilogram of weight gain per month; each year
of job experience may be worth an additional
$1,000 in yearly salary; and a president is
more likely to be re-elected when the
economy is strong.
 Regression
• Regression is concerned with specifying the
relationship between a single numeric dependent
variable (the value to be predicted) and one or
more numeric independent variables (the
predictors). As the name implies, the dependent
variable depends upon the value of the independent
variable or variables. The simplest forms of
regression assume that the relationship between the
independent and dependent variables follows a
straight line.
• You might recall from basic algebra that lines can be
defined in a slope-intercept form similar to y = a +
bx. In this form, the letter y indicates the dependent
variable and x indicates the independent variable.
The slope term b specifies how much the line rises
for each increase in x. Positive values define lines that
slope upward while negative values define lines that
slope downward. The term a is known as the
intercept because it specifies the point where the
line crosses, or intercepts, the vertical y axis.
Regression analysis is commonly used for modeling
complex relationships among data elements, estimating
the impact of a treatment on an outcome, and
extrapolating into the future. Some uses are:
• Examining how populations and individuals vary by their
measured characteristics, for use in scientific research
across fields as diverse as economics, sociology,
psychology, physics, and ecology
• Quantifying the causal relationship between an event
and the response, such as those in clinical drug trials,
engineering safety tests, or marketing research
• Identifying patterns that can be used to
forecast future behavior given known criteria,
such as predicting insurance claims, natural
disaster damage, election results, and crime
rates
 Simple Linear Regression
• On January 28, 1986, seven crew members of the United States
space shuttle Challenger were killed when a rocket booster
failed. In the aftermath, experts focused on the launch
temperature as a potential culprit. The rubber O-rings
responsible for sealing the rocket joints had never been tested
below 40ºF (4ºC) and the weather on the launch day was
unusually cold and below freezing. With the benefit of hindsight,
the accident has been a case study for the importance of data
analysis and visualization. Although it is unclear what
information was available to the rocket engineers and decision
makers leading up to the launch, it is undeniable that better
data, utilized carefully, might very well have averted this disaster.
• The rocket engineers almost certainly knew that cold
temperatures could make the components more
brittle and less able to seal properly, which would
result in a higher chance of a dangerous fuel leak.
However, given the political pressure to continue
with the launch, they needed data to support this
hypothesis. To build the regression model, scientists
might have used the data on launch temperature and
component distresses from 23 previous successful
shuttle launches.
• Since the shuttle has a total of six primary O-rings,
up to six distresses can occur per flight. Though the
rocket can survive one or more distress events, or
fail with as few as one, each additional distress
increases the probability of a catastrophic failure.
• The following scatterplot shows a plot of primary
O-ring distresses detected for the previous 23
launches, as compared to the temperature at
launch:
• Examining the plot, there is an apparent trend. Launches
occurring at higher temperatures tend to have fewer O-ring
distress events. Additionally, the coldest launch (53º F) had
two distress events, a level which had only been reached in
one other launch. With this information at hand, the fact
that the Challenger was scheduled to launch at a
temperature over 20 degrees colder seems concerning.
• Suppose we know that the estimated regression
parameters in the equation for the shuttle launch data are:
a = 3.70 and b = -0.048. Hence, the full linear equation is y
= 3.70 – 0.048x.
• As the line shows, at 60 degrees Fahrenheit, we predict
just under one O-ring distress. At 70 degrees Fahrenheit,
we expect around 0.3 failures. If we extrapolate our
model, all the way to 31 degrees—the forecasted
temperature for the Challenger launch—we would
expect about 3.70 - 0.048 * 31 = 2.21 O-ring distress
events. Assuming that each O-ring failure is equally likely
to cause a catastrophic fuel leak means that the
Challenger launch at 31 degrees was nearly three times
more risky than the typical launch at 60 degrees, and
over eight times more risky than a launch at 70 degrees.
 Ordinary Least Square Estimation
• In order to determine the optimal estimates of α
and β, an estimation method known as Ordinary
Least Squares (OLS) was used. In OLS regression,
the slope and intercept are chosen so that they
minimize the sum of the squared errors, that is,
the vertical distance between the predicted y
value and the actual y value. These errors are
known as residuals, and are illustrated for
several points in the following diagram:
• In mathematical terms, the goal of OLS
regression can be expressed as the task of
minimizing the following equation:
• In plain language, this equation defines e (the
error) as the difference between the actual y
value and the predicted y value. The error
values are squared and summed across all the
points in the data.
 Correlation
• The correlation between two variables is a number
that indicates how closely their relationship follows
a straight line. Without additional qualification,
correlation typically refers to Pearson's correlation
coefficient, which was developed by the 20th
century mathematician Karl Pearson. The correlation
ranges between -1 and +1. The extreme values
indicate a perfectly linear relationship, while a
correlation close to zero indicates the absence of a
linear relationship.
 Multiple linear regression
• Most real-world analyses have more than one
independent variable. Therefore, it is likely
that you will be using multiple linear
regression for most numeric prediction tasks.
The strengths and weaknesses of multiple
linear regression are shown in the following
table:
• It is the most common form of Linear
Regression. Multiple Linear Regression
basically describes how a single response
variable Y depends linearly on a number of
predictor variables.
• The basic examples where Multiple Regression
can be used are as follows:
• The selling price of a house can depend on the
desirability of the location, the number of
• bedrooms, the number of bathrooms, the year
the house was built, the square footage of the
lot and a number of other factors.
• The height of a child can depend on the height
of the mother, the height of the father,
nutrition, and environmental factors.
 Polynomial Regression
• Polynomial regression is a special case of Linear
regression where the relationship between X and
Y is modeled using a polynomial, rather than a
line.
• Polynomial Linear Regression is similar to the
Multiple Linear Regression but the difference is, in
Multiple Linear Regression the variables are
different whereas in Polynomial Linear Regression,
we have the same variable but it is in a different
power.