Predictive Methods: Chapter 3

Predictors
Christophe Muller
December 2024

Aix-Marseille University, AMSE, France. [Link]@[Link]

Contents
1 Statistical Decision Theory 2
1.1 Optimal parametric prediction . . . . . . . . . . . . . . . . . . . 2
1.2 Decision theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Optimality of conditional expectation . . . . . . . . . . . . . . . 6

2 Linear Predictors 7

3 Evaluating Predictive Power 9

3.1 Confidence intervals of forecasts . . . . . . . . . . . . . . . . . . . 9
3.2 Gauss-Markov theorem . . . . . . . . . . . . . . . . . . . . . . . . 11
3.3 Model instability . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.4 Measuring prediction performance . . . . . . . . . . . . . . . . . 12

4 Identification 15

5 Nonlinear Predictors 17
5.1 The Tobit model . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
5.2 Conditional moments . . . . . . . . . . . . . . . . . . . . . . . . . 18
5.3 Impacts of changes in independent variables . . . . . . . . . . . . 19
5.4 Selectivity Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
5.5 Proportionality of the effects . . . . . . . . . . . . . . . . . . . . 21
5.6 Heterogeneity of the effects . . . . . . . . . . . . . . . . . . . . . 22

6 Simulations 24
6.1 Analysis effects of variable by using simulations . . . . . . . . . . 24
6.2 Policy prediction using simulations . . . . . . . . . . . . . . . . . 24

1
1 Statistical Decision Theory

The methods of prediction are encompassed in statistical inference. Diverse

inference approaches are possible that are related to different risk functions,
as described below. They all correspond to using information about a certain
population and periods to infer something about the distribution of another
population (prediction), or another period (forecast).

1.1 Optimal parametric prediction

Definitions: For parametric models:

(i) A statistical loss function L(., .) is a mapping from D × Θ onto R+ ,

where D is a set of decision rules (estimators, tests, confidence intervals) and
Θ a set of parameters that define probability functions.

The two sets in the definition of the loss function are sometimes associated
each with their own σ -fields to better specify their random dimensions, and
justify applying to them stochastic operators like the expectation. Although,
we favor the use of loss functions for choosing predictors, these functions are
typically used to choose: point and set estimators, tests, confidence interval
estimators, and models within a set of models.

(ii) A statistical risk function R(., .) is a mapping from D × Θ onto R+

defined by

R(d, Pβ ) = E P [L (d (Y, X, Z) , Pβ )]

for any d ∈ D and any Pβ probability function defined by β ∈ Θ.

(iii) A Best Predictor , p∗ , which depends only on a random vector X, for a

random variable Y and a loss function L(.), is the solution to the minimization
program

min E(L(Y − p(X)) | X).

p(.)

(iv) If the predictor p is parameterized as p = h(X, β) with a parameter

vector β ∈ Θ, and h a given real-valued function, then the parameter β0 ∈ Θ
of the best parameterized predictor is obtained from :

2
 β = Arg min E(L(Y − h(X, β)) | x) = Arg min EY,X (L(Y − h(X, β))) .
β∈Θ β∈Θ
m.d. first min depend on x

(v) Empirically, this corresponds to the M-estimator of β:

n
1X
β̂ = Arg min L(yi − h(xi , β)).
β∈Θ n
i=1

(vi) The model is said to be misspecified when the parameter set Θ is such
that there is no parameter value β0 ∈ Θ such that h(x, β0 ) is the best predictor
for all X.

When the loss function L and the parameterized function h are twice differ-
entiable, the best predictor seen as an M-estimator will have usual asymptotic
properties.

1.2 Decision theory

IIUBeyond the above fundamental definitions, if we want to design loss func-

tions adapted to the prediction and forecast problems, we should first provide
a quick review of the general statistical decision theories. These definitions just
below can be skipped in a first reading.

Definition:
(i)IIU h : (Θ, σT heta → (D, σD ) a measurable function, with respective
sigma-fields σT heta and σD
is said to be a good decision function for the loss function L if and only if
for all β ∈ Θ, d = h(β) is equivalent to L(d, β) = 0.

(ii) Let be a statistical model (X, A, Pθ ; θ ∈ Θ)

and a measurable space of decisions (D, σD ), a pure decision rule is a mea-
surable function linking observations in X with decisions from D :

d : (X, A) → (D, σD ).

(iii) The set of pure decision rules is denoted: ∆.

The pure decision rule d is defined before the observations are observed.
When the latter come from a random variable X, one can denote it d(X)..

3
In that case, d(X) is random and its probability distribution depends on the
probability distribution of X. Once the observations x are available, the ex post
pure decision is d(x), which is a number in the case of predictions.
Different types of statistical decision problems can be envisaged: point or
set estimations, tests and model selections, predictions, etc. For predictions,
the pure decision rule is called a predictor, and its empirical result a prediction.
The name forecast, reserved for predictions about the future, is often employed
both for the rule and its result.
For a given economic problem, a given loss function and a given set of prob-
ability functions associated with the random space that represent the observa-
tions, one searches for the best decision rules, in our case, the best predictors.
However, optimal decision making with pure decisions is often hard. Instead,
one may try to use probabilities over diverse pure decisions.

Definition

(i) Let be a statistical model

(X, A, Pβ ; β ∈ Θ)
and a measurable space of decisions (D, σD ), a mixed decision rule m is a
measurable function linking observations with probabilities on decisions:

m : (X, A) → (PD , σPD ),

where PD is a set of probability functions over the set of decision rules D,

and σPD is an associated sigma-algebra of events.
For any empirical result x , one gets a probability function, mx , that yields
the probability of selecting each pure decision in D . This choice must be con-
sistent with the definition of the events of σD .

(ii) The set of mixed decision rules is denoted: ∆M .

An issue is that the true probability distribution, Pβ0 , of the observations

is unknown. Therefore, the good decisions cannot be reached. Moreover, the
loss function L applied to d(X) is a random variable, which cannot be opti-
mized. This implies that to find optimal rules one uses instead the following
risk functions instead of the loss functions.

Definition

R D ×Θ to ℜ+
(i) A risk function for pure decision rules d is a function from
, such that for all d ∈ D, β ∈ Θ, R(d, β) = EPβ L(d(x), β) = X L(d(x), β)dPβ (x).

(ii) d1 is preferred to d1 if and only if for all β ∈ Θ, R(d1 , β) ≤ R(d2 , β).

(iii) A risk function for mixed decision rules is a function from ∆M ×Θ to ℜ+ ,

such that for all m ∈ ∆M , β ∈ Θ,

4
 R R
R(m, β) = X D
L(d, β)dmx (d)dPβ (x).

(iv) Let be a function g from Θ to ℜ+ the quadratic risk considered for the
estimation of g(β) is:

R(d, β) = EPβ (d(x), g(β))2 .

(v) A Bayes risk function for a pure decision rule d, with a prior distribution
π over the measurable state (Θ, T), where T is a sigma-algebra associated to the
parameters in Θ , is:
R R
Rπ (d) = Θ X
L(d(x), β)dPβ (x)dπ(β).

(vi) A Bayes risk function for a mixed decision rule m is:

R R R
Rπ (m) = Θ X D
L(d, β) dmx (d) dPβ (x) dπ(β).

Bayesian risk ordering for mixed rules are total orderings. The other risk
function ordering are typically only partial orderings. However, they are still
useful to choose a rule. For this, one search for admissible rules in the following
sense:

Definition:

(i) A mixed decision rule m∗ is said admissible if there is no other rule m

such that for all β ∈ Θ, R(m, β) ≤ R(m, β),
with at least a strict inequality.

To choose a decision rule, one restricts the rules to consider by using statis-
tical principles. For example, one uses a Bayes rule, that is: to minimize the
Bayes risk. Another principle is to choose a minimax rule dM that protects at
best against the worst risk values:

sup inf sup

β∈Θ R(dM , β) =d∈∆ β∈Θ R(d, β).

Another principle is to restrict the rules to consider to unbiased rules exclu-

sively. Another principle is to consider invariant rules to changes in one-to-one
transformations of parameters. Another principle is to consider functions of
sufficient statistics. Several of these principles can be combined.

to add something on sampling theory loss functions

5
1.3 Optimality of conditional expectation

Let Y and X be random vectors on the same probability spaces.

Proposition

If the mean of square of errors (MSE) is used as a loss function for a pre-
diction of Y based on observed data X , then the expectation E(Y |X) is the
unique optimal predictor among square integrable and twice differentiable func-
tions f (X).

Proof:
Y and f (X) can be random vectors, for example when a sample of size n is
observed.
The optimal predictor is a function f (.) of X , assumed square integrable
and twice derivable, that minimizes the quadratic loss:
′
min E[(Y − f (X)) (Y − f (X)])
f (.)

Using the law of iterated expectations to condition on X :

′

min EX [EY |X (Y − f (X)) (Y − f (X)) |X ]
f (.)

Therefore, a sufficient condition is, that for any x ∈ Rk

′
min EY |X [(Y − f (X)|X = x) (Y − f (X)|X = x)],
f (.)

which, by developing the product, is

min EY |X (Y ′ Y − Y ′ f (X) − f (X)′ Y + f (X)′ f (X)|X = x)
f (x)
which, noticing that Y ′ f (X) is scalar, eliminating terms not depending on
f , regrouping terms and dividing by two, is
min EY |X (−2Y ′ f (X) + f (X)′ f (X)|X = x)
f (.)
which is
′
min[−2EY |X (Y |X = x) f (X) + f (X)′ f (X)]
f (.)
discuter le calcul avec scalaire f(x) pluis integration par rapport a
la distribtion de X plus tard pour EY
The first-order conditions, by matrix derivation with respect to f (X) , are
therefore, in terms of gradients
−EY |X (Y ) + f (X) = 0n ,
which delivers the optimal solution

f ∗ (X) = E(Y |X),

since the second-order conditions are trivial in this case. QED.

6
2 Linear Predictors

Proposition:

When the real random vectors Y and X ≃ [X1 , ..., Xk ]′ are jointly Gaussian
in a sample of observations, then E(Y |X) is the linear regression of Y over
1, X1 , ..., Xk .

Proof:
Let LR(Y |X) denote the linear regression of Y over 1, X1 , ..., Xk , and u ≡
Y − LR(Y |X) be the corresponding residual.
Using the characterization of the linear regression as an orthogonal projec-
tion in an Euclidean space of random variables, we need to prove that for all
square integrable Borelian function h : Rn → R, E[h(X).u] = 0. av notation
et dimensions
We have already seen that, for such linear regressions, the error term satisfies:
E(u) = 0 and E(X.u) = 0, and therefore X and u are orthogonal.
Since (Y, X ′ )′ is Gaussian, the linear combination of components (X ′ , u)′ is
also Gaussian. In that case, since orthogonal Gaussian variables are indepen-
dent, X and u are independent. Therefore, this is also the case for any Borelian
function h of them, and
E[h(X).u] = E[h(X)].E(u) = 0 since E(u) = 0. QED.

An issue with this approach is that it does not distinguish the data used
to estimate the model, the data used for predictions, and the data to predict.
However, as a first stage of analysis, it allows us to ensure that the estimation
errors are minimized in terms of the quadratic loss.

For simplicity, or because the Gaussian hypothesis looks attractive, one may
want to assume that E(Y |X) is linear in the coefficients of the variables in X .
It is therefore easy to use it for predictions in that case.

The simplest linear model is:

yt = α + βxt + ϵt

where yt , xt and ϵt are random variables varying with time.

In practice, the realizations of the random variables yt , xt can be observed,
whereas ϵt remains unobserved. In addition, the error terms are assumed to
follow independently a centered normal distribution with variance σ 2 .
The parameters α, β and σ 2 must be estimated for the model to use for
predictions. Let us denote the OLS estimators of these parameters, respectively,
as: α̂ , β̂ and σ̂ 2 .

7
 In that case, a simple forecast formula for the next period, which we will
justify later on, is:

ŷt,1 = α̂ + β̂xt+1

A forecast formula after h periods is

ŷt,h = α̂ + β̂xt+h

The same formula can be used for non-time series observations, for example
if t is the index of individuals.

Note however, that this approach is based on assuming that the studied
context or economy keeps the same structure, that is that the used equations
for the predictions keep the same linear functional form with the same variables
and the same coefficients.

As we have seen before, the forecast with the smallest Mean Square Predic-
tion Error, E(Yt+1 − Ŷt,1 )2 , is the conditional expectation E(Yt+1 |Xt ). In the
case of a time series model, the predicting variables at t, and based on the last
observed m periods, Xt , are Xt = [1, Yt , Yt−1 , . . . , Yt−m+1 ]′ .
Moreover, it has been seen that, in the Gaussian case, one can restrict the
′
attention to forecasts that are linear in Xt , that is: Ŷt,1 = Xt+1 β , with β a
vector of parameters to estimate.

think In that case, the linear projection of Yt+1 on Xt , denoted P̂ (Yt+1 |Xt )
′
is the linear forecast solution to the equation: E((Yt+1 − Xt+1 β)′ Xt ) = 0k ,
where k is the number of columns of Xt . Remember that an intercept term has
been included in the definition of Xt here. verif dim
This linear projection reaches the smallest mean square projection error
among all the linear predictors of this type.
Therefore, in that case, the OLS formula applies
β = [E(Xt′ Xt )]−1 E(Xt′ Yt ).
Of course, this formulae apply for any other linear projectors based on other
definitions of Xt .
When the joint process (Yt+1 , Xt ) is covariance-stationary and ergodic for
the second
PT moments, then:
′
T
1
X
t=1 t Xt → P
E(Xt′ Xt )
andPT
1 ′ P ′
T t=1 Xt Yt → E(Xt Yt ).
Therefore,
β̂OLS →P β.

Generalizations to k predicting variables in X is straightforward: to do

Indeed, in most cases, several regressors are included in the prediction model.

8
 In particular, in the case of time series, examining the shape of their timeline
plots can suggest to include specific prediction components. Before to consider
particular explanatory variables, one often tries to deal with trends, seasonal
effects and cycles by using time variables.
For example: yt = a + bt + cSt + dyt−1 + ϵt ,
where St is a seasonal component, may look like a reasonable prediction
model.
It is a good idea to examine the timeline graph of the variable to predict
to detect these features. Trends correspond to a long term rise or decrease in
the plotted data. Seasonal patterns emerge when period factors are at play, for
example by season, month, week, or day of the week. They are characterized
by constant periodicity and relatively stable magnitude. Cycles rather describe
rises and falls in the timeline plot with no fixed period and often middle term
spans and variable magnitude. Investment cycles are important in economics,
and electoral cycles matter in politics.

3 Evaluating Predictive Power

3.1 Confidence intervals of forecasts

Predicting or forecasting is only useful if the predicting power is good enough.

There are diverse formulae to evaluate the predicting power of a predictor, while
aiming at centered and low variance of prediction errors is a good idea.
There are several stages in evaluating the predictive power of a predictor.
Firs, one measures the precision of the method with respect to the errors coming
from the estimation. Second, one measure the specific accuracy of the predic-
tion by mimicking out-of-sample observations by keeping aside a few observa-
tions that are predicted from the other ones. Third, one compares alternative
predicting models.
In our above example of the simplest linear model, the corresponding forecast
error, for the next period forecast, is:

et,1 = yt − ŷt,1 = (α̂ − α) + (β̂ − β)xt+1 + ϵt+1 .

Therefore, it combines:
- gaps between the estimated and true coefficients;
- observed predicting variables for future periods;
- an unobservable error term.

Under the classical OLS regression hypotheses, which we will remind us later,
the prediction error has a null expectation: Eet,1 = 0,

9
 and its variance is

2
σt,1 = E(e2t,1 ) = V ar(α̂) + V ar(β̂)x2t+1 + 2Cov(α̂, β̂)xt+1 + σˆε2 . iired

Since, under the hypothesis, E(ut |xt ) = 0, and with xt+1 non uniformly
equal to a constant, the prediction based on OLS is unbiased, then the risk
function MSE applied to the prediction problem for the next period reduces to
the variance of the prediction error. The variance of the prediction error of the
forecast ŷt,1 is
V (et,1 ) = V (yt+1 − ŷt,1 ) = V (yt+1 + V (α̂ + β̂xt+1 ) − 2Cov(yt+1 , α̂ + β̂xt+1 ) =
V (yt+1 + V (α̂ + β̂xt+1 ) ,
since Cov(yt+1 , α̂+ β̂xt+1 ) = 0 because the predictor was based on past data
used to estimate the model, while yt+1 is independent of these data, under the
iid assumptions of errors. to discuss
On the other hand, V (α̂ + β̂xt+1 |xt+1 ) = V (α̂|xt+1 ) + V (β̂xt+1 |xt+1 ) +
2Cov(α̂, β̂xt+1 |xt+1 ) = V (α̂) + V (β̂)x2t+1 + 2Cov(α̂, β̂)xt+1 ,
and neither α̂ nor β̂ depend on xt+1 .
In the case of the OLS estimator under the classical hypotheses, and esti-
mated from the sample up to  observation t, 2oneobtains
−x̄)
V (α̂ + β̂xt+1 |xt+1 ) = σ 2 1t + Pt(xt+1(xt+1 −x̄)2
.
i=1

Let V (yt+1 ) = σc2 , an unknown parameter. In the case of the classical OLS
assumptions, if one assumes furthermore that these assumptions are extended
2
to out-of-sample, one has σt+1 = σ 2 , which can then be estimated from the
sample data.

In that case, the variance of the error of the forecast is

!
1 (xt+1 − x̄)2
V (et,1 ) = V (yt+1 − ft,1 ) = σ 2 1 + + Pt
t i=1 (xi − x̄)
2

.
av calcul

Note that, in that case, the forecast variance is increasing in the square
of gap xt+1 − x̄. For predictions that are not based on time series, the same
calculations apply by substituting an observation xc for xt+1 in this formula. In
that case, xc can be in or out of sample, depending on the prediction problem
considered.

Under the usual assumptions, asymptotic confidence intervals based on cen-

tral limit theorems can be used for the predictions, on the ground that t , or n,
is large enough. Typically, when one can use the Student’s law approximation,
the confidence interval of the forecast, at the 5 percent level, is :

10
 [ŷt,1 ± V (et,1 )1.96]
.
iit

3.2 Gauss-Markov theorem

Gauss-Markov theorem can be applied to linear predictors. Let us first recall

it. Let be the model Y = Xβ + ϵ with E(ϵ|X) = 0 and V (ϵ|X) = σ 2 In . Then,
the OLS estimator is the best unbiased linear estimator for this model. This
optimality is in term of the MSE loss function, which means that the OLS
has the lowest variance-covariance matrix among all unbiased linear estimators.
The class of linear estimators in this model means estimators linear in the
yi , i = 1, . . . , n. A similar theorem can be stated with the GLS by replacing the
hypothesis on the variance of errors by an hypothesis V (ϵ|X) = Ω, where the
matrix Ω is symmetric and definite positive.

The same argument applies to the linear combination of the OLS estimator
(α, β)′ , for the forecast ŷt,h = α̂ + β̂xt+h . Indeed, the ranking of variance-
covariance matrices in the theorem holds by conditioning on xt+h , and the
linear transformation in the formula of ŷt,h . Then, an integration over the
distribution of xt+h delivers the result.

Therefore, forecasts often become quickly inaccurate over time.

3.3 Model instability

However, the above forecast or predictor formula of a linear predictor based

only on one variable is likely to yield a mediocre performance in many cases.
There are several issues. First, this model should be a bad approximation of the
true data generating process. This is the ’model error’. Second, the economy
and the context evolve. So, the model can become obsolescent. Third, the
data are not accurate and biased. Fourth, the empirical variables are only
imperfect proxies for the unobservable theoretical indicators. Fifth, the data
are not contemporary. They are produced with delays, and often revised over
time. Sixth, there are unobserved non-stationary structural shocks that are not
included in the model.
These issues imply that a forecasting expert should strive to complete the
data. For this, he must made assumptions about what the exogenous variables
to include are. But this may yield a seventh issue of ’input errors’ if these as-
sumptions are incorrect, notably if useless variables are included and important
correlates are omitted.

11
 Nonetheless, all these issues must be faced and dealt with. A model is always
an approximation of the truth.

3.4 Measuring prediction performance

Still, sophisticated risk functions that could account for some of these issues
may improve the performance of the selected forecast.
Let us first consider the two most often used risk functions for the forecast
at period t based on all the past data ŷt|t−1 are:
PT
M SE = T −1 t=1 (yt − ŷt,1 )2

and
PT
M AE = T −1 t=1 |yt − ŷt,1 |,

with in our simple model ŷt,1 = β̂0 + βˆ1 xt+1 , where β̂0 and βˆ1 are OLS
estimates over the span of periods from 1 to t. However, other estimators could
also be considered and plugged in this risk function.
But when evaluating a model, considering only its goodness-of-fit does not
guarantee that it will provide good predictions. In particular, an over-fitted
model, for example with too many parameters, has always a great fit, but may
forecast poorly. So, using the R2 as a criteria to choose between linear models
may not always be appropriate.
Therefore, one must measure out-of-sample forecast accuracy. One way to
do this is to divide the data in ’training observations’ and ’test observations’.
The model is estimated using the training observations, and forecasts are made
for the test observations. Then, the accuracy measures, such as above, are
calculated by using the errors for the test observations only.
In principle, one would like to use out-of-sample observations for evaluat-
ing predictions. Since they are not available, one often use instead the last m
observations and generated predictions for them from the other observations.
The first subsample can be called the ’training’, ’learning’, ’reference’ or ’es-
timation’ subsample, from periods t = 1 to T − m ≡ τ ; while the second
one can be denoted the ’test’, ’prediction’ or ’forecast’ subsample, from periods
t = T − m + 1 ≡ τ + 1 to T. By changing the parameter m, one can investigate
the effects on prediction of how the model parameters may change over time; or
the effects on prediction of how the model changes over time if one allows for
more general changes in the model.
In context where the observations are not naturally ranked, one can just draw
m observations at random, or otherwise (put them at the end of the sample list
to keep the same notation if wished).
Cross-validation is a balanced and systematic implementation of this kind
of approaches. Cross-validation with cross-section data with observations i =

12
1, . . . , n can be implemented by first selecting an observation i that will be
tested with a prediction attempt on the bases of all the other observations. The
prediction error can then be calculated for the test observation. Second, this
procedure can be applied to each of the observations i = 1, . . . , n. Then, the
prediction performance criteria, i.e. the above risk functions, can be computed
from all the obtained prediction errors.
However, it is not entirely clear that the evaluation of the prediction per-
formance should be ’symmetrized’ in that way. The observations that must be
predicted can also be intrinsically different from the other ones. In that case,
one can define ex ante a test set of observations and a training set of observa-
tions. In the first stage of the cross-validation the observation is selected only
from the test test, and the prediction is calculated only by using the training
set.
These considerations related to the separation of the stages of estimation
and prediction implies to adjust the definition of the above risk functions, as
follows.

T
X
P M SE = m−1 ∗ 2
(yt − ŷt,1 )
t=T −m+1

and

T −m+1
X
P M AE = m−1 ∗
|yt − ŷt,1 |,
t=1

where the new predictors, denoted with a star, are now estimated only over
the first T − m observations. One issue is how to choose m. Typically, the ratio
m/T is kept small. Of course, the definition of ŷt,1 must be adjusted to the
definition of the subsamples, for example for each choice of m.
One can also use weighted PMSE with weights increasing with t to account
for a changing over time, including gains in accuracy along with the increasing
sample size used for the estimation. AV
If all goes well, the PMSE is consistent to the variance of the theoretical
prediction error E[Yt − E(Yt |X1 , ..., Xt−1 )]2 ,
which can also be denoted by substituting E(Yt |It−1 ) for E(Yt |X1 , ..., Xt−1 ),
where It−1 is the past information history used for the prediction.
Similarly, under regularity conditions to allow the use of a law of large num-
bers, the PMAE is consistent to E|Yt − E(Yt |It−1 )|.
The method for choosing parameter m , or in general for splitting the sample
in an estimation subsample and a prediction subsample can be refined, and
developed. to do

13
 Of course, splitting the sample should also be done with other risk func-
tions, including MAE and PMAE to deal with outliers, which involves an esti-
mation based on the least-absolute estimator. There is a connected literature
on structural break in predictions (Clements and Hendry, 2006; Pesaran, Pick
and Pranovich, 2013), in which the sample splits can be estimated or tested.
To recap, there are roughly three approaches to evaluating prediction per-
formance: (1) using risk functions only over the global observed sample, which
bestows a lot of confidence in the estimated model; (2) splitting the sample in
an estimation sample and in a prediction sample, with possibly different risk
functions used for these two stages; (3) other more sophisticated approaches.

The are common issues with all these criteria of prediction power. First, they
depend on the scales used for measuring the random variables. This is only a
moot point, since nothing wrong happens if one always keep the same scales in
the analysis. Besides, other similar indicators are constructed by standardizing
these measures by diverse scale statistics. To avoid the scale effect, one can
divide the loss kernel by a corresponding variable. For example, for the PMAE
one can divide by yt , while for the PMSE one can divide by yt2 . However,
allocating a large weight to observations close to zero with these approaches is
hard to justify.
Second, MSE and PMSE are non-robust to outliers, as opposed to MAE and
PMAE. However, in the popular case of normal distributions, some precision
is lost when using MAE and PMAE, instead of MSE and PMSE. PMEA if
preferred to the PMSE if one accepts occasionnally large prediction failures,
provided the predictions are rather good on average.
Third, and more worrying, nothing guaranties that these risk functions well
correspond to the risk function that a good expert should use. In particular,
it may be reasonable to assume that overpredicting errors and underpredict-
ing errors have not the same cost for the expert. exps from Elliotts and
Timmermann
Moreover, predictions considered for the last m observations may be biased,
which also suggest to use instead an asymmetric loss function. A measure of
the prediction bias is, in that case, the average of the prediction errors on the
last m observations. For example, if it is found positive, it may be the sign of
an excess of overestimations over underestimations.
One could instead use a loss function such that L(0) = 0 and L(y) is strictly
increasing for positive values of y , and strictly decreasing for positive values of
y , but is also asymmetric with respect to y = 0 . But, it is hard to choose the
kind of asymmetry to impose.
One could also wonder if numerical criteria looking much like extensions of
goodness-of-fit measures is the way to go, when considering the interests of the
decision-makers that will be using the prediction results. For example, macroe-
conomist may be more interested in the occurrence and timing of a recession
that precise GDP forecasts. There have been attempts to categorize variables
to respond to this concern. For example, one can construct a trichotomic vari-
ables indicating: clearly positive growth, clearly negative growth, approximate

14
stagnation. Then, the variable to predict would lead to nonlinear models, which
we discuss in a next subsection.
Similar formulae and considerations apply for prediction errors of higher
horizons. However, the mentioned issues can worsen when the horizon increases.

4 Identification

Overfitting can be related to identification problems with the model. So, it is

useful to recall what identification is.

Definition: Let be a parametric model (Ω, β), with β ∈ Θ.

(i) An M-estimator β̂ of a parameter β in the parametric model (Ω, P β ) is
the solution to an optimal problem. For a sample of size n, it is associated to a
given real-valued objective function Sn (., .):

β̂ ≡ arg min Sn (Y n , X n , Z n , β),

β
′
where Y , X , Z are the data in the [Link] example, Y n = (y1 , ..., yn )
n n n

regroups the realizations of a dependent variable Y, X n denotes the observations

of independent variables, and Z n denotes the observations of instruments.

(ii) (Point Identification): A value β0 ∈ Θ of the parameter β of the model

is identifiable if there is no other value β ′ ̸= β0 , such that Pβ0 = Pβ ′ .

The point-identification condition is weaker, and therefore easier to obtain

for a certain value β0 , than the above identification of the whole model as for
any possible value of β.

The next proposition provides some analytical rules to apply to check iden-
tification.
Proposition:
(i) (General identification, or point identification for M-estimators):
A sequence of minimized criterion functions, Sn , identifies asymptotically
β0 , which denotes the ‘true value’ of β under model (Ω, β)
if:
For any model (Ω, β), for any parameter value β ∈ Θ, for any Y n , X n , Z n ,
a sample of n observations of random variables valued in Ω,

S(Ω, β) ≡ p lim Sn (Y n , X n , Z n , β) exists,

n→∞

where the limit is taken with respect to model ( Ω, β), and satisfies the in-
equality

15
 S(Ω, β0 ) < S(Ω, β ′ ), for any β ′ ̸= β0 .

(ii) Moreover, if Θ is not compact, then there is no sequence β m without

limit point and such that lim S(Ω, β m ) = S(Ω, β0 ).
m→∞

Proof: todo

That is, the minimization problem has a unique solution that is the true
value, asymptotically.

Thus: (1) A parameter is identifiable is there is no other parameter that

yields the same probability distribution;
(2) A sequence of criterion functions identifies a (true) value β0 of a param-
eter if β0 is the unique value minimizing the limit of the criterion.
Therefore, M-estimators should identify the true parameter value as long as
their criteria function converges well towards a well-behaved criterion function,
and their asymptotic minimization program not only has a unique solution, but
this solution is the true value of the parameter.

As mentioned before, it is not necessary that the model used for prediction
be a causal model. Everything that works will do, even if no clear causal mech-
anisms are related to it. As a matter of fact, most high quality predictions are
not calculated using a causal structural model.
Moreover, it is not even necessary to impose identification of the used model.
A parametric model is identified when its parameters can always be uniquely
obtained from the data. In some cases, an unidentified model may generate
several values for some parameters, which may make the model less reliable
for predictions or forecasts. However, statistical identification may fail and the
prediction model still perform well for out-of-sample data. This is typically the
case with machine learning and neural networks used for predictions. In this
sense, accuracy of prediction is preferred to the capacity to interpret parameters.
If the model predicts well and the predictions are robust to the use of new data,
then one does not need to worry about identification. Besides, machine learning
models are often nonparametric without specifying an a priori given functional
form. In that case, identification is generally elusive. In the case of neural
networks that have typically more parameters than observations, there is no
question of identifying statistically the parameters anyway. In all cases, one
should rather focus on the predictions, and on the loss function minimization
that has generated them than on parameter estimation. If the loss function
is well minimized numerically, and well specified, then it should deliver proper
predictions.
Still, the model must have some minimal structure to ensure that the pre-
dictions are meaningful. Unobserved confounders such as omitted variables can

16
weaken the prediction solidity, as can do quasi-colinearity in predicting vari-
ables. These issues can generate instability of estimated forecasts, while not
always.
If the model is too complex, it may be overfitting the data. In that case,
it may fit noise rather than true robust patterns of interactions between the
predicting variables and the variable to predict. In the end, what matters is the
out-of-sample performance.
However, too highly correlated predicting variables and overfitting can be
dealt with lasso methods or ridge regressions. Dropout, batch normalization and
weight decay are methods that can mitigate overfitting and stabilize training
stages, even if parameters are not identified.
An issue is when one want to interpret the prediction results, for example
in terms of the relative importance of some factors. In that case, this requires
some kind of identification of the model.
Numerical problems can arise when the parameters are too numerous, with
difficulties to find global minima with neural networks of maximum likelihood
methods, and therefore to converge correctly. These numerical problems differ
from identification issues, but may jeopardize the predictions.

5 Nonlinear Predictors

When facing nonlinear models, and therefore nonlinear predictor, one can first
linearized it. One kind of linearization is through stepwise algorithm, such
as with the Kalman filter. However, some results obtained from performance
criteria no longer apply. For example, least-square estimation, even stepwise,
does not usually deliver good minimal-variance properties.

5.1 The Tobit model

Nonlinear predictors arise for example for the Tobit model:

Let yi be a dependent variable of interest, for any individual i.
A latent (i.e., partially observed) variable yi∗ is assumed. This is an imagined
device that assists in specifying the model. It serves two purposes simultane-
ously: (1) providing a model based on a theory for the level of yi when fully
observable; (2) providing an explanation for the It is common to specify a linear
equation as a device to introduce some explanatory variables, xi , of yi∗ .

yi∗ = x′i β + ui
The independent variables xi and the errors ui are assumed to be inde-
pendent. Moreover, the errors ui are assumed to follows a centered Gaussian
distribution N (0, ‘sigma2 ).

17
 Second, the link of yi and yi∗ must be specified. In the case of the Tobit
model, the relationship between the observed yi and the latent yi∗ boils down
to choose a threshold of yi∗ , and of yi , typically equal to zero, under which yi∗
is no longer observed.

yi = yi∗ if yi∗ > 0 ; else yi = 0; i = 1, ..., n .

Example: A wage equations in which wages are observed only for working
individuals who have a positive wage. Non-working individuals have a zero
wage. The latent variable is the individual marginal productivity. Interesting
regressors are the education and the experience of the individual.

Proposition: The log-likelihood of the observed censored sample is

n 
(yi − x′i β)2
X  
LL = 1[yi =0] log (1 − Φ (x′i β/σ)) + 1[yi >0] − log σ .
i=1
2σ 2

Then, estimates of the parameters β and σ 2 can be obtained by using the

maximum likelihood method.

5.2 Conditional moments

If one is interesting in the unobserved latent variable yi∗ , its conditional expec-
tation is

E(yi∗ | xi ) = x′i β.

From this model, by imposing normal errors, one can also derive the following
truncated moment of the observed variable yi .

x′i β
 
ϕ σ
E(yi | xi , yi > 0) = x′i β + σ , (1)
x′i β

Φ σ
 ′ 
xi β
ϕ
x′i β
  σ
for any β and any X, where λ σ =  ′ 
x β
> 0 is called the inverse
Φ i
σ

Mills’ ratio.

18
 The conditional expectation of the latent dependent variable yi∗ , which em-
bodies the main theoretical interest, is overestimated when it is computed over
the non-censored sample only when β and σ are known or estimated. The for-
mula of this truncated moment, can be exploited to provide predictions of the
moment for various values of xi . As the formula is nonlinear and varies with the
levels of the dependent variables

Other useful nonlinear formulae of moments for the Tobit model are:

P r(yi > 0) = Φ(x′i β/σ).

and
Eyi = x′i βΦ(x′i β/σ) + σϕ(x′i β/σ).

These moments can be used to generate predictors by plugging in these

formulae the values of the independent variables at the prediction points, and
the estimated values of the parameters.

5.3 Impacts of changes in independent variables

To exploit these moments’ formulae for predictions, one may also want to know
how changing values of independent variables xi would affect the considered
expectations.

Proposition:

In the case of the latent linear model, E(yi∗ | xi ) = x′i β, these changes are
fully described by the marginal effects on the mean latent variable Y ∗ :
∂E(yi∗ | xi )
∂xji
= βj ,

for any independent variable xji , any value of it, any value of the other
regressors and any value of the latent variable yi∗

That is: the small changes in the conditional expectation of Y ∗ are propor-
tional to the coefficient of the considered independent variable. However, this
extremely simple case does not extent to nonlinear conditional expectations, as
we will see.
Marginal effects, for any continuous regressor X j can be obtained by differ-
entiating Eq. (1):

x′i β
 
∂E(yi | xi ; yi > 0) ∂λ σ
= βj + σ .
∂xji j
∂xi

19
 x′i β x′i β x′i β x′i β βj x′i β x′i β βj
   
∂λ σ Φ( σ ) − σ ϕ( σ ) σ −ϕ( σ )ϕ( σ ) σ
Now, ∂xji
=  ′
x β
 2
Φ( σ i )

x′ β
i ) j
β
x′ β x′ β x′ β
n o
−ϕ( σ
= 
x′ β
σ2 Φ( σi ) σi + ϕ( σi ) , and therefore:
i )
Φ( σ

Proposition:

x′i β x′i β x′i β

n o
∂E(yi | xi ;yi >0)
∂xji
= βj − βj λ( σ ) σ + λ( σ ) .

The partial marginal effects are therefore much more complex than the coef-
ficient βj . Notably, their values vary
n across observations. It can be shown that
x′i β x′i β x′i β
o
the adjustment factor, 1 − λ( σ ) σ + λ( σ ) , strictly lies between 0 and 1.
Of course, for discrete variables xji ,
one can replace the derivatives with
finite variations, which does not change anything fundamentally but makes the
expression of the bias more tedious. For example:

E(yi | x1i , ..., xj∗ k 1 j∗ k

i + 1, ..., xi ; yi > 0) − E(yi | xi , ..., xi , ..., xi ; Y > 0)
j∗ ′ j∗ k ′
(x1 +1,...,xk
i ,...,x i) β (x1
i ,...,xi ,...,xi ) β
ϕ( i ) ϕ( )
= βj + σ j∗
σ
(x ,...,x +1,...,x )′ β
1 k −σ 1 j∗
σ
(x ,...,x ,...,xk )′ β
.
Φ( i i i ) Φ( i i i )
σ σ

Let us now turn to the marginal effect for the full data. Again, as a conse-
quence of the calculus performed in Chapter 1:
 ′   ′ 
xi β xi β
E(yi | xi ) = Φ x′i β + σϕ .
σ σ
Therefore:

Proposition:

The marginal effect of xji on the conditional expectation of yi conditional

on xi , for the whole population is:
 ′ 
∂E(yi | xi ) xi β
j
=Φ .βj .
∂xi σ
Proof:

′ x′ β
We know that E(yi | xi , yi > 0) n =′ xi β + σλ( oσ ),
i

∂E(yi | xi ,yi >0) x′i β xi β x′i β ∂ Pr(yi >0 | xi ) βj x′i β

∂xj
= βj [1−λ( σ ) σ + λ( σ ) ] and ∂xji
= σ ϕ( σ ).
i
Since E(yi | xi , yi = 0) = 0, E(yi | xi ) = E(yi | xi , yi > 0). Pr(yi > 0 | xi ).
Then, the previous formulae can be plugged into
∂E(yi | xi ) | xi ,yi >0) i >0 | xi )
∂xj
= ∂E(yi ∂x j Pr(yi > 0 | xi ) + ∂ Pr(y∂x j E(yi | xi , yi > 0)
i i i

20
 to yield n
x′i β x′i β x′i β x′i β
n ′
x′i β x′ β
o o
βj ′ xi β
σ ϕ( σ ) x i β + σλ( σ ) +Φ( σ )β j [1−λ( σ ) σ + λ( σ ) ] = Φ( σi )βj .
QED.

5.4 Selectivity Bias

In the case of a variable xji that contributes positively to the selection (e.g.,
education for participation in the labor market in the wage equation example),
we usually have β̂ j > 0, asymptotically.
a revoir If moreover, most individuals are uncensored (e.g., work in the ex-
ample), and therefore x′i β̂ > 0 asymptotically, and if the participation threshold
is at zero, then on average x̄′ β̂ j > 0. In that case, ∂x∂ λ̂
j < 0, at the mean of
i

the sample. Here, the selection diminishes the effect of xji (education in the
example).
AV page sur calcul explicite de derivative of lambda in xj

a revoir In the case of the wage equation example and its return to educa-
tion, since β j > 0 is expected in a wage equation and the usual missing factors
in the two equation should be positively correlated (social family background,
motivation, ability), which implies ρ > 0, definir rho then the selection atten-
uates the effect of the education on wages (that is: when comparing the selected
sub-population with the latent one).

x′i β
x′i β ϕ( σ )
NB: The bias σλ( σ ) =σ x′ β
is the bias of E(yi | xi ; yi > 0), not the
Φ( σi )

bias of the OLS β̂. For investigating the latter, one can compute
−1 −1
(E [X ′ X]) E (X ′ Y ) = β+ σ (E [X ′ X]) E (X ′ λ),
since Y = Xβ + σλ + v, where v is an error term assumed to be orthogonal
to the other terms. Then, each component of the bias can be examined. Of,
course this is akin to the examination of the usual omitted variable bias, with
the correlations among all variables intervening in a combined and complex way.

5.5 Proportionality of the effects

A few remarks are of order. First, we can see that both kinds of parameters,
β and σ, play logically independent roles, as opposed to the case of the Probit
model.
Second, the marginal effect of a continuous variable xji on the conditional
mean of yi given xi is measured by its coefficient βj , rescaled by the probability
of observing a positive response given xi (i.e., by P r(yi > 0|xi ) = Φ(x′i β/σ) .

21
This implies that the sign of the effect, although not its magnitude, is known
as soon as the sign of the coefficient βj is known.
Third, because of this rescaling feature, the ratios of the marginal effects
of any two distinct continuous regressors, say xl and xj , for anyone moment
taken among: E(yi | xi ), E(yi | xi , yi > 0) and P r(yi > 0 | xi ), are all equal
to βl /βj for any given values of yi and xi . Indeed, any formula of the marginal
variations of these moments, truncated of not, for a marginal variation of xji ,
is the product of βj by a scaling factor, and the same scaling factor applies for
the corresponding marginal variations with respect to a different xli . That is:
the relative marginal effects for any two distinct variables xli and xji is βl /βj .
This implies that the Tobit model is actually not a flexible model as it does
not allow to study how these distribution characteristics vary independently
with the levels of the independent variables. For example, for a government,
the probability of worker of being employed, the mean wage of workers and
the mean earnings of people in age of working are of distinct policy interest,
and are believed to enjoy each specific variations with respect to divers policy
parameters. exp
Consider again the developed formula of E(yi | xi ), obtained by conditioning
on selection: E(yi | xi ) = E(yi | xi , yi > 0). Pr(yi > 0 | xi ) since E(yi | xi , yi =
0) = 0. Then, as noted before, the elasticity of E(yi | xi ) with respect to any
variable xji , is the sum of the elasticity of E(yi | xi , yi > 0) and of the elasticity
of P r(yi > 0 | xi ). This property is satisfied for all censored regression model
that do not impose Gaussian errors, instead of being specific to the Tobit model.

5.6 Heterogeneity of the effects

The effects depend on all the values of the independent variables, and on the
values of all the other parameters, including on the parameter σ . They depend
nonlinearly on all these elements, and also nonlinearly on the level of the
variable xji . The formulae become still more complicated when interactions
between the independent variables are introduced in the model. This is also the
case, when dependent or independent variables are transformed by quadratic,
exponential or logarithmic transformations.
The complexity of the conditional expectations and conditional probabili-
ties for censored regressions implies that one should go much beyond the usual
practice of reporting only the estimated coefficients β and their standard errors.
Indeed, these coefficients do not clearly reveal the effects of the variables at
first sight. This is because the effects of the independent variable on moments
of interest are almost never constant in this kind of nonlinear model. In many
analysis, the latent variable is a mere intermediate device in the model con-
struction, and has little interest in itself. Usually, one would rather know the
effects of the variables on the mean of the observed variable (e.g., the wages),
whether for the selected or the full population. This is the typical perspective
for policies.

22
 However, the estimate of β are still useful to report as they inform on the
latent model estimates that may be of interest, in particular if the latent model
is directly linked to some economic theory. When yi∗ is a theoretical variable
of interest, one may want to pursue its analysis through the latent model. For
example, in the wage equation example, yi∗ may be interpreted as the latent
marginal productivity of the individual, thus allowing for negative productivity,
and it may well be worth analyzing.
A way to account for the heterogeneity of the effects is to estimate the
marginal effects at interesting values of the independent variables. For ex-
ample, one can perform this estimation for the virtual individual that would
have all its characteristics equal to the mean characteristics of the sample.
Thus, the heterogeneity of effects across individuals can be reported through
distinct estimated values (or some of their marginal effects, or some elastic-
ities) of P r(yi > 0 | xi ), E(yi | xi , yi > 0) and E(yi | xi ) at different fixed
values of the regressors xi . One may choose to report these values of the diverse
marginal effects at the mean of xi , or at some quantiles of the x′i s, or even for
some subpopulations of interest (for example, the censored and non-censored
subpopulations).
As a matter of fact, the partial effect at the average (PEA), is a popular
choice. Since the same adjustment factor is the same for all the coefficients βj ,
in the case of the Tobit model, there is no need to compute different adjustment
factors for each of the independent variables. However, one issue is that the
special individual defined by the mean characteristics is only an abstract con-
struct that may have little analytic or policy relevance. For example, when the
sex of the individual is an explanatory variable, this individual is characterized
as having a certain probability to be female and otherwise to be male.
When there are nonlinear transformations of independent variables in the
model (e.g., logarithmic), one may rather use the mean of the transformed
variables as a benchmark in the calculus, so as to remain consistent with the
linear form of the latent model. But in that case, interpreting this mean may
be less straightforward than when using the mean of the variable before its
transformation. As a matter of fact, the two approaches (mean of levels and
mean of logarithms) can be used in practice.
In part owing to these conceptual difficulties, one may want to estimate
instead average partial effects (APE, sometimes denoted average marginal ef-
fects), which is the analog of the variations of the Average Structural Function
which is advocated in Blundell and Powell (2006). it corresponds to the option
asf in Stata commands. Note that by default, the margins command in Stata
gives the asf, not the marginal effect at the mean value of the variables. The
APE is the empirical mean of the individual effects, each individual effect be-
ing calculated with the respective observed individual characteristics. In that
case, for any independent variable xji , the adjustment factor for βj becomes the
empirical mean of the individual adjustment factors. Again, for the considered
moments, this mean adjustment is strictly between 0 and 1.
Another dimension of heterogeneity in the model is the heteroscedasticity of
the errors that is generated by the selection.

23
6 Simulations

6.1 Analysis effects of variable by using simulations

One can also conduct the analysis of the effects of variables by using simula-
tions. This not only avoids entering in the explicit nonlinear formulae for the
marginal effects for nonlinear models, but simulations also allow for examining
non-marginal effects (e.g. , a change in several education years would affect the
wages of working individuals). This may be important for major policies that
entail discrete changes in specific variables. Most exogenous shocks or policy
interventions are non-marginal in practice.
Simulation analyses are also useful to draw the distribution density of the
effects.
to dvlop a lot, inc. with simulators perhaps?

If one wish to run simulations or predictions, one may want to impose that
the predicted values for y are nonnegative if the variable Y is nonnegative (e.g.,
wage rate). This feature is generally not satisfied with OLS for example. How-
ever, positive predicted values can be obtained for y by using the formulae
for E(Y |X, Y > 0) and E(Y |X) obtained for the Tobit model, instead of the
−1
OLS formulae XP (X ′ X) X ′ Y , where XP are the values of the independent
variables at which the prediction is calculated.

6.2 Policy prediction using simulations

Using simulations based on a model, that is predictions using this estimated
model, is an alternative approach to statistical impact analysis for assessing the
effects of policies.

AD tout
Let us consider the study by Muller and Bibi (2010) who developed a novel
methodology to predict poverty under targeting policies. The method is based
on estimation methods that focus on the poor. Using data from Tunisia, ‘fo-
cused’ transfer schemes are estimated that highly improve anti-poverty targeting
performances. Post-transfer poverty can be substantially reduced with the new
estimation method. For example, a one third reduction in poverty severity from
proxy means test transfer schemes based on OLS method to focused transfer
schemes requires only a few hours of computer work based on methods available
on popular statistical packages. Finally, the obtained levels of under-coverage
of the poor are particularly low.

Although living standards are measured with household surveys, they are
generally badly known for the households that are not surveyed. Cash transfers
based on PMT are generally found to provide the best results, although there
is an enormous variation in targeting performances.

24
 Two indicators, Leakage and Undercoverage, are popular for measuring tar-
geting performance. With imperfect targeting, only people among the poor who
are predicted as poor can benefit from poverty alleviation. On the other hand,
non-poor people predicted as poor receive transfers. Thus, two types of errors
characterize imperfect targeting. The Type I error (Undercoverage) is that of
failing to reach some members of the targeted group. It is estimated by the
probability of not receiving any transfer, while poor. The Type II error arises
where benefits are awarded to ineligible people under perfect targeting. The
Leakage of program benefits is a monetary assessment of this error, obtained
by adding (1) the transfers given to those whose pre-transfer living standard is
above the poverty line, and (2) the transfers received by pre-transfer poor that
are unnecessary because the post-transfer living standards exceeds the poverty
line. Unnecessary transfers are those in excess for all households lifted up above
the poverty line, since from this level of living standards they are no longer
poor. If the aim is minimizing poverty, there is no justification to transfer cash
to households once they have been lifted up to the poverty line since this would
not change the final poverty level. The Leakage ratio is obtained by dividing
the Leakage indicator by the available budget. A last measure of the program
efficiency is the reduction in a poverty measure due to transfers.*
In practice, anti-poverty targeting can be based on predictions of household
living standards, generally obtained from ordinary least squares (OLS) regres-
sions on observed characteristics. However, the OLS method is centered on the
mean of the dependent variable (e.g., household living standard) and should
provide accurate predictions around this mean only, which is often located far
from the poverty line. Then, the predicted living standards of the poor and
near poor may be inaccurate.
Two simple restrictions of the predictive regressions are mobilized: (i) cen-
soring the dependent variable to eliminate observations located far from the
poverty line; (ii) using quantile regressions. Good results may be obtained by
just trying one quantile around the poverty line. Then, focusing on the poor
means that the predictions are calculated by defining the quantile regression
or the censorship threshold in terms of living standard levels representative of
the poor. Assume that the equation used to predict living standards has the
form yi = x′i b + ui , where yi is the living standard of household xi is a vec-
tor of exogenous correlates of living standard for household i, ui is an error
term, b is a vector of parameter to estimate. OLS estimates corresponds to
imposing the restriction E(yi |xi ) = x′i b, which implies E(ui ) = 0. Quantile
regression estimates centered in quantile θ correspond instead to the restriction
qθ (yi |xi ) = x′i b, where function qθ denotes the conditional quantile function of
order θ, conditional on the values of the variables xi . This restriction implies
qθ (ui |xi ) = 0. That is: the quantile on which a quantile regression is centered
relates to error quantiles. Thus, the vectors b differ when different quantiles are
specified.
In that case, what is predicted is a chosen quantile of the distribution of the
living standards conditionally on the correlates. This method has two short-
comings. Firstly, if the error terms are approximately normal, some efficiency

25
may be lost as compared with OLS (the MLE under normality). This matters
for targeting purposes as large variance of predictions may yield poor predictive
performance for any social indicator. Secondly, the focus is conditional on the
set of correlates. That is, the chosen quantile is not that of the dependent vari-
able, but the quantile of the error in the estimated equation. However, that is
precisely the quantile of the error that may matter most if one is interested in
the prediction error that affects the transfer scheme performance.
Let z be a pre-specified poverty line, f (.) is the density function of household
income y (or household living standard) and α is a poverty aversion parameter.
Given an anti-poverty budget, one must design transfers that optimally allocate
this budget across households. Let us first consider the situation when Y (the
vector of incomes in a population before applying the transfers, ti , i = 1, . . . , N
) is perfectly observed. The optimal transfer allocation is the solution to:

N  2
1 X z − yi − ti
min I[yi +ti <z]
ti N z
i=1

subject to
N
X
ti = B
i=1

and

ti ≥ 0, ∀i.
where N is the population size, B is the budget to allocate, ti is the non-
negative cash transfer to household i and yi is its pre-transfer income. When
Y is perfectly observable, the solution to this problem can be referred to as
‘perfect targeting’ and denoted ti for household i.

We use data from the 1990 Tunisian consumption survey conducted by the
INS (National Statistical Institute of Tunisia). The survey provides information
on expenditures and quantities for food and non-food items for 7734 households.

Definition of the predicting variables:

Set I: Area: Great Tunis Northeast Northwest Middle East Middle west Sfax
Southeast Southwest
Complement for Set II: Demographic information: Number of children in
household old less than 2 years old. Number of children aged between 3 and 6
years. Number of children aged between 7 and 11 years. Number of adults aged
between 12 and 18 years. Number of adults old more than 19 years. Age of the
household head (HH). Squared age of the HH.

26
 Type of house: Number of rooms per capita 1 if household lives in a detached
house, 0 otherwise. 1 if household lives in a flat, 0 otherwise. 1 if household
lives in an Arab house, 0 otherwise. 1 if household lives in a hovel, 0 otherwise.
Accommodation Mode: 1 if household is owner of the house. 1 if household
is renting a house. 1 if household has a leasing agreement for his house 1 if
household lives in a free of charge house.
Complement for Set III: Occupation of HH: Dummy variable for HH is un-
employed. Dummy variable for HH living in the Southeast and agricultural
labourer. Dummy variable for if HH living in the Southwest and agricultural
labourer. Dummy variable for if HH living in another region and agricultural
labourer. Dummy variable for if HH is an industry worker. Dummy variable
for if HH is a farmer. Dummy variable for if HH living in the Northwest and
agricultural farmer. Dummy variable for if HH is self-employed or manager.
Dummy variable for if HH has another type of job. Number of participants in
the household’s budget. Number of female workers. Number of male workers.
Schooling level of HH: Dummy variable for HH is illiterate. Dummy variable
for HH has a primary schooling level. Dummy variable for HH has a junior sec-
ondary schooling level. Dummy variable for HH has a senior secondary schooling
level. Dummy variable for HH has a higher educational level. Number of stu-
dents. Number of children in private secondary school. Number of children in
public secondary school. Number of children in private primary school. Number
of children in public primary school.
HH = ‘household head’.

The censored quantile regression estimator for dependent variable yi and

quantile θ is obtained as the solution to the minimization of
′
P
i ρθ [yi – sup(0, xi γ)],

where ρθ [u] = (θ–I[u<0] )|u|, xi is a matrix of regressors and γ is a vector of

parameters. Quantile regressions correspond to replacing sup(0, Xi′ γ) with Xi′ γ
. The estimation is obtained by using a linear programming algorithm and sub-
sample selection at each iteration of the optimization. The confidence intervals
of the censored quantile regression estimates are estimated with the bootstrap
method and 1000 bootstrap iterations.

The results show that quantiles regressions (centred at the first decile) per-
form much better than the other methods for predicting the logarithms of living
standards of the poorest households (here defined as belonging to the first or
second decile of the living standard distribution), to the exception of censored
quantile regressions that perform better for the households with living standards
below the first quintile. In contrast, the best method for predicting the mean of
the logarithms of living standards in the whole population is the OLS method.
We use a poverty line equal to TD 250 to estimate targeting efficiency
measures, consistently with the most credible poverty line in The World Bank

27
28
29
30
(1995), corresponding to a head-count index of 14.1 %. This poverty line corre-
sponds to an equivalent poverty line of TD 280 without subsidies.

Three important points may be noted. First, the gaps between the estimated
reductions in P 2 with different prediction methods are considerable. The sta-
tistical method used to design the transfer scheme is a crucial ingredient of the
performance of the scheme. When compared with other cash transfer methods,
substantial improvement of the poverty situation measured by P 2 can be ob-
tained (with our preferred estimation based on Set II: from 0.385 % with the
best OLS method to 0.272 % with the best quantile regression method – cen-
tered in the first decile). Moreover, the percentage of excluded poor households
from the scheme dramatically falls (to 8.6 %) as compared with what is obtained
with OLS predictions based on geographical dummies (for which it is 41.6 %).
Second, the usually employed method, based on OLS estimates, appears as the
worst performing approach in contrast with methods focusing the predictions
on the poor. However, when considering only the number of the poor, the OLS
may provide acceptable predictions for the richest of the poor that are not in
that case discounted when compared with the poorest.

References:
Clements, M.E., and D.F. Hendry (2006), ’Forecasting with Breaks’, Chapter
12 in G. Elliott, C.W.J. Granger and A. Timmerman, Handbook of Economic
Forecasting, Vol. 1, pp. 605-657, Elsevier B.V.
Muller, C. and S. Bibi (2010), ’Refining Targeting agains Poverty: Evidence
from Tunisia,’ Oxford Bulletin of Economics and Statistics, Vol. 12(3), 381-410.
Pesaran, M.H., A. Pick and M. Pranovich (2013), ’Optimal Forecasts in the
Presence of Structural Breaks,’ Journal of Econometrics.

31
32
33