Online appendices for: Explaining Fixed Effects: Random Effects

modelling of Time-Series Cross-Sectional and Panel Data

Andrew Bell and Kelvyn Jones

School of Geographical Sciences and Centre for Multilevel Modelling, University of Bristol

These appendices provide the Stata code and datasets required to replicate both our simulations (in
section 8) and our example analysis (below). As such these appendices include the following:

 Simulations (section 8)
o A do file that loads to xtfevd program ('[Link]')
o 16 do files, which each run simulations for different DGP scenarios (eg ‘[Link]’)
o A do file which combines the 16 datasets and creates the results presented in table 2 and
3 (‘collapse and [Link]’)
o The dataset which results from this do-file showing the complete detailed results of the
simulations (‘[Link]’)
 Example analysis
o A reanalysis of Milner and Kubota’s (2005) study of democracy and free trade (see
below)
o The data used by Milner and Kubota (cut down for the purposes of our analysis
(‘[Link]’)
o The do file used to run our analyses and produce tables 4 and 5 (‘Example [Link]’).

Example: the effect of democracy on trade liberalism

This example uses TSCS data to look at the effect of democracy on trade openness in

developing countries. The data consists of a measure of a country’s statutory tariff rate as

the dependent variable (with low tariffs reflecting trade openness), and independent

variables including a polity score (measured between -10 and +10, where high values

indicate greater democracy), GDP per capita, the natural log of the country population, and

A1
the year of measurement1, measured on occasions (level 1) between 1980 and 1999 for 101

countries (level 2)2 (see Table 4). Milner and Kubota (2005) use FE estimation (see model 2

in table 5) and argue that their findings show that “more democratic regimes tend to have

lower tariff rates” (p126). Of course this is an over-interpretation, as their FE model can

only measures within-country effects – their results only actually suggest that a country

becoming more democratic leads to lower tariff rates. Here, we reanalyse Milner and

Kubota’s data under a RE framework and show that even that conclusion is subject to

considerable doubt.

Our models are as follows (see Tables 5 and 6)3:

1. A null RE model with no predictors (a simplification of equation 1)

2. A FE model, similar to that used in Milner and Kubota (2005) (equation 6)

3. A standard RE model which takes no account of heterogeneity bias (equation 1)

4. RE model with the within-between specification4 (equation 12)

5. As 4 but with outlying intercepts included as a single dummy variable

6. As 5 but with the coefficient for within polity score allowed to vary at both level 1

and 2 (equation 15)

7. As 6 but with an outlier polity effect included as a differential slope in the fixed part

1
Note that Milner and Kubota also have more complex models with more control variables. Here we use their
most simple model (model 1 in their table 2, p127) to illustrate our methodological argument as clearly as
possible. We intend to make a more definitive critique in a later paper (Bell et al., 2014).
2
Milner and Kubota’s article suggested that their data ran from 1970. In fact, for all countries the data was
subject to missingness until 1980. Note that the rest of the data is also subject to missingness or imbalance.
However, more appropriate methods for dealing with missing data (Carpenter et al., 2011) are beyond the
scope of this paper, so here we use listwise deletion on all cases with missing values in the predictors and
outcome that we use.
3
Milner and Kubota additionally use an AR1 correction to allow autocorrelated residuals. We ran the simpler
of our models with autocorrelated residuals and found that it did not affect our substantive conclusions. In
order to keep this model as simple as possible for illustrative purposes, we therefore do not report the results
with auto-correlated residuals.
4
Note that the ‘between’ country means were calculated using the full data, prior to listwise deletion. The
within components were calculated using the country means of the cut down data, to preserve orthogonality.

A2
 8. As 7 with a cross level interaction between the within and between components of

democracy included (equation 18).

These models were fitted using MLwiN version 2.275 (Rasbash et al., 2013) with RIGLS

restricted maximum likelihood estimation.6

Looking at Model 1, we can calculate the VPC from the two variance terms, and see that

58% of the variance in the response occurs at the higher-level, between individuals. As FE

models only look at the occasion-level variance, they therefore can only consider 42% of the

interesting variation that is going on in the dependent variable. Context, in this case

individual difference, is being controlled out when it is at this higher level that most of the

variance lies, meaning the majority of the variation in the data is effectively being ignored.

Comparing model 2 and 3, we see that the mis-specified standard RE model without level 2

means suffers from bias, particularly in the population variable (lnpop), the effect of which

is vastly underestimated. However, the FE results (model 2) are identical to the within part

of the RE estimates of model 4 which models the cause of this bias (different within and

between effects) explicitly. In addition, including the mean term of polity in model 4 shows

us that, in fact, there is no evidence for an effect of a country’s average level of democracy

over the period of measurement on free trade. Milner and Kubota’s (2005, p126)

conclusion that “more democratic regimes tend to have lower tariff rates” is in fact

unsupported by this analysis.

5
These models can also be easily estimated in most major statistical software packages, including Stata, R and
SAS. Code to implement the models in Stata using the ‘runmlwin’ command (Leckie and Charlton, 2013) can
be found at the end of this appendix.
6
With the exception of the FE model, which was estimated using the xtreg command in Stata. MCMC results
were largely the same, as would be expected due to the large number of higher level units.

A3
One of the characteristics of TSCS data is an interest in how individual entities, in this case

countries, operate differently from each other. Milner and Kubota express this interest

early in their article, drawing attention to specific countries that have experienced

democratisation and trade liberalisation simultaneously. However, their method is unable

to consider the heterogeneity of individual countries because their FE analysis controls out

all country effects. In contrast the shrunken higher-level residuals in a RE model can be

estimated to consider variation between countries. Figure 1 (obtained from model 4) shows

this – there are three clear South Asian outliers with much higher differential intercepts

than other countries. These cannot be thought of as part of the overall distribution of

countries, so they are ‘dummied out’ as a set in model 5 to have their own differential

intercept and preserve the assumption of Normal residuals for those countries that remain

in the random part as part of a common distribution7.

In Model 6 the coefficient associated with the within polity score is allowed to vary, and the

associated random coefficient shrunken residuals are plotted in Figure 2. Again, we find

that Bangladesh is a substantial outlier in its effect, having a much steeper negative slope

than other countries; it is much more difficult to find these outliers with FE models. We

included an interaction between this country’s dummy and the within democracy variable

(model 7) to allow it to have its own differential slope and remove it from the common

distribution of higher-level effects. This caused the overall ‘within’ polity effect to become

insignificant. The overall mean effect found by Milner and Kubota appears to be solely the

result of a single outlying country, and this is made clear by figure 3. Their use of FE to get

“rid of proper nouns” (King, 2001 p504) misleads because it is a specific entity (Bangladesh)

7
The dummied variables are now effectively fixed effects. As they are no longer shrunk based on a common
variance (see equation 13), the value of the dummy coefficient is greater than the values of the points plotted
in figure 1.

A4
rather than a common global effect that is driving the supposedly causal relationship. This

shows the importance of assessing outliers in the effect as well as the constant, and this is

difficult to do in a FE framework. In contrast, RCMs do this almost automatically, and by

using dummy variables to model these outliers “the specifics of people and places are

retained in a model, which still has a capacity for generalisation” (Jones, 2005 p255). Whilst

India would be an even more extreme outlier in terms of its raw slope residual, it has very

little variation in its within polity score, making its unusual slope much less reliable than that

of Bangladesh. There is thus substantial shrinkage for India’s slope residual (see equation

13), and the result is shown in figures 2 and 3, reflecting the fact that its effect on the mean

coefficient is minimal in comparison to that of Bangladesh. FE dummy coefficients are un-

shrunken so it is not possible to consider distinguishing between a reliably unusual country-

specific effect, and an unreliably unusual one, in this way.

A further advantage of random coefficient models is the potential to use variance functions

to ascertain how variance changes with polity score (see Figure 4). We see that there is

much greater variation (conditional on the fixed part of the model) between countries with

a low within polity score than those with a high within polity score; assuming a general

trend towards democracy over time, this suggests that countries tariff rates become more

alike as they move towards democracy. At level 1, there was evidence of a linear variance

function8, whereby countries are slightly more volatile between occasions in their trade

policy where there has been a move towards democracy.

8
Including the term associated with within polity did not reduce the deviance, so there was no evidence
for a full quadratic variance function. The linear variance function equation reduces from equation 17
to: 2 . See Bullen et al. (1997).

A5
In model 8, a cross level interaction was included to attempt to explain the variation in the

slopes with the within polity score seen in Figures 2 and 3. Whilst the overall effect of

within-country democracy was and still is insignificant, there does appear to be differential

effects for different countries. In fact Figure 5 shows that, for countries that are generally

(historically, over the long term) undemocratic, the effect of an increase in democracy is in

the opposite direction to that suggested by Milner and Kubota – as they become more

democratic they tend to increase tariff rates. This is an interesting result, which suggests

very different causal explanations to the uniform effect posited by Milner and Kubota. The

world is messier and more heterogeneous than a FE model allows it to be, and that

messiness needs to be considered before researchers can be sure of the substantive

meaning of their results.

Acknowledgements

Thanks to Helen Milner and Keito Kubota for making their data available to us, and again to all those

acknowledged in the full paper. None are responsible for what we have written.

Stata code for the models

RE models can be fitted easily in Stata using the xtmixed command. However, for the most
complex models (for example with complex variance at the occasion level), the command
‘runmlwin’ (Leckie and Charlton, 2013) can be used. This requires MLwiN to be installed on
the computer; Stata specifies the model, runs it in MLwiN and transfers the results back to
Stata. Below is the code for the models in tables 5 and 6, including the generation of within
and between variables and interaction variables:

*to install runmlwin

ssc install runmlwin, replace

global MLwiN_path "[pathway to MLwiN program for your computer]"

*load the dataset

A6
use Milner1, clear

*generate mean variables

egen gdppc_mean = mean(gdppc), by(ctylabel)

egen polity_mean = mean(polity), by(ctylabel)
egen lnpop_mean = mean(lnpop), by(ctylabel)

*remove missing values

drop if missing(tariff)
drop if missing(gdppc)
drop if missing(polity)
drop if missing(lnpop)

*generate within variables

egen date_mean_new = mean(date), by(ctylabel)

egen gdppc_mean_new = mean(gdppc), by(ctylabel)
egen polity_mean_new = mean(polity), by(ctylabel)
egen lnpop_mean_new = mean(lnpop), by(ctylabel)

gen datew = date - date_mean_new

gen gdppcw = gdppc - gdppc_mean_new
gen polityw = polity - polity_mean_new
gen lnpopw = lnpop - lnpop_mean_new

drop gdppc_mean_new
drop polity_mean_new
drop lnpop_mean_new

*center variables

sum gdppc, meanonly

gen cgdppc = gdppc - r(mean)
sum date, meanonly
gen cdate = date - r(mean)
sum lnpop, meanonly
gen clnpop = lnpop - r(mean)

sum gdppc_mean, meanonly

gen cgdppc_mean = gdppc_mean - r(mean)
sum lnpop_mean, meanonly
gen clnpop_mean = lnpop_mean - r(mean)

*generate cross-level interactions etc

generate politywxpolity_mean = polityw*polity_mean

generate SAsia = 0

A7
replace SAsia = 1 if ctylabel == 68
replace SAsia = 1 if ctylabel == 61
replace SAsia = 1 if ctylabel == 76

generate bangla = 0
replace bangla = 1 if ctylabel == 61

generate BanglaXPolityw = bangla*polityw

*generate the matrix for the linear variance function at level 1 in models 6-8

matrix A = (1,1,0)

*set the nature of the data (needed for xtreg)

tsset ctylabel date, yearly

*run the models

runmlwin tariff cons, level2(ctylabel: cons) ///

level1(date: cons) nopause rigls
estimates store REnull

xtreg tariff polity clnpop cgdppc cdate, fe

estimates store FE

runmlwin tariff cons polity clnpop cgdppc cdate, ///

level2(ctylabel: cons) level1(date: cons) nopause rigls
estimates store RE

runmlwin tariff cons polityw lnpopw gdppcw datew polity_mean ///

clnpop_mean cgdppc_mean, level2(ctylabel: cons) ///
level1(date: cons) nopause rigls
estimates store REwb

runmlwin tariff cons polityw lnpopw gdppcw datew polity_mean ///

clnpop_mean cgdppc_mean SAsia, ///
level2(ctylabel: cons) level1(date: cons) ///
initsprevious nopause rigls
estimates store mod5

runmlwin tariff cons polityw lnpopw gdppcw datew polity_mean ///

clnpop_mean cgdppc_mean SAsia, ///
level2(ctylabel: cons polityw) level1(date: cons polityw, elements(A)) ///
initsprevious nopause rigls
estimates store mod6

runmlwin tariff cons polityw lnpopw gdppcw datew polity_mean ///

clnpop_mean cgdppc_mean SAsia BanglaXPolityw, ///
level2(ctylabel: cons polityw) level1(date: cons polityw, elements(A)) ///
initsprevious nopause rigls

A8
estimates store mod7

runmlwin tariff cons polityw lnpopw gdppcw datew polity_mean ///

clnpop_mean cgdppc_mean SAsia BanglaXPolityw politywxpolity_mean, ///
level2(ctylabel: cons polityw) level1(date: cons polityw, elements(A)) ///
initsprevious nopause rigls
estimates store mod8
estimates table REnull FE RE REwb, se stats(deviance)

estimates table mod5 mod6 mod7 mod8, se stats(deviance)

A9
 References:

Bell, Andrew, Kelvyn Jones, and Malcolm Fairbrother. 2014. Does democracy encourage
globalisation? A random effects reanalysis of Milner and Kubota's (2005) study of free trade.
In preparation.
Bullen, Nina, Kelvyn Jones, and Craig Duncan. 1997. Modelling complexity: Analysing between-
individual and between-place variation - A multilevel tutorial. Environment and Planning A,
29 (4): 585-609.
Carpenter, James R., Harvey Goldstein, and Michael G. Kenward. 2011. REALCOM-IMPUTE: Software
for multilevel multiple imputation with mixed response types. Journal of Statistical Software,
45 (5): 1-14.
Jones, Kelvyn. 2005. Random reflections on modelling, geography and voting. Chap. 28 In Research
methods in the social sciences, edited by Bridget Somekh and Cathy Lewin. 252-55. London:
Sage.
King, Gary. 2001. Proper nouns and methodological propriety: Pooling dyads in international
relations data. International Organization, 55 (2): 497-507.
Leckie, George, and Chris Charlton. 2013. runmlwin: A program to run the MLwiN multilevel
modelling software from within Stata. Journal of Statistical Software, 52 (11).
Milner, Helen V., and Keito Kubota. 2005. Why the move to free trade? Democracy and trade policy
in the developing countries. International Organization, 59 (1): 107-43.
Rasbash, Jon, Chris Charlton, William J Browne, M Healy, and B Cameron. 2013. MLwiN version 2.28.
Centre for Multilevel Modelling, University of Bristol.

A10
 Table 4: Variables in the trade liberalism analysis, including the amount and proportion of

variance that occurs at level 2.

Variable Explanation Data Type Level 2 Variance VPC

Tariff Unweighted statutory tariff rate Continuous 117.220 0.582
Polity Summary measure of regime type - values Ordinal (but 37.575 0.717
between -10 (autocratic) and 10 (democratic) treated as
[lagged 1 year] continuous)
GDPpc Per capita real GDP [lagged 1 year] Continuous 1.55e7 0.940
LnPop Natural Log of population [lagged 1 year] Continuous 2.281 0.993
9
Date Year of tariff measurement Continuous 1.986 0.071

Table 5: The estimates for trade liberalism analysis

1. null 2. FE 3. Standard RE (with 4. Within-between

10
heterogeneity bias) RE
Coefficient S.E. Coefficient S.E. Coefficient S.E. Coefficient S.E.
Fixed Part
Constant 19.672 1.162 21.954 0.283 21.868 0.960 20.892 0.990
Polity -0.227 0.086 -0.210 0.076 -0.227 0.086
Lnpop –gm 37.788 6.257 3.322 0.618 37.788 6.240
GDPpc –gm 0.001 0.000 -0.001 0.000 0.001 0.000
Date –gm -1.813 0.162 -0.996 0.066 -1.813 0.161
Polity mean –gm -0.055 0.161
Lnpop mean –gm 3.202 0.638
GDPpc mean -0.001 0.000

Random Part
Level 2: country
117.220 19.093 74.016 12.134 77.838 12.581
Level 1: date
84.342 4.590 56.300 3.064 53.581 2.917

-2*loglikelihood: 5854.063 5532.410 5499.771

9
With balanced data, the variable Date would have zero level 2 variance. However because of the imbalance
of the dataset, there is a small amount of between-variation.
10
Note that the within estimates were calculated using the variables of the form in model 4. Note
also that the ‘between’ means were calculated using the full data, prior to listwise deletion. The within
components were calculated using the unit means of the cut down data, to preserve orthogonality.

A11
 Table 6: extensions to the RE model for the trade liberalism analysis

5. with SAsia 6. RCM polity with 7. With Bangladesh 8. With cross level
dummy L1 linear variance dummy interaction
Coefficient S.E. Coefficient S.E. Coefficient S.E. Coefficient S.E.
Fixed Part
cons 19.004 0.779 19.144 0.784 19.016 0.777 19.096 0.780
Polity W -0.227 0.086 -0.143 0.187 -0.015 0.132 -0.135 0.138
Lnpop W 37.788 6.247 45.380 6.303 42.916 6.081 40.367 6.155
GDPpc W 0.001 0.000 0.001 0.000 0.001 0.000 0.001 0.000
Date W -1.813 0.161 -2.000 0.159 -1.942 0.155 -1.901 0.155
Polity mean –gm -0.203 0.123 -0.227 0.123 -0.246 0.122 -0.203 0.123
Lnpop mean –gm 1.654 0.519 1.687 0.508 1.643 0.498 1.741 0.505
GDPpc mean -0.001 0.000 -0.001 0.000 -0.001 0.000 -0.001 0.000
11
SAsia dummy 36.107 4.267 30.566 4.071 33.543 4.038 33.985 4.103
[Link] W -3.605 0.657 -3.614 0.634
Polity [Link] mean -0.078 0.036

Random Part
Level 2: country
40.281 7.116 41.676 6.984 40.941 6.894 40.836 6.877
-3.429 1.233 -2.778 0.805 -2.389 0.751
(Polity W) 1.102 0.300 0.378 0.123 0.306 0.106
Level 1: date
53.693 2.919 38.820 2.196 39.775 2.237 39.646 2.229
1.220 0.158 1.223 0.170 1.193 0.181
(PolityW)

-2*loglikelihood: 5445.315 5284.145 5265.084 5260.55

11
SAsia Dummy includes three countries: Bangladesh, India and Pakistan. They could be fitted as a single term
(rather than as three separate dummies) without any significant increase in the model deviance.

A12
Figure 1: Plot of level 2 shrunken (intercept) residuals from model 4 of the trade liberalism
analysis, with 95% confidence intervals

40
Residual associated with the constant (u0j)

Bangladesh
30 India
Pakistan

20

10

-10

-20

-30
0 20 40 60 80 100

Country Rank

A13
 Figure 2: Plot of level 2 random slope shrunken residuals associated with the within polity

coefficient from model 6, with 95% confidence intervals.

4
Residual associated with the polity coefficient (u1j)

Bangladesh
India
Pakistan
2

-2

-4
0 20 40 60 80 100

Country Rank

A14
Figure 3: Predictions of the within-effects of polity on each country’s tariff rate, from model

7 (with other variables kept constant).

100

Bangladesh
80

Pakistan
Predicted Tariff Rate

60
India

40

20

-20 -15 -10 -5 0 5 10 15

Polity Score (within)

A15
Figure 4: Variance functions at level 1 and level 2 for the within polity effect, from model 7.

With 95% confidence intervals.

350

300

250
Variance

200

150

100
Level 1
50

Level 2
0
-15 -10 -5 0 5 10 15

Polity Score (within)

A16
Figure 5: Cross level interaction between the within and between effects of polity, with lines

for countries with a mean polity score of +6 and -6 over the period of measurement. With

95% confidence intervals.

34

32

30

28
Predicted Tariff Rate

Mean polity score of +6 Mean polity score of -6

26

24

22

20

18

16

14

12
-15 -10 -5 0 5 10 15

Polity Score (within)

A17