Nonlinear Modeling
Nonlinear Relations wrt X – Linear wrt bs
1) Polynomial Models: E Yi = b 0 + b1 X i + b 2 X i2
E Yi
= b 0 + b1 X i + b 2 X i2 = 0 + b1 + 2b 2 X i = h ( X i )
X i X i
b0
E Yi E Yi E Yi
=1 = Xi = X i2 None are functions of β = b1
b 0 b1 b 2
b 2
1
2) Transformed Variable Models: E Yi = b 0 + b1 ln ( X i1 ) + b 2
X i2
E Y=b 1 E Y 1
= h1 ( X i1 ) = h2 ( X i 2 )
i i
1 = −b 2
X i1 i1
X X i 2 X2
i2
b0
E Yi E Yi E Yi 1
=1 = ln ( X i1 ) = None are functions of β = b1
b 0 b1 b 2 X i2 b 2
In each case: E Yi = f ( Xi , β ) = X'i β
1
Case 1: X'i = 1 Xi X i2 Case 2: X'i = 1 ln ( X i1 )
X i2
Nonlinear Regression Models
Nonlinear Regression models often use γ as vector of coefficients to distinguish from linear models:
Exponential Regression Models (Often used for modeling growth, where rate of growth changes):
E Yi E Yi
E Yi = 0 exp ( 1 X i ) = exp ( 1 X i ) = 0 X i exp ( 1 X i ) functions of γ
0 1
f ( Xi , γ ) = 0 exp ( 1 X i ) X'i γ
More general exponential model (with errors independent and N 0, 2 ) : ( )
Yi = 0 + 1 exp ( 2 X i ) + i Typically, 0 0, 1 0, 2 0
Intercept: E (Yi | X i = 0 ) = 0 + 1 (1) = 0 + 1
Asymptote: E (Yi | X i → ) = 0 + 1 (1) = 0
0.693 0.693
"Half-way" Point: E Yi | X i = = 0 + 1 exp 2 = 0 + 1 exp ( −0.693) = 0 + 1
2 2 2
Data Description - Orlistat
• 163 Patients assigned to one of the following doses
(mg/day) of orlistat: 0,
60,120,150,240,300,480,600,1200
• Response measured was fecal fat excretion (purpose is
to inhibit fat absorption, so higher levels of response
are considered favorable).
• Plot of raw data displays a generally increasing but
nonlinear pattern and large amount of variation across
subjects.
Fecal Fat Excretion vs Orlistat Daily Dose
60
50
40
Percent FFE
ffe
30
meanffe
20
10
0
0 200 400 600 800 1000 1200 1400
Dose
Nonlinear Regression Model - Example
1x
Y = 0 + +
2 + x
Simple Maximum Effect model:
0 ≡ Mean Response at Dose 0
• 1 ≡ Maximal Effect of Orlistat (0+ 1 = Maximum Mean Response)
• 2 ≡ Dose providing 50% of maximal effect (ED50)
Nonlinear Least Squares
1 X i
f ( Xi , γ ) = f i ( γ ) = f ( 0 , 1 , 2 ) = 0 +
2 + Xi
f ( Xi , γ ) f i ( γ ) Xi − 1 X i
= Fi ( γ ) = = 1
γ ' γ ' + ( 2 + X i )
2
2 X i
1 X1
+
f1 ( ) 2 + X1
0
Y1
Y= f (γ) = =
Yn f n ( ) X
0 + 1 n
2 + X n
X1 − 1 X 1
1 2
F1 ( ) 2 + X 1 ( 2
+ X n)
F(γ) = =
Fn ( ) Xn − 1 X n
1
+X 2
2 n ( 2
+ X n)
F acts like the X matrix in linear regression (but depends on parameters)
Nonlinear Least Squares
^ ^ ^
Goal: Choose 0 , 1 , 2 that minimize error sum of squares:
2
n
1X i
Q = SSE ( γ ) = Yi − 0 + =
+
i =1 2 X i
= ( Y - f ( γ )) ( Y - f ( γ ))
'
n
Q 1X i
= −2 Yi − 0 + Fi ( j ) j = 0,1, 2
j 2 + X i
i =1
Q set
= −2 Y - f ( γ ) F ( γ ) = 0 0 0
T
γ '
Estimated Variance-Covariance Matrix
-1
^ ^' ^
s 2
γ = s F F
2
T
^
^
Y-f Y-f
s2 =
n− p
-1
^
^' ^
s i = s F F
( i+1,i+1)
^ ^
Note: KNNL uses g for γ and D for F
Orlistat Example
• Reasonable Starting Values:
– 0: Mean of 0 Dose Group: 5
– 1: Difference between highest mean and dose 0
mean: 33-5=28
– 2: Dose with mean halfway between 5 and 33: 160
• Create Vectors Y and f (0)
• Generate matrix F(0)
• Obtain first “new” estimate of
• Continue to Convergence
Orlistat Example – Iteration History (Tolerance = .0001)
iteration g0 g1 g2 SSE Delta(g)
0 5.0000 28.0000 160.0 13541.6
1 6.2379 28.5863 140.9 12945.5 365.5745418
2 6.1771 28.1281 133.7 12942.9 52.82513814
3 6.1507 27.9163 129.9 12942.2 14.44158887
4 6.1361 27.7967 127.8 12942.0 4.506448063
5 6.1277 27.7272 126.5 12941.9 1.510150161
6 6.1227 27.6861 125.8 12941.9 0.526393989
7 6.1197 27.6615 125.4 12941.9 0.187692352
8 6.1180 27.6467 125.1 12941.9 0.067822683
9 6.1169 27.6377 125.0 12941.9 0.024703325
10 6.1162 27.6323 124.9 12941.9 0.009040833
11 6.1158 27.6291 124.8 12941.9 0.003318268
12 6.1156 27.6271 124.8 12941.9 0.001220029
13 6.1155 27.6259 124.8 12941.9 0.000449042
14 6.1154 27.6251 124.7 12941.9 0.000165379
15 6.1153 27.6247 124.7 12941.9 6.09317E-05
^ 27.62 X
Y = 6.12 +
124.7 + X
Fitted Equation, Raw Data - FFE vs ODD
60
50
40
fecal fat excretion
ffe
30
f(odd,bhat)
20
10
0
0 200 400 600 800 1000 1200 1400
orlistat daily dose
Variance Estimates/Confidence Intervals
2
163
^
i
Y − f i γ
s 2 = i =1 = 80.89
163 − 3
1.1594 −0.7219 15.609
' ^ -1
2
s γ = s F F = −0.7219 130.14
^ ^
2
12.081
15.609 130.14 2238.76
Parameter Estimate Std. Error 95% CI
0 6.12 1.08 (3.96 , 8.28)
1 27.62 3.48 (20.66 , 34.58)
2 124.7 47.31 (30.08 , 219.32)
Notes on Nonlinear Least Squares
Large-Sample Theory:
When i ~ N ( 0, )
^
2
independent , for large n: γ is approximately normal
-1
^ ^ ^
^ ^
E γ γ Approximate 2
γ estimated by s γ = MSE F 'F
2
( )
^
γ ~ N γ , 2 ( F'F )
-1
(Approximately)
For small samples:
• When errors are normal, independent, with constant variance, we can often use the
t-distribution for tests and confidence intervals (software packages do this implicitly)
• When the extent of nonlinearity is extreme, or normality assumptions do not hold,
should use bootstrap to estimate standard errors of regression coefficients
Example
Many patients get concerned when a medical test involves an injection of
radioactive material. For example, to scan a gallbladder, a few drops of
Technetium-99m isotope are used.
Half of the Technetium-99m would be gone in about 6 hours. However, it takes
about 24 hours for the radiation levels to reach what we are exposed to in day-to-
day activities.
Below is given the relative intensity of radiation as a function of time.
If the level of the relative intensity of radiation is related to time via an
exponential formula
The value of the regression constants 𝐴 and λ,
Example contd…
Example contd…