week y -

2
.

One .

Drmensconcl Search method '

want to minimize f : → / R one- dimensconal search .

Def f is unrmodal f f has only one local minimizer .

e Newton s '
method .

want to
minimize a unimodal funetron f : IR - →IR over a closeTntervaldenored
by
I
=
fao .
bo]

Want a
sequence
xk that totheminTmtr xA
converges

Find xt sach that fl ) |

(x
x= x
*
=
0
.

FONC

Assumptron We
t
:
can evaluatef ( xe ,
(
f
'
p el ),f
(

"
( xlk ) '

t procedures .

Setstep
↑) k = o andchoose initidpoint
.
,

2) At each dlk) , approximate fea) using quodrattc functron .

qe () = f( xk)
π ( a fayor
)
( + 4 - - series expansion ,

.n .

Properes . a
)
qk ( x(
k )
) = f (xk ))

qk (xk = ' (xk )

) )
b) f

qe (xtk ) A
"
c ) ) "

( xk )
= '

3) Set xkt by minTmizer of fk (.

the

k
'

f' (
0
"
=
)←
fk '
) [x
(
( >Stcfe,FoNC .
p
π
)
=
0 - -
) f
x ,
.

④
=
p
.

]π

(
k ( - f
'
( 7 = k

…
*)
"()

4) set k kt and
go to step 2)
=

.
 qo (x) f(
a)
.

qikxl .
⑨

: '
(l )
Aminimizeroffix ) xI
minrnizer
(
x x x .
.
minimTzerofGox ! .

)
intalpont A minTmizer xA

f
Convergence .

) when f " (
x) ) o for all xEI ,
tworkswell .

2) when fl () [ ofor some x ε I, itmay

taltoconverge t
. minins
.
xA
( Eu)

…
,
⑨ )
( ) ( 2)
x π
()
xA (
@) x x

>

moves to Maximizer
.

Newton' sMethods of Tangents .

Let g (x) flx )

Then the Newton method (*)

equal
'

,
s τs to

Find xA sach that *

g (( ) = 0 LFONC
I of equatrong( =o
at the .
)
τs root

Then
xkl )
k1 y (
,

) =
x 71
(AA)
-

'

g (xk)

L) which es called Newton '

s method of tangent .
 gx)?

'

fio
'
=
gbiky slope
k)
:
rs
.

glxk) )
-

." n

: ""

ome
)

A Gradcent Methods
1.
F : 1R
"
→ R]

o Gradieat Algorrthm .

Consider minimizef(=)

where tRn , R τs the objervetunetton and EBn esdecisonvector.

FACT] DfI 1τ smax-rate ascendingdirectron of f at ( for a small

displacement ≥)
b)

and IIpfall is the rate .

proof Consider diretcon of I wenlll

= I and
any

φ (+) = f (e + t )
φ
'
Df ( attd] d
(+ ) :
.

$ % = Df ( ) ( directronah
) derivative
schwartE Inequality upperbounded
.

r f
.

Rate of
change ε.| Df
ns Yns
<
= xf( ), >
)
≤ fl

rate of charye ≤ ( / pf ) ll

Therefore ifwe choose =

f matimun roteascendang
directon
.

lIIs the
.

Evaluate fat anewpoint taIsing a

Taylo . serres form .

small o
functon of α
- .

we
conseder φ ( × )= f ( +α )= f( )+ Df). d
_
α + o ( x)

× ×
LJ ]
=
IT ☆f( )α

EIXnJ CnX.
]

If we choose I = a
f
7 (y)
 + small o of α
.

then ,
f (2 -α 7f ( ) = f( )- 11f ( |β + 0 (l

Thus ,
if |7 f( ] t, for sufficientysmalla 5o .

we have fC - α Df( )] < f( )

TI terns of mininiaation
A
.

Thrs means that - XIDf( 7)esan improvement point over the current point -

)
improvement is
guaranteed .

Then
,
we obtantheTteratrve algorithm called GradcentAlgorithm .

) xk) α 7 (kf
)
) tool
k if step size τs not
guaranteed
-

is
arge rmprovement
-

ystepsize Xk
.

Choosing step size Xk is called Iine search .

4
along the direccron -I f(≥) 7

Whea to the ?
stop cteration
1

FONC 11 pf( = 0 .
>
snot practical

Practrcal Condition .

.
Gradrent conditron : | / π f( , 1I < E

o . 1f "
.
Sucesgileobjestivedifference … (
|
-f
(
|L .
C
-
.

(
. Saccecsive pornt

| -
k
II LE or . 1
LE .

k)
11 11