Deterministic & Stochastic Statistical

Methods
(20A54404)

Lecture Notes

II – BTECH

Prepared by

[Link]

Department of Humanities & Sciences

VEMU INSTITUTE OF TECHNOLOGY

(Approved by AICTE, New Delhi and Affiliated to JNTUA, Ananthapuramu)
Accredited by NAAC, NBA ( EEE , ECE & CSE) & ISO 9001-2015 Certified Institution
Near Pakala. P. Kothakota, Chittoor-Tirupati Highway
Chittoor, Andhra Pradesh -517112
Website: [Link]
 R20 Regulations
JAWAHARLAL NEHRU TECHNOLOGICAL UNIVERSITY ANANTAPUR
(Established by Govt. of A.P., ACT No.30 of 2008)
ANANTAPUR – 515 002 (A.P) INDIA

Course Code Deterministic & L T P C

Stochastic Statistical
20A54404 3 0 0 3
Methods
Course Objectives
 Study of various Mathematical Methods and Statistical Methods which is needed for Artificial
Intelligence, Machine Learning, and Data Science and also for Computer Science and
engineering problems.
Course outcomes (CO) : After completion of the course, the student can able to
CO-1: Apply logical thinking to problem-solving in context.
CO-2: Employ methods related to these concepts in a variety of data science applications.
CO-3: Use appropriate technology to aid problem-solving and data analysis.
CO-4: The Bayesian process of inference in probabilistic reasoning system.
CO-5: Demonstrate skills in unconstrained optimization.
Syllabus
UNIT - I- Data Representation
Distance measures, Projections, Notion of hyper planes, half-planes. Principal Component Analysis-
Population Principal Components, sample principal coefficients, covariance, matrix of data set,
Dimensionality reduction, Singular value decomposition, Gram Schmidt process.
UNIT - II - Single Variable Distribution
Random variables (discrete and continuous), probability density functions, properties, mathematical
expectation Probability distribution - Binomial, Poisson approximation to the binomial distribution and
normal distribution their properties-Uniform distribution-exponential distribution.
UNIT – III- Stochastic Processes And Markov Chains:
Introduction to Stochastic processes- Markov process. Transition Probability, Transition Probability
Matrix, First order and Higher order Markov process, step transition probabilities, Markov chain,
Steady state condition, Markov analysis.
UNIT – IV- Multivariate Distribution Theory
Multivariate Normal distribution – Properties, Distributions of linear combinations, independence,
marginal distributions, conditional distributions, Partial and Multiple correlation coefficient. Moment
generating function. BAYESIAN INFERENCE AND ITS APPLICATIONS: Statistical tests and
Bayesian model comparison, Bit, Surprisal, Entropy, Source coding theorem, Joint entropy,
Conditional entropy, Kullback- Leibler divergence.
UNIT – V- Optimization
Unconstrained optimization, Necessary and sufficiency conditions for optima, Gradient descent
methods, Constrained optimization, KKT conditions, Introduction to non-gradient techniques,
Introduction to least squares optimization, Optimization view of machine learning. Data Science
Methods: Linear regression as an exemplar function approximation problem, linear classification
problems.
Textbooks:
1. Mathematics for Machine Learning by A. Aldo Faisal, Cheng Soon Ong, and Marc Peter
Deisenroth
2. Dr.B.S Grewal, Higher Engineering Mathematics, 45th Edition, Khanna Publishers.
3. Operations Research, S.D. Sharma
Reference Books:
1. Operations Research, An Introduction, Hamdy A. Taha, Pearson publishers.
2. A Probabilistic Theory of Pattern Recognition by Luc Devroye,. Laszlo Gyorfi, Gabor Lugosi.
Page 2 of 131
Page 3 of 131
Page 4 of 131
Page 5 of 131
Page 6 of 131
Page 7 of 131
Page 8 of 131
Page 9 of 131
Page 10 of 131
Page 11 of 131
Page 12 of 131
Page 13 of 131
Page 14 of 131
Page 15 of 131
Page 16 of 131
Page 17 of 131
Page 18 of 131
Page 19 of 131
Page 20 of 131
Page 21 of 131
Page 22 of 131
Page 23 of 131
Page 24 of 131
Page 25 of 131
Page 26 of 131
Page 27 of 131
Page 28 of 131
Page 29 of 131
Page 30 of 131
Page 31 of 131
Page 32 of 131
Page 33 of 131
Page 34 of 131
Page 35 of 131
Page 36 of 131
Page 37 of 131
Page 38 of 131
Page 39 of 131
Page 40 of 131
Page 41 of 131
Page 42 of 131
Page 43 of 131
Page 44 of 131
Page 45 of 131
Page 46 of 131
Page 47 of 131
Page 48 of 131
Page 49 of 131
Page 50 of 131
Page 51 of 131
Page 52 of 131
Page 53 of 131
Page 54 of 131
Page 55 of 131
Page 56 of 131
Page 57 of 131
Page 58 of 131
Page 59 of 131
Page 60 of 131
Page 61 of 131
Page 62 of 131
Page 63 of 131
Page 64 of 131
Page 65 of 131
Page 66 of 131
Page 67 of 131
Page 68 of 131
Page 69 of 131
Page 70 of 131
Page 71 of 131
Page 72 of 131
Page 73 of 131
Page 74 of 131
Page 75 of 131
Page 76 of 131
Page 77 of 131
Page 78 of 131
Page 79 of 131
Page 80 of 131
Page 81 of 131
Page 82 of 131
Page 83 of 131
Page 84 of 131
Page 85 of 131
Page 86 of 131
Page 87 of 131
Page 88 of 131
Page 89 of 131
Page 90 of 131
Page 91 of 131
Page 92 of 131
Page 93 of 131
Page 94 of 131
Page 95 of 131
Page 96 of 131
Page 97 of 131
Page 98 of 131
Page 99 of 131
Page 100 of 131
Page 101 of 131
Page 102 of 131
Page 103 of 131
Page 104 of 131
Page 105 of 131
Page 106 of 131
Page 107 of 131
Page 108 of 131
Page 109 of 131
Page 110 of 131
Page 111 of 131
Page 112 of 131
Page 113 of 131
Page 114 of 131
Page 115 of 131
Page 116 of 131
Page 117 of 131
Page 118 of 131
Page 119 of 131
Page 120 of 131
Page 121 of 131
Page 122 of 131
Page 123 of 131
Page 124 of 131
Page 125 of 131
Page 126 of 131
Page 127 of 131
Page 128 of 131
Page 129 of 131
Page 130 of 131
Page 131 of 131
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
Page 2 of 51
Page 3 of 51
Page 4 of 51
Page 5 of 51
Page 6 of 51
Page 7 of 51
Page 8 of 51
Page 9 of 51
Page 10 of 51
Page 11 of 51
Page 12 of 51
Page 13 of 51
Page 14 of 51
Page 15 of 51
Page 16 of 51
Page 17 of 51
Page 18 of 51
Page 19 of 51
Page 20 of 51
Page 21 of 51
Page 22 of 51
Page 23 of 51
Page 24 of 51
Page 25 of 51
Page 26 of 51
Page 27 of 51
Page 28 of 51
Page 29 of 51
Page 30 of 51
Page 31 of 51
Page 32 of 51
Page 33 of 51
Page 34 of 51
Page 35 of 51
Page 36 of 51
Page 37 of 51
Page 38 of 51
Page 39 of 51
Page 40 of 51
Page 41 of 51
Page 42 of 51
Page 43 of 51
Page 44 of 51
Page 45 of 51
Page 46 of 51
Page 47 of 51
Page 48 of 51
Page 49 of 51
Page 50 of 51
Page 51 of 51
 LAY Page 2 of 83

stsiburt
uli varnlade Norma

T aivenioe Mocm datsbtn

X NU ).

- V2T
15-Vosate onal cl'stobst

(x, )
, y,,y,
Si-Vasiate

Moltivagte
-
Morma distoibotin

Noema dstoibtiy

The K Nosiate Norma dsiutn sie b

uohene
- - ik

Lx
A
 Page 3 of 83

M Noreiate Nosmal dtbut

Normal dstobstt Ce hae
Bi-Vaxade
Bi-

fc Cu)«(au)
whna k=
1

Le 3 Cadastha Kenden voba

nd

P
-Vooenm mato E) = =

Us take X-u
A
Eapatio ( beCone ag

AC
K x-u)Al1-u
 Page 4 of 83
that
u roe to d K Soch

t- dpdk, d

Pdf tie

Cor)

K e
dp d,dy
d+p

deite
*

le a oitue
ngeaue defmte )
CieNon-
T A is aoste ceinite
XZ0 Noo-nseztih
matsi ten a No)-neFti
A|Eo
Nepahue
Oon- Bihgdas motoi y
AlZ0
Such ha c'AC L

T Aanktradsi
 Page 5 of 83

e - = c nea =
(-Au) A (-u)= cg)ACey
yc' A @
=y'&Y
A y
(-a)' A (*-u) «'I
Cenade t Jocobis mato
le

ean (3)

keh'y
ke d
dd cl -

+4 +
.

TTe
 Page 6 of 83

'y 9t1,t- +9p

Tun egn (4)becanal a

ek e

P
le.K TT d
P
: dyh=
JT
lel.(y=
el (T
K
 Page 7 of 83

cAcl=

Allellel Asscde k)
A el lel C'ACI
C'acl IIl=|
AT

tel
. Ia
Cam
ean () Seoene
Substtute Valsu

(%-u)
-ala
P2
obobilit dent
TLas
Nooma ckstaibhn
functan uti Varide
 Page 8 of 83

De
A Padimnina Vectos Kndar
Vanable

fos 2P
Sad to hawe a mwlti-vontete nom distibty
4s clany octtn )

f) P(xi, ¥, Kr)

( 21 1 e oE-n)
whne m (mi mp) Vectos
is tha VectoS

is a naco Co-volonee
matot mutt olare nma distsbutey

T shaotcstnotokento sg enhs
Np m
 Page 9 of 83

Muti-Voroiate Nosmadkstsibortic)
ropehie

C
Jont Ta mwti-Nosake ona dkstobutty
Jont adenshy
MN (<) has
.t(ta-)
A
afstobuto-n
Rstsbuterns ano
ShopeTa Corctoun 5 tua Jormt
D- menena e0pgnis
dRstsbuto
Meag Canoc: Speck

Iaaistobutin s
pecfied
Te MN' (
and

C4 Moenant Garnat uckiko

MGF
MN (H14)distbtio ho
(t+at).
m(t) =

anatis
e a Se OAI Vectos

ChoRackealstiefuncttn
MN (4) dsteltttieo a CF

e CiTt
t'A+)
VectoS
shena tts a
 Page 10 of 83

C6) Cambinatind

Ca) tet be
Yis MN ) An
mbnett) aT hos a vanate Ooma

distbutten

Le be
A

dstibst a is (a'a aa)

e a be mA
-
B be mgo
o ndan
lstobtt-
a+BY íg MN (a+ Bu, B8
3tandand
d LotZbe n hdpendart
ndoroVan'obl
Tha Ya: + LZ, alistlbty
coit LL hos a M (4,2)
LLT=
(y-u) hos a MN (0,T
Z
cltetoibutn
 Page 11 of 83

4) ndspedana

Ca) On
ndaprdont
Cb) [Link]
Coenpeke Srdapardana

stabutte
mangha seoibtt-)
denna
t M (, 2)
endtten distobutn
damendna stbstee

,
Te m-

MN( 4 +d, 2- )
 LAYL Page 12 of 83

Dstssbutin c CDmbrnota)

Distbutico aXt+btb*a |

X fs Norma
0
whene

dkstlbted kh anttus , o;
indeperdant

fonckin
by Mon
M e +
Y

Mt) e

mornerd genea)
Ml4) Mlar+bt ' functrenn
) Kindap Ka)

Ma4) M(bt)

e o t 4bt +a"o

e et 2
N(A) )
 Page 13 of 83

YN(a (4t») d'o a")

3
van'nu

Y a+bt
N (113)
X + N(4,4)
N (2.2+3., 2+3"3)
3*2
N(lo,31)
 Page 14 of 83

ne mbinota) .
betuoo ko arsable
Lot X2
oith a b a Canstorts
Me ax s E Car)
=a ECKI)

Vovend Var (as) = a v (x1)

E lax E (ar]
FCaCk ECKI)]
aE Cxi-ECK

v Nogina a a iAdri V

Co vosnca bekwea) ak a
Cov C a , ak)= a- ECax)} asi-Eloxi)
E

E Ca- ECKnari- ECK)3

E- ECx)]J
a )

CCov Cai,axi) = ° VCEI)

 Page 15 of 83

Me Line nbiooktc) axtbX is

E (ar+bt) E [a+ E [bx

a EC1)+ bE(*)
EC
E CJ)
E'*)= «'e (x)

E] -EC)
ECK)
Vorsianoa ax +b2 s
a"vC«) +v(x) +2ab Cov(K1 X»)
VCartba)
ob 612

whea
22

tbo+atb
tab ta" ab ]|
abo+ E'ot ab + B .
 Page 16 of 83

et be a Kon) ariabla

PJpl
C +Ga + +Cip P
tCap Rp
Lnea
Y3Ca + Ca kat +CapP
Conbinotka.

CP
Ca Ca CP

y
Y=Cx Canbinatn s
The men the ien

ECy)E Cex) = CECx)

Tha Vorghnca ta Lnedn mbiactèn s
vC)= Cex)= c"v(x)
 Page 17 of 83

fY-Et)} Y-ECY)}
Cov )- GC)= E

E x-E ex-FC)|
Cex)

CECx C-CECK)}|
Cx
eCE)} c'S e- E)3
ECx-
C E» c
i

N
 Page 18 of 83

Jit oobity madfuncti Descete)

e X,y be tuo dimenuro kordern vargblas

thao
tha thn
leastedd
TotatSBbabiltt
PCxY PCx
=, Y=)

Satsfies te olloush ndtibnd

C PCx (-, =Y)

Jatn baiGr lenit furtt Conttrias)

et [Link] an direntna andon Vorsabo takk

Valy Xes hea ae Eb,cyEd
Ta ndkie) Cx,) Cor) fCx= Y#s) Co
y Said to be Tayit bab
funcho olla
Satsftes sgollo Codttd
funct)

fC ) dd
 Page 19 of 83
Fuoekde).

Mangina.
sbl fonctoe X
defnod

PC) PER-)-
T

(PCactimdnu

hamgiroa obobiefsrecho
The 27
clefnod a
) PCx-,-) (- alesoee)

d (A Conthind
ty s) AC9)

Pu Ra -
P
A

PimPx
Pa
P

fom

Ta Ps) Ps) * P) Pn
obabitty dtstutn tabla
Bi-Vane
 Page 20 of 83

Hene P, Y=)
PCa) PCx Y=«%)
P()- PCx=, Y= i,
Tu Cnditira obabile den functhn
2X
i3 e tat = iscdelhad o

furctto
rridtina de
X 1s deAnad as

y(sk) y
nen a unctian
Cmgm
T MF a krdbro Vosbo about

obilie tunctn Fx) s

Mt) od

Mt)E Le

-o
(-for deacck
. e a)
wheat fs e ametey
 Page 21 of 83

honactesistic nck C
a anderr Vanoa
Chonactesiste
dletnad a
dotedb O4) and

D)E
tf) di Ctiovua)

2tpa)d A deoe)
wneng onomet
fsa
Nole
s a dm Vanbka ,and fs a
A X

Mt My Cet)
L Xfs a andan Varsiablo dC fs a
Content

t)
 Page 22 of 83
I3obler)

Vanglabla Xåy haue

) T tuoo dern
oll oebidir den torcttn

a- o(ts)
Oustse taa

Tod C Mag deny unctin

funtder
Crdstten de
Vogienea Voenc

(4 ovarienet Cx

Sol Gay -- Os)e|

, otaaustye

MMagina den tunctun2X_ k 2

pCe)d
-
Pec)da

Ca-74)d o
O
 Page 23 of 83

-
(a-1)

ac)- xc) o7-Co-o-

a--
-
H c) 2

Cohen OeE|

Mo kobbilH de
)d

C) d+4y)d+ fe)d

Co-
2c) CD(Ds]
- -
 Page 24 of 83

Ostt

uoct
Ca)
ondt JenatH

Csndtro dlen orckom X Y=y

C2- )
3-2
4-2-24
3-
oe)e
Condan dei
)
(/)= a(ax-N)
3-24

2 24
3-2
oeCt)E|
 Page 25 of 83

amnce

v() E C EC]
E (x) =
C) d

( )d d

eC9-
E)
12
 Page 26 of 83

Ex) ) ) d
-
d + )d +

d«
)-
[+8)-]=- o

-
FC
v(x) E (x)- [ecC)|

(E
25

144
36 25
14-44
 Page 27 of 83

Varance

V()- E(Y*) - [e (Y))

E () d
- + , t)d+ ) dy

(-)

)
(4)#)-)

F
 Page 28 of 83

.F C)
-0
4,u

()

+8)-" y4

[(4)-(] 2

E CY

LEC0
(
5 36-2
44 144 14
vs)=
4-4
 Page 29 of 83

C4 Coveanco Cx

FLY]
CGu C) F y) EC).

ECeY) ) erdg

-) dxd

4-].
- 3-2
 Page 30 of 83

o:moptel hae tetollon

dersa fonct

otuwr
nd Vaue K

a) Ma densty fncht }
Covarln CxY)
3)

Cxeg)= kl4--) 0 ()£2

O otite

Ceu) did
dd
22
kC4--)dadg=| C4k-K-K)ddy=
6

4 K4
- d

8K
k Ltok4=
 Page 31 of 83

Mag deoit o t
C)dg+ Cu)cyt
-
)d

--
9)]
LLo

3-
4
Moga laa Funetio aiven a

s) d
C4--)d
+)t1
 Page 32 of 83

ovosTnc0

22
E )dkdg
2 2

ddy

(-
=

E CX)
 Page 33 of 83

EC= () d
-o
( - (-)d«
O

[A)
)
E CY)=

- (

Cev CxY)=
y3
34
elr]E
6

C) EY

2
3

36-236
 Page 34 of 83

tha istelbutt
Coditiona stsbutn2
Cesceke ordasn Vaniaba)
Tlogha dotobot
PR

É
) dee
) (dsee ero Voaeba)
)

C =°, fy9= oio/2

C3)

Vorsjable
Moai dtibtè entiniou& kon

C Cu a) d

d
 Page 35 of 83

opeie

d=
m Vaslabla
ndapedant

).)
T
Ceodtne etobot
PCAnB) PCe/A) PAnB)
P CAle) Pre) PA

FIY
 Page 36 of 83

opete Csdttteno

c-slu)

y)

deperdent

()
 Page 37 of 83

_ondttr obebi ec
Le (Y) be descee o dmeodena
Than the Croditer obobiity
rdo
masfunctiy
asbbia
, ae
densted
bt
Cxl) denad a
P
PC=, Y=v
ohua Py-4) #o
PtyCl) =
P)
fcto)
tha

8.0en X
clensted Py () and defnd as
b

PT) Pl, Y-) oheo Plx=1)=o

PCx )
Suppos 23
23,4
Ho Y s red
Cendsten at (y= Ard Y2
eo ne Con hang
Py () PCx-,7sa) Vab X
PCY ) CardebeRob
SPP P C)- PCt1,Y=)
PCY-)
 Page 38 of 83

t
C Jstt obsitbilty Cstbuno
endanb ormobo betn

2 3 T
1

2
36

3
36 3 3
36 a
Ta 36
dkatobutrs
Value

tha Volug
nd and tho

Ta
EP)
P 36 36 3
 Page 39 of 83

Tla Magha dat 2 X

T
PCk-) PR Y)
PCx=t)= £ p(x=1, Y=*).
P=1,Y»1) +Pltlhy) Y-4)
PCx=1, Y=a) + PKI

32,+ 34 3

PCx
Plk3) 6t 3

2 3

P 36
 Page 40 of 83

. T Mgide
defad
P-)= Px Y: »)
PY)= PCx-Y Y-)

P )Y 1) + (aY) +
PCx= =1)+ PCxa4

36

PCY-=
PY=
pt-4) 36

Tue meaCis
VotuyYs
PCY-) 36 36
 Page 41 of 83

Te Condt
dlekrad a
Value
Y=
follo
Y
= PCx
Px-/y-i) PCY1

PCx-1/-)= Px=l Y) A
36
PCY-)

PCx-/Y)PCxa Ye1) 3

PCt-a/y-) PC3, Y)3

PY 1 34

P4/y) P4 Y1)
PCY 1) U
36
Canditadt t YI
2 3

Pk-N-

PCy f ) Py:, a
 Page 42 of 83

onen na tunchèn

omend8
Vabiabo
Vartiabo X
Tha moment a ndam
de-hnad a
about Pata A
E [C-A]
The moment abot ponl A

E LCx-AS C-A) P) Ca

t '-A)PCa)d tian Co

he morne erder) Ngioba Xabot

coopted and adefad os

P) dec E)= Pa)

Ca

fCd Cth
Ca

At oogn)
Cle EC a Cal mean
ondero Vasoba X od denoted
o
Ale
 Page 43 of 83
Krow thpt

Var (x) E(x) -ECH)"

Vow
C)4-( X
hiu Voyabto
Th mment onden
obout the

Csuny dendted

_ Cdindnu
tC) d Co

E C-"aVar (x) =1-a°

van
Secand mant
 Page 44 of 83

-funckt (MaF)
Momenthetnah
moment hananat fancite (Ma)
Prdeo
Krdon Vambba abat Ongn) ho

eca) os descacte

M E (**)
for Conkirn
e Pc)d

Smrmoton ef exendad
Tho tegaatuo
to he ertie 9onge
the Ne and
Pbnometeq s
tke ke he
ber umed that the RHS is absoutcl
Csvege«t

Thos Mte)- E()= EC+t+

Cmtet 1+xt *
(El)= e erpen
M) EC)+tE() +ECx) +
+E(+
M) 1t u+ u
M,
 Page 45 of 83

enna the menend enanatg

9tndon Vorablo abouk tte Pan a

defrud a
+-a)
M,) aboot a)

OnnF(e")
Mt(obout aso

M E(
 Page 46 of 83

y too 9mdan
be oriabla3
densit fanckrn
with Tont ob
<
P(9) Azy oeE
t
the mangina danab ekay
ind A, Ao nd
e know te4

fC) dd ded=

A dd I

d=I a

A8& A8
A 9 ()

(
Ct-ot) A-
 Page 47 of 83

.PCes)=
Other

de finchwo
Mg

( )
fc)= 41tt- o
Ot
M dentaht

As) fCus)d 8xy d

8(

43 (-o= 43
)- S4 osl
 Page 48 of 83

(6-x-v); OCX2
Pts) F
Ousrq

s a Jo obabilt de
PCx e o Y23), PCx4Y 23
23)
PCxe/y<a)
P P (xel9. Y<3) =
3
C6--)d dr
O

-t- 8
(¢y3 dd
-4)d
(a--
6C-4
(6-

1)-L(-7.
].() 8
 Page 49 of 83

PCx+y23) X+ E3
C Y 3-
23-
3-
fCy)dy d:
dy e
6--s)
3-X
3-X
3-

2.

6C-1
[a-z-J-l6--4]de
l1+-674 d
C1-)-
2
-xC)

6-6 -2 -d
*+a) d

-4s
X
4-

+ a) (4-)+(8-)
C-+ l-*
 Page 50 of 83

)
PCxeY es)
PCxel 0Y3
PCY3)
3/8
PCya)
mogina de nchen

(6-)d
8
O

e-( -s ()

et) -(a-o)-4
C--)8 to-4 Co-2
ly)-(lo-24
4344
Otr
 Page 51 of 83

PC 3) ftvdy

o2 dy

to ls)(-4)
to-6
8

PCye3
m
PCYea)
 Page 52 of 83
epecbed Val
langin distibsttnd

ae
oe jrto oity
Korhn sbtes
Cistsbutte
PC, Y= 8) en
Cohot an -ta
2
34| /24a4 0

and Fspect ed Vab

EC) EY)
Sol
By te okou tobla ha-e

PCx- 6/24 /4 to

54

Te Mag iyktin
 Page 53 of 83

EG-))+)+ ()
d

F: @)()+ () ) + )
E) =1& 3
24 4
C&fobsten,doperdon
oobebiitY
ondteno
dgtobutem
snditna
8Y epoday.
Cx-/Y-1) Stare 2

PCx=1)
pardan besR

Octu- chg e biat dsttr)

 Page 54 of 83

Pe 4
2-

Pae TApedrt
beloC
otSIdpedat
clange
obabit Catobofr.
Y=1 ach
 Page 55 of 83
S
Ctabut
ndttna itabstan.
the
Glon fenaaa
belae ShnwS
tobla
Ta
Po bos
Blen Ta.
Bo

tor anda
distosbutnoCandag
Csbutm
tc maa
Ha Vamable

Boys 0.5|8S s18

0 481T= 48 17,
G

b) H t otha Variobe
Cplruy efnana Vaab
Gnea)

3333
0. 2513 ADTt7
33 83 77
35-13
 Page 56 of 83

Cmdt ditsb t
Chando Vagioble
Bo

Cardu 2303
Cir s46-17s 33-
a Cell

Varnbe

3 29-277|
C
444 ro
CoQ0
Ea

auatr
e
e) Bosed 00 hoia Calclatn, ttans
is
ndag expan
betue)

Sel betunen

fenana ond da 2 yt
Cmponae

5 5 7 boys blne Cornpanad

4444
f2. 43 7. fo
 Page 57 of 83

)tal Moltpe Coelatt

ectert
Co9oelatde is Cad Net Gnolatia)
anh elatonship betuoealn one
s a stody
and One fodaperdan anabla
dependent Vaneba
Keep tta Ots indeperdant Nooaba

Cantent
tao ondbas s
ondblg
[Link] beseen: Pacdds
hen nofacos
Caalad eno OdaCoeltarert,
held Cant
betuoeen
betuoea)
Cneatt0 is stdied
a a thivd Nonsbe
-tuo osobl asP Csmdath
st de
Ordaj neta
Cnaterdt tt Cold be
CoAciea. heis Oua Vorabla s koptGsod

1a3, 3 3 iÚ13.2
13.
stadied betusea
C a PanhtCoeot tis
Ondy Vastabtu
tuwo ulebte b py Onde!
Cnaterd ould be Collrd SeCand

Ccudatio) Coelcat
12 34
 Page 58 of 83
foLou
Cooa at-
onki

ia:3

Mulite Corulat)
orulatte) se Study Co7oulake
mutipa
beoea 3 or mor Vasabla at a
D Cos muttplo onalcen-
q deperdaat fackost s stdied
stadied
daperdeat a
rmoltapa Conaelatiao fs
T Caeficient
9epsededby R
thua CoeMrcent
Cmedate
Ri23 is uottipla
as aependent Vonaba,
Ndapendant Nanobfeg

31
Ka os deperderthi
moltia Coeln
Vanuabta ,
CoeAAcrent

a
uith
ana

Voaabtau
Mdaperank
mottpCa Coelaten Coeffet uith
Ras Is
dopedant Vanabi
ndopendent oabla
 Page 59 of 83

orneloatt) CoeAent

as -3

a13

3.12

CoeAAoet
lla Zow ndas CsrnelaAten

ani2
13 O 44 O5T
O.98

Calctale utipe relakto Coernt bet

Vanbbla is dependent Vaatabto ond
1s
Second and Thid as dapardont Vanbbe
 Page 60 of 83

Co98 (o.44)" (o-99)(o.44) (o.s

Co.54
9604 O. 1936 O.4657

O. 6883

O:186 0.99
mnsuna d tom
,f3 ane thoua onite
Nelo,Z 10) Zx,"= |60,
thaln meons ustth
thain
EA1?2 = 60, 23 6o, ZX3 40
3 = 40
-t mutpo Culake Cerent
altae
 Page 61 of 83

So tstwe Calcuate

60 0 5
VE2x
O.6

C)+S1s

Co6+(o. 67)"-2-(o-6) (o ) (o-7)

Co.(o.75)
O6989- 0.5025
o.4-37sS

o4489 O.67
4375
 Page 62 of 83

Coet cen dadrn betuwear

Zno
eno Ode oralotn
No votaba
tuo Vanabl i, 13893
held Cretond
has bee
rde CeeAioak a Vauabco

held Cnt Si2: D13 2 23.1

beag
bea9
Naabet hos
Vaable ho3
Second Ordu CAn Tuso
12-34
held ra2Oa3.14 0

Da-24
Dv
34 V
3 Votabl
Das 4
D.v

3l4 Tv
1t2 3

I Dv Dv
3
3 L.v 131 24 Ty
t3 2-
D-V 2O-V
312 9.v 31TV
 Page 63 of 83

Coraletian
pfuance betseanMti d
Cotasadin

MultitkaCmah Cnalaca)

9TuRotaship betusea. Jastndkp bekweao

Vaslaba and a a ostabas

Cooned Vastabo 1s negachl he eect
Vanaba is Ca0od a
Called o molttbla Coralahe
Cooelati

T Ef
Cerorbira
avab

&VasbbCe Varbe
 Page 64 of 83

aletattn Coraelcta)

oblar) Cden daa

) 3 4
O

Cenoeakrni
Clesee ta Cee4Aend nh
toe

23
3

I-13.
T13

I- I-
 Page 65 of 83

sTOelat)
betwean an Jaobla
Tue enArdkip
eac otu Vosab
neelacki
Ce a Coalati

4
5 5 LO

( 2
a4 5
5 3 64 5
15
5 4 5

Z3
720 9 167 92 56s3
e hoe
I S hous 1
29
Shaus Et= 198 E¥ 2 153
Sns 3 2
Z*3 1

13 9
Aa392
 Page 66 of 83

9112 A13
i2.23 23-se

123 2

(24) (2) (2«)

(5Ci5s) () (3o)-

u)-(aa|®)¢1)- ("
&lo
12 ~0-46
VI (23s721) (1t0 -20
45 -45=-0 46
946T
to &)Co)
 Page 67 of 83

9 S - (14) (2»)

VCn(e )oas-(1
9113
VCna(a] [»E*s(t7

(s)(sc)- ()a)
Vun)-][()a)-17)
8o 4-3

(135 81)
835-92)
3 3
(1os) (5) 574
3 -O:48
 Page 68 of 83

E (2«) (29)

2113, (2)(2s)
V[Cnzw(] loa-(a)
()Ca) (s) t)

o5 aHo.

V910-9) 35-81

65 -65

C)(5) A%0

-093
[Link]
,

- O.93
 Page 69 of 83

e hu

911 46

9113 O:4%

O93

12
12-33 2 s 9123

1- V1-s
(-O46)(o.4) (-o.93

1-(o48) V1-o93)
o46 044 64

(1-0 30q)(-o.G679)
-0.ol36
V
Co7616) (o-1351)
O ol3 -O. O13 S
o 1039 322

3 -O004
 Page 70 of 83

(a) 9113 91a l23

I- I-
O.4--(-o-44) (-o93)
I-o4) 1-(-o.13)

- -o 3649

05

v Co.+ea)Otas1)
OS2.
Vo-98e4) Co-1351)
OS22

oO.l06s O 3263
O.159

:2 O59
 Page 71 of 83

9113
913

(o)(-o.44)(o48)
VA-C-o-46 -(o -42)

O. 93 O:2908

o884 (o7616

-Oto92
O 606

O9189
o 91oS
a3
 Page 72 of 83
itisJaundd
(4D n a -vandhe stobutr
tat
13061
a3-040
Vabne i3 2

VC1-(my (1-(
6
O4 07 xO:6
(1-t%+)C1-Co61)
O4 O 429
(-0-4a) -o37
C
51 o.617

02tE
.566
O.048
 Page 73 of 83

VC1-( (1-(%)

-o7) (1-(o.4
61-0 28
CICo.1t)
C-41)
O 33

.51 0.84

O. 33 O33
O6
4-2-84

50t1/
 Page 74 of 83

(6) sn tha olla* data

2
3
3

Connelaton CoeAre
te s4rto molto
3.12
23
Sol
3

6 9 as 41 50
a9

5
3 4 a5 42 30 3
32 32 1

3
64 36 4
2
398 43o 23
34 40 33

34 a98
,430 2X173 2a
Ž 3 = 289
3 33
 Page 75 of 83
Xz
a The olotre) betwea)

EKKa(Exi)(£K)
Vnr-(4 E-(Exy
(s) (2oe) - (34)C4
C+3.)-(40)
C5)()-(34)(5)
819

Cb Ta Coelat)

(s)(2)-(34) (a3
(aar)-(a)
(a7)-(4 J()
O-155
 Page 76 of 83

Ce Th bekuean R X3

() (a)

-
(s )C4a) (s3)
) nEt-(£Rs)"

(5)Caso)-(4) (s)(a3r)-(as)
5748
Nou e oue
(a
a3
3
-81)+.15T1)=2 (-.892)(otss1) (o.5348

(o574)
.66067
O.666

9333
 Page 77 of 83

a.13

(0:5948)2(0.81)([Link]) (o.5948)
039+

O285lo
O.9-59

Ka 3.12

89)(o.1s)(0.5748)
(-IS1)+(06248)-(-o
rL -o 81)

O5134
O.204-336

582
 Page 78 of 83

Valne Ra18

bi3 C.58 ba= O 88

bi3 = O.53 ba 68 b32 -30
we no that

bu ba Co7) ([Link]) 81

D1 bis xb31 Co5)6) :11

)B23 =
ba3 b32 O83

}23

3
A

Co-e1)+ Co 1)-a (o1) (o9) (o.83)

Co.e3)
O 3050
O 3111 O:99
 Page 79 of 83

C
R 13 = G-a Ys Y

(o.s)( 99) (o 83)

Co 1) (o e)-
(o 12)

O13846
ol1 83
 Page 80 of 83

anhr Cooalan. Ce4ftet

Stao13 3

C1-

O23

obear
te Men daa

3
3
lo
A
54 42 30

tad al
onne Iatuy Cetc
 Page 81 of 83

sol

3 69 5%loo 48 440
72 85 loo s184 360

54 36 41316 42 324 398

64 68
442 1269 336
| 990 360
8 |430
188% dOE8 650 7%
34 40 a88
-/4) E)

et-(2 nE-(ar
(5)2o8)-(34) C40)
-0.892
(29%) -(34 (s) (490)(4)
(5)
-(£«)(e¥s)
nE- (e«)Vn£x-(£Ys)
(5)(165o)-(34) (28e) 94-5s
Cs)7-C34)t5) (18s64)--(ase)
 Page 82 of 83

Ks-(x)(z)

(5(76t)- (4) (is)

VCs) (188%4) -(128)
at (5)(43)-(4)

O18355
he ako)
ePrcenH
12-3 O32 Das. .

a 3 O12

VG-i)VG-)
984
0 89a .24
CC-o145) (1- (o-re4y")
O 03788
05829
2 6500
 Page 83 of 83

C 182

-0945-(-o.892) (o98a5)
-(-e1) 1-(o983)
-O 067H&
O 0817o
- B8

a3-1
C N-
O.835-(-o.Bq2) (-o 945

---812) V
I-(-0.945)
[Link]||

.14183
3F754
 Page 2 of 64
ptirmgatièn

at-5
pttmtten OPhm2atia) oblarn eqiru us to
a tunche)
deesmie marimur) ard YUnimum alue
Ophimatktey
Te To
CCngtsonad Optimgatey
Q)UoCantsoinad ria
CD JaConstoaha.d OPtimotia)
Robeng
Tuang ano
Ta ana C) nasmiztn
Cost mndniatin
O
rackhea Coerputatiena tagk 3 odi im
a moy slablas
Cetsad

te desvatie aonette oth sexpecto

StepC nd
ond tn Put it. egual to eso
cohich fs
to d te valsua x and &n
Ca0led a gtattonan t

df
By selu O&0 e houe n ()
stationeng. Prtt
 Page 3 of 64
Stcp-2 tep s rd Seeand OPrdas
Second
functte) taolth eapeck to x ord
desivae
destate a

and z,en Solue t ewatt Ac-8

wsto Cput
Ac-e Ac-e o, thon îs a wstto 6Pf

C exksen ds
C Ac-e=o
(exkeerne prnk
C &) Ac-B'2 O No exttna
)Ac-B= o X Prntt
B

Masma ar) mama , aCasdy

Step
P we Co rd
to,th vate AA
Co)
skep5 Laat step fs to od ta arinuo)

Mahimun Vaue t* ths

extoeno Value
 Page 4 of 64
obler)

Extse2 Vabe
nd tie
and aetesman

hethe tey a aXinum r) ninurm

ien o) -+ -6
3 O-6 0

= 3x"-6t =o

-
3 -61o
3 ) =o

Saln

O
2

( - 2X 0

-8 -o
A
 Page 5 of 64

4 8 = 0

3-8) o

-8 so

3e=

&hen Eo

cwhan

(o,o) (12)
in ang.

SCondOnda nditi
6
3-6 5

A
6

C
 Page 6 of 64

A= 61 B 6, e 6
a)

A = 6to) = o A=o
=E:-6
C=Go) =
0 Ce o
Ac- B
o)Co) -C-6) =- (36) = -36

No Extserna t

A 6) = l2 A 2
B 6
C 6C2) =l2 C 2
Ac- e
Ca) t) -(-6) 144-36 lo>
fs an Extoema t
A 12 0, ate untrna

Extoeene Value a
) 13+ 6 O
a323- 6())
88- 6 C4) = 1634 * =8
So,-8 fs huuwm Value at BH=S
So,fundt fs
 Page 7 of 64

onsfraned Oimitekèo

genna Cavdfronud Obimatian talk is to

Magimge (or) Minine a funcken )

en Centon Congtmnt On

xernpla
MGohmum
ind
he
Comme) to endsukeg hug
Ne
foskcat Vehiclo t a
tor exemga

Constron
eten
Cawested
Cmateand Con be
Canatron

Mnimie ta a,a =|
S ect fo +2t3
neap Censtan&

M
Sbjec- to X1++3 I 2
 Page 8 of 64

Co e stte a
A optimpatn barn
to Centtront ak
xes
Miniizefa) Sbje
sbjec
eaa Rn

S- ,a )= o Ord

Conttsony mtpa
epaly hennty Censta
mwtipla

ton en ()
Con be csten
ua Constoo-nt

ea Cnatsa.n
o
+%-l
heaH CmtsoendG9 e osttt a
h(4) = +72+X3- 4
s a Combthahn ea
Set
Sreaa tsn
S
 Page 9 of 64
fundin
henunadze

Cnstemna.d inatiy
nckt ho enu
mnimiz
disGaenad bolangs
toa
feasiba Set
teehnhpel at achee tte
an
Seana
TT ,a PAat tokm
but

nn pPsDoch
fg to eAa a
AYe- gn
functt-o Ca0l4 aled

L ( =Ca)+ 1

tuoo veCtok

Lagorg Cratgro
fureh
muntur)
Cnatsend

a)=m
A

ond deto che

u ail t koof
(e Corna to oeekj
 Page 10 of 64

inca egRia

Dependant Vosioblo s Contthiou in Dctua

Sope.

unaa) daka
PohB

Ct Mx
iebe
&isLhdapecdot Vameba Lndepedet
Vornobo
dapedat
Yis -etenH
Anean ockun

casttte a
Ts Can be Co

- is
C+mx
ndapedant Voriaba
m
is aSope:
Cis tecapt
7is deperdar Nothabe
moltip nea) gteia)
C

oi Co

i Lrdaperdurt Vastabe
doperdan Vasabo!
3snove)
X3 is moR
 Page 11 of 64

nive X 2, 3, 4
b+a
3 341S*

Y
3
A
3

54 30
lo 19

|a () (e*)- (41) E«)

(e (e
Cio) C(s4)
a (1)(3)- asb o

a 3)- Ctoo) aknwn

59o 540
20
20 0o

a 5 a=5
b 3
b nCeH)- (e)Es)
nE)-(e1)
b)s)- (1) (11). al6-2o- I1
[oo
C3)- loo
 Page 12 of 64

Line Keus Anay8is

Ts se io Uodegtondn
heation betueo -tuo more
the
Vorbla C Multifa ghenn)
to Cas ndenstendag to byiabo
ohen
rdaperdant Varnabt2

t Oti Votabe is cepadart. ala

Redieted roobla)
botb?
bo +bi
e
aals X
RBB
ind a Stsogd he hueh dpedd+

ms e bsesuatiad
bo y-ta
bi Sope

Ts Ca is Cats a hea
inca
fs Obtahad by
ts and
Sauao metued
Lest
han Slope fs
slope e (= betb)
bo-bjx)
hen s Sope fs-ve (=
 Page 13 of 64

rob{arn

2 4 4

| 2
- O O
0%O:3

O 36

s
= 3

bot bt
S bot0*63

E-T)-) 1 (4)
b (-7) 3

2
 Page 14 of 64

2 (-) (y-y)
b

mn Coosdehoe
s Claulaed Isi
C34)

bo t b
= bo t (o 6) 3
bo A-Co 6)3

bo= *3

+0.6

-4 O-B9
(-) S-2
=
n-
Let
Caltae te
actua + sttrted.
 Page 15 of 64

Kegers)
A senptind

o muolth Coltnenty

[Link]+
eron3
A) No Auto COUrelah

Nosm dtstoibsten
ndependes fo cacl oty
A ta ewetiang
 Page 16 of 64

ioea

egnac ate
A Sinplo eernpta
og
Glcoge le»el ie

Sobject Agel loo

A-3
3 &5
42
57
5
55
fs Ldaperdant toa 6 data Poeds
s deperdand

FRndien rorid o
Ta Sinra

X;4 O
be+b crveduen i

Estimate4 Ehmate of tte ra Sope

Cor Redeed) Pguse etapt

Y is dependust
ESe obSeptedi) s Irdaperdat
 Page 17 of 64

Stnple eempto aegre

epatio to edrt tael
uen

C)C)
-(e)4)
Do

Ce)-()

by
C))
Ce)- (e)

AE(x) C)
Subjec
84 98p
99 A257
43
65 13 6S 441 4225
2

3 a5 625624|
3150 5625
42
S 57 A951 3249
59 81 348 6s6
2
34 L486 20485 401 40022
 Page 18 of 64

b(4) () ér)
(2)- |

)
(4
(47) (aoqas)
bo(4-86) (140)-
6 C1401) - (247))Y
.V

bo 66 1
H4-4S

bo65-14

bb )-(<)Ca)
n() -(4
(486)
62048s)- (47)
6CI1401)-(247)

b a868
O.885335
444-5

bo +b
 Page 19 of 64

=bo b

9 = 65-14+0 a852r
Val 5
T Vale Val

=65-14- + (o 3es22r) (35

86. 32A

tenco qgwcse ho

de 5S is 86 32

Hetegoscedastiaty
whad is tta roblarn
+Hetesoscedastict, ekss to
So T blar 1exidualg a
a Studio
do n hae Uatom Vacsnco

Arise han Voraltta S Ooee acsossS

Obsetiad
Teds to e tnetttiend 9eged seu
hcaea 3tendanadego

Heososeedatiry
 Page 20 of 64
LAL

s Choptes we sho Cancex)

Optimpadien
OU Scluel ott t Clawica thea
Tuls theon desl wit te
Use
u's
the
dIfnhia Caluus to deksaa in
Miniroafo both nCstrainad
atmna
Cantiau uneton
and antsolnad

ts Chopte topi&incde
h Csdfine
Suffctendt sdiog
aqd
SulfCent
dleselepraet
ecoda Ao oGngronad
-fos oCatinp ettne Pa

toettment Ta Snsolnad
Tla
ad ta
roblan U oi med
denrpenat khoickao Cmdta
tneunlity ottak.
 Page 21 of 64

robCacns&

we sha
sessg
ttn
Celesr t extsee Pont (tta
laira nma JoCanstrohad

Continioug unctang

lathernatcolly
f-)hos aimun) at-a
A fonett

C)

a unetin ) o a tndmurm at

Potn

+Go+)- ) >o
 Page 22 of 64
1)
Point of 1fleckn

Cornda aCantniaus
as

fianctte£G) detna
inkea(ab)

Cnot s) eeru
al Masla Mnna
(Called te Stodiorony )
T ocAudas
X, 'o Maxim
4 Prvns MGala

lobaakedte asryum
F)ma PC), fC1a), -f(v6)
Sn
ACa) fes
Cae a globa o) abedde xim

Loca(cdote) maxina
P) ang a00a
n Ot hand. fCK) ,

oCa o) relatie maxina

PC is a oca ah
MAnmunh,
Munmsm

Shoud be noted ttat tta Paint A Corasprda

is Caldad ftectn
 Page 23 of 64

Crnd tns Optina

Necetlon SfAcerd

Necegony Condtia)

A eceion Cdttte for a orttndau

O.
Certtndn4 Porst ad SeCond

denuatie hoe estoee Pbtot, at X

Each s desiveee )
eualunte at Yo, Vonish

Caeo
s
Araddent veckos

SRAdedt Csodte)

Suffaed Csrdtè Psra Stotionup

be an fgthat
to exkuerePt
Hession mats evoluated atx s
is
C1 Negade dAnte eahagYo aa Maxionun
t
whan faa
ca) ate deadle iimwent
 Page 24 of 64

roof for sditios

os eceiany Cndti
) raof
heer,foc o <e<||
B oylos
+ hHh I4o4eh

(ha
OD
h Chh-. hj- ha) f.A

h s Smalenough Jal2**9
Smodl tha eondu
heron +ailtend to Zo as hjo
Oncdan h
F(zoth) -f) = v£C)h + O()

Vfh hft) ) +hp

= h th
FRoe hak xo fs on eteme Paî, nsu Ce shal
shl
theos b Csntadt)

Posgia, let us pme that Ona 0f Ponti

dessties Soy pth, does Vonsh
act Vonish

beCsme2 o+h) -f*)*

3)
hp
e
 Page 25 of 64

Sinee )
ethe

No Suppose fC) hen oth) -#(xo)

hoe the Sorne s as he

hp

PCoth) txa)o ushen hp>o

Coth)- fC*o) hen hp
This Cetsadec
estoecnee Pniot
csonen x)
ment amNay to tte

ien abau

Ths Ce Csnclda tht uohan On

desvodies na et detta00 eaua to Zen
a Xo e exoene
Ths, t follas thot or o to be n eKtena
anod s necoi3one tha

thaoem
Cmptase -t
 Page 26 of 64

a) roof Suftint Sdti

oyon thesem +
Bt
ue houe

Co+h) f()=V #)h t h

Yoth

Sinca x sa 9toticneny P. then b akley

theotm (ccouo
Theotn ndHiNheotfn) e haue

Thue Foth)P) =H th XotOh

le o be a latioun rohan bydefottn

a
Xo beto be a Manum
tho
Co)
2
yo+eh

Wth yeeh
padvore -foo th in ezprdadfore)
e houe
hhi
 Page 27 of 64

eves Slaca the xd dervato

Crodindnl
o a eghknuahod

isil hoe Sarre Sign a

s80 ohe)
ndeenty h+h mos Yield tho See
0
euatsted at koth Xo Xo+h
Thus ue hoe

hh

detres a wadrete

epna and herea h

acde- defhte a Xo
eHenien onotss t.
makimettn
Cmgtutt
This fa
mahtmizoctièn Cae
A siila)oof Canbe estokisd for
Hemn mato
shaoa. Couspnd
s stve defne ot
 Page 28 of 64

(1 ntke Maicur amur)

fnctic
+ + X3"- 4 -8X2- 12Xs +5%
.fC)=
eca
APt th C6ndltie

Co)

) = (o, o, o)
73
thi's e
ax- 4- O

12 8 2o

Tha SolAts thex Smotorem egyata Ae by 1

tta Ont faindho

tho oisfe
3disfes
C2, 4 6) hch fs
ecakaid andttn

Nos Crecey tte

Checey suttan ardta, ve ust
Cleeemiaa ashothes o is a Maxum annuy
 Page 29 of 64

Hession moto ewalued at (2,46)

Uen by

T nss eksminand

hose tha Valey 4 8

mnes delegmnat+ s
Trus eeh

)
A
 Page 30 of 64
achihe Kcane.
tiongatienVie
Mach
do ue need OPimiotn
ht
keannin

opie, ae tN a
Ca
Oevigotio fs an
on Aderad
O ete
etc
but ld mony modelk ezpesireat

eushat do yu leaan in optimzztè?

function
Calds
how
Udestmday
t nd MaRia
, Mama tcttng

mandena C)= o
2 2ao

funchin Coud Moe Campicated

usith loss fonctians ML Models

Cloed fs soluttand

Opticnighte) on e Cmeup csth alosttmg

oximamnima 9 thau fonctnY
n elftcent and electe
 Page 31 of 64

onetien&
C) Joderstrdu
moet ML daiven
athne
ose Oectve farcttn lssuoctia
Pind he best Ponoretess -tnot ndnione
Coes

Line

StoothtHCo

Laatte

a ( (t-) a (1-
Jodesstardap ah Soweg 9e
Do os uont to e
RMSRoP Ns ADAM .Vs ocmertm 9
to
Badch Grvadientt Descar Vs Stochatte radied
Cescant
Oescortt s uni odch vadient
 Page 32 of 64

oclion&

Ya hoe a nauw oblenm that doe ndt in

the Setne
deep Leonn
Fasic in the Cndext
loee Poncti
Coston oefoncta) leae ha
ou Con osit a
Soluip to ude plattorm

uiteos fucton
tuckeng&
Optindin leann to
Optimti aves -te
O
Undesstrden ho e
Con ursile Coss tuncten
Fobser
4) wst yetes Soluesos Oie2att
pdate for d8fenart
ndested srth tha
nds Save
Gradent Deicant
nea Loss (u)

Sunmam
phihoo For ach
OPasp*
Joderstdee
Solues
Plck te ighd

4)Snplarerh
Loas
Solueg
furcten&

fos Costom usS

.
ncten
 Page 33 of 64

Non-nesin Dornrmn_

Cn Constsaned
C Divect
--
Optingcdtn

metod
echnpa

Sanch
Descant etuads Co) Gradient metad.

teapext decant (ahy) mekaod

Neatsng mettad
ciiy Fletche eves mettuod

ayaict metg

C Steepeg descant (Cacht meuod

OCadne
C)3tont uwith the a>bitiany init PointX
ser te itesato urnbe) 1=|

Seaneh ditey

S V£ -V-£(*i)
)ind te Opra skep length ia tha diectay
S
Set
it + S =
->vf
OPtinaH
sOptimuarn stopOt
Step-5
T
) et the nes itestie) nunbe)
step-
 Page 34 of 64

Tas metod Coo to bea Ve efecte

UnCansteohed Optt
atienechnipo
O degcant dieehin is
But Soco Steepext
Veny eftctae
not Ven efecti
is
Coca e mtd
opehd, te
oar
ob&mn

CO MGrimze
and
Stocap onte Poto
O8tnp Ceant metd
teatin-:.
DRA

wto X

Substttte

, uened ACta OPtinaStp legt

S0 e mrm

sttsn
 Page 35 of 64

Sinco ues

)+/
oation-2

S-v= (\)
(+S) = (+,1+)

ta. )():)
-02

is not Optin
K3
Sa
 Page 36 of 64
+(3+s Sa) - f-0.2 o2. a, l240Aa)
0.04s-0.08 a-12

o
4

0-2
-0-

4ts nef O
aaten-
0-
Sq Vf4 =

14+oa)
+C4+ a S4) = -+o-ah,
So 09
and
ue odiniue e Undto

x* =/-lo
5
is und4
 Page 37 of 64

Noth

wnea
Cs-()1 iste east motox

(mototy S2Cd da dodt) 24

euunted at

Gnuwm )
R)=
eUaluate tte
te ciesve
Ze

C) J=2,3 - 9

te abaue eusatug (3)

Con be Osed t impsmu afPsoras

it -[J] v
 Page 38 of 64
oble

by
F( ) = Xi-X2t2x"+21 X2
te Us Neutang
stod
ettod
So
42
J:
-2

2+31JCo o)
by en (4) (*1= vi -Cs,J vf)
- vf.

V - 3J
t+41+ |

V
 Page 39 of 64

-Oobtoined )nzatèn
Tha genna Condtroined Oinzotin tolk
MGaimize a tnctien p)
-foncic) P(x) by
moxien2
asicnize
Vo en
en Certam Cotoottt 0n X

oe fonple.
f,2,13) = fa +
tind minimu

toenGaute
CommaOto ths fo engoeersinQ

Fostest Rhtle corh

eronpa, den the
Coistornt fuel etPclen
All Canstooind Con be anested to a
Canta
Ctsots
Crnshroink
Fualt
Cangtson
a) Ia
Cotsotd

E Micnize+(Ma,X3)
8ojet to X+X,+3 l AtZa+Xa-o

dnieaalt antainB
hme +C,¥3)
 Page 40 of 64

A optinato.n oblaras
Con be tten aag

the
ís Peasba. ait
nenerale groge tunckn
egpda s to
T Conatooad Optimtn bao
, ha ennde, that
th functin a)
Stde
to tua -featita Stote
Pnt dscoved bedons
ths
achlewe ths
Thena a Semena technleSthat
bot ts
tt+ togennal, a dPRaut obar
defina aa
dene
A Ve'Cmmer) oRsach to
Caltacd tha reaged agogto
n furctn
h
LG f+gt)+a

C ) aggo
f) = toncttn
han
hantha [Link]

mio foa) mas

ES o>o
 Page 41 of 64
Oecekon ord Soficent
kk Dnditton&
-Pos OPtina
tho Konsh
Kanush
Madternadicad Optit
(kkT) Coditiondal nan as to
Kuhn-TUik
ano rst devatlee
Kuhn-Tuikeg Cordttn,
necaany Cendtins
rdkns)
Sone-ttnas Ca0ad As ndes
Selutèn in on-Gneas rnmipoetohe2
fora a Candiend
Condicn
OpRma, ided tht Some egdant
9atised
Cordtind Solve
Tla NecesBon and oHtert
Non Linea ormu

Sneality Constooit

we Foao that ohon - Lneo onm b

with ety atsont
t) Sxhtholt
Magtnugg Minne
We Con Sole t obon 'shLogazko
stipey metod
*
tha Noo- hay
Mouo Cedo
eg Constsain

MhnRe ) soohthat :)<bi

3C) b
 Page 42 of 64
e Con Solue tha
Sotue koblor) rh kKT Mttnd.

aplenhatinobac) e,fi

maxim f) Soch that

Coaidos
[Link] ta NLPP

i() b
Canttoank toto
veiot ech th inegalty
eayatdn adday e Ratie Slack Varakay

ra

Slaak bsioble

)3=b
Cndlo hi) )4 blo
Tus e NLPP

r3dos Suh thas hiCa)o

CrnateontE So eCon e
1

medod

osmiote Loghongo un a

L%,S, fo)- h t)
Pc)- NCa0+S-s)
C: ton
 Page 43 of 64

Cood tien fo 3totionont

necenu
an

L -=0 C) +S"-bi =o

s bi-:)
=
-2 2 Sio
Solue cb(A) ue e! Statinana
Fs
utip eqpaten S

aCbj (bi-a:)) = t
, b-i6)o
Stra Mmeouny he Moke arsnc
oto

POen Go
dio (4) e hae eth io|

o )boh Vonish a
 Page 44 of 64

Si
meon Ctooit
Satisfed a Stsidt

neayanlt
OCSio) Csteont (make b angea)
4 e 9elaxed
te Statènany Priot l not be affected

Thas inptae Si 0
Constoont Stsf aseqjalt

Bha Objche
Tus impte thod alb sinces ed,
functtn decseoe &ehcnd
ttoee9 a bi nCstaie moe Spaa
feasibla and the pkina o 9 a Obyecte
d
funct f), Ceay Conno deceo
Henca oo Optitra &olutur)

mnumttg a3
1nceoe

1
Conne tacwcon uuch mptie th
 Page 45 of 64

.
mank
Constsoin and eqwatèn
tho
bi
A:)=
Uoveatsicted in
thon beCam

Heafor Non- Linean goonon Cor)

Mnimi
Such thet 3och th ai)£bi
The Mecossen Cardttery T necorkou Candttn
J
(
(zc-) - b:) =o i Caics) -bi) = o

C)b
 Page 46 of 64

NLPP
the
Maximie Z36-
Maten 0:4X+ I:6 a-O2x
Such that
XI X2 20
Cardtin be Oecouoe
T the KKT
and SuSfcenat
to a maxiusm
Shou be (ancaa
and ) 20 is Convex
02x, to be Cacae
612-
Fa)= 3.6 O.4+
Costott e

-0
O -o-4

TL p mnoos Di 0:8 o
Da -o8 e O-82
-04
Thus D20D
ere# cae
0 herca ís CnCal
feOProee
OPpo¥e Bizn wh
Als the CanstsointF
Zincay t
Convex
and e ken en nea functin is
Cnditian9
Haco te kKT dttnd and SAcient
maxtmum
 Page 47 of 64
DeAna the lograpo unctin a2

L=
+-lo)

,
(367-o-4 I67-O-)-A(1x
Ta necano Crdtin ano
, KZ°
0

o &6-O-241-2A=0- O
0
6-04-7-=
= (+- )-o

lo,
+Z

Tollo- Ca
e ae tha

vohar Ao
se houe2

hich de nt Satisty en )
discondad
henca thrs Cax s
 Page 48 of 64

C
Ue

6-2a 6-
6-
3.6-2 1:6- | lo
4

Ao4
tt = 3-5, X3 C* x Sub Volu
in Sts
NLPP
ole t
Minir
8 z=-Ia-*
Such het

fo e
t t» te
tha KKT Cordtnd to be
mnmnm
-fa) Shnul be Ce g) Z s Cnex.
Pca) to be Convey, e Constouct heai
matsix a
H=
H
 Page 49 of 64

Mnos OnR
D

Als tha Cantoort 2 is nea)hctig

and heco it fs Coriea alto

Tus KKt Crdtttn (aoll e mhmu)

Detea ogpongi fuct a

-l) > (ts-)

Tko necoi Crditiend ang
o, X20
.
Lo)

A(X1+3a)
(G)
rt (6
e have te olawt Cos
Ca
 Page 50 of 64

n O ) ve houe

ord herco this

this o
Co s
whch Violatc en
egn
desCanded

o2
so n
(5) 1+= 2

ye

Hece
-=
uadtu e houe
which as e necaiAo Cdtiny
Henco tra Stotinp

Ca ) = C-)
is tte Optina solotu and blsae

4,9
 Page 51 of 64
NLPP
ole 1,
Mapeie Z= 81+loX2X
Sch that 31t 2
s

Condtd to ke NCse
te KKT
Zto be axtur)

f) Shald be orane
Cone
.
a()o fs
e Consts a
toe ConCaua

aman Mata
H
,

mho
Da--o O 2
4

whch s Cpoie Storste Csot

Ts s ConCoe
Atso te Csnatsont B Zs s a
Lnea hegndy Funct
hanco Coaextnctty
 Page 52 of 64

Ths te KKT ondttdcn il be ecao

SPAeendtor aa

e aghargio tunatin o

L + lory-x/- (3t a-6)

Cdtins ang
neceo
X20
dsi,
Cndttengon
The neceulo

-2=o (
fo-

T follaun Cay aive

wAo 41 X 5
egn ) d
ushch violak
Coe s Ciscada
g32

en8 e
3r 2X2= 6
 Page 53 of 64

) ea
= 8-32 = 5-

Thus
B-3)+{=
3

ece X
Codtins
ushch >atsfy a
Satisf theneCaRes
eco tue Stattno

( ( )

Z
ptina

13
Solutu

(
Vake

- K

(4) e NLPP
X

mayP) = 4 +6-7/-x,

)
 Page 54 of 64
KKT Condfienl to be necedo
Sol
Soffcrnt for to to be maxionm
ard
Cootaue ond 2 a s Convex
-f) Shood be

ofc) to be Condae
ue Carstout a temn motsx a

o
H

maso an

D3- o o

oo-
uohich s OPfoaie &gn Sidnp uts
Thus fs Csove
-tu
312
nea
newity- tunctno
and e one
 Page 55 of 64
Thus -ta KET Corditt
aimun

Defne tte dogargforctè a

L (41-+61-x, ) - (+"
-(ax, t3-1
OcCak9u Cedttds

4- 2)-A-22 AL0
)
3 d2 co
6-2-2-3

t -)=o
ax 3
(T)
shn
Ts dos ot Satt3 &?)
d nce dincandac
 Page 56 of 64

4 a)
27 =
-22+6 =2
fron(A) uee +X3

+6-A3 6-2
ea
suth alo

Stotino

T Optima&oluto nd Vale s

Z +4)-(9- A-9
 Page 57 of 64
esano Matin 4}

&itio
)f) is ConvC
mnirmu Anot
Semi-defhie (Coen-
ard
attaat
DI>o, D z° Ds
Dio
mnsos

a)t) ts Stsdy oney

Psto-e eAe C
O>0, D >0,, Ds

antae. -fa) is Canueg.

Conve axium Prt
D, D co, .D,>° DaCo
altenotle 8go, but Aethep
Shauld ke -ve
 Page 58 of 64
GRADIEN METHop

teepat scent metd

onckien

FCa ) 41+6,-2 272-2x2

Qbedute Opttomm OcC a
C ) (Ys,4/s) -333 -33
So le the tnit nt be
e b

VC)='C+- 4122 , 6-24-4-7a) )

Co dtonantidan

isevati X
C
in
v#() (-a)

Cn)+ T0)
C)+-as, )

h C C)-2(C1-25)4
4Ct-2+6
 Page 59 of 64

h()4 er+ 4
Sskstitfe in ean (3)

-16+ 4=0 (1-2, )

-16=-4 (1-(4).)
SeGand
tecete)
esn )

#C)= (o

(Y1)+r (o,)
(Va1)+ (0, tt
X"( I+T)
h() (x")
m)
()+6(t+v)--(-
--(i+
6+6--f- T-2T-4
6+ 2-t"--F
 Page 60 of 64

-2+ 1
2

hl) -2t

-4 0
-|
, 1+
4-
(a, 5/)
Thid kesetien
= (V, 5/4) ean
Sub

VCr")+ (-o)

(Y, 5/4) tr(-)

( ( 4)+ (-Y
(-v,54
 Page 61 of 64

h()- +Cx")
=
(-4)+4(h)--)-
E-^(
- --
(
--
-

-
at- (t

h'l) hls)o en

( )
 Page 62 of 64

(, 5/+

Sob 3e 54) in egn)

Ya)
vf) flo,

(e 1)+ (o

h() 4")
4(he)+E+)-+{)-(%))
6Y

+- +
501&
6
 Page 63 of 64

- 4 +6Y -208
32
6

h() 32

Let h()=o

(sle 5/4 )
-

ahio).

"vf (*")

(le )+ (-e®) h t
(
+ (3)+(TE)- -() H)
 Page 64 of 64

ht
Y
597
3
S egn
4
64 ( )
/s-
(
)
Sigh atia)..
x'- ( )
vfC) = (o
BeCaup2
VfC) o, thao Cen e
teminde atthte Patot

.The aoorek makiuum

. 34-395, 131

(
T ac Opticws
0:3333, 3333
X