8 Module -@ RS angus Ble ae Cutwentet: s+ x) Bivar cate data mvolues a_vartables and focuses on grading gla tonshtps blw them exe Tempe nature tn a shop and sales data, Sales of sucartess 9 To analyze cnelattonshtps, a scaHes plo ts used. DE helps visualtee fends ,dtffesences and selattonsheps btw 2 Vantables +) Shows strength, shape, dtvecitor and presence of outlters lorleng covselation (ox) eUseful foo caployatory data befove calcu regression. > scatter plotVwastate Sdaitsttcs Wn wen atest t cs 4) Covastance and Cosselatton ave eaamples of bivaacate # Covartance'- %) Denoted by cov X¥) x) Ti ts used to measure the vastance blw two demenstons. toomula- eee gece eee ’ covey = Ll (x1 ECK)) Cyr ECY)) N tet where, 24 and yy data valu rom Xand y. ECK) and ELY)—> mean values Of % and y; N> numbes of gtven darta. ; Ftnd the covartance of dota, X= {12,3495 and YL, 4.4, 18 Sl- Mean C= ELK) IS Lg and ECY) = 52 -y 5 Crdladton - Cadenoted by s). ¥) Peasson Covselatton Coef{tctem Ca) measuses the ghrength and dear!’ of the dinead yelaitonshtp belween Q vartables, | |Foomula:- tile > re Note Tp WO X and -Y tncsease toget hes T{ vo, one th TH veo, they ave tndependent. ® Ftnd the conyelortion fos aata, X= {h203/b Sele EGO= yx 3 and ECY)=Jy=!! Txt : = (ay Ca-svt a-3Pt (4-374 (8-3 a ry= C1184 CRN 19746-1194 (25-1 V 5 my = 3-646 CovCX.y)= 12 oo HIB [Link] populaaton Tx, ~y *tandard deveations creases whule the othes deceases. ge yand Y= {bHe4 16.25}OO —°E!Sl A LULU oe Mulivartate Sdattsites:— Worn Warren x) Mutfivasiate data tnvolves mane than fo vantaies and ts commonly used tn ML. %® Stmtlon 4o bivantate data but with mutftple dependent vantables, ®) Common analysts techntques - Reqresston analysts, patnctpal Component vinalysts ceca), Path analysts. E gq AHytbutel Atfutbute 2 Atfotbude 5 i \ y \ + 2 y 2 | 3 3 6 \ | *) Example data has 3 aHstbutes, and theta mean ts Called a cenbotd, | | *) Vantance of muuitvastate data ts sepsesented as a covastance mat te. — Heotmap wa *) A qaaphecal vepresentation of Q 2D mahrx ustng coloss, *) Helps tn peacetving patieans eastty. *) Darker colors = darges values , Leghtes coloss =smalley values, x) EC Traffic analysts (-hegh vs tow faffte aeqtens) peotth data (wetght vs health stotus) p - paren %) A vtsyaltzatton tool fix mutivastate data. x) Conststs of scatter plots fox each pats of variables.eee x itetps tn ftndtng qelattonshtps and cov elottons | ber fA yandom mabtx of 3 columns ts ploHed as a pata plot | 4p Show velattonshtps. taantea! Maghemates for gattvascets Bees” pts from Ieneas allgebaa, stottsitcs probabtlety xy ML aequtses conce and tnfosrmaiton theory. 4) Lincon algebva ts esenital fos mL, dealtng wrth ltneay equattons, vectoss, matetces And pansfoomearitons. ta denen Sales ont Gaus. unten far Ewes ASS SA henean systern of equattons ts @ group of equaitons wtth unknown yastables. Wouter as AL=y and Solved 5 2p hares 4) F a untque sotuiton exsts, the system consistent {nctepeadent EF multiple soluttons cxtst, Hs consistent dependent. EF no selutton citsts tH ts tnconststert. * Gaussian Klemtnatron solves large systems USING YOU OPLd attons: 1) Convest the system tnto an augmented mabix tt) Use g prvot Clement jo eltminate Vavtables th othes sows, (tO Convest fo wow echelon fox, ty) Use back-substitulton fo {tnd unknowns,I _ ODTDDIOOOI TE ——————— &) Gtven, set OF eqns, arid 4xr=6 ALI 4312 =F. Solve by Gaurstan kitmenodten method Sele ket A= augumented mad are a Ae fie 0 -5 Fe fa 4 | ye al * hz 1 213 Rua Ry 4 3]F a Ra=Ra-ARi - -] Rane (elt ey Ri— Ri- QR2 meenecompostttons = nighie ASERPRLLTS mane decompostiton breaks mahix tnto smaties pasts fox eastes catcwattons oOktge decomposttton'= A 15 decomposed tnto egenvalues shut gretgervectoe mattk A> dtagonal manne A mabie and ergenvectoas: gis onspose oy 8. Lv Decompostiton:- oa Wawa x) dU decompostiten sptts a marx Unto. whee L— Lowes fstangulox mabtr VU Uppy frangwlon matt. SAPS 1) Sart wtth on tdenttty mahix Tt) Rpply Gausstan eltmmatten tn) Reduce the mante to get & and U, 8) Find LU decomposttton Of (Given. mater: Aft 2Se TAS 393 2 Bene { | , 2 Suet, BEM => {A = Machine ww helps In undeosiandeng dala destotbultons, whtle slattstres asstst COE OH) (Hg o lo 4.3 oo 13 4 ‘ Ro = Ra- 3Ra ro off! 2 st || One eos, Ra= Ra -3Ri vo offi 2 3.1 O]lo -3 a Ot 0 -2 ks = Ra- 2 Re 3 Vo offi 2 10 0-3 3 43 1 JLo 0 L*O canning and Enputane OF cbabrley aoa seats -10 10 -10 -10 - 1013, ML 15 closely (tnked tp Prbabtitty and sterttsttes. Psobabtir'y data analysts. Many me datasets follow cleffeaen proba be lety atsfitbuftons, maktng theta Knowledge essenttal. 7pelity Bishi ulons” tsttbutton descotbes how po.0bs ; probably d values of A vastable CX) are jopiuled a ts pabritty of deffenen: dts htbutions= Cae a mathemat cal fundiren that detesmmes the 4 values (NO dataset. pol rypss 9 of probablicty fy. dishaybulten {Nous Poo bo. belfy Dishtbulten> Deals with ¢ Deals with countable values sy Discrete Probabtlt agente posst ble values, 2) Contenous fey concerter junctten (POF)= Deteomenes proba bitty for 4) Probabtltty Oens ty comtnous vartables oCummutadive Otsbebutton funciton (CDF)> Probabclety of a value ummutadttve OtsADUNED Y= being less than Coa) equal to a 4 Drobablitiy mass functten (PMF)+- Used for dtscaete distetbutrons given numbed . Cequivatent 4 PDF) Comtnous Probabttcty Dtstotbultons'- LOONENE sANnAnnne 2 WELL 4) Neamnal Desherbutton CAausstan dtsptb mos} common dtshtbulfon ution) 4) Bell-shaped Curve, ¥)Mean,medtan and mode are equal 6) Used foo hetghts, blood. paessure and esl scoxes Avormulay aNSete ee ee ee a eermrer eer ter] 9) fa &) Raclargulon (untfonre) detrbuttn *) All values tn dhe range (ait) have equal porobabrltty. t %) Used for modeling furs dece volts, sandom number genesaiten | ‘wamuta:- 3) Exponenital dtsbt buiton- BARA new : %) Used fow modeltng {ime unitl an event happens Ceg-cwatting tme fora bus) . xx) spect al case. 04 the Gamma dist rbutton. Foomula:- AL a FCx,ads Ae » 220 ’ *)|Mean= 1 and [Link]= 1 A t » “ \ I en Probab tity deststbutten - wenn nnn Semen 4) Binomtal Distt button + 6) Models the poobablitty of x Successes tn 'n’ trals [Link] jteps) 4) Fach fat} has two outcomes: success(p) ov fatture Ci-p)—q fuomula = 2 tae P(kex) = NC, p*q™ where, qe tp Vastanie = Npq3) Poisson Dtsbtbutton y models the Numbes of events tn a fired ttme Ceg., NUMbes of * coatls Ped hows) used when tyents ave independent. formulas 4) (Mean =a, Standard devlatten = Vy )Besnoullt_Dishtbu tton:- #) Models an expeotment with onty & outcomes ¢ suecess C1) ov farkaeld 4) Used 1 DINaaY classrytcattor problems. %) Aoymulas- xS(Mean =p, vastance= p@ | Bensty Kalgrngheon= ohet X= {1 2,---Eny be a ced OF obseoved values from aa unknown dtshtbujton, Deastfy estimation ts the psocess of est(mating the probabtitty denstty functten plx frm fhese obsesvattons. OY pG) ts small, + may be an anomaly . 4) Theve area matn tu pes = Here are a math ty pes :»y Posametste Density Estimatten a) Non~ Paramette Denstty Esitmatron. i a ane Ravametite Denstty Fsitmattony Wn AN known probabt itty disbrbutten pcxle), * ASSUMES data follows a whese & 15 the pavameter set. Maxtmum Itkeltnood _ Esttmatton (MLE) ‘un damental sratts{tcal: method used to esitmate parameters *) MLE ts o ; nimtatag the (tkelt hood functton, of a probability dtsprbuften by Mme “aythe idea behend MLE 1540 choose fhe posametes 6 that Wakes. the observed data X= {xuXe,--- model . Xny most probable under the assumed MLE deftnrhon:- Fox a given dataset K, the Itkelthood L(X36) ts defined as, this functton vepsesents the paobabtltty’ of obseaving the given data “under a spect{re chotce oF Parameters 6. tht goal 1s to find 6 thad maxtm(zes LCK30): mar £(X;8) 4og-tke Ithoed Functton *) SInce comput tng the pooduct of probabtleltes can lead to very sma numbers Ccaustng numertcal instabtltty), we take the natural fogartthin. of the [tkelthood functton. This fansfooms the povblem jmebed eae_irkelthoed maxtmezatron: r qe © max = dog pl%q58) tet frame {he pooblem. as mtotent2ing the Negative aneonaitvcly we can h ts commonly done th opltmizatten tasks ieqrtketchood (NLL), whte e NULL ts offen prefesved to ML, as loss Jancirons Mintmeatng thi strral models Ave drstgned (1 fhis in newsal netwosks and sfott {oom MLE tn Reguesston (Gauss tan Assumpiten):- me that the output vareable y follo. Mm vegyesston problems, we assul ven an tpput x. Mathematrcal a gausstan CNoomal) dishtbutten g apts assumption 1S watten as. whese. hCx) ts the model functton (eqs Lneas v¥egresston fanctton ACx) = pet Bid) 2 o™ ts the vaatance of the Gausstan notse. doq- Ut kelthood th Requesston= Taktng the Aeqartthm of the Gausstan, Jtkelthood funcften: ane oq plyt)Expectatton - Maxtmteaiten (Em) Algoatthm:- See AEC ch ¥) Used foo MLE when data has latent yastables Chtdden 2 105). a ts genesated by Ex Gausstan Metre model mm), whes® dat multiple Gausstans. EM steps : 1) E-step Esttmate exptcted PDF and pasarn eles fos @ g the MLE funciten.» ch latent Vartabi: | a) steps opltmee pasameters ustn teraitvely fo refine pasamedeas. This process wepeats Non-Pagametste Denstty Esttmatton:- eh A an ¥) Does not assume m spectftc dishtbutton, . Pavten Window. Esttmatton:- Gtven 1 samples, le Kk be the no of potnts tn a degton R: mB A paod=_ bk nv whese V ts the volume op the region. Rts a hypecube wth | Benstty estemate: length h, then» V=h}} (fox 30) for 2D) ,} aazen window functton jm" #(2-) . {\) tylx-xql< hie h 0, ofheswise nt proba ttty density fanciton esitmate: q 8 Gaussian functton Instead of a sdmple wstndow gtves the tan Penstty Function, ust’ Gauss genemnest_Hetghbours CKNN) Denstty Esttmatton- uses ® neasest netghboos Instead of fired regton Stee. x») Density asttmate ts the aveoage of values setusned by k-neavest netghbours. ature Engineencng and Demenstonaltty Reductten Techntques- 4) Feature Enginecn(ng > Selecting 00 transfoomrng fearkes Heature Trans fer mattony Coeattng new es ty Tmpmove model peaformance. features Ceq » BMI fom hught & werght) ~— Feature Selectton> Choostng relevant features whtte remevlNg unnecessary ones. Vtature Subset Seleeftom- Reduces dataset stze by removing traeievant and yedundant features. = fvutds the cusse of dtmenstonattty, where too many features Increase Complextty . | Us Sa greedy search approach to find the best subset.Feature Hera SRE Utena! Que: +.Relevangy Important fectuses conhrtbute move Cegamole vs hose tn face detect ton). A. Redundancy Features provedtng duplecate Info can be senoved. Ceg., age vsdate of, broth) Setectton Porocess'- Wakes WAS 1. Genevate al posstble substts | 2. Evaluate subsets and model peofesmance 3. Select the hest subset. HTL -based Uses stattsfteal measuses (comvelat ton, mutual tnfosmats) %) Wrapper-basedt Uses classtfters fav feature selectton, offering beter pesfovmance but higher computatton. Alganithms:- ; 1) Stepwise Forwaad Selectton- X) Sfarts With no atftbutes and adds the best one ot each siep anttl a good set ts foomed. 2) Stepwtse_ Backword Eltmtnatton’> *) Hovis wrth all atetbutes and vemoves the woast one at each step unitl a good set ts formed Nete (om bined appyoach: Uses both methods ~ adds the best athtbute and vemoves the worst one,p ctpal_Com Component fnalystsCecny'- pre y rch meduces dimens(ons by hans fooming data unto Nevo features wth martmum vartance. ee mean from dataset” =ppcompake covartance Matyi , wp catcatade eigenvalues & etgenvectoss. ect fop etgenvectoss foo pansfoomatton y) sel fosm dataset ustng selected etgenuectors ) Trans 4) denen. piscrtmthant Analysts (LDA) Lar Psoereees x) LOA viedaces dimenstons while presesveng class Sepaa abt tty . deyst ty Compute class means. ty Compute blw-Class and wtthin-class scattes matetces. int) Find ergenv ectors and eigenvalues. w)Transfoam data ustng fop etgen vectors 5) Stagulay value Decomposttton (SVD): Nts a decomposttto techntque fhet vepsecents a mar(x A as here, )}Uand Vane oathegonal matsices | Sts a Atagonal mabrx wth stngutas values. | Ye ’ Gmpute AATI Ul WW) Find us etgematues and eigenvectors fo -fesw’ marta U: {tt) Sort eagenvatues tr dascendtng oader . W) Take the square wool of etgenvatues to form dragonal mabre $ V) ‘lind etgenvebites and etgenvectoas of ATA fe get mati V. ¢ + — def fhe data potnis be [?] ul Apply PCA and find the ransfamed data, solr Act BP2 owe - PU] oy we] i 6 |? 4 6 F uae means Coal Lap [eo 6s Certts | fo les 6.5 2x2 xa Xeentes = fe 1 teal 6 7 6.5 Covontance T center « Mere) ns no of peatures (hug na) = 4 [05 -05]fo5 -0 at 105 05 F £05 0.5 Keoy = [OF -0.5 05 0.5—rerrnrnmnrrrr er iii te ee To pind. etgen vatues (x), solve det(x-at)=0 el 05 0.4 > O5-A - 0.5 = 0 7H 05a | (5-7 - (0.5)" =o OSE? QAO - (oso) = ° At azo AQ-1)=0 wane the etgen values Use these A values, fo get clgen vectors. => det (K-a1).Ve0 (fo a=\) (cE De -0.5 0.5 0:5 -0.5 Vv 20 -05 -05/ LY? =) -0.5V1-0.5V2 =0 0 -0.5V) -0.5V9 = 0 9 @ 7 -0.5 V1 -0.5Va -0.9Vi - 0.5 VE OD -[Link]- 0.9V3 =0 5 Vi = - No Viz -Va =>ee aera Lt YY PUgEN VeChoy foo ast [wy aE a why {or Ato, 0.5 -05 a =o 25 05) [V2 ov aaa -0.5Vi40.5V2 | => p.5Vi-omve =v —>® ! -0.5ViI 405Ve= 0 @ ®> o5Vi-05Ve=0 pov= 0f5V2 Vw 1 ve of Sage me mn, Noamalteation Of etgen vectoss, Note! tn general, roe pat? sees Vara Eig Now er Teo, VG oo (eacy TWdy r= Wwe Va qmbine G and e2 and name A. Ae | UG VG Moy UG Tusansfoomed data Ye A® Keenterea 2 [iy 4]vaypos -05 “Ufo US2)-0.5 0.5 =o o : F-oa0% 0.409 heat en.) Hs {-0.904 0.409), Veatty th ts Cowrect , X= avy + Mean Ke { (We | o 0 HY 1s 15 Udo oy J 0-409 0-709 6.5 65 = fyaaa 1.001 ee) aaaeeel C.000) 6.9998 c oF vatjted =. Metengstepre 4 6 5 4 3 * \ Sfep dr Note Feacerj and PCACEs) ore always Lr to each othes Fmaby poacer) PCR Ce)