Scribd
Upload
20 views43 pages

BDA Practical

The document discusses Walmart's use of big data analytics to enhance operations, customer personalization, and supply chain optimization across its extensive network of over 11,000 stores. It highlights the challenges faced in inventory management and the implementation of advanced analytics techniques to improve decision-making and operational efficiency. Additionally, it covers the integration of data manipulation tools like Pandas and the ETL process for effective data handling and analysis.

Uploaded by

justaniket30
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
20 views43 pages

BDA Practical

The document discusses Walmart's use of big data analytics to enhance operations, customer personalization, and supply chain optimization across its extensive network of over 11,000 stores. It highlights the challenges faced in inventory management and the implementation of advanced analytics techniques to improve decision-making and operational efficiency. Additionally, it covers the integration of data manipulation tools like Pandas and the ETL process for effective data handling and analysis.

Uploaded by

justaniket30
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
‘A group of Academic & Research instutions Practical Now n= Core Study 00 big deret & big dase Ancibysis t- coalmart's Data -daven Soccer. ystem “Requixernent Operasing _ Suster Softwoart Editor Nawotk Browser windows to Duthon sehip Any Chrome. & Introduction |- @almarr y+he coortd's largett retailer , has suceestPulby embyacd big dara anak (+0 teens form tr operation , Gutomer expctiencer ,and erive businas growth . this big dara and %& advance enhance ccuc study extplora how Wwatmarxe olize sophisticaared analysis 4cchaiques +o sicey Peompctitive un the cver -cvolving serail landscape walmar: handle o vart amount of data daily, from cutomex transactions and wentoru management +o supply chain logistic. . with over 11,000 Stora 4h 94 counties | the Sheer Scale oP coalmarr's operations nesitatates efFicicat dee handling — for decition- maning . MIT r c Aoi ol Academic & Research mations — — | * Chatlenges © trventory Management waimarr face the chatlenges of managing daventory across a massive neteoork of store fPiciently. @ Gstome Personatization = meening the diverse needs of dk curtomer bare requerc understanding dndividual py cferences- + Implementarion of Big dara + ill a [ - Wdimart Leverager big data across Various aspect of the business t @ Supply Chain Optimizaron + _ “The company employs ana. Ta yale advanced anatytia to ercamline it, sopply Chain. prediaive anatuticr betp Fucuuattens , enabling better inventory anicparc demand management & veducing steckwuts insights into caurtomes Dchewiay —4e Gujsmee Tnsights + watmast otilizes big dasa to gosn fincclssin.g Purchare hitronice and ontine inracerion: vows the company to tailor markering Sirategics Re pexronaltize promotions, eS ee a5 Bu optimizing sopply chain @Operasio operations and impreving store operations Page No. management, Walmart optimize Invent’ MIT ‘Naraup of Academie & Resoarch institutions on historical dara improved inventory management , -redsice esr and hanced cartomer experiences Gnctusion nregration of big dota & anatutict of era) incluthry «By Q@almact’s successful 4h bans Forming JrensPormanve pewer exemplifics data -dvven dcasion - harnciing ‘the insights dexived Goalmers not only cvacome operational chalieng ch bot auo sett a benbhmarh fox the entiye ‘eral sector Wing +he cro bracing Big dara da key to sustained making in the Gorn massive dateuers, MIT AN group of Acadomic& Reaearch Instone Practical No; 2 Demoruvrare he caparititia of panda: in purmon execs dara bu performing various operation Uke importing data, dara type explorer oo .enleurmn and ye seteaton , summary srastirea , and Basa manipolasion Sustem Reavivement 1 Windows tt Puthon serup Any Browsez powerful data manipolaren Libya oo pithen Gidely Used Por working with. sructare data, indudin FeeaPia. Ti provida tect fu br preside dara structure Bye Serics and daramame thar make dara anawysis Br Franipviarion efrcient. ker Function alivies include dara femerting, dare type exelevation « colsumn Bw Sacchon, statistical analysis and data manipulerion , the procus of working with excel Pandas simplifi ct wd an ind&pensabie tool for data un Puthon, matong TB scicnricrt B anakystt. The provided care , study Page No Mendsatcd ane application of pan dou Panckors pulating exceed data, importing, etploving and mary versatility & chricicney . IShococcuing the Ubreey's nether cut deta import » ere loyeetion , Sciection, or meanipolation, pandas provicles a Comprehemsive ses Tek tools for handling diverse data anateysis totk- bering excd dara into a Panda daafsame : Suse the Pandorread- cece 0)! Punctton to import exict dara dnro a datafaume. ® Gating Deve tupo of Pict » Mose the (-dtupu’ atmbor to veiove the dare typu of cach fied ao data Pycume - Reading Spedfic Coloomm t- = SpediPy the duixcd coloomn nome én the ‘osceols’ Parameter ok ‘ycad-—ercei)' function , Fomor Srasitria: of specitic Coloumn: eOrtize Pander | Staristcat Function: Ghe Som) tmeand)', mart) , & mint)’ on the dstived coloomn, porting data skipping row! B Colouomnu + Page No.: MIT Agroup of Academe & Research nstitions Sse the Skipreawt’ oy ‘useclt’ paramerees dn the rcad-erca ()' Panction +o ship epceilled recos coloummni, _ Spccificd Cosine & Pass fant soritike data Rame indesing & sctection methoctt to & vat choove specific” eploom 5 & Coloomna'- anc ‘drop O' method to duet yous ey coloumns from the dataPrame bSerdation t= & 2 @ The pando: program socceriFully Gmported , explored RB manipolated the exces ‘dara. 2 Dara tupa, specific ColOOMNS , SoMMoRsy + STeAtistiC and scccted dara cocks obtained as derived Rootes The cur study demoartyeun the effectiveness Fe of panda: in handting tect dara dé Pow vadous dera meanipvlation tout Gncuteon + Ta capabilities in dearer importing, ———— _ cxploratton, succtton bo menzpolation make a Powerful toot for dara anatusi¢ & manipulation tcuks, MIT ci (" [Link]") pu of Ereel data sect c_ ce [eoicomn"s Specs Fic — colon dP“ Cotoumns "I Colam - sur [Link]) Column—-mecn cific column meant) soccific- column. marl) Fic. column. min 0 Mam (Summary Stasis Poy Coleone Print (" Sum |__wlemp sum) : piemm. mcar ) umn - mor) in) Col ons ‘Coloumn 2 *Goiumn 2" I] al. drop (colorant ct = (" Golumat” i @iumnt" |, Ted, places Truc ) Pied DotaPrame ofrer Deserion?® ) __ Page No. MIT Academic & Research Instutons Data Pietctas Giemn2 dtupe + cb) Summory Empty Columns “Practical No.3 <—O1im ot he rt —Pamtoru i to ethics cx oe Joke peony multi -yontrowm iit accorrling +9 Speciticd. busin rite: ox _xequiarmen ts the tanfomned dara ints o nogc oy dati ararchouce = ___ windows 10 bare Editor Purhon sctup Network | Ary Browre Chyome- Theory *- erticed Components ad weatm of data uwcqration & data wazrchsutin Ti Gnvolver three main sage Doron the ettrucHon phate ,doro ci gathered yom various Source sustems. Sovrca may Undude dasaharu , sprcadthcett \logs , APT oy ftar filer, “This Phare Poceuer om yetrser the required data without attaing ut Structure. Page No. oY struducre 2 This phere includ e __8ggrcgating + enviching the ASS eS ery meet the businar yequiremc Common tyuniformationt dndude ad tupe convertion, handling Missing s % appluing inert ater @ load e the Fina! Phare wi the Loacing PGs co een O data warctheuic or o databace solver ePFidientty ston SES aN CE mas aonabyticas Pr procs that ud and whe Format fox | mabling ganization to derive Mraningrut r make informnee dnights Bs required Moclele Source File he Zp Pile @ Bears form am 0) Fur @_twod Dasa Cou import _zipfiie import pandas ov pel - ae isn port __tngging Into) Loging ebosicl fig Le 5 = Logging Ge conad- (zip-Ric-path, exhod- Poth )4 with wpfile ZipPic (up_Pire- posh, '¥ ‘) os tips: zip och, chactaul (extract path ) __lwaging- tafe (* Exactatt completed :) form (dase \) Pa. + goretime (dara Cdare’ ne hancPormetion eneopleted ee ral 'dare'] Logging inf _réexen dare. _—_ \(agbo enero igo path , index = false) Loading Completed!) __ dara. to -C8 caging une _Obsezuation coxmaktons moety ineludt mani tan Gor Succes Put evecution cheeking turact Pic Yo a Conclusion TheETL poorer dy essential Rox maintaining Jose courts sr epiacocti on, tytn tran loading of dat fram the 0 the terget Hic. quality % conictency oats divers < “Practicle No. 4 Pimi- Study any one tHadoop tte Cc. System: Requirement so Os = indo oPreneat ¢ Purthon soup Metwork - Etheenct Prow rex Chrome an open-source Ramewerk > procs Laxge Volumes of tn a dinvibured computing envronment conivs of two man component. doc Divvpuiced Pie Sustem Por Stovodige reduce For Pprocetiy Hadoop Use Care: fmaluring Sedat Medio Dasa Roblem Stosement media Company want to Anatuze ah foci ats omount one oie =genercetcd content. to druighrt Gnto user behawlor , Prckearence vour ce gare open dis i ir ‘Nu of Acaderte & Reseach Watton S : “ Molurion with Hadoop © “Dara Core cian The company colieds dato Prom voxout Source, dinclucling user Port , comm cars Uker, By shearts @ Dare Tngertion F The eniected data iw Jngested dinto Hadoop ecormstem, where dt ay sored | a divvributed Pile sussem deigned ver hans large datouer ® Data processing Hadoop MapReduce cu used to proces the dara cth scale @ Par Amel! ¢ a | p « analus' is pPtcx preproceting, the anct ed dara { _ coved buck In HDRES ov deve coatehoUts Por Rearher anatucis Hadoop Clase, Roceduse + O Semin @®_Dara_Ingution— ue MapRedssce Tob —____— @®_ Dare Analysis. ——___— @. I rs MIT ‘croup of Acadenic & Research insttutons longldstable Ke) Text va consext ) throws TOExcepHion : tr Gxception § 1 ae Redlda = Vatuc stostying ( length ye4 2s RadiCs] - gh vite (coord, onc Pablic ext, void vecluce (Text ken) Tetabtc se Value, Conrext consecy) throws crereupicdException £ veut )t : | MIT Ler ooo Obsermation + During the cxecertion § MapRecbuce —— z jobs; Observe the clisinbution oP taiks clerors the Hadoop Ctuster | Revit The MapReduce job will count the number on at up the social media dara _ Procuice output Uke! Ker 1000. Hence ,we Successlby steacly MIT ‘NQr00p of Academic & Research nations — faze No.5. —Crcare = O Hive table + Greate Hive Extenal “Table —bitead dar into tive tabic —C._ Create Hive Tntemat “bie lcci Rhauiement te OS = esindows SRAM AGA Hive eft a datorarchouse infartyercture belt __] an top of tHadoop Dor provid ng—data_ Summaxi zation, query % anatutis ft ence ier vee dorrctetr, __| ater “to guerty ond cincitay2e ors 5 stoyféd in Hodoop Ditind buted ire Suctem Ct wring oa S@)> bite ones —catied. tive i uve" "supports too types ol —teblers exreenot a a table wher Page No. Ome} ot ‘Acroup of Asti & Research istiions ei This_mcans ther: __ -£ = itm: = 2. Internal Table. a ae x cparchoute divectose; managed by Hive. ——_ Y 4 athe tate deletes” both the metadata sth data MIT ‘AGO of Academic & Research inttutons CREATE EXTERNAL TRBLE TE_NoT £xTSTScircenal -tuble id INT G y Name STRING ‘Rou FORMAT _DELLMTTED ETELOS TERMINATED By |! LOCATION ' user [hive /external table! i LOAD DaTA_INPATH ' /Parh [to | extenol [Link]’ ©. —_INTo TABLE _crtrrnel tobe i CREAT ABLE IP id INT ___ name STRING Row FoRMAT_ “DELIMITED — “PrELOs TERMINATED BY '.' 5 Not EXISTS internal table & | OutPur Seare Five ExFeEna Load Data into Hive ¢rirexnat Tobie Cb) + ___ [oading. date to fable caeahy Sea Eno Tauro extenatrable tah + Crum chet, nuimRows «\y a OF. — = “Time taken t ‘Lai seconda MIT OO cece & Remarc ration Tntctmat “Tebi\e Cc) uP execution of the code would ndicate thar the Hive table were created sou loaded ou excepted. IT GoM ot Aca & Research nat “Practice No.6 the dara vieto HIVE Table: ——Saload dats Pom Lecat Pile syctem os load dora fom Hors Mie suc —&: Croy data to tive anbic Loman 4.— sous : 7 —2. Sqaop tive import to import table data. | Rims: Lead ; | System, GRaunern ent eee eae eine AA 4aR, a Proctor + iW SoPrnare HIVE cio dato corehoure inPrectacictrive built o-top of tHladocp fox providing date someesri- “zation, quety 2s Analerc tt car deuctaped oy.) | Gorcebask. cond laser apentourced on port of the | T ppdcne Softencre Saundasion. Mius enabler o6ers ro _quau % analuze —Larg+—datascks—steaced—in— “Hadoop Dismabuted — Gir clceem ovvcia em eemmeate “pie ile suites using o SQL clic Langeage | cated —HiveQ tL. ———— eas ee, Component of HIVE, OArerartoxe Oi @ Hise Query oyocesar 7 = Execution. Engine SB Driveet 9s Gy od Dosa _Tonen locar Sire Sastemn Ca) > A Dara From 1toPs Sie suctern Cp) __ Dara to Hive Tahir Uncation Ce), Syoop Hive Tmpert (d)_ Giese ewer Load DATA Local INPAIH {Ph /-t0 | tecal data cy 4 __CTNTo “TABLE pie teil be § | LoaD “para INPaATH /Pathl 4tolhd& [Link]' INTO _ ZZ JAaSLE hive. table ¢ a fadonp i — topyGemLocat [fash |-42.|[Link] L _pcer hive [usarchout ¢fhive sable 4 MIT @ Loves p ¢ TSE od Dara Prom tecat Cire Suctern Co) t Leading —data to tobie defoutt, hive tabbi¢ Tobie “Ocfott nive tebie Stars + Coumbiicae ts —nuimRaais =X ,totalsizc: xx xomMora tines 10 Nfo hi cin pox uploaded data into Hive So INGo hive Hivelenpot SLEAT ___ faired to load chou * org, SIP4i imal = Hig Aa Hive Textfite format b. Hive sequence Pie fermed & Hive RO File format d. Hive PURO Pie formar &. Wve ORC Fie Ramat Po Hive Parquet fie formar rom Requixement! 22 Os: _eéindosos 10 Apache Hadenp Hardecas < RAM 4GR intel ia See ox od (@) Hi exit ror ae ghic my -tobie te oumn N TRIM c Pile 1 CREATE TABLE © Hive Qc Pre Somat CREATE “TABLE my table ve § : coloumnt “TNT + colocmn? TRING STORED As RcCPT{F ; AVRO SitePormat @_Hive s creare

You might also like