DYPTC,Talsande.

EXPERIMENT NO: 01

Title: Introduction of Big Data & Hadoop

Aim: Study of Big Data & Hadoop.

Theory:

Data:

The quantities, characters, or symbols on which operations are performed by a computer, which
may be stored and transmitted in the form of electrical signals and recorded on magnetic, optical,
or mechanical recording media.

Big Data:

Big Data is a collection of data that is huge in volume, yet growing exponentially with time. It is
a data with so large size and complexity that none of traditional data management tools can store
it or process it efficiently. Big data is also a data but with huge size.

Examples of Big Data:

Following are some of the Big Data examples-

New York Stock Exchange:

The New York Stock Exchange generates about one terabyte of new trade data per day.

Social Media:

The statistic shows that 500+terabytes of new data get ingested into the databases of
social media site Facebook, every day. This data is mainly generated in terms of photo
and video uploads, message exchanges, putting comments etc.

Jet engine

A single Jet engine can generate 10+terabytes of data in 30 minutes of flight time. With
many thousand flights per day, generation of data reaches up to many Petabytes.

Types Of Big Data:

Following are the types of Big Data:

1. Structured

Computer Science & Engineering Page 1

 DYPTC,Talsande.

2. Unstructured
3. Semi-structured

Structured:

Any data that can be stored, accessed and processed in the form of fixed format is termed
as a 'structured' data. Over the period of time, talent in computer science has achieved
greater success in developing techniques for working with such kind of data (where the
format is well known in advance) and also deriving value out of it. However, nowadays,
we are foreseeing issues when a size of such data grows to a huge extent, typical sizes are
being in the rage of multiple zettabytes.

Examples of Structured Data

An 'Employee' table in a database is an example of Structured Data

Employee_ID Employee_Name Gender Department Salary_In_lacs

2365 Rajesh Kulkarni Male Finance 650000

3398 Pratibha Joshi Female Admin 650000

7465 Shushil Roy Male Admin 500000

7500 Shubhojit Das Male Finance 500000

7699 Priya Sane Female Finance 550000

Unstructured:

Any data with unknown form or the structure is classified as unstructured data. In
addition to the size being huge, un-structured data poses multiple challenges in terms of
its processing for deriving value out of it. A typical example of unstructured data is a
heterogeneous data source containing a combination of simple text files, images, videos
etc. Now day organizations have wealth of data available with them but unfortunately,
they don't know how to derive value out of it since this data is in its raw form or
unstructured format.

Examples of Un-structured Data:

The output returned by 'Google Search'

Computer Science & Engineering Page 2

 DYPTC,Talsande.

Semi-structured:

Semi-structured data can contain both the forms of data. We can see semi-structured data
as a structured in form but it is actually not defined with e.g. a table definition in
relational DBMS. Example of semi-structured data is a data represented in an XML file.

Examples Of Semi-structured Data

Personal data stored in an XML file-

<rec><name>Prashant Rao</name><sex>Male</sex><age>35</age></rec>

<rec><name>Seema R.</name><sex>Female</sex><age>41</age></rec>

<rec><name>Satish Mane</name><sex>Male</sex><age>29</age></rec>

<rec><name>Subrato Roy</name><sex>Male</sex><age>26</age></rec>

<rec><name>Jeremiah J.</name><sex>Male</sex><age>35</age></rec>

Characteristics of Big Data:

Big data can be described by the following characteristics:

Computer Science & Engineering Page 3

 DYPTC,Talsande.

 Volume
 Variety
 Velocity
 Variability

(i) Volume – The name Big Data itself is related to a size which is enormous. Size of data
plays a very crucial role in determining value out of data. Also, whether a particular data can
actually be considered as a Big Data or not, is dependent upon the volume of data.
Hence, 'Volume' is one characteristic which needs to be considered while dealing with Big
Data.

(ii) Variety – Variety refers to heterogeneous sources and the nature of data, both structured
and unstructured. During earlier days, spreadsheets and databases were the only sources of
data considered by most of the applications. Nowadays, data in the form of emails, photos,
videos, monitoring devices, PDFs, audio, etc. are also being considered in the analysis
applications. This variety of unstructured data poses certain issues for storage, mining and
analyzing data.

(iii) Velocity – The term 'velocity' refers to the speed of generation of data. How fast the data
is generated and processed to meet the demands, determines real potential in the data. Big
Data Velocity deals with the speed at which data flows in from sources like business
processes, application logs, networks, and social media sites, sensors, Mobile devices, etc.
The flow of data is massive and continuous.

(iv) Variability – This refers to the inconsistency which can be shown by the data at times,
thus hampering the process of being able to handle and manage the data effectively.

Benefits of Big Data Processing:

Ability to process Big Data brings in multiple benefits, such as-

o Businesses can utilize outside intelligence while taking decisions

o Improved customer service

o Early identification of risk to the product/services, if any

o Better operational efficiency

Computer Science & Engineering Page 4

 DYPTC,Talsande.

Apache Hadoop:

Apache Hadoop is an open source software framework used to develop data processing
applications which are executed in a distributed computing environment.

Applications built using HADOOP are run on large data sets distributed across clusters of
commodity computers. Commodity computers are cheap and widely available. These are mainly
useful for achieving greater computational power at low cost.

Similar to data residing in a local file system of a personal computer system, in Hadoop, data
resides in a distributed file system which is called as a Hadoop Distributed File system. The
processing model is based on 'Data Locality' concept wherein computational logic is sent to
cluster nodes(server) containing data. This computational logic is nothing, but a compiled
version of a program written in a high-level language such as Java. Such a program, processes
data stored in Hadoop HDFS.

Hadoop Architecture:

High Level Hadoop Architecture

Hadoop has a Master-Slave Architecture for data storage and distributed data processing
using MapReduce and HDFS methods.

NameNode:
NameNode represented every files and directory which is used in the namespace
DataNode:
DataNode helps you to manage the state of an HDFS node and allows you to interacts
with the blocks
MasterNode:
The master node allows you to conduct parallel processing of data using Hadoop
MapReduce.

Computer Science & Engineering Page 5

 DYPTC,Talsande.

Slave node:
The slave nodes are the additional machines in the Hadoop cluster which allows you to
store data to conduct complex calculations. Moreover, all the slave node comes with Task
Tracker and a DataNode. This allows you to synchronize the processes with the
NameNode and Job Tracker respectively.

In Hadoop, master or slave system can be set up in the cloud or on-premise

Features Of Hadoop:

• Suitable for Big Data Analysis

As Big Data tends to be distributed and unstructured in nature, HADOOP clusters are
best suited for analysis of Big Data. Since it is processing logic (not the actual data) that
flows to the computing nodes, less network bandwidth is consumed. This concept is
called as data locality concept which helps increase the efficiency of Hadoop based
applications.

• Scalability

HADOOP clusters can easily be scaled to any extent by adding additional cluster nodes
and thus allows for the growth of Big Data. Also, scaling does not require modifications
to application logic.

• Fault Tolerance

HADOOP ecosystem has a provision to replicate the input data on to other cluster nodes.
That way, in the event of a cluster node failure, data processing can still proceed by using
data stored on another cluster node.

Conclusion:
Sample Questions:
1) What is Hadoop.

2) Explain Hadoop Architecture.

3) Explain components of Haddop.

Computer Science & Engineering Page 6

 DYPTC,Talsande.

EXPERIMENT NO: 02

Title: Installation of Hadoop.

Aim: Study of Installation procedure of Hadoop.

Theory:

Procedure for Install Hadoop with Step by Step Configuration on Ubuntu:

This is 2 part process

 Part 1) Download and Install Hadoop

 Part 2) Configure Hadoop

There are 2 Prerequisites

 You must have Ubuntu installed and running

 You must have Java Installed.

Part 1) Download and Install Hadoop

Step 1) Add a Hadoop system user using below command

sudo addgroup hadoop_

sudo adduser --ingroup hadoop_ hduser_

Computer Science & Engineering Page 7

 DYPTC,Talsande.

Enter your password, name and other details.

NOTE: There is a possibility of below-mentioned error in this setup and installation

process.

"hduser is not in the sudoers file. This incident will be reported."

This error can be resolved by Login as a root user

Execute the command

Computer Science & Engineering Page 8

 DYPTC,Talsande.

sudo adduser hduser_ sudo

Re-login as hduser_

Step 2) Configure SSH

In order to manage nodes in a cluster, Hadoop requires SSH access

First, switch user, enter the following command

su - hduser_

This command will create a new key.

ssh-keygen -t rsa -P ""

Computer Science & Engineering Page 9

 DYPTC,Talsande.

Enable SSH access to local machine using this key.

cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

Now test SSH setup by connecting to localhost as 'hduser' user.

ssh localhost

Computer Science & Engineering Page 10

 DYPTC,Talsande.

Note: Please note, if you see below error in response to 'ssh localhost', then there is a
possibility that SSH is not available on this system-

To resolve this -

Purge SSH using,

sudo apt-get purge openssh-server

It is good practice to purge before the start of installation

Install SSH using the command-

sudo apt-get install openssh-server

Computer Science & Engineering Page 11

 DYPTC,Talsande.

Step 3) Next step is to Download Hadoop

Select Stable

Computer Science & Engineering Page 12

 DYPTC,Talsande.

Select the [Link] file ( not the file with src)

Once a download is complete, navigate to the directory containing the tar file

Computer Science & Engineering Page 13

 DYPTC,Talsande.

Enter,

sudo tar xzf [Link]

Now, rename hadoop-2.2.0 as hadoop

sudo mv hadoop-2.2.0 hadoop

sudo chown -R hduser_:hadoop_ hadoop

Part 2) Configure Hadoop

Step 1) Modify ~/.bashrc file

Add following lines to end of file ~/.bashrc

#Set HADOOP_HOME
export HADOOP_HOME=<Installation Directory of Hadoop>
#Set JAVA_HOME
export JAVA_HOME=<Installation Directory of Java>
# Add bin/ directory of Hadoop to PATH
export PATH=$PATH:$HADOOP_HOME/bin

Computer Science & Engineering Page 14

 DYPTC,Talsande.

Now, source this environment configuration using below command

. ~/.bashrc

Step 2) Configurations related to HDFS

Set JAVA_HOME inside file $HADOOP_HOME/etc/hadoop/[Link]

Computer Science & Engineering Page 15

 DYPTC,Talsande.

With

There are two parameters in $HADOOP_HOME/etc/hadoop/[Link] which need

to be set-

1. '[Link]' - Used to specify a directory which will be used by Hadoop to store

its data files.

2. '[Link]' - This specifies the default file system.

Computer Science & Engineering Page 16

 DYPTC,Talsande.

To set these parameters, open [Link]

sudo gedit $HADOOP_HOME/etc/hadoop/[Link]

Copy below line in between tags <configuration></configuration>

<property>
<name>[Link]</name>
<value>/app/hadoop/tmp</value>
<description>Parent directory for other temporary directories.</description>
</property>
<property>
<name>[Link] </name>
<value>hdfs://localhost:54310</value>
<description>The name of the default file system. </description>
</property>

Navigate to the directory $HADOOP_HOME/etc/Hadoop

Computer Science & Engineering Page 17

 DYPTC,Talsande.

Now, create the directory mentioned in [Link]

sudo mkdir -p <Path of Directory used in above setting>

Grant permissions to the directory

sudo chown -R hduser_:Hadoop_ <Path of Directory created in above step>

sudo chmod 750 <Path of Directory created in above step>

Step 3) Map Reduce Configuration

Before you begin with these configurations, lets set HADOOP_HOME path

sudo gedit /etc/profile.d/[Link]

And Enter

export HADOOP_HOME=/home/guru99/Downloads/Hadoop

Next enter

Computer Science & Engineering Page 18

 DYPTC,Talsande.

sudo chmod +x /etc/profile.d/[Link]

Exit the Terminal and restart again

Type echo $HADOOP_HOME. To verify the path

Now copy files

sudo cp $HADOOP_HOME/etc/hadoop/[Link]
$HADOOP_HOME/etc/hadoop/[Link]

Open the [Link] file

sudo gedit $HADOOP_HOME/etc/hadoop/[Link]

Add below lines of setting in between tags <configuration> and </configuration>

<property>
<name>[Link]</name>
<value>localhost:54311</value>
<description>MapReduce job tracker runs at this host and port.
</description>
</property>

Computer Science & Engineering Page 19

 DYPTC,Talsande.

Open $HADOOP_HOME/etc/hadoop/[Link] as below,

sudo gedit $HADOOP_HOME/etc/hadoop/[Link]

Add below lines of setting between tags <configuration> and </configuration>

<property>
<name>[Link]</name>
<value>1</value>
<description>Default block replication.</description>
</property>
<property>
<name>[Link]</name>
<value>/home/hduser_/hdfs</value>
</property>

Computer Science & Engineering Page 20

 DYPTC,Talsande.

Create a directory specified in above setting-

sudo mkdir -p <Path of Directory used in above setting>

sudo mkdir -p /home/hduser_/hdfs

sudo chown -R hduser_:hadoop_ <Path of Directory created in above step>

sudo chown -R hduser_:hadoop_ /home/hduser_/hdfs

sudo chmod 750 <Path of Directory created in above step>

sudo chmod 750 /home/hduser_/hdfs

Computer Science & Engineering Page 21

 DYPTC,Talsande.

Step 4) Before we start Hadoop for the first time, format HDFS using below command

$HADOOP_HOME/bin/hdfs namenode –format

Step 5) Start Hadoop single node cluster using below command

$HADOOP_HOME/sbin/[Link]

An output of above command

$HADOOP_HOME/sbin/[Link]

Computer Science & Engineering Page 22

 DYPTC,Talsande.

Using 'jps' tool/command, verify whether all the Hadoop related processes are running or
not.

If Hadoop has started successfully then an output of jps should show NameNode,
NodeManager, ResourceManager, SecondaryNameNode, DataNode.

Step 6) Stopping Hadoop

$HADOOP_HOME/sbin/[Link]

$HADOOP_HOME/sbin/[Link]

Conclusion:

Computer Science & Engineering Page 23

 DYPTC,Talsande.

Sample Questions:
1) Explain Hadoop distributed file system.

2) Explain Installation steps of Hadoop.

Computer Science & Engineering Page 24

 DYPTC,Talsande.

EXPERIMENT NO: 03

Title: Introduction of Hadddop MapReduce.

Aim: Study of Hadoop Mapreduce.

Theory:

MapReduce is a software framework and programming model used for processing huge
amounts of data. MapReduce program work in two phases, namely, Map and Reduce. Map tasks
deal with splitting and mapping of data while Reduce tasks shuffle and reduce the data.

Hadoop is capable of running MapReduce programs written in various languages: Java, Ruby,
Python, and C++. The programs of Map Reduce in cloud computing are parallel in nature, thus
are very useful for performing large-scale data analysis using multiple machines in the cluster.

The input to each phase is key-value pairs. In addition, every programmer needs to specify two
functions: map function and reduce function.

MapReduce Architecture in Big Data:

The whole process goes through four phases of execution namely, splitting, mapping, shuffling,
and reducing.

Now in this MapReduce tutorial, let's understand with a MapReduce example–

Consider you have following input data for your MapReduce in Big data Program

Computer Science & Engineering Page 25

 DYPTC,Talsande.

MapReduce Architecture
The final output of the MapReduce task is

bad 1

Class 1

good 1

Hadoop 3

is 2

to 1

Welcome 1

The data goes through the following phases of MapReduce in Big Data
Computer Science & Engineering Page 26
 DYPTC,Talsande.

Input Splits:

An input to a MapReduce in Big Data job is divided into fixed-size pieces called input
splits Input split is a chunk of the input that is consumed by a single map

Mapping

This is the very first phase in the execution of map-reduce program. In this phase data in each
split is passed to a mapping function to produce output values. In our example, a job of mapping
phase is to count a number of occurrences of each word from input splits (more details about
input-split is given below) and prepare a list in the form of <word, frequency>

Shuffling

This phase consumes the output of Mapping phase. Its task is to consolidate the relevant records
from Mapping phase output. In our example, the same words are clubed together along with their
respective frequency.

Reducing

In this phase, output values from the Shuffling phase are aggregated. This phase combines values
from Shuffling phase and returns a single output value. In short, this phase summarizes the
complete dataset.

In our example, this phase aggregates the values from Shuffling phase i.e., calculates total
occurrences of each word.

MapReduce Architecture :

 One map task is created for each split which then executes map function for each record
in the split.
 It is always beneficial to have multiple splits because the time taken to process a split is
small as compared to the time taken for processing of the whole input. When the splits
are smaller, the processing is better to load balanced since we are processing the splits in
parallel.
 However, it is also not desirable to have splits too small in size. When splits are too
small, the overload of managing the splits and map task creation begins to dominate the
total job execution time.
 For most jobs, it is better to make a split size equal to the size of an HDFS block (which
is 64 MB, by default).
 Execution of map tasks results into writing output to a local disk on the respective node
and not to HDFS.

Computer Science & Engineering Page 27

 DYPTC,Talsande.

 Reason for choosing local disk over HDFS is, to avoid replication which takes place in
case of HDFS store operation.
 Map output is intermediate output which is processed by reduce tasks to produce the final
output.
 Once the job is complete, the map output can be thrown away. So, storing it in HDFS
with replication becomes overkill.
 In the event of node failure, before the map output is consumed by the reduce task,
Hadoop reruns the map task on another node and re-creates the map output.
 Reduce task doesn't work on the concept of data locality. An output of every map task is
fed to the reduce task. Map output is transferred to the machine where reduce task is
running.
 On this machine, the output is merged and then passed to the user-defined reduce
function.
 Unlike the map output, reduce output is stored in HDFS (the first replica is stored on the
local node and other replicas are stored on off-rack nodes). So, writing the reduce output

How MapReduce Organizes Work?

Hadoop divides the job into tasks. There are two types of tasks:

1. Map tasks (Splits & Mapping)

2. Reduce tasks (Shuffling, Reducing)

as mentioned above.

The complete execution process (execution of Map and Reduce tasks, both) is controlled by two
types of entities called a

1. Jobtracker: Acts like a master (responsible for complete execution of submitted job)
2. Multiple Task Trackers: Acts like slaves, each of them performing the job

For every job submitted for execution in the system, there is one Jobtracker that resides
on Namenode and there are multiple tasktrackers which reside on Datanode.

Computer Science & Engineering Page 28

 DYPTC,Talsande.

How Hadoop MapReduce Works

 A job is divided into multiple tasks which are then run onto multiple data nodes in a
cluster.
 It is the responsibility of job tracker to coordinate the activity by scheduling tasks to run
on different data nodes.
 Execution of individual task is then to look after by task tracker, which resides on every
data node executing part of the job.
 Task tracker's responsibility is to send the progress report to the job tracker.
 In addition, task tracker periodically sends 'heartbeat' signal to the Jobtracker so as to
notify him of the current state of the system.
 Thus job tracker keeps track of the overall progress of each job. In the event of task
failure, the job tracker can reschedule it on a different task tracker.

Computer Science & Engineering Page 29

 DYPTC,Talsande.

Conclusion:

Sample Questions:
1) What is MapReduce.
2) Explain Terminology of MapReduce.

Computer Science & Engineering Page 30

 DYPTC,Talsande.

EXPERIMENT NO: 04

Title: Study of Hadoop distributed file system.

Aim: Study of architecture & operations of HDFS

Theory:
HDFS:

HDFS is a distributed file system for storing very large data files, running on clusters of
commodity hardware. It is fault tolerant, scalable, and extremely simple to expand. Hadoop
comes bundled with HDFS (Hadoop Distributed File Systems).

When data exceeds the capacity of storage on a single physical machine, it becomes essential to
divide it across a number of separate machines. A file system that manages storage specific
operations across a network of machines is called a distributed file system. HDFS is one such
software.

HDFS Architecture:

HDFS cluster primarily consists of a NameNode that manages the file system Metadata and
a DataNodes that stores the actual data.

 NameNode: NameNode can be considered as a master of the system. It maintains the file
system tree and the metadata for all the files and directories present in the system. Two
files 'Namespace image' and the 'edit log' are used to store metadata information.
Namenode has knowledge of all the datanodes containing data blocks for a given file,
however, it does not store block locations persistently. This information is reconstructed
every time from datanodes when the system starts.
 DataNode: DataNodes are slaves which reside on each machine in a cluster and provide
the actual storage. It is responsible for serving, read and write requests for the clients.

Read/write operations in HDFS operate at a block level. Data files in HDFS are broken into
block-sized chunks, which are stored as independent units. Default block-size is 64 MB.

HDFS operates on a concept of data replication wherein multiple replicas of data blocks are
created and are distributed on nodes throughout a cluster to enable high availability of data in the
event of node failure.

Read Operation In HDFS:

Data read request is served by HDFS, NameNode, and DataNode. Let's call the reader as a
'client'. Below diagram depicts file read operation in Hadoop.

Computer Science & Engineering Page 31

 DYPTC,Talsande.

1. A client initiates read request by calling 'open()' method of FileSystem object; it is an

object of type DistributedFileSystem.
2. This object connects to namenode using RPC and gets metadata information such as the
locations of the blocks of the file. Please note that these addresses are of first few blocks
of a file.
3. In response to this metadata request, addresses of the DataNodes having a copy of that
block is returned back.
4. Once addresses of DataNodes are received, an object of type FSDataInputStream is
returned to the client. FSDataInputStream contains DFSInputStream which takes care
of interactions with DataNode and NameNode. In step 4 shown in the above diagram, a
client invokes 'read()' method which causes DFSInputStream to establish a connection
with the first DataNode with the first block of a file.
5. Data is read in the form of streams wherein client invokes 'read()' method repeatedly.
This process of read() operation continues till it reaches the end of block.
6. Once the end of a block is reached, DFSInputStream closes the connection and moves on
to locate the next DataNode for the next block
7. Once a client has done with the reading, it calls a close() method.

Computer Science & Engineering Page 32

 DYPTC,Talsande.

Write Operation In HDFS:

In this section, we will understand how data is written into HDFS through files.

1. A client initiates write operation by calling 'create()' method of DistributedFileSystem

object which creates a new file - Step no. 1 in the above diagram.
2. DistributedFileSystem object connects to the NameNode using RPC call and initiates
new file creation. However, this file creates operation does not associate any blocks with
the file. It is the responsibility of NameNode to verify that the file (which is being
created) does not exist already and a client has correct permissions to create a new file. If
a file already exists or client does not have sufficient permission to create a new file,
then IOException is thrown to the client. Otherwise, the operation succeeds and a new
record for the file is created by the NameNode.
3. Once a new record in NameNode is created, an object of type FSDataOutputStream is
returned to the client. A client uses it to write data into the HDFS. Data write method is
invoked (step 3 in the diagram).
4. FSDataOutputStream contains DFSOutputStream object which looks after
communication with DataNodes and NameNode. While the client continues writing

Computer Science & Engineering Page 33

 DYPTC,Talsande.

data, DFSOutputStream continues creating packets with this data. These packets are
enqueued into a queue which is called as DataQueue.
5. There is one more component called DataStreamer which consumes this DataQueue.
DataStreamer also asks NameNode for allocation of new blocks thereby picking desirable
DataNodes to be used for replication.
6. Now, the process of replication starts by creating a pipeline using DataNodes. In our
case, we have chosen a replication level of 3 and hence there are 3 DataNodes in the
pipeline.
7. The DataStreamer pours packets into the first DataNode in the pipeline.
8. Every DataNode in a pipeline stores packet received by it and forwards the same to the
second DataNode in a pipeline.
9. Another queue, 'Ack Queue' is maintained by DFSOutputStream to store packets which
are waiting for acknowledgment from DataNodes.
10. Once acknowledgment for a packet in the queue is received from all DataNodes in the
pipeline, it is removed from the 'Ack Queue'. In the event of any DataNode failure,
packets from this queue are used to reinitiate the operation.
11. After a client is done with the writing data, it calls a close() method (Step 9 in the
diagram) Call to close(), results into flushing remaining data packets to the pipeline
followed by waiting for acknowledgment.
12. Once a final acknowledgment is received, NameNode is contacted to tell it that the file
write operation is complete.

Conclusion:

Sample Questions:
1) Explain HDFS.
2) Describe read operation of HDFS.
3) Explain write operation of HDFS.

Computer Science & Engineering Page 34

 DYPTC,Talsande.

EXPERIMENT NO: 05
Title: Installation R-Programming environment.
Aim: Study of R- Programming Languages.

R programming is a very popular language and to work on that we have to install RGui.
Installing R to the local computer is very easy. First, we must know which operating system we
are using so that we can download it accordingly. The official site [Link]
[Link] provides installer files for major operating systems including Windows, Linux, and
Mac OS.

Install R in Windows:

To install R on your Windows, just follow these steps:

1. Download the R Installer File

2. Run the Installer
3. Install R
4. Add R to the PATH Environment Variable Manually
5. Verify the Installation

Step 1: Download the R Installer File

Go to the official R Project website ([Link] and download the latest version
of R for Windows.

Computer Science & Engineering Page 35

 DYPTC,Talsande.

To proceed, please click the link labelled “Download R for Windows”.

To proceed, please click the link labelled "install R for the first time."

Computer Science & Engineering Page 36

 DYPTC,Talsande.

To proceed, click the link labelled Download R-4.4.2 for Windows.

Subsequently, the R Installer File will begin to download.

Step 2: Run the Installer

Open your downloads folder and double-click on the R installer file that you just downloaded.
You may be required to grant access based on your security configurations. Please permit it and
continue.

Step 3: Install R

After executing the installer, you will encounter this screen.

Select your desired language. (Here English) and click on OK button.

Computer Science & Engineering Page 37

 DYPTC,Talsande.

And then click on Next button to accept licenses agreements,

You will be prompted to specify the installation directory. The default location is C:\Program
Files\R\.

Computer Science & Engineering Page 38

 DYPTC,Talsande.

You may retain this setting and continue with the installation by clicking on Next button.

The installer will then prompt you to choose components and where to place shortcuts.
Click Next button to continue installation.

Computer Science & Engineering Page 39

 DYPTC,Talsande.

Select Start-up options, then click Next button.

Select Start Menu Folder, then click Next button.

Computer Science & Engineering Page 40

 DYPTC,Talsande.

Select Additional Tasks, then click Next button.

Computer Science & Engineering Page 41

 DYPTC,Talsande.

Setup has finished installing R for Windows 4.4.2. Click Finish to exit the installer.

Step 4: Add R to the PATH Environment Variable Manually

It is essential to manually set that the PATH environment variable of your system is configured
properly.
To add the PATH environment variable, please follow these steps.

 Copy the path where R is installed to the bin folder.

 (Usually, it is in C:\Program Files\R\R-4.4.2\bin)
 Right-click on This PC and select Properties.
 Click on Advanced System Settings and then Environment Variables.
 Under User and System Variables sections, find and select Path, then click Edit.
 Click New and paste the path.

Computer Science & Engineering Page 42

 DYPTC,Talsande.

Click OK to close all dialogs and apply the changes.

Step 5: Verify the Installation

After the installation is finished, you can confirm that R has been installed correctly by opening a
command prompt (cmd) and entering the following command:
R --version

Computer Science & Engineering Page 43

 DYPTC,Talsande.

EXPERIMENT NO: 06

Title: Introduction of R Language and Study of datatypes, variables & logical operator in R
Aim: Study of fundamentals of R Language.

Theory:

R Software:

R is a programming language and free software developed by Ross Ihaka and Robert Gentleman
in 1993. R possesses an extensive catalog of statistical and graphical methods. It includes
machine learning algorithms, linear regression, time series, statistical inference to name a few.
Most of the R libraries are written in R, but for heavy computational tasks, C, C++ and Fortran
codes are preferred.

R is not only entrusted by academic, but many large companies also use R programming
language, including Uber, Google, Airbnb, Facebook and so on.

Data analysis with R is done in a series of steps; programming, transforming, discovering,

modeling and communicate the results

 Program: R is a clear and accessible programming tool

 Transform: R is made up of a collection of libraries designed specifically for data
science
 Discover: Investigate the data, refine your hypothesis and analyze them
 Model: R provides a wide array of tools to capture the right model for your data
 Communicate: Integrate codes, graphs, and outputs to a report with R Markdown or
build Shiny apps to share with the world

R used for:

 Statistical inference
 Data analysis
 Machine learning algorithm

Comparison R with other language:

Years ago, R was a difficult language to master. The language was confusing and not as
structured as the other programming tools. To overcome this major issue, Hadley Wickham
developed a collection of packages called tidyverse. The rule of the game changed for the best.
Data manipulation become trivial and intuitive. Creating a graph was not so difficult anymore.

Computer Science & Engineering Page 44

 DYPTC,Talsande.

The best algorithms for machine learning can be implemented with R. Packages like Keras and
TensorFlow allow to create high-end machine learning technique. R also has a package to
perform Xgboost, one the best algorithm for Kaggle competition.

R can communicate with the other language. It is possible to call Python, Java, C++ in R. The
world of big data is also accessible to R. You can connect R with different databases like Spark
or Hadoop.

Finally, R has evolved and allowed parallelizing operation to speed up the computation. In fact,
R was criticized for using only one CPU at a time. The parallel package lets you to perform tasks
in different cores of the machine.

Data Types in R:

Following are the Data Types or Data Structures in R Programming:

 Scalars
 Vectors (numerical, character, logical)
 Matrices
 Data frames
 Lists

Basics types:

 4.5 is a decimal value called numerics.

 4 is a natural value called integers. Integers are also numerics.
 TRUE or FALSE is a Boolean value called logical binary operators in R.
 The value inside " " or ' ' are text (string). They are called characters.
 Example 1:

Example 1:

# Declare variables of different types

# Numeric

x <- 28

class(x)

Output:

## [1] "numeric"

Computer Science & Engineering Page 45

 DYPTC,Talsande.

Example 2:

# String

y <- "R is Fantastic"

class(y)

Output:

## [1] "character"

Example 3:

# Boolean

z <- TRUE

class(z)

Output:

## [1] "logical"

Variables:

Variables are one of the basic data types in R that store values and are an important
component in R programming, especially for a data scientist. A variable in R data types can
store a number, an object, a statistical result, vector, dataset, a model prediction basically
anything R outputs. We can use that variable later simply by calling the name of the variable.

To declare variable data structures in R, we need to assign a variable name. The name should
not have space. We can use _ to connect to words.

To add a value to the variable in data types in R programming, use <- or =.

Here is the syntax:

# First way to declare a variable: use the `<-`

name_of_variable <- value

# Second way to declare a variable: use the `=`

Computer Science & Engineering Page 46

 DYPTC,Talsande.

name_of_variable = value

In the command line, we can write the following codes to see what happens:

Example 1:

# Print variable x

x <- 42

Output:

## [1] 42

Example 2:

y <- 10

Output:

## [1] 10

Example 3:

# We call x and y and apply a subtraction

x-y

Output:

## [1] 32

Vectors:

A vector is a one-dimensional array. We can create a vector with all the basic R data types
we learnt before. The simplest way to build vector data structures in R, is to use the c
command.

Computer Science & Engineering Page 47

 DYPTC,Talsande.

Example 1:

# Numerical

vec_num <- c(1, 10, 49)

vec_num

Output:

## [1] 1 10 49

Example 2:

# Character

vec_chr <- c("a", "b", "c")

vec_chr

Output:

## [1] "a" "b" "c"

Example 3:

# Boolean

vec_bool <- c(TRUE, FALSE, TRUE)

vec_bool

Output:

##[1] TRUE FALSE TRUE

We can do arithmetic calculations on vector binary operators in R.

Example 4:

# Create the vectors

Computer Science & Engineering Page 48

 DYPTC,Talsande.

vect_1 <- c(1, 3, 5)

vect_2 <- c(2, 4, 6)

# Take the sum of A_vector and B_vector

sum_vect <- vect_1 + vect_2

# Print out total_vector

sum_vect

Output:

[1] 3 7 11

Example 5:

In R, it is possible to slice a vector. In some occasion, we are interested in only the first five
rows of a vector. We can use the [1:5] command to extract the value 1 to 5.

# Slice the first five rows of the vector

slice_vector <- c(1,2,3,4,5,6,7,8,9,10)

slice_vector[1:5]

Output:

## [1] 1 2 3 4 5

Example 6:

The shortest way to create a range of value is to use the: between two numbers. For instance,
from the above example, we can write c(1:10) to create a vector of value from one to ten.

# Faster way to create adjacent values

c(1:10)

Output:

## [1] 1 2 3 4 5 6 7 8 9 10

Computer Science & Engineering Page 49

 DYPTC,Talsande.

Arithmetic Operators

We will first see the basic arithmetic operators in R data types. Following are the arithmetic
and boolean operators in R programming which stand for:

Operator Description

+ Addition

- Subtraction

* Multiplication

/ Division

^ or ** Exponentiation

Example 1:

# An addition

3+4

Output:

## [1] 7

You can easily copy and paste the above R code into Rstudio Console. The output is
displayed after the character #. For instance, we write the code print('Guru99') the output will
be ##[1] Guru99.

The ## means we print an output and the number in the square bracket ([1]) is the number of
the display

The sentences starting with # annotation. We can use # inside an R script to add any
comment we want. R won't read it during the running time.

Example 2:

# A multiplication

Computer Science & Engineering Page 50

 DYPTC,Talsande.

3*5

Output:

## [1] 15

Example 3:

# A division

(5+5)/2

Output:

## [1] 5

Example 4:

# Exponentiation

2^5

Output:

Example 5:

## [1] 32

# Modulo

28%%6

Output:

## [1] 4

Logical Operators:

With logical operators, we want to return values inside the vector based on logical
conditions. Following is a detailed list of logical operators of data types in R programming

Computer Science & Engineering Page 51

 DYPTC,Talsande.

Logical Operators in R

The logical statements in R are wrapped inside the []. We can add many conditional
statements as we like but we need to include them in a parenthesis. We can follow this
structure to create a conditional statement:

variable_name[(conditional_statement)]

With variable_name referring to the variable, we want to use for the statement. We create the
logical statement i.e. variable_name > 0. Finally, we use the square bracket to finalize the
logical statement. Below, an example of a logical statement.

Example 1:

# Create a vector from 1 to 10

logical_vector <- c(1:10)

logical_vector>5

Output:

## [1]FALSE FALSE FALSE FALSE FALSE TRUE TRUE TRUE TRUE TRUE

In the output above, R reads each value and compares it to the statement logical_vector>5. If
the value is strictly superior to five, then the condition is TRUE, otherwise FALSE. R returns
a vector of TRUE and FALSE.

Computer Science & Engineering Page 52

 DYPTC,Talsande.

Example 2:

In the example below, we want to extract the values that only meet the condition 'is strictly
superior to five'. For that, we can wrap the condition inside a square bracket precede by the
vector containing the values.

# Print value strictly above 5

logical_vector[(logical_vector>5)]

Output:

## [1] 6 7 8 9 10

Example 3:

# Print 5 and 6

logical_vector <- c(1:10)

logical_vector[(logical_vector>4) & (logical_vector<7)]

Output:

## [1] 5 6

Conclusion:

Sample Questions:
1) Describe R Environment.
2) Explain uses of R
3) Explain advantages of R.
4) Describe Data types in R.
5) Explain logical operator in R.
6) Explain variables in R.

Computer Science & Engineering Page 53

 DYPTC,Talsande.

EXPERIMENT NO: 07
Title: Implementation of Different Data Structures in R

Aim: To study and implement different data structures in R such as Vectors, Lists, and Data
Frames.

[Link]

A vector is the basic data structure in R. It stores elements of the same data type (numeric,
character, logical, etc.).

In R, a vector is a basic data structure used to store multiple values of the same [Link] is a one-
dimensional data structure and can hold numeric, character, logical, or other atomic types.
In R, To create a vector, we use c() function(combine function) and in this, the elements are
separated by a comma(,).
Syntax:

vect_name<- c(e1,e2,e3,..)

Example:

Filename: VectorEx.R

# Numeric vector
numbers <- c(10, 20, 30, 40)
print("Numeric Vector:")
print(numbers)

# Character vector
subjects <- c("Math", "Science", "History")
print("Character Vector:")
print(subjects)

# Logical vector
flags <- c(TRUE, FALSE, TRUE)
print("Logical Vector:")
print(flags)

Computer Science & Engineering Page 54

 DYPTC,Talsande.

Output:

[1] "Numeric Vector:"

[1] 10 20 30 40
[1] "Character Vector:"
[1] "Math" "Science" "History"
[1] "Logical Vector:"
[1] TRUE FALSE TRUE

2. Lists

A list is a data structure that can store elements of different data types such as numbers, strings,
vectors, and even data frames.

A list is a collection of elements of different types. Lists are particularly useful when you need to
store heterogeneous data.
In other words, A list is a flexible data structure that can store elements of different types,
including numbers, characters, vectors, matrices, other lists, and even functions.
In R, To create a list, we use list() function and in this, the elements are separated by a comma(,).
Syntax:

list_name<- list(e1,e2,e3,..)

Example:

Filename: ListEx.R

# Creating a list with different types

student <- list(
name = "John",
age = 21,
scores = c(85, 90, 95),
pass = TRUE
)

print("List Example:")
print(student)

Computer Science & Engineering Page 55

 DYPTC,Talsande.

# Accessing list elements

print(paste("Student name is", student$name))

Output:

[1] "List Example:"

$name
[1] "John"

$age
[1] 21

$scores
[1] 85 90 95

$pass
[1] TRUE

[1] "Student name is John"

3. Data Frames

A data frame is a two-dimensional tabular data structure where each column can have a
different data type, but all columns must have the same length.

In R, A data frame is a two-dimensional array-like structure, or we can say it is a table in which

each column contains the value of one variable, and row contains the set of value from each
column. Data Frames are two-dimensional, heterogeneous data structures.
In R, To create a data frame we use the [Link]() function.
Syntax:

Computer Science & Engineering Page 56

 DYPTC,Talsande.

df <- [Link](vector1, vector2, ..)Example:

Filename: DataFrameEx.R

# Creating a data frame

students <- [Link](
Names = c("Madhu", "Durga", "Naveen"),
Ages = c(22, 23, 21),
Scores = c(85.5, 90.0, 78.5)
)

print("Data Frame Example:")

print(students)

# Accessing a column
print("Names of students:")
print(students$Names)
Output:

[1] "Data Frame Example:"

Names Ages Scores
1 Madhu 22 85.5
2 Durga 23 90.0
3 Naveen 21 78.5
[1] "Names of students:"
[1] "Madhu" "Durga" "Naveen"

Computer Science & Engineering Page 57

 DYPTC,Talsande.

EXPERIMENT NO: 08

Title: Study of Line graph in R

Aim: Understanding of data visualization in R.

Theory:
R – Line Graphs:

A line graph is a chart that is used to display information in the form of a series of data
points. It utilizes points and lines to represent change over time. Line graphs are drawn by
plotting different points on their X coordinates and Y coordinates, then by joining them together
through a line from beginning to end. The graph represents different values as it can move up
and down based on the suitable variable.

R – Line Graphs:

The plot() function in R is used to create the line graph.

Syntax: plot(v, type, col, xlab, ylab)

Parameters:

 v: This parameter is a contains only the numeric values

 type: This parameter has the following value:
 “p” : This value is used to draw only the points.
 “l” : This value is used to draw only the lines.
 “o”: This value is used to draw both points and lines
 xlab: This parameter is the label for x axis in the chart.
 ylab: This parameter is the label for y axis in the chart.
 main: This parameter main is the title of the chart.
 col: This parameter is used to give colors to both the points and lines.

Creating a Simple Line Graph:

Approach: In order to create a line graph:

It is created using the type parameter as “o” and input vector.

Below code to describe the line graph.

Example:

Computer Science & Engineering Page 58

 DYPTC,Talsande.

# Create the data for the chart.

v <- c(17, 25, 38, 13, 41)

# Plot the bar chart.

plot(v, type = "o")

Output:

Adding Title, Color and Labels in Line Graphs in R

Approach: To create a colored and labeled line chart.

Take all parameters which are required to make line chart by giving a title to the chart and add
labels to the axes.

Computer Science & Engineering Page 59

 DYPTC,Talsande.

We can add more features by adding more parameters with more colors to the points and lines.

Example:

# Create the data for the chart.

v <- c(17, 25, 38, 13, 41)

# Plot the bar chart.

plot(v, type = "o", col = "green",

xlab = "Month", ylab = "Article Written",

main = "Article Written chart")

Output:

Computer Science & Engineering Page 60

 DYPTC,Talsande.

Multiple Lines in a Line Graph in R Programming Language

Approach: To create multiple line graphs.

In above example, we created line graphs by only one line in each graph.

Now creating multiple lines to describe it more clearly.

Example:

# Create the data for the chart.

v <- c(17, 25, 38, 13, 41)

t <- c(22, 19, 36, 19, 23)

m <- c(25, 14, 16, 34, 29)

# Plot the bar chart.

plot(v, type = "o", col = "red",

xlab = "Month", ylab = "Article Written ",

main = "Article Written chart")

lines(t, type = "o", col = "blue")

lines(m, type = "o", col = "green")

Output: When we execute the above code, it shows the following result-

Computer Science & Engineering Page 61

 DYPTC,Talsande.

Conclusions:

Sample Questions:
1) Describe Data frame in R.
2) Explain line graph in R.

Computer Science & Engineering Page 62

 DYPTC,Talsande.

EXPERIMENT NO: 09

Title: Study of Bar chart in R

Aim: Understanding of data visualization in R.

Theory:
R – Bar Charts:

A bar chart is a pictorial representation of data that presents categorical data with
rectangular bars with heights or lengths proportional to the values that they represent. In other
words, it is the pictorial representation of dataset. These data sets contain the numerical values of
variables that represent the length or height.

R uses the function barplot() to create bar charts. Here, both vertical and Horizontal bars can be
drawn.

Syntax:

barplot(H, xlab, ylab, main, [Link], col)

Parameters:

 H: This parameter is a vector or matrix containing numeric values which are used in bar
chart.
 xlab: This parameter is the label for x axis in bar chart.
 ylab: This parameter is the label for y axis in bar chart.
 main: This parameter is the title of the bar chart.
 [Link]: This parameter is a vector of names appearing under each bar in bar chart.
 col: This parameter is used to give colors to the bars in the graph.

Creating a Simple Bar Chart:

Approach: In order to create a Bar Chart:

A vector (H <- c(Values…)) is taken which contain numeral values to be used.

This vector H is plot using barplot().

Example:

# Create the data for the chart

A <- c(17, 32, 8, 53, 1)

Computer Science & Engineering Page 63

 DYPTC,Talsande.

# Plot the bar chart

barplot(A, xlab = "X-axis", ylab = "Y-axis", main ="Bar-Chart")

Output:

Creating a Horizontal Bar Chart:

Approach: To create a horizontal bar chart:

Take all parameters which are required to make simple bar chart.

Now to make it horizontal new parameter is added.

barplot(A, horiz=TRUE )

Example: Creating a horizontal bar chart

# Create the data for the chart

A <- c(17, 32, 8, 53, 1)

Computer Science & Engineering Page 64

 DYPTC,Talsande.

# Plot the bar chart

barplot(A, horiz = TRUE, xlab = "X-axis",

ylab = "Y-axis", main ="Bar-Chart")

Output:

Adding Label, Title and Color in the BarChart:

Label, title and colors are some properties in the bar chart which can be added to the bar by
adding and passing an argument.

Approach:

To add the title in bar chart.

barplot( A, main = title_name )

X-axis and Y-axis can be labeled in bar chart. To add the label in bar chart.

barplot( A, xlab= x_label_name, ylab= y_label_name)

To add the color in bar chart.

barplot( A, col=color_name)

Computer Science & Engineering Page 65

 DYPTC,Talsande.

Example :

# Create the data for the chart

A <- c(17, 2, 8, 13, 1, 22)

B <- c("Jan", "feb", "Mar", "Apr", "May", "Jun")

# Plot the bar chart

barplot(A, [Link] = B, xlab ="Month",

ylab ="Articles", col ="green",

main ="GeeksforGeeks-Article chart")

Output:

Computer Science & Engineering Page 66

 DYPTC,Talsande.

Creating Stacked and Grouped Bar Chart:

The bar chart can be represented in two form group of bars and stacked.

Approach:

Take a vector value and make it matrix M which to be grouped or stacked. Making of matrix can
be done by.

M <- matrix(c(values...), nrow = no_of_rows, ncol = no_of_column, byrow = TRUE)

To display the bar explicitly we can use the beside parameter.

barplot( beside=TRUE )

Example 1:

colors = c("green", "orange", "brown")

months <- c("Mar", "Apr", "May", "Jun", "Jul")

regions <- c("East", "West", "North")

# Create the matrix of the values.

Values <- matrix(c(2, 9, 3, 11, 9, 4, 8, 7, 3, 12, 5, 2, 8, 10, 11),

nrow = 3, ncol = 5, byrow = TRUE)

# Create the bar chart

barplot(Values, main = "Total Revenue", [Link] = months,

xlab = "Month", ylab = "Revenue",

col = colors, beside = TRUE)

# Add the legend to the chart

legend("topleft", regions, cex = 0.7, fill = colors)

Computer Science & Engineering Page 67

 DYPTC,Talsande.

Output:

Example 2:

colors = c("green", "orange", "brown")

months <- c("Mar", "Apr", "May", "Jun", "Jul")

regions <- c("East", "West", "North")

# Create the matrix of the values.

Values <- matrix(c(2, 9, 3, 11, 9, 4, 8, 7, 3, 12, 5, 2, 8, 10, 11),

nrow = 3, ncol = 5, byrow = TRUE)

# Create the bar chart

barplot(Values, main = "Total Revenue", [Link] = months,

Computer Science & Engineering Page 68

 DYPTC,Talsande.

xlab = "Month", ylab = "Revenue", col = colors)

# Add the legend to the chart

legend("topleft", regions, cex = 0.7, fill = colors)

Output:

Conclusions:

Sample Questions:
1) Describe data visualization in R.
2) Explain bar charts in R.

Computer Science & Engineering Page 69

 DYPTC,Talsande.

EXPERIMENT NO: 10

Title: Study of Pie chart in R

Aim: Understanding of data visualization in R.

Theory:
R – Pie Charts:

A pie chart is a circular statistical graphic, which is divided into slices to illustrate
numerical proportions. It depicts a special chart that uses “pie slices”, where each sector shows
the relative sizes of data. A circular chart cuts in a form of radii into segments describing relative
frequencies or magnitude also known as a circle graph.

R – Pie Charts:

R Programming Language uses the function pie() to create pie charts. It takes positive numbers
as a vector input.

Syntax: pie(x, labels, radius, main, col, clockwise)

Parameters:

 x: This parameter is a vector that contains the numeric values which are used in the pie
chart.
 labels: This parameter gives the description to the slices in pie chart.
 radius: This parameter is used to indicate the radius of the circle of the pie chart.(value
between -1 and +1).
 main: This parameter is represents title of the pie chart.
 clockwise: This parameter contains the logical value which indicates whether the slices
are drawn clockwise or in anti clockwise direction.
 col: This parameter give colors to the pie in the graph.

Creating a simple pie chart:

To create a simple pie chart:

By using the above parameters, we can draw a pie chart.

It can be described by giving simple labels.

Computer Science & Engineering Page 70

 DYPTC,Talsande.

Example:

# Create data for the graph.

geeks<- c(23, 56, 20, 63)

labels <- c("Mumbai", "Pune", "Chennai", "Bangalore")

# Plot the chart.

pie(geeks, labels)

Output:

Pie chart including the title and colors:

To create color and title pie chart.

Take all parameters which are required to make pie chart by giving a title to the chart and add
labels.

We can add more features by adding more parameters with more colors to the points.

Example:

# Create data for the graph.

Computer Science & Engineering Page 71

 DYPTC,Talsande.

geeks<- c(23, 56, 20, 63)

labels <- c("Mumbai", "Pune", "Chennai", "Bangalore")

# Plot the chart with title and rainbow

# color pallet.

pie(geeks, labels, main = "City pie chart",

col = rainbow(length(geeks)))

Output:

Slice Percentage & Chart Legend:

To create chart legend and slice percentage, we can plot by doing the below methods.

There are two more properties of the pie chart:

Computer Science & Engineering Page 72

 DYPTC,Talsande.

slice percentage

chart legend.

We can show the chart in the form of percentages as well as add legends.

Example:

# Create data for the graph.

geeks <- c(23, 56, 20, 63)

labels <- c("Mumbai", "Pune", "Chennai", "Bangalore")

piepercent<- round(100 * geeks / sum(geeks), 1)

# Plot the chart.

pie(geeks, labels = piepercent,

main = "City pie chart", col = rainbow(length(geeks)))

legend("topright", c("Mumbai", "Pune", "Chennai", "Bangalore"),

cex = 0.5, fill = rainbow(length(geeks)))

Output:

Computer Science & Engineering Page 73

 DYPTC,Talsande.

3D Pie Chart:

Here we are going to create a 3D Pie chart using plotrix package and then we will use pie3D()
function to plot 3D plot.

# Get the library.

library(plotrix)

# Create data for the graph.

geeks <- c(23, 56, 20, 63)

labels <- c("Mumbai", "Pune", "Chennai", "Bangalore")

piepercent<- round(100 * geeks / sum(geeks), 1)

# Plot the chart.

pie3D(geeks, labels = piepercent,

main = "City pie chart", col = rainbow(length(geeks)))

legend("topright", c("Mumbai", "Pune", "Chennai", "Bangalore"),

cex = 0.5, fill = rainbow(length(geeks)))

Output:

Computer Science & Engineering Page 74

 DYPTC,Talsande.

Conclusions:

Sample Questions:
1) Describe data visualization in R.
2) Explain Pie charts in R.

Computer Science & Engineering Page 75

 DYPTC,Talsande.

EXPERIMENT NO: 11

Title: Study of central tendency in R

Aim: Understanding of statistics in R.

Theory:
The measure of central tendency in R Language represents the whole set of data by a single
value. It gives us the location of central points. There are three main measures of central
tendency:

 Mean
 Median
 Mode

Prerequisite:

Before doing any computation, first of all, we need to prepare our data, save our data in external
.txt or .csv files and it’s a best practice to save the file in the current directory. After that import,
your data into R as follow:

Get the CSV file here.

# R program to import data into R

# Import the data using [Link]()

myData = [Link]("[Link]",

stringsAsFactors=F)

# Print the first 6 rows

Computer Science & Engineering Page 76

 DYPTC,Talsande.

print(head(myData))

Output:

Product Age Gender Education MaritalStatus Usage Fitness Income Miles

1 TM195 18 Male 14 Single 3 4 29562 112

2 TM195 19 Male 15 Single 2 3 31836 75

3 TM195 19 Female 14 Partnered 4 3 30699 66

4 TM195 19 Male 12 Single 3 3 32973 85

5 TM195 20 Male 13 Partnered 4 2 35247 47

6 TM195 20 Female 14 Partnered 3 3 32973 66

Mean in R Programming Language:

It is the sum of observations divided by the total number of observations. It is also defined as
average which is the sum divided by count.

Where, n = number of terms

Example:

# R program to illustrate

# Descriptive Analysis

# Import the data using [Link]()

myData = [Link]("[Link]",

stringsAsFactors=F)

Computer Science & Engineering Page 77

 DYPTC,Talsande.

# Compute the mean value

mean = mean(myData$Age)

print(mean)

Output:

[1] 28.78889

Median in R Programming Language:

It is the middle value of the data set. It splits the data into two halves. If the number of elements
in the data set is odd then the center element is median and if it is even then the median would be
the average of two central elements.

Where n = number of terms

Syntax: median(x, [Link] = False)

Where, X is a vector and [Link] is used to remove missing value

Example:

# R program to illustrate

# Descriptive Analysis

# Import the data using [Link]()

myData = [Link]("[Link]",

stringsAsFactors=F)

Computer Science & Engineering Page 78

 DYPTC,Talsande.

# Compute the median value

median = median(myData$Age)

print(median)

Output:

[1] 26

Mode in R Programming Language:

It is the value that has the highest frequency in the given data set. The data set may have no
mode if the frequency of all data points is the same. Also, we can have more than one mode if we
encounter two or more data points having the same frequency. There is no inbuilt function for
finding mode in R, so we can create our own function for finding the mode or we can use the
package called modest.

Creating user-defined function for finding Mode

There is no in-built function for finding mode in R. So let’s create a user-defined function that
will return the mode of the data passed. We will be using the table() method for this as it creates
a categorical representation of data with the variable names and the frequency in the form of a
table. We will sort the column Age column in descending order and will return the 1 value from
the sorted values.

Example: Finding mode by sorting the column of the dataframe

# Import the data using [Link]()

myData = [Link]("[Link]",

stringsAsFactors=F)

mode = function(){

return(sort(-table(myData$Age))[1])

Computer Science & Engineering Page 79

 DYPTC,Talsande.

mode()

Output:

25: -25

Using Modest Package:

We can use the modest package of the R. This package provides methods to find the mode of the
univariate data and the mode of the usual probability distribution.

Example:

# R program to illustrate

# Descriptive Analysis

# Import the library

library(modest)

# Import the data using [Link]()

myData = [Link]("[Link]",

stringsAsFactors=F)

# Compute the mode value

mode = mfv(myData$Age)

print(mode)

Output:

[1] 25

Conclusions:
Sample Questions:
1) Describe mean, median and mode.
2) Explain measure of central tendency in R.

Computer Science & Engineering Page 80

 DYPTC,Talsande.

EXPERIMENT NO: 12

Title: Study of linear regression in R

Aim: Understanding of statistics in R.

Theory:
R - Linear Regression:

Regression analysis is a very widely used statistical tool to establish a relationship model
between two variables. One of these variable is called predictor variable whose value is gathered
through experiments. The other variable is called response variable whose value is derived from
the predictor variable.

In Linear Regression these two variables are related through an equation, where exponent
(power) of both these variables is 1. Mathematically a linear relationship represents a straight
line when plotted as a graph. A non-linear relationship where the exponent of any variable is not
equal to 1 creates a curve.

The general mathematical equation for a linear regression is −

y = ax + b

Following is the description of the parameters used −

 y is the response variable.

 x is the predictor variable.
 a and b are constants which are called the coefficients.

Steps to Establish a Regression:

A simple example of regression is predicting weight of a person when his height is known. To do
this we need to have the relationship between height and weight of a person.

The steps to create the relationship is −

Carry out the experiment of gathering a sample of observed values of height and corresponding
weight.

Create a relationship model using the lm() functions in R.

Find the coefficients from the model created and create the mathematical equation using these

Computer Science & Engineering Page 81

 DYPTC,Talsande.

Get a summary of the relationship model to know the average error in prediction. Also called
residuals.

To predict the weight of new persons, use the predict() function in R.

Input Data

Below is the sample data representing the observations −

# Values of height

151, 174, 138, 186, 128, 136, 179, 163, 152, 131

# Values of weight.

63, 81, 56, 91, 47, 57, 76, 72, 62, 48

lm() Function:

This function creates the relationship model between the predictor and the response variable.

Syntax

The basic syntax for lm() function in linear regression is −

lm(formula,data)

Following is the description of the parameters used −

formula is a symbol presenting the relation between x and y.

data is the vector on which the formula will be applied.

Create Relationship Model & get the Coefficients

Live Demo

x <- c(151, 174, 138, 186, 128, 136, 179, 163, 152, 131)

y <- c(63, 81, 56, 91, 47, 57, 76, 72, 62, 48)

Computer Science & Engineering Page 82

 DYPTC,Talsande.

# Apply the lm() function.

relation <- lm(y~x)

print(relation)

When we execute the above code, it produces the following result −

Call:

lm(formula = y ~ x)

Coefficients:

(Intercept) x

-38.4551 0.6746

Get the Summary of the Relationship

Live Demo

x <- c(151, 174, 138, 186, 128, 136, 179, 163, 152, 131)

y <- c(63, 81, 56, 91, 47, 57, 76, 72, 62, 48)

# Apply the lm() function.

relation <- lm(y~x)

print(summary(relation))

When we execute the above code, it produces the following result −

Call:

lm(formula = y ~ x)

Computer Science & Engineering Page 83

 DYPTC,Talsande.

Residuals:

Min 1Q Median 3Q Max

-6.3002 -1.6629 0.0412 1.8944 3.9775

Coefficients:

Estimate Std. Error t value Pr(>|t|)

(Intercept) -38.45509 8.04901 -4.778 0.00139 **

x 0.67461 0.05191 12.997 1.16e-06 ***

---

Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 3.253 on 8 degrees of freedom

Multiple R-squared: 0.9548, Adjusted R-squared: 0.9491

F-statistic: 168.9 on 1 and 8 DF, p-value: 1.164e-06

predict() Function:

Syntax

The basic syntax for predict() in linear regression is −

predict(object, newdata)

Following is the description of the parameters used −

object is the formula which is already created using the lm() function.

newdata is the vector containing the new value for predictor variable.

Predict the weight of new persons

Live Demo

Computer Science & Engineering Page 84

 DYPTC,Talsande.

# The predictor vector.

x <- c(151, 174, 138, 186, 128, 136, 179, 163, 152, 131)

# The resposne vector.

y <- c(63, 81, 56, 91, 47, 57, 76, 72, 62, 48)

# Apply the lm() function.

relation <- lm(y~x)

# Find weight of a person with height 170.

a <- [Link](x = 170)

result <- predict(relation,a)

print(result)

When we execute the above code, it produces the following result −

76.22869

Visualize the Regression Graphically:

# Create the predictor and response variable.

x <- c(151, 174, 138, 186, 128, 136, 179, 163, 152, 131)

y <- c(63, 81, 56, 91, 47, 57, 76, 72, 62, 48)

relation <- lm(y~x)

# Plot the chart.

plot(y,x,col = "blue",main = "Height & Weight Regression",

Computer Science & Engineering Page 85

 DYPTC,Talsande.

abline(lm(x~y)),cex = 1.3,pch = 16,xlab = "Weight in Kg",ylab = "Height in cm")

When we execute the above code, it produces the following result –

Conclusions:

Sample Questions:
1) Describe linear regression.
2) Explain procedure to perform linear regression in R.

Computer Science & Engineering Page 86