Ex.

No: 01 : INSTALL APACHE HADOOP

AIM:

To initialize and start Hadoop in pseudo-distributed mode, configure key Hadoop files, and
check the execution status of Hadoop.

PROCEDURE:

Step 1: Installation

1. Install Java JDK 1.8 on the system.

2. Install Apache Hadoop by extracting the Hadoop zip file.

3. Place Hadoop in the directory:

C:\hadoop

4. Verify that [Link] is present inside:

C:\hadoop\bin

5. Set the environment variables:

JAVA_HOME = C:\Java\jdk1.8.0_202

HADOOP_HOME = C:\hadoop

6. Add the following paths to the system PATH variable:

%JAVA_HOME%\bin

%HADOOP_HOME%\bin

%HADOOP_HOME%\sbin

7. Create directories for NameNode and DataNode:

C:\hadoop\data\namenode

C:\hadoop\data\datanode
Step 2: Configure Hadoop Files

Navigate to:

C:\hadoop\etc\hadoop

Edit the following configuration files.

[Link]

<configuration>

<property>

<name>[Link]</name>

<value>hdfs://localhost:9000</value>

</property>

</configuration>

[Link]

set JAVA_HOME=C:\Java\jdk1.8.0_202

[Link]

<configuration>

<property>

<name>[Link]</name>

<value>1</value>

</property>

<property>

<name>[Link]</name>

<value>file:/C:/hadoop/data/namenode</value>

</property>
 <property>

<name>[Link]</name>

<value>file:/C:/hadoop/data/datanode</value>

</property>

</configuration>

[Link]

<configuration>

<property>

<name>[Link]</name>

<value>yarn</value>

</property>

</configuration>

[Link]

<configuration>

<property>

<name>[Link]-services</name>

<value>mapreduce_shuffle</value>

</property>

</configuration>

Step 3: Format the NameNode

Format the NameNode using the command:

hdfs namenode -format

Navigate to Hadoop home directory and start services:

[Link]

[Link]

Step 5: Check Hadoop Status

1. Open Command Prompt as Administrator.

2. Verify running Hadoop daemons using:

3. jps

Step 6: Access Hadoop Web Interfaces

• NameNode Web UI:

[Link]

• ResourceManager Web UI:

[Link]

OUTPUT:

Formatting NameNode:

hdfs namenode -format

Starting Hadoop services:

[Link]

[Link]

Running processes:

NameNode

DataNode

ResourceManager
 NodeManager

Jps

RESULT:

Thus, the Hadoop cluster has been successfully constructed in a single-node (pseudo-
distributed) setup. Configuration files were set, the NameNode was formatted, Hadoop services
were started, and all Hadoop components were verified using the jps command. The cluster was
also monitored using Hadoop web interfaces.

Ex. No: 02 : MAP REDUCE PROGRAM TO CALCULATE THE FREQUENCY OF A GIVEN WORD IN A
GIVEN FILE

AIM:

To execute a MapReduce program to calculate the frequency of a given word from a text file
using Hadoop.

PROCEDURE (SIMPLIFIED – CMD BASED):

Step 1: Start Hadoop

[Link]

[Link]

Check:

jps

Step 2: Create Text File using CMD

Go to Hadoop directory:

cd C:\hadoop
Create text file:

notepad [Link]

Enter content and save:

big data analytics

hadoop big data

big data Hadoop

Step 3: Upload File to HDFS

Create HDFS directory and upload the file:

hdfs dfs -mkdir /ex2

hdfs dfs -put [Link] /ex2

Verify upload:

hdfs dfs -ls /ex2

Step 4 Run Grep MapReduce Program

Execute the Grep MapReduce program to count the given word “big”:

hadoop jar %HADOOP_HOME%\share\hadoop\mapreduce\hadoop-mapreduce-examples-

Step 5: View Output

hdfs dfs -cat /ex2_out/part-r-00000

OUTPUT:

big 3
RESULT:

Thus, the MapReduce program was executed successfully and the frequency of the given word
was calculated using Hadoop.

Ex. No: 03 : CREATE HDFS COMMANDS

AIM:

To study and execute basic Hadoop Distributed File System (HDFS) commands for file and
directory operations.

PROCEDURE (CMD BASED):

Step 1: Start Hadoop Services

[Link]

[Link]

Verify:

jps

Step 2: List Root Directory in HDFS

hdfs dfs -ls /

Step 3: Create a Directory in HDFS

hdfs dfs -mkdir /hdfs_test

Step 4: Create a Text File in Local System

Go to Hadoop directory:

cd C:\hadoop
Create file:

notepad [Link]

Enter content and save:

Hadoop is a framework for big data processing

Step 5: Copy File from Local System to HDFS

hdfs dfs -put [Link] /hdfs_test

Step 6: List Files in HDFS Directory

hdfs dfs -ls /hdfs_test

Step 7: View File Content in HDFS

hdfs dfs -cat /hdfs_test/[Link]

Step 8: Copy File from HDFS to Local System

hdfs dfs -get /hdfs_test/[Link] C:\hadoop

Step 9: Delete File from HDFS

hdfs dfs -rm /hdfs_test/[Link]

Step 10: Remove HDFS Directory

hdfs dfs -rmdir /hdfs_test

/hdfs_test

[Link]

Hadoop is a framework for big data processing

RESULT:

Thus, basic HDFS commands were successfully executed to create, list, view, copy, and delete
files and directories in Hadoop Distributed File System.

Ex. No: 04 : STUDY OF SERIALIZES AND DESERIALIZES OF INTEGER TYPE IN HADOOP

AIM:

To study and implement serialization and deserialization of integer data type using Hadoop’s
IntWritable class.

PROCEDURE:

Step 1: Start Hadoop Services

[Link]

[Link]

Verify:

jps

Step 2: Create Java Program

Navigate to Hadoop directory:

cd C:\hadoop
Create Java file:

notepad [Link]

Step 3: Program Code

import [Link].*;

import [Link];

public class IntSerialize {

public static void main(String[] args) throws IOException {

IntWritable number = new IntWritable(100);

// Serialization

FileOutputStream fos = new FileOutputStream("[Link]");

DataOutputStream dos = new DataOutputStream(fos);

[Link](dos);

[Link]();

// Deserialization

IntWritable readNumber = new IntWritable();

FileInputStream fis = new FileInputStream("[Link]");

DataInputStream dis = new DataInputStream(fis);

[Link](dis);

[Link]();
 [Link]("Deserialized Integer Value: " + [Link]());

Step 4: Compile the Program

javac -classpath
"%HADOOP_HOME%\share\hadoop\common\*;%HADOOP_HOME%\share\hadoop\common\l
ib\*" [Link]

Step 5: Execute the Program

java -classpath
".;%HADOOP_HOME%\share\hadoop\common\*;%HADOOP_HOME%\share\hadoop\common
\lib\*" IntSerialize

OUTPUT:

Deserialized Integer Value: 100

RESULT:

Thus, serialization and deserialization of integer data type was successfully implemented using
Hadoop’s IntWritable class.

Ex. No: 05 : RUN A BASIC WORD COUNT MAPREDUCE PROGRAM TO UNDERSTAND

AIM:

To run a basic Word Count MapReduce program in Hadoop to understand the working of the
MapReduce paradigm.
PROCEDURE:

Step 1: Start Hadoop Services

[Link]

[Link]

Verify:

jps

Step 2: Create Input Text File

Navigate to Hadoop directory:

cd C:\hadoop

Create file:

notepad [Link]

Enter content and save:

hadoop mapreduce framework

hadoop big data

big data Hadoop

Step 3 Upload File to HDFS

Create HDFS directory and upload the file:

hdfs dfs -mkdir /ex5

hdfs dfs -put [Link] /ex5

Verify upload:

hdfs dfs -ls /ex5

hadoop jar %HADOOP_HOME%\share\hadoop\mapreduce\hadoop-mapreduce-examples-

Step 5: View Output

hdfs dfs -cat /ex5_out/part-r-00000

OUTPUT:

big 2

data 2

framework 1

hadoop 3

mapreduce 1

RESULT:

Thus, the Word Count MapReduce program was successfully executed in Hadoop, and the
MapReduce paradigm was understood by counting the occurrences of each word in the input
file.