Scribd
Upload
3 views3 pages

Hadoop Code Bundling and Execution Guide

This document provides two options for creating MapReduce code in Java - using Eclipse or the build tool ANT. It explains how to set up a MapReduce project in Eclipse, include the necessary libraries, export the code as a JAR file, and run it. For ANT, it describes downloading the wordcount example, deleting the existing JAR, running ant to build a new JAR, and then running the code. The document states that individual programs have readme files with more details and is intended to answer how to code MapReduce jobs.

Uploaded by

SERGIO COUTINHO
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
3 views3 pages

Hadoop Code Bundling and Execution Guide

This document provides two options for creating MapReduce code in Java - using Eclipse or the build tool ANT. It explains how to set up a MapReduce project in Eclipse, include the necessary libraries, export the code as a JAR file, and run it. For ANT, it describes downloading the wordcount example, deleting the existing JAR, running ant to build a new JAR, and then running the code. The document states that individual programs have readme files with more details and is intended to answer how to code MapReduce jobs.

Uploaded by

SERGIO COUTINHO
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

One reducer—> Everything is sorted..

10 reducers—>Output in each reducer is sorted but its


not globally

Total Order Partitioner…

if k1 > k2 then reducer for key 1 > reducer for key 2

reducer reducer
a
b d
c

Dude, how do i write this code


Dude, how do i bundle this code…

How to create the code:

OPTION 1 USING Eclipse:


- Take the code from LMS
- Unzip it
- Take the wordcount folder and put in WorkSpace
- Open Eclipse
- New Java Project
- Include libraries from
/usr/lib/hadoop, /usr/lib/hadoop/lib, /usr/lib/hadoop/h
adoop-mapreduce, /usr/lib/hadoop-yarn
- Export the code as JAR file
- Run the JAR
Bang you are done

OPTION 2 USING Build Tool called ANT:


- Install ANT: sudo yum install ant
- Copy wordcount folder from the LMS code files to
say a new location of your choice, we did it in america
- cd wordcount and delete existing jar
- Run $>ant
- Bang your JAR is ready..
- Run your code…

If you want more details each program has a read me


file…

At this moment, we are clear with:


- HDFS
- How code gets submitted
- How code gets bundled from Eclipse as well as from
command line using ANT
Last question to be answered today is… HOW TO
CODE ! MR…
Lets jump on to it..
Before that lets take up any questions…

You might also like