Scribd
Upload
10 views4 pages

MapReduce WordCount Assignment

The document outlines a MapReduce Word Count assignment involving three movie review files. It details the steps for preprocessing the text, mapping key-value pairs, shuffling and sorting the data, and reducing to final word counts, along with a reflection on common words. The assignment emphasizes the importance of common words in English movie reviews.

Uploaded by

adarshhalse17
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
10 views4 pages

MapReduce WordCount Assignment

The document outlines a MapReduce Word Count assignment involving three movie review files. It details the steps for preprocessing the text, mapping key-value pairs, shuffling and sorting the data, and reducing to final word counts, along with a reflection on common words. The assignment emphasizes the importance of common words in English movie reviews.

Uploaded by

adarshhalse17
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

MapReduce Word Count Assignment

Question (10 Marks):

You are given the following 3 input files, each containing two-line movie reviews:

File 1 - [Link]

The visuals were stunning and the music was mesmerizing.

I loved every frame of the movie.

File 2 - [Link]

The plot was engaging but the pace was slow.

Still, the performances were top-notch.

File 3 - [Link]

Not the best movie I have seen, but it was entertaining.

Dialogues were crisp and the direction was tight.

Instructions:

Using the MapReduce framework, perform a word count on the given input.

Your answer should include:

1. (2 Marks) Preprocessing: Convert all text to lowercase, remove punctuation, show cleaned and

tokenized version of each file

2. (3 Marks) Map Phase: List all (key, 1) pairs generated by each file's mapper

3. (2 Marks) Shuffle & Sort: Group the key-value pairs by key

4. (2 Marks) Reduce Phase: Show the final word count for each word
5. (1 Mark) Reflection: Mention two common words that appear across all three files, and explain

why they are likely common in reviews

Solution:

Step 1: Preprocessing (2 Marks)

File 1:

the visuals were stunning and the music was mesmerizing

i loved every frame of the movie

File 2:

the plot was engaging but the pace was slow

still the performances were top notch

File 3:

not the best movie i have seen but it was entertaining

dialogues were crisp and the direction was tight

Step 2: Map Phase (3 Marks)

Mapper Output - File 1:

(the,1), (visuals,1), (were,1), (stunning,1), (and,1), (the,1),

(music,1), (was,1), (mesmerizing,1), (i,1), (loved,1), (every,1),

(frame,1), (of,1), (the,1), (movie,1)

Mapper Output - File 2:

(the,1), (plot,1), (was,1), (engaging,1), (but,1), (the,1),

(pace,1), (was,1), (slow,1), (still,1), (the,1), (performances,1),


(were,1), (top,1), (notch,1)

Mapper Output - File 3:

(not,1), (the,1), (best,1), (movie,1), (i,1), (have,1), (seen,1),

(but,1), (it,1), (was,1), (entertaining,1), (dialogues,1),

(were,1), (crisp,1), (and,1), (the,1), (direction,1), (was,1),

(tight,1)

Step 3: Shuffle and Sort (2 Marks)

Grouped Values (Key -> List of Counts): [Sample]

the: [1,1,1,1,1,1,1,1]

was: [1,1,1,1,1]

were: [1,1,1]

and: [1,1]

i: [1,1]

but: [1,1]

movie: [1,1]

Step 4: Reduce Phase (2 Marks)

Final Word Counts:

the: 8

was: 5

were: 3

i: 2

and: 2

but: 2

movie: 2
(other words occur once each)

Step 5: Reflection (1 Mark)

Common Words: "the", "was"

Reason: These are common descriptive or grammatical words that frequently appear in English

movie reviews.

You might also like