Scribd
Upload
11 views1 page

HDFS Data Processing Steps Guide

The document outlines a data processing workflow consisting of three main steps: Map, Shuffle, and Reduce. It assumes the use of four data partitions and provides a structured representation of data distribution across these partitions. The data is stored in HDFS and involves various numerical values associated with each partition.

Uploaded by

rashiqueparvez8538
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
11 views1 page

HDFS Data Processing Steps Guide

The document outlines a data processing workflow consisting of three main steps: Map, Shuffle, and Reduce. It assumes the use of four data partitions and provides a structured representation of data distribution across these partitions. The data is stored in HDFS and involves various numerical values associated with each partition.

Uploaded by

rashiqueparvez8538
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

STEP 0 –

1 - MAP 2 – SHUFFLE 3 - REDUCE


STORE TO
and SORT
HDFS
Assume 4 data
partitions. ( , 2 ) ( , 2 )
( , 2 ) ( , 8 )
( , 1 ) ( , 3 )
( , 1 ) ( , 1 ) ( , 4 )
( , 2 ) ( , 5 )
( , 3 )
( , 4 )
( , 2 ) ( , 2 )
( , 2 ) ( , 3 )
( , 1 )
( , 1 )
( , 1 ) ( , 2 )
( , 2 ) ( , 2 )
( , 2 )
( , 1 ) ( , 1 )
( , 2 )
( , 2 ) ( , 1 )
( , 2 ) ( , 1 )
( , 1 ) ( , 1 )
( , 1 ) ( , 1 )

You might also like