NoSQL Data Management
MODULE 3-
Key-Value Stores and Map-Reduce Framework
Department of Computer Science Engineering
School of Engineering
Module 3 : Key-Value Stores and Map-
Reduce Framework
Map-Reduce, Basic Map-Reduce, Partitioning and Combining,
Composing Map-Reduce Calculations, A Two Stage Map-
Reduce Example, Incremental Map-Reduce. Key-Value
Databases, What Is a Key-Value Store, Key-Value Store
Features, Consistency, Transactions, Query Features, Structure
of Data, Scaling, Suitable Use Cases, Storing Session
Information, User Profiles, Preference, Shopping Cart Data,
When Not to Use, Relationships among Data, Multi operation
Transactions, Query by Data, Operations by Sets.
CSE3513 NoSQL Data Management 2
Understanding Map-Reduce
How Computers Work Together to
Handle Big Data
CSE2024 NoSQL Databases 3
Why Do We Need Map-Reduce?
• When we have a lot of data, one computer can’t handle it
all.
• So we use many computers (a cluster) to share the work.
• Map-Reduce helps make these computers work together
efficiently.
What Is Map-Reduce?
• Map-Reduce is a method used by big companies like Google
to handle huge amounts of data.
It has two main parts:
1. Map – Break the big job into small parts.
2. Reduce – Combine all the small results into one final
answer.
Example: Everyone counts their apples (Map), and the teacher
adds them up (Reduce).
Process of MAP REDUCE
Step 1: Map (Divide and Work in Parallel)
• Each computer takes one small piece of data (like an order
record).
• It looks for key information like product name, price, and
quantity.
• It sends this as a key-value pair such as tea → {price: $26,
quantity: 8}.
Step 2: Reduce (Combine the Results)
• The Reduce function collects all pairs with the same key
(same product).
• It adds up all their prices and quantities to get the total.
• Example: Prices = $26, $36, $44 → Total $106; Quantities =
8, 12, 14 → Total 34.
How It Works Together
1. Data is stored on many computers.
2. Each runs the Map function on its data.
3. Results go to a central computer for the Reduce function.
4. Final combined result is obtained efficiently!
Real-Life Example – Counting YouTube Views
• Each computer counts video views in one region (Map).
• All results are added up to find total global views (Reduce).
• This helps count billions of views quickly!
Summary
• Map: Divide big data into smaller parts and process them in
parallel.
• Reduce: Combine all results into one final output.
• Map-Reduce helps computers work together efficiently!
Understanding Map-Reduce
Partitioning, Combining, and
Reducing Made Easy
CSE3513 NO SQL DBMS 13
Example of Reduce
CSE3513 NO SQL DBMS 14
Map Step (Splitting Work)
Each mapper processes a portion of this input data (like
dividing work among multiple computers).
•Mapper 1 might process:
puerh 26
genmaicha 12
dragonwell 18
dragonwell 13
puerh 36
•Mapper 2 might process:
genmaicha 18
genmaicha 10
dragonwell 38
hojicha 9
puerh 44
Shuffle & Sort Step (Grouping by Key)
Now, the MapReduce system automatically groups together
all pairs with the same key (tea name), no matter which
mapper they came from.
So all puerh values go together, all dragonwell values go
together, etc.
It rearranges them like this:
CSE3513 NO SQL DBMS 16
Reduce Step (Combining / Aggregating)
Each reducer now gets one group of items (one tea type) and
processes it.
For example:
•Reducer 1 gets all puerh values and combines them
→ (26 + 36 + 44) = 106
•Reducer 2 gets all dragonwell values
→ (18 + 13 + 38) = 69
•Reducer 3 gets all genmaicha values
→ (12 + 18 + 10) = 40
•Reducer 4 gets all hojicha values
→ (9) = 9
After reduction, you have one summarized result per tea:
puerh 106
dragonwell 69
genmaicha 40
hojicha 9
CSE3513 NO SQL DBMS 17
Combining
• After mapping, we may have repeated data for the same
tea.
• Instead of sending all data to reduce, we combine locally
first.• Example:
• puerh = 26 + 36 + 20 + 40+6 = 128
• genmaicha = 12 + 8 + 16 = 36
• 🎯 Combining reduces data transfer and saves time.
Combining + Partitioning Together
Combining + Partitioning Together
Each mapper:
1. Maps data
2. Combines totals
3. Sends smaller results to reducers
Reducers finish the final totals.
Result: Less data movement + faster processing.
Non-Combinable Reducers
Some reducers cannot combine early.
Example: Counting unique customers per tea.
• Map step: (tea, customer)
• Reduce step: count unique customers.
• Combining early may lose uniqueness info.
Summary
✅ Map – Breaks data into small pieces.
✅ Partition – Divides work by key.
✅ Combine – Reduces repeated data.
✅ Reduce – Produces final result.
💡 Goal: Process big data quickly and efficiently.
Composing Map-Reduce
Calculations
CSE3513 NO SQL DBMS 23
How Map-Reduce Works
Map: Operates on each input record
independently.
Reduce: Works on grouped results of
the same key.
Helps in aggregating, counting, or
summarizing data.
Calculating Averages
Calculating Averages
Averages are not directly composable.
To calculate averages:
Combine total sum and count first.
Then compute average = total sum /
count.
Counting with Map-Reduce
• Each Map emits a count of 1 per
record.
• Reduce step sums these counts for
total results.
Two-Stage Map-Reduce Example
Two-Stage Map-Reduce Example
• Used to compare sales of products
year-over-year.
• Step 1: Aggregate product sales by
month.
• Step 2: Compare monthly results
between years.
Stage 1 – Monthly Product Sales
Stage 1 – Monthly Product Sales
• Reads original order records.
• Outputs key-value pairs for sales of each
product per month.
Stage 2 – Year Comparison
• Maps current and prior year
quantities.
• 2011 → Current year
• 2010 → Prior year
• Earlier years are ignored.
Stage 2 – Year Comparison
Reduce Step – Merging Records
• Reduce merges records by summing quantities.
• Computes percentage increase or decrease between
years.
Why Multi-Stage Map-Reduce?
Easier to write and debug small
steps.
Enables reuse of intermediate
results.
Saves time in programming and
execution.
Incremental Map-Reduce
Allows updates without reprocessing
all data.
Maps can be rerun only for changed
inputs.
Reduce can be optimized if partitions
are unchanged.
Supports additive updates efficiently.
Tools for Map-Reduce
Hadoop – Open-source Map-Reduce
system.
Apache Pig – High-level scripting for
Hadoop.
Apache Hive – SQL-like syntax for
Map-Reduce.
Used widely for large-scale data
processing.
Key Takeaways
Map-Reduce simplifies large-scale
parallel computations.
Decomposing steps increases clarity
and reusability.
Incremental and composable Map-
Reduce make data pipelines efficient.
Key-Value Databases (Riak
Example)
Understanding structure,
features, and use cases of
Key-Value stores
What is a Key-Value Store?
A simple NoSQL database using key-
value pairs
Acts like a hash table: {Key → Value}
Used for high-speed primary key
access
Example systems: Riak, Redis,
DynamoDB, Memcached, Berkeley DB
Applications store and retrieve data
via keys only
Structure of Data in Key-Value Stores
• Data stored as Key-
Value pairs
• Value can be any
blob (text, JSON,
XML, binary)
• Example structures
in Riak (right image)
Alternate Key Design
• Avoids storing multiple object types
in one bucket
• Appends object name to key (e.g.,
sessionID_userProfile)
• Allows retrieval of individual objects
easily
Features of Key-Value Stores
• Consistency: Eventual consistency
using replication
• Transactions: Based on quorum (N, R,
W values)
• Query: Retrieve only by key
• Structure: Value can be any format
• Scaling: Achieved through sharding
and replication
Suitable Use Cases
Session Information storage
User Profiles and Preferences
Shopping Cart Data
These use cases benefit from fast
access and single-key retrieval.
When Not to Use Key-Value Stores
When data relationships are
important
When multi-key transactions are
needed
When queries on value fields are
required
When set-based or multi-record
operations are needed
Summary
Key-Value stores are the simplest
NoSQL systems
Provide high scalability and
performance
Ideal for simple lookups, sessions,
and caching
Not suitable for relational or
complex query scenarios
CSE3513 NoSQL Data Management 47