📘 Apache Pig (Simplified 7-Mark Answer)

🔹 Introduction:
Apache Pig is a high-level data flow platform built on top of Hadoop for processing large data
sets. It uses a simple scripting language called Pig Latin.
Pig scripts are internally converted into MapReduce jobs and run on data stored in HDFS
(Hadoop Distributed File System). Pig can also run on Apache Tez or Apache Spark engines.
It can handle structured, semi-structured, and unstructured data. Any task that can be done using
Java in MapReduce can also be done using Pig in an easier way.
🔹 Features of Apache Pig:
1. ✅ Ease of Programming:
Pig Latin is easy to write compared to complex Java MapReduce code. Non-
programmers can also use it easily.
2. ✅ Optimization:
Pig automatically optimizes the execution plan, so users don’t have to worry about
performance.
3. ✅ Extensibility:
Users can write their own functions (UDFs) in Java, Python, etc., to apply custom logic
on data.
4. ✅ Flexibility:
Pig can handle all types of data — structured and unstructured.
5. ✅ In-built Operators:
Pig has many built-in operators like sort, filter, and join for easy data processing.
🔹 Advantages of Apache Pig:
 ✏️Less Code: Pig Latin needs fewer lines than Java MapReduce.
 ✏️Reusability: Scripts can be reused.
 ✏️Supports Nested Data Types: Like tuples, bags, and maps — useful
for complex data.
🔹 Limitations of Apache Pig:
 ❌ Not suitable for real-time processing.
 ❌ Slower for small data sets.
 ❌ Requires learning Pig Latin (new language).
 ❌ Cannot handle failure recovery automatically like advanced tools.
✅ Conclusion:
Apache Pig is an easy and efficient tool to process large-scale data using simple scripts. It
reduces development time and supports all types of data, though it’s not ideal for real-time or
small-data tasks.

Here Is a simplified and exam-ready 7-mark answer for “MapReduce” in easy English:
📘 MapReduce – Simplified 7-Mark Answer

🔹 What is MapReduce?

MapReduce is a programming model used for processing large amounts of data across many
computers (called nodes) in a Hadoop cluster. It works in a distributed and parallel way and is
written using Java. The main goal is to process data fast and reliably using low-cost (commodity)
hardware.

MapReduce has two main functions:

1. Map – Breaks input data into key-value pairs (tuples).

2. Reduce – Combines and summarizes these key-value pairs into a smaller set of
results.

The reduce task alwayss runs after the map task.

🔹 Key Features:

Processes huge data sets across many machines.

Automatically handles data transfer, task management, and failure recovery.

Makes scaling easy — just change configuration to run on more machines.

🔹 How MapReduce Works (3 Stages):

1. Map Stage:

Input data (from HDFS) is given to the mapper function line-by-line.

Mapper processes the data and outputs it as key-value pairs.

 2. Shuffle Stage:

The framework sorts and transfers the mapper output to the reducer.

Ensures that data with the same key goes to the same reducer.

3. Reduce Stage:

Reducer processes the grouped data.

Final output is written back to HDFS.

🔹 Benefits of MapReduce:

Parallel processing = faster performance.

Runs on local disks = less network traffic.

Scalable = can run on hundreds or thousands of machines easily.

Fault-tolerant = handles failures automatically.

🔹 Conclusion:

MapReduce is a powerful data processing technique in Hadoop. By dividing tasks into map and
reduce phases, it helps in fast, reliable, and scalable analysis of big data.
Absolutely, Dhanvi! Here’s your easy-to-remember, 7-marks-ready answer for Election
Algorithms (Bully + Ring) — written in simple English with enough detail, structure, and length
to fill at least 1.5–2 handwritten pages.

📘 Election Algorithms – (Bully and Ring)

Used in Distributed Systems when the current coordinator fails.

✅ What is an Election Algorithm?

When the coordinator (leader) in a distributed system crashes or becomes unresponsive, an

election is conducted to select a new coordinator.

Two most common algorithms are:

🔶 1. Bully Algorithm 😤

This algorithm is based on the idea that higher-numbered processes are stronger and will “bully”
lower-numbered ones.

🔹 Steps:

1. A process (say P4) notices that the coordinator is not responding.

2. It sends an ELECTION message to all higher-numbered processes (P5, P6, P7…).

3. If no one replies, P4 declares itself the new coordinator.

4. If a higher-numbered process replies, P4 drops out.

 5. That higher process now starts its own election.

6. This continues until the highest active process wins.

🔹 Example from Figure 6.20:

(a) Process 4 starts election.

(b) Process 5 and 6 respond → 4 drops out.

© Both 5 and 6 now hold elections.

(c) Process 6 tells 5 to stop.

€ Process 6 wins and tells everyone.

🔹 Special Cases:

If Process 7 (a higher process) comes back up, it can send a new election message and become
the coordinator.

We can also use “Are You Alive?” messages to quickly detect failures.

🔶 2. Ring Algorithm 🔁

Here, processes are arranged in a logical ring, and each one knows only its next neighbor.

🔹 Steps:

1. A process (say P3) notices the coordinator has failed.

 2. It sends an ELECTION message with its number to the next process in the ring.

3. Each process adds its number to the message and forwards it.

4. When the message comes back to the starter (P3), it finds the highest number.

5. That process becomes the new coordinator.

6. A new message COORDINATOR is sent around the ring to inform everyone.

🔹 Key Points:

All processes are equal (unlike Bully where stronger ones dominate).

Takes longer than Bully but uses fewer messages.

Works well in systems with circular topology.

📝 Table: Bully vs Ring Comparison

Feature Bully Algorithm Ring Algorithm

Type Hierarchical Equal / Round-robin

Speed Fast Slower
Message Load More messages Fewer messages
Knowledge Required Knows all processes Only next neighbor
Coordinator Selected Highest active processHighest in ring message
✅ Conclusion:

Election algorithms help distributed systems to recover from coordinator failure.

Bully is faster and works best when all processes are known.

Ring is simple, uses fewer messages, and ensures fairness.

Both ensure the system continues running without interruption.

Sure! Here’s a simplified and neat version of your notes on Grid Computing:

Grid Computing – Simplified Notes

What is Grid Computing?

A network of computers that work together to perform tasks too large for a single computer.

All computers follow the same protocol, acting like a virtual supercomputer.

Mainly used for analyzing large data or simulations.

Computers share processing power and storage.

A form of distributed computing, often used across locations.

Why is Grid Computing Important?

Scalability: Easily add more machines as workload increases.

Resource Utilization: Uses idle or underused machines, reducing waste.

Solves Complex Problems: Ideal for large-scale problems (e.g., weather models).

Collaboration: Supports teamwork across locations.

Cost Savings: Reuses existing hardware, reducing expenses.

Working of Grid Computing

Control Node: Manages the network and resources.

Provider: Shares its resources with the network.

User: Uses resources from the grid.

How it works:

Control node checks available resources and assigns them based on requests.

Providers contribute resources when idle to avoid performance issues.

The network can be homogeneous (same OS) or heterogeneous (different OS).

Middleware (software/protocols) helps manage smooth operation.

Grid computing is structured and relies on control node + middleware.

Types of Grid Computing

1. Computational Grid: High-performance processors for heavy calculations.

2. Scavenging Grid: Uses many normal computers for available computing power.

3. Data Grid: Connects computers for large-scale data storage.

Use Cases

Genomic Research

Drug Discovery

Cancer Research

Weather Forecasting

Risk Analysis

CAD (Computer-Aided Design)

Animation and VFX

Collaborative Projects
Advantages

High resource utilization

Supports parallel processing

Built for scalability

Disadvantages

Software is still evolving

More complex to manage

Limited flexibility

Security concerns

Let me know if you want a printable version or diagram to go with this!