Question Bank: Big Data and Analytics[22CD751]
MODULE 1 and few questions from MODULE 2
SI. NO. Question Bloom's Level Mark
s
1 Define digital data and explain its types with examples. Level 1: 5
Remember
2 List different types of data sources used in analytics. Level 1: 3
Remember
3 Explain the term Big Data with a suitable definition. Level 1: 3
Remember
4 Differentiate between Big Data and Analytics. Level 2: 5
Understand
5 Describe the characteristics (5Vs) of Big Data with Level 2: 6
examples. Understand
6 Give domain specific examples where Big Data is Level 2: 4
applied. Understand
7 Compare Traditional Business Intelligence and Big Data. Level 3: 6
Apply
8 Illustrate a typical data warehouse architecture. Level 3: 5
Apply
9 Illustrate a Hadoop environment with neat diagram. Level 3: 6
Apply
10 What is Big Data Analytics? Give its significance. Level 1: 4
Remember
11 Classify Analytics into types and explain each. Level 2: 6
Understand
12 Explain the importance of Big Data Analytics in business Level 2: 5
decision making. Understand
13 Analyze technologies used in Big Data environments. Level 4: 6
Analyze
14 Compare few top analytical tools used in industry. Level 4: 6
Analyze
15 Explain NoSQL and its advantages over RDBMS. Level 2: 5
Understand
16 Summarize Hadoop components and its ecosystem. Level 2: 6
Understand
17 Design a use case where Big Data Analytics can solve a Level 5: 8
real-world problem. Evaluate
18 Evaluate the challenges faced in Big Data Level 5: 7
implementation. Evaluate
19 Propose a Big Data solution for healthcare or finance Level 6: 8
sector. Create
20 Formulate a data pipeline combining warehouse and Level 6: 8
Hadoop for analytics. Create
21 Differentiate between structured, semi-structured, and Level 2: 5
unstructured data with examples. Understand
22 Explain different types of data sources such as IoT, social Level 2: 5
media, and enterprise systems. Understand
23 Describe the 5Vs of Big Data with real-world examples. Level 2: 6
Understand
24 Illustrate the Big Data Life Cycle with an example use Level 3: 6
case. Apply
25 Explain the stages of Big Data Analytics: Collection, Level 2: 6
Cleaning, Analysis with tools. Understand
26 Compare Data Warehouse vs Hadoop environment with Level 4: 6
proper justification. Analyze
27 Discuss the challenges of Big Data Analytics. Level 4: 6
Analyze
28 Explain types of NoSQL databases with suitable Level 2: 5
examples. Understand
29 Analyze how Spark is used in real-time analytics with Level 4: 7
industry examples. Analyze
30 Design a Big Data architecture for a smart city project. Level 6: 8
Create
31 How would you differentiate OLAP and OLTP systems in terms Level 3: 8
of their data processing roles and use cases? provide examples of Apply
applications that typically use each type?
MODULE 2
Bloom’s
[Link] Question Marks
Level
Explain the reasons behind the emergence of Hadoop as a
1 Understand 8
solution for processing big data.
Compare RDBMS and Hadoop with respect to scalability, data
2 Analyze 8
handling, and fault tolerance.
Explain the architecture of HDFS with NameNode and DataNode
3 Understand 8
roles.
Analyze the Hadoop Distributed File System (HDFS) and discuss
4 its key features, explaining how each feature contributes to Analyze 8
efficient big data storage and processing.
Describe the history and evolution of Hadoop. Highlight its major
5 Remember 8
milestones.
Explain the role of Hadoop YARN in managing resources and
6 Understand 8
applications.
Describe the core components of Hadoop ecosystem and explain
7 Understand 8
their functions.
Explain the working of MapReduce programming with an
8 Understand 8
example.
Differentiate between Mapper, Reducer, Combiner, and
9 Analyze 8
Partitioner in Hadoop MapReduce.
Bloom’s
[Link] Question Marks
Level
Describe the process of sorting and searching in MapReduce
10 Understand 8
with a suitable example.
Explain how compression improves the efficiency of Hadoop
11 Understand 8
processing.
Analyze a real-world use case where Hadoop provides a better
12 Analyze 8
solution than traditional RDBMS.
Explain the process of data replication in HDFS and discuss its
13 Understand 8
importance in fault tolerance.
Discuss how Hadoop handles large-scale data analytics across
14 Analyze 8
distributed systems.
Explain the role of NameNode, Secondary NameNode, and
15 Understand 8
DataNode in HDFS.
Describe the process of resource scheduling and job execution in
16 Understand 8
Hadoop YARN.
Compare MapReduce and traditional batch processing
17 Analyze 8
techniques, highlighting the advantages of MapReduce.
Explain the concept of a Partitioner in MapReduce and give an
18 Apply 8
example scenario where custom partitioning is useful.
Discuss the importance of fault tolerance and data reliability in
19 Analyze 8
HDFS with examples.
Explain the end-to-end process of running a MapReduce job on
20 Understand 8
Hadoop, from input to output.
MODULE 3
SI.
Question Bloom’s Level Marks
No
Define the CAP Theorem and explain its three components Level 1:
1 10
with an example. Remember
SI.
Question Bloom’s Level Marks
No
Classify the types of NoSQL databases and mention two Level 2:
2 10
use-cases for each type. Understand
Compare the terms used in RDBMS and MongoDB (e.g., Level 2:
3 8
table vs collection, row vs document). Understand
Illustrate CRUD operations in MongoDB with syntax and
4 Level 3: Apply 6
examples.
Demonstrate how to find documents using search criteria
5 Level 3: Apply 5
(e.g., AND/OR, range queries) in MongoDB.
Explain how null values are handled in MongoDB with Level 2:
6 4
examples. Understand
Write and explain queries using count(), limit(), sort(), and
7 Level 3: Apply 6
skip() in MongoDB.
Describe the use of aggregate functions in MongoDB with
8 Level 4: Analyze 7
appropriate examples.
Analyze how the CAP theorem influences MongoDB’s
9 Level 4: Analyze 6
design choices.
Compare MongoDB with other NoSQL databases in terms of
10 Level 4: Analyze 6
data model and consistency.
Design a MongoDB collection structure for an e-commerce Level 5:
11 8
application including CRUD queries. Evaluate
Formulate an aggregation pipeline to group and summarize
12 Level 6: Create 8
data in MongoDB.
SI.
Question Bloom’s Level Marks
No
Write MongoDB commands to insert a single document and
13 Apply 5
multiple documents into a collection named Students.
SI.
Question Bloom’s Level Marks
No
Demonstrate how to update one document and update many
14 Apply 6
documents with appropriate MongoDB syntax.
How do you delete one document and delete many Understand /
15 5
documents in MongoDB? Give examples. Apply
Write a query to retrieve only the first 5 documents from a
16 Apply 4
collection using limit(). Explain the syntax.
Show how to sort documents in ascending and descending
17 Apply 5
order using sort(). Use a price or age field as an example.
Explain the purpose of the skip() function. Write a query to skip Understand /
18 5
the first 10 documents and return the next 5. Apply
Combine sort() and limit() in a single query to display the top 3
19 Analyze 6
highest paid employees from the Employee collection.
Write a CRUD-based scenario: Insert student records, update
20 one record, delete one record, and display the remaining data Create 8
using sort() and limit().
Write MongoDB queries to: i) Insert three new documents into a
Student collection. ii) Retrieve students with marks > 75. iii) Update
21 Apply 10
the department of a given student. iv) Delete students with marks <
35.
Using a sample collection, demonstrate the commands for Create,
Understand /
22 Read, Update, and Delete in MongoDB. Explain how each command 10
Apply
affects the data.
With a Student collection, perform: i) insertOne() ii) insertMany() iii)
23 updateOne() iv) updateMany(). Provide appropriate examples with Apply 10
syntax.
For an Employee collection, write a query to: i) Retrieve documents
where the Skills array contains both “Communication” and
24 Analyze 10
“Recruitment”. ii) Sort the results by Dept in ascending order iii) Limit
the result to 5 documents. Explain each part.
Explain the CAP theorem in the context of NoSQL databases like
25 MongoDB. How do consistency, availability, and partition tolerance Understand 10
influence design choices?
Compare and contrast key-value, document, column-family, and Understand /
26 10
graph-based NoSQL databases with examples. Analyze
MODULE 4
SI Bloom’s CO
Question Marks
No Level Mapping
Describe the key features and purpose of Hive in Big Data
1 L1 8 CO1
processing.
2 Explain the architecture of Hive and its main components. L2 8 CO1
List and describe the various data types supported by Hive.
Hive supports several data types that can be applied to store
structured and semi-structured data efficiently:
1. Numeric Data Types
● TINYINT, SMALLINT, INT, BIGINT – used for storing
integer values of different ranges.
● FLOAT, DOUBLE, DECIMAL – used for decimal and
floating-point numbers.
Example: INT for age, DECIMAL(10,2) for salary.
3 L1 8 CO1
2. String Data Types
● STRING – general-purpose text.
● CHAR(n) – fixed-length string.
● VARCHAR(n) – variable-length string with a limit.
Example: STRING for customer name, CHAR(2) for
state codes.
3. Date and Time Types
● DATE – stores date values.
SI Bloom’s CO
Question Marks
No Level Mapping
● TIMESTAMP – stores date and time.
Example: DATE for joining date, TIMESTAMP for
order timestamp.
4. Complex Data Types
● ARRAY<type> – ordered collection of elements.
● MAP<key_type, value_type> – key-value pairs.
● STRUCT – collection of named fields.
Example: ARRAY<STRING> for multiple phone
numbers, MAP<STRING, INT> for product quantity
mapping, STRUCT<name:STRING, age:INT> for
storing nested customer info.
5. Usage Application
● Hive data types help ensure data integrity, efficient
storage, and query optimization.
● By applying the correct type to each column, large
datasets can be processed faster and more accurately
in Hive queries.
4 Describe the different Hive file formats and their uses. L2 8 CO1
Explain the role of Hive Query Language (HQL) in querying
Big Data.
Hive Query Language (HQL) is a SQL-like language designed
to query, manage, and analyze large datasets stored in
Hadoop. Applying HQL allows users to perform data
5 operations without writing complex MapReduce programs. L2 8 CO2
1. Role of HQL in Big Data Analytics
● Data Retrieval: Enables querying large datasets
stored in HDFS using familiar SQL syntax (SELECT,
WHERE, GROUP BY, ORDER BY).
SI Bloom’s CO
Question Marks
No Level Mapping
● Data Manipulation: Supports operations like
INSERT, UPDATE, DELETE, and LOAD DATA to manage
tables and partitions.
● Data Aggregation: Provides aggregation functions
(COUNT, SUM, AVG, MAX, MIN) to summarize large
datasets efficiently.
● Data Transformation: Supports joins, subqueries,
and views for transforming raw data into meaningful
insights.
2. Advantages
● Simplifies Big Data querying without needing
low-level MapReduce coding.
● Integrates with other tools like Spark for advanced
analytics.
● Supports structured, semi-structured, and partitioned
data efficiently.
3. Example
-- Count total orders per customer
SELECT customer_id, COUNT(*) AS total_orders
FROM orders
GROUP BY customer_id
ORDER BY total_orders DESC;
● This HQL query efficiently aggregates millions of
order records to provide insights on customer
activity.
4. Application
● HQL is widely applied in reporting, analytics, and
ETL pipelines in Big Data ecosystems, enabling quick
insights from huge datasets.
SI Bloom’s CO
Question Marks
No Level Mapping
Demonstrate the use of RC File implementation in Hive for
6 L3 8 CO2
data storage.
Apply a User Defined Function (UDF) in Hive on a sample
7 dataset and explain its effect. L3 8 CO2
Compare Hive and Pig in terms of purpose, execution, and
data handling.
In the Hadoop ecosystem, Apache Hive and Apache Pig are
two high-level tools built on top of MapReduce to simplify
Big Data processing.
● Hive is primarily used for data analysis and querying
large structured datasets.
● Pig is designed for data transformation and
manipulation using a simple scripting approach.
Both tools convert user commands into MapReduce jobs
internally but differ in purpose, language, and usage pattern.
8 L4 8 CO3
SI Bloom’s CO
Question Marks
No Level Mapping
Analyze the given E-commerce Sales dataset and identify
9 suitable Hive data types for each attribute.[minimum 10 L4 8 CO3
attributes]
10 Explain the concept and benefits of Pig in Big Data analytics. L1 8 CO1
11 Describe the anatomy of Pig and its main components. L2 8 CO1
Explain the philosophy behind Pig and its advantages over
12 L2 8 CO2
traditional MapReduce programming.
Demonstrate the use of relational operators in Pig with an
13 L3 8 CO2
example dataset.
Apply eval functions in Pig to transform data in a given
14 L3 8 CO2
dataset.
Analyze complex data types in Pig and provide suitable use
cases.
1. Complex Data Types in Pig
Pig supports several complex data types to handle nested or
structured data:
1. Tuple
o Definition: An ordered set of fields, which
can hold multiple values of different data
types.
15 L4 8 CO3
o Example: (name, age, city) → ('John', 25,
'Bangalore')
o Use Case: Representing a single record with
multiple attributes.
2. Bag
o Definition: An unordered collection of tuples;
allows duplicates.
o Example: {(‘John’, 25), (‘Alice’, 30)}
SI Bloom’s CO
Question Marks
No Level Mapping
o Use Case: Storing multiple records of a
dataset, such as all orders by a customer.
3. Map
o Definition: Key-value pairs; keys are strings,
values can be any data type.
o Example: [ ‘product’: 'Laptop', ‘price’: 50000 ]
o Use Case: Storing attributes of a record with
dynamic or variable fields.
2. Analysis
● Complex data types allow hierarchical and
semi-structured data to be stored and processed
efficiently.
● They enable Pig to handle nested JSON or XML-like
data, which is common in Big Data environments.
● Using tuples, bags, and maps reduces the need to
flatten data before processing.
3. Practical Use Cases
● E-commerce analytics: Use bags to store all items
purchased in a transaction.
● Web logs: Use maps to store key-value pairs of user
actions (page, timestamp, duration).
● Social media data: Tuples to represent user profiles
with multiple attributes (name, location, interests).
4. Summary
● Complex data types in Pig provide flexibility,
efficiency, and scalability for Big Data processing.
● Proper use of these types simplifies data
transformation, aggregation, and analysis on large
datasets.
SI Bloom’s CO
Question Marks
No Level Mapping
16 Explain Piggy Bank and its use in extending Pig functionality. L2 8 CO2
Compare the execution modes of Pig and discuss when each
17 L4 8 CO3
mode is preferred.
Design a solution using Hive to query a large dataset
18 L6 8 CO4
efficiently.
Develop a Pig script to process a dataset and extract
19 L6 8 CO4
meaningful insights.
Propose a combined Hive and Pig workflow to handle a Big
20 L6 8 CO4
Data analytics task.
MODULE 5
SI Bloom’s CO
Question Marks
No Level Mapping
1 Describe the key features and components of Apache Spark. L1 8 CO1
Explain the concept of a unified Spark platform and its
2 L2 8 CO1
advantages.
Identify the main users of Spark and their roles in Big Data
3 L1 8 CO1
analytics.
Describe the different storage layers in Spark and their
purposes.
Apache Spark has multiple storage layers that allow
efficient handling of Big Data across memory and disk.
4 These layers optimize computation, storage, and retrieval in L2 8 CO2
distributed environments.
1. Memory (RAM)
● Description: Spark primarily stores data in memory
(RAM) for fast access.
SI Bloom’s CO
Question Marks
No Level Mapping
● Purpose: Enables low-latency computations and
supports iterative algorithms (e.g., machine
learning, graph processing).
● Example: Caching intermediate RDDs in memory
speeds up repeated transformations.
2. Disk Storage
● Description: Data that does not fit in memory is
stored on disk.
● Purpose: Provides reliability and fault tolerance;
acts as a backup when memory is insufficient.
● Example: Large datasets in HDFS can be processed
without exhausting memory resources.
3. External Storage Systems
● Description: Spark can integrate with external
storage like HDFS, S3, Hive, or JDBC-compliant
databases.
● Purpose: Allows persistent storage and access to
distributed datasets across clusters.
● Example: Reading from an S3 bucket or writing
processed data to Hive tables.
4. Caching and Persistence Layers
● Description: Spark allows persisting RDDs or
DataFrames in memory, disk, or both with different
storage levels:
o MEMORY_ONLY, MEMORY_AND_DISK,
DISK_ONLY, etc.
● Purpose: Optimizes performance for iterative
computations and repeated data access.
● Example: Persisting a dataset used in multiple ML
algorithm iterations.
SI Bloom’s CO
Question Marks
No Level Mapping
Demonstrate how to create RDDs from existing datasets in
5 L3 8 CO2
Spark.
6 Apply common RDD transformations on a sample dataset. L3 8 CO2
7 Apply RDD actions to extract insights from a sample dataset. L3 8 CO2
8 L2 8 CO2
Analyze the differences between RDD transformations and
9 L4 10 CO3
actions.
Compare the performance of Spark RDDs with Hadoop
10 L4 10 CO3
MapReduce for large datasets.
Evaluate the benefits of persisting RDDs for iterative
algorithms.
In Apache Spark, RDDs (Resilient Distributed Datasets)
can be persisted (cached) in memory or disk, which is
highly beneficial for iterative algorithms such as machine
learning, graph processing, and iterative data
transformations.
1. Faster Computation
● Persisting RDDs avoids recomputing the same
11 dataset across multiple iterations. L5 8 CO4
● Iterative algorithms, like gradient descent, often
reuse the same dataset multiple times.
● Example: MLlib’s K-Means clustering requires
repeated access to feature data. Persisting the RDD
reduces computation time.
2. Reduced I/O Overhead
● Without persistence, Spark recomputes RDDs from
the source or parent transformations, causing
additional disk and network I/O.
SI Bloom’s CO
Question Marks
No Level Mapping
● Persisting RDDs minimizes repeated data
shuffling across the cluster.
3. Fault Tolerance
● Persisted RDDs maintain lineage information,
allowing Spark to recompute only missing partitions
in case of node failure.
● This ensures reliability without fully recomputing
all iterations.
4. Flexible Storage Levels
● Spark provides multiple persistence levels:
o MEMORY_ONLY – fastest, keeps data in RAM
o MEMORY_AND_DISK – falls back to disk if
memory is insufficient
o DISK_ONLY – stores on disk for large datasets
● Users can choose based on data size and cluster
resources.
Design a Spark program using RDDs to perform a specific
12 L6 8 CO4
data analysis task.
Explain the basics of a Data Lakehouse and its role in
13 L2 8 CO1
modern data architectures.
Analyze different data loading strategies in a Data
Lakehouse.
In a Data Lakehouse, selecting the right data loading
strategy is critical to ensure efficient storage, fast
14 L4 10 CO3
querying, and high-quality analytics. Data can be
structured, semi-structured, or unstructured, requiring
different approaches.
SI Bloom’s CO
Question Marks
No Level Mapping
1. Batch Loading
● Description: Load large volumes of data
periodically (daily, weekly).
● Use Case: Historical transactional data, periodic ETL
jobs.
● Advantages:
o Simple to implement
o Suitable for large, static datasets
● Disadvantages:
o Not real-time
o Higher latency for analytics
2. Streaming/Real-Time Loading
● Description: Data is ingested continuously as it
arrives using tools like Kafka, Spark Streaming, or
Flink.
● Use Case: Clickstream data, IoT sensor data,
real-time logs.
● Advantages:
o Low-latency analytics
o Supports near real-time dashboards and
alerts
● Disadvantages:
o More complex infrastructure
o Higher processing overhead
SI Bloom’s CO
Question Marks
No Level Mapping
3. Incremental Loading
● Description: Only new or changed data is loaded
into the Lakehouse.
● Use Case: Customer or product master tables
updated daily.
● Advantages:
o Reduces data movement and storage
requirements
o Faster than full batch reloads
● Disadvantages:
o Requires tracking of changes (CDC or
timestamps)
4. Hybrid Loading
● Description: Combines batch and streaming
approaches.
● Use Case: Large historical data is loaded via batch,
new data is ingested via streaming.
● Advantages:
o Optimizes performance and freshness
o Flexible for diverse workloads
Evaluate best practices for performance optimization in a
Data Lakehouse.
A Data Lakehouse combines the benefits of Data Lakes and
15 Data Warehouses, allowing analytics on structured, L5 8 CO4
semi-structured, and unstructured data. Optimizing
performance is crucial for fast, reliable queries and efficient
data processing.
SI Bloom’s CO
Question Marks
No Level Mapping
1. Data Partitioning
● Description: Dividing large tables into smaller
partitions based on key columns (e.g., date, region).
● Benefit: Reduces query scan size and improves read
performance.
● Example: Partitioning sales data by month for faster
aggregation queries.
2. Data Clustering/Sorting
● Description: Organizing data within partitions
based on frequently queried columns.
● Benefit: Speeds up filtering and range queries.
● Example: Clustering logs by user ID for faster
retrieval of user activity.
3. Efficient File Formats
● Description: Use columnar formats like Parquet
or ORC instead of row-based formats.
● Benefit: Reduces I/O, enables predicate pushdown,
and improves compression.
● Example: Storing transactional tables in ORC format
for analytics.
4. Caching Frequently Accessed Data
● Description: Store hot data in memory or faster
storage layers.
● Benefit: Reduces latency for repeated queries and
analytical workloads.
● Example: Caching a product catalog in Spark for
repeated analytics.
5. Data Compaction
SI Bloom’s CO
Question Marks
No Level Mapping
● Description: Combine small files into larger ones to
reduce metadata overhead.
● Benefit: Improves query planning and reduces
storage system load.
● Example: Merging multiple daily log files into
monthly files.
6. Query Optimization
● Description: Use query engines (Spark SQL, Presto)
with cost-based optimizers and predicate
pushdown.
● Benefit: Efficient execution plans reduce CPU,
memory, and I/O usage.
7. Monitoring and Tuning
● Description: Regularly monitor query performance
and tune cluster resources.
● Benefit: Ensures consistent performance under
varying workloads.
Anlyze the advantages and limitations of using RDDs
16 L4 8 CO2
compared to DataFrames and Datasets in Spark.
Analyze Spark’s storage layers for efficiency and scalability
17 L4 10 CO3
in a Big Data project.
Explain the architecture and key characteristics of Resilient
Distributed Datasets (RDDs) in Spark. Discuss how
immutability and fault tolerance are implemented.
1. Architecture of RDDs
18 RDDs are organized as a distributed collection of L2 8 CO4
partitions, where each partition is processed on different
nodes of a cluster. The architecture includes:
● Driver Program: Creates SparkContext and defines
RDD operations.
SI Bloom’s CO
Question Marks
No Level Mapping
● Cluster Manager: Allocates resources (YARN, Mesos,
Kubernetes, or Standalone).
● Executors: Execute tasks on data partitions.
● Partitions: Logical chunks of data distributed across
the cluster.
Spark applies lazy evaluation, meaning transformations are
not executed until an action is called.
2. Key Characteristics of RDDs
✅ Distributed
Data is automatically split across multiple nodes, enabling
parallel computation.
✅ Immutable
Once created, an RDD cannot be modified. Any change
results in the creation of a new RDD.
✅ Lazy Evaluation
Transformations (e.g., map, filter) build a lineage graph
rather than executing immediately.
✅ In-Memory Computation
RDDs can be cached in memory for faster iterative
processing.
✅ Typed Data
RDDs can store any kind of data, like strings, objects,
numbers, etc.
3. Immutability in RDDs
Immutability ensures better consistency and parallel
execution:
● When a transformation is applied, a new RDD is
generated instead of altering the original.
SI Bloom’s CO
Question Marks
No Level Mapping
● Immutability avoids conflicts in distributed
execution.
● It makes RDD lineage tracking easier for fault
recovery.
Example:
If rdd1 undergoes map(), Spark creates rdd2 instead of
changing rdd1.
4. Fault Tolerance in RDDs
RDDs are fault tolerant due to lineage:
● Spark tracks the sequence of transformations used
to create an RDD.
● If a partition fails, Spark recomputes only the lost
partition using the lineage graph, not the entire
dataset.
● No need for data replication like Hadoop.
For additional safety, developers may cache or persist
RDDs in memory or disk.
Example storage levels:
● MEMORY_ONLY
● MEMORY_AND_DISK
● DISK_ONLY
19 L6 8 CO4
20 L5 8 CO4