Discuss Hive architecture and explain its
working nature with a neat diagram.
Introduction
Apache Hive is a data warehousing framework built on top of Hadoop that provides a SQL-
like query language (HiveQL) to process and analyze large datasets stored in HDFS or
HBase. Hive simplifies Big Data analytics by allowing users to write queries similar to SQL
without dealing with low-level MapReduce programming.
Hive Architecture
Hive follows a layered architecture that separates user interaction, query processing,
execution, and storage.
Main Components of Hive Architecture
• User Interfaces
• HiveQL Process Engine
• Execution Engine
• Metastore
• Storage Layer (HDFS / HBase)
(Draw the given diagram showing User Interfaces, Metastore, HiveQL Process Engine,
Execution Engine, MapReduce, and HDFS/HBase.)
Components of Hive Architecture
1. User Interfaces
Hive provides multiple interfaces for user interaction:
• Command Line Interface (CLI)
• Web UI
• HDInsight
These interfaces allow users to submit HiveQL queries.
2. Metastore
The Metastore stores metadata information, such as:
• Table names
• Column names and data types
• Table locations
• Partition details
It is usually backed by an RDBMS.
3. HiveQL Process Engine
• Parses HiveQL queries
• Performs semantic analysis
• Creates logical execution plans
• Interacts with the Metastore for metadata
4. Execution Engine
• Converts logical plans into physical plans
• Executes queries using MapReduce
• Manages job execution and monitoring
5. Storage Layer
• Stores actual data in HDFS or HBase
• Hive does not store data itself; it only queries the data
Working of Hive (Query Execution Flow)
1. User submits a HiveQL query using CLI or Web UI
2. HiveQL engine parses and validates the query
3. Metadata is retrieved from the Metastore
4. Logical plan is converted into MapReduce jobs
5. Execution engine runs jobs on Hadoop
6. Results are returned to the user
Hive Data Types
1. Primitive Data Types
• INT, BIGINT
• FLOAT, DOUBLE
• STRING, BOOLEAN
• TIMESTAMP
2. Complex Data Types
• ARRAY
• MAP
• STRUCT
• UNIONTYPE
Hive File Formats
Hive supports multiple file formats for storage:
• TEXTFILE – Default format
• SEQUENCEFILE – Binary key-value pairs
• ORC – Optimized Row Columnar format
• PARQUET – Columnar storage format
• AVRO – Schema-based serialization
Querying Data in Hive using HiveQL
1. Data Definition Language (DDL)
CREATE TABLE customers (
cust_id INT,
name STRING,
city STRING
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ',';
Used to create and define table structure.
2. Data Manipulation Language (DML)
LOAD DATA INPATH '/data/[Link]'
INTO TABLE customers;
Used to load or manipulate data in tables.
3. Data Query Language (SELECT)
SELECT city, COUNT(*)
FROM customers
GROUP BY city;
Used to retrieve and analyze data.
Salient Features of Data Manipulation in Hive
• SQL-like syntax
• Supports large-scale data processing
• Automatically converts queries to MapReduce
• Suitable for batch analytics
• No need to write Java MapReduce code
Retail Use Case – HiveQL Workflow
DDL
CREATE TABLE sales (
cust_id INT,
product STRING,
amount DOUBLE,
city STRING
);
DML
LOAD DATA INPATH '/sales/[Link]' INTO TABLE sales;
Query
SELECT city, SUM(amount)
FROM sales
GROUP BY city;
✔ Helps analyze customer buying behavior.
Advantages of Hive
• Easy to learn (SQL-like)
• Scalable
• Integrates with Hadoop
• Supports structured and semi-structured data
Conclusion
In conclusion, Hive provides a powerful abstraction layer over Hadoop, enabling efficient
querying and analysis of large datasets using HiveQL. Its layered architecture, metadata
management, and support for various data types and file formats make it ideal for data
warehousing and batch analytics. Hive is widely used in real-world applications such as retail
analytics, log processing, and customer behavior analysis.
ARCHITECTURE OF Hive :