Scribd
Upload
2 views2 pages

Spark vs Hive: Key Concepts & Techniques

Example – There are two files A.txt and B.txt which are stored in a cluster having 5 nodes. When these files are put in HDFS, as per the applicable block size, let's say both of these files are divided into two blocks.

Uploaded by

rajsreerama.s
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
2 views2 pages

Spark vs Hive: Key Concepts & Techniques

Example – There are two files A.txt and B.txt which are stored in a cluster having 5 nodes. When these files are put in HDFS, as per the applicable block size, let's say both of these files are divided into two blocks.

Uploaded by

rajsreerama.s
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd

SPARK

rdd vs dataframe vs dataset


features of rdd
spark vs MAP Reduce
Diff between Transformations & Actions ??
difference between parquet, avro & orc file formats
which file format is native to spark ??
what version u r using ??
Spark context Vs Spark Session ??
Can we have multiple spark sessions ??

Spark optimisation techniques


[Link] selection
[Link] vs Persist
[Link] vs repartition
[Link] vs GroupBy
[Link] variable vs Accumulator variable
[Link] serializer vs kryo serilaiser??
[Link] Collection Tuning
[Link] of File Format

what is Data Skewness ??


what is Broadcast Join ?? when to use ??
Rank, Denserank & row no

Spark Architecture
what is lineage graph ??
What is lazy evaluation??
what happens in the background when Spark job is submitted ??
Spark Submit command -- components ??
Deployment modes ??
how u will determine the no of executors for a spark job ??
What is container, application manager & application master ??

Spark read -- from a database eg : Mysql


Out of Memory Issues
Oozie scheduler uses ??

what is scrum methodology??


CICD pipeline brief

HIVE

native file format


orc fileformat
Managed vs external table
syntax for creating an external table
how to convert external table to a managed table
how to add a new column to a hive table
how to rename column name of a partitioned column
Hive optimisation techniques
Partitioning vs Bucketing
Types of row format
Hive SERDE
what is Hive metastore ?? Types of it ??
MAPPING VS REDUCING
DML VS DDL
sql vs hql
Hive architecture ??
what is Mapside Join ??

sortby vs orderby

3 5
8 2
9 4
5 10
1 6

1 2 3 4 5 6 7 8 9 10 orderby whole data irrespective of data

3,5 - 2,8 - 4,9 - 5,10 - 1,6 sortby in partition level data

You might also like