Scribd
Upload
6 views4 pages

Top Movie Genres for Ages 25-30

The document outlines a Spark application that processes IP addresses and analyzes movie rental data to find the top three genres rented by customers aged 25-30. It includes details on the structure of the Customers, Movies, and Rentals tables, along with example SQL queries for filtering and grouping data. Additionally, it provides examples of input and output formats for data processing tasks.

Uploaded by

Sunith Chethan
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
6 views4 pages

Top Movie Genres for Ages 25-30

The document outlines a Spark application that processes IP addresses and analyzes movie rental data to find the top three genres rented by customers aged 25-30. It includes details on the structure of the Customers, Movies, and Rentals tables, along with example SQL queries for filtering and grouping data. Additionally, it provides examples of input and output formats for data processing tasks.

Uploaded by

Sunith Chethan
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd

IP

IP ------> [Link]

val dstram = sparkstramingcontext("/ /")

dstream----> stream of ip [Link](x=> Geolite(x))

[Link] it to output file

Geolite(rdd:

ip -> latitude
longitude
city
state
country

Write an spark to find the top 3 most popular movie genres among customers aged 25-
30 (inclusive).
Display the genre along with the count of rentals.

Consider the following tables representing a movie rental database:

Customers:

customer_id name age gender


1 Alice 25 F
2 Bob 30 M
3 Charlie 28 M
4 David 5 M

Movies:

movie_id title genre release_year


1 The Matrix Sci-Fi 1999
2 Inception Sci-Fi 2010
3 The Godfather Crime 1972
4 Pulp Fiction Crime 1994

Rentals:

rental_id customer_id movie_id rental_date


1 1 1 2023-01-10
2 1 3 2023-02-15
3 2 2 2023-03-20
4 3 4 2023-04-25
5 4 1 2023-05-30
6 2 4 2023-06-05
7 1 2 2023-07-10
8 3 3 2023-08-15
9 4 2 2023-09-20
10 1 4 2023-10-25

Expected Output:

genre num_rentals
Sci-Fi 4
Crime 3

[Link]($"movie_d").count

[Link]($"age" >=25 & $"age"<=30)

val intermedaitedf = [Link](Rentals, Customers.customer_id ====


Rentals.customer_id)

intermedatedf join Movies

movie genre rental_id

.groupBy($"genre").count().as($"num_rentals")

input :-
ID, Name, catgry, fileName
101,emp1, A, [Link]
102,emp2, B, [Link]
103,emp3, C, [Link]
104,emp4, A, [Link]
105,emp5, C, [Link]

output:
[Link]
101,emp1, A
103,emp3, C
105,emp5, C
A:1,B:0,c:2

[Link]
102,emp2, B
104,emp4, A
A:1, B:1,c:0

[Link]($"filename" === "[Link]").drop($"filename").saveAsTextFile("


/abc,txt ")

.groupBy($"catgry").count

A 1
B 0
C 2

udaf (x ,y)
cat x : y

TableA table B

1 1 15 1 s 4 2 s 1 3
1

1 1

1 1

2 1

2 1

3 2

null 2

null 3

null

null

null

group by salary order by salary desc limit 2

empid name managerid


1 samarth null
2 aman 1
3 abhishek 2
4 utkarsh 3
5 shreya 4

emptable self join

name manager_name
samarth null
aman samarth
abhishek aman
utkarsh abhishek
shreya utkarsh

for (1<5)
(tablename)

select [Link], e2.manager_name from emp e1 JOIN emp e2 ON e1.emp_id =


e2.manager_id)

Hi My name is Sunith
Sunith is an engineer

[Link](x=>[Link]("
")).map((word)=>(word,1).reduceByKey(_+_).orderByValue()

You might also like