Scribd
Upload
8 views1 page

Spark SQL Dataset Operations Guide

This document shows how to use Spark SQL to create a Dataset, perform operations like sorting and displaying results, and compute descriptive statistics. It creates a Dataset using spark.range, shows the items, sorts in descending order and shows the first 5, and calculates descriptive stats.

Uploaded by

Sai Gopi
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
8 views1 page

Spark SQL Dataset Operations Guide

This document shows how to use Spark SQL to create a Dataset, perform operations like sorting and displaying results, and compute descriptive statistics. It creates a Dataset using spark.range, shows the items, sorts in descending order and shows the first 5, and calculates descriptive stats.

Uploaded by

Sai Gopi
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd

import [Link].

SparkSession

//val spark = SparkSession


.builder()
.appName("Spark SQL basic example")
.config("[Link]", "some-value")
.getOrCreate()

//create a Dataset using [Link] starting from 5 to 50, with increments of 5


val numDS = [Link](5, 50, 5)

//import [Link]._
[Link]()

// reverse the order and display first 5 items


[Link](desc("id")).show(5)

//compute descriptive stats and display them


[Link]().show()

You might also like