Scribd
Upload
35 views11 pages

Step-by-Step Spark Installation Guide

The document provides steps for installing Spark on Windows: 1. Download and install Hadoop and Java from provided links 2. Download Spark from the Apache website 3. Unzip Spark and set the SPARK_HOME environment variable to point to the Spark folder 4. Add SPARK_HOME/bin to the system Path variable and grant permissions to the temp folder 5. Test the Spark installation by running the spark-shell command

Uploaded by

Charles W Gitahi
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
35 views11 pages

Step-by-Step Spark Installation Guide

The document provides steps for installing Spark on Windows: 1. Download and install Hadoop and Java from provided links 2. Download Spark from the Apache website 3. Unzip Spark and set the SPARK_HOME environment variable to point to the Spark folder 4. Add SPARK_HOME/bin to the system Path variable and grant permissions to the temp folder 5. Test the Spark installation by running the spark-shell command

Uploaded by

Charles W Gitahi
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd

FACULTY OF COMPUTING AND INFORMATION

MANAGEMENT

Installing Spark
Steps for Installing spark
 [Link] and Install Hadoop
 Hadoop can be downloaded from the link below:
 [Link]
 Because this version does not include windows 10 binary, refer the
to tutorial for installing hadoop in Windows environment:
Steps for Installing spark
2. Download and Install Java virtual machine.
The latest version can be downloaded from
[Link]

[Link]

3. Download Spark from [Link]

4. Unzip and extract your download into a local folder.


Winzip or winrar can be used to unpack spark zip file.

Spark folder within drive and be used to storing the unzipped file.
Steps for Installing spark
 5. Create SPARK HOME variable

 From windows "Search advanced system settings" program-> click

the button of "Environment Variables"


Add new user variable called SPARK_HOME for storing a path that
points the location Spark binary files
SPARK_HOME --> C:\Spark\spark-2.3.2-bin-hadoop2.7

This is an example.
Steps for Installing spark
 6. edit the Path and add SPARK_HOME variable as follows
 %SPARK_HOME%\bin

 Example
Steps for Installing spark
 5. Grant permission to temp folder
 Create a temp folder under c:\tmp
 Execute below command as administrator.
 [Link] chmod 777 C:\tmp
Steps for Installing spark
 6. Test spark by typing "spark-shell“ command on the prompt
 Example
Closing Spark Shell
 To close Spark shell, press Ctrl+D or type in :q 
 Example
Connecting spark with python
 Pyspark command is used to connect spark with python
 Example
References for installing Spark

 [Link]
installation-on-windows-10/

 [Link]
2.1.0-in-Windows-10-environment

You might also like