0% found this document useful (0 votes)
6 views42 pages

MongoDB and Hadoop Installation Guide

The document outlines practical exercises for installing and configuring MongoDB and Hadoop, including steps for setting up databases, collections, and executing various MongoDB queries. It also covers the installation of a single-node and multi-node Hadoop cluster, detailing necessary prerequisites, configurations, and commands for successful setup. The document serves as a comprehensive guide for students in a Big Data Analytics course at SSAIET, Navsari.

Uploaded by

tinalclg
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
6 views42 pages

MongoDB and Hadoop Installation Guide

The document outlines practical exercises for installing and configuring MongoDB and Hadoop, including steps for setting up databases, collections, and executing various MongoDB queries. It also covers the installation of a single-node and multi-node Hadoop cluster, detailing necessary prerequisites, configurations, and commands for successful setup. The document serves as a comprehensive guide for students in a Big Data Analytics course at SSAIET, Navsari.

Uploaded by

tinalclg
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Enrollment No.

221230107020 [ BIG DATA ANALYSTICS (3170722)]

PRACTICAL: - 1
AIM:- To Demonstrate Installation and Configuration of MongoDB client
Server.

Step 1 — Download the MongoDB MSI Installer Package

Download the current version of MongoDB. Make sure you select MSI as the
package.

Step 2 — Install MongoDB with the Installation Wizard

A. Navigate to your downloads folder and double


click on the .msi package you just
[Link] will launch the installation
wizard.

B. Click Next to start installation.

SSAIET, Computer Dept, Navsari Page 1


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

C. Accept the licence agreement then click Next.

[Link] the

Complete setup

E. Select “Run service as Network Service


user” and make a note of the data directory,

SSAIET, Computer Dept, Navsari Page 2


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

we’ll need this later.

[Link] won’t need Mongo Compass, so deselect it and click Next.

[Link] Install to begin installation

SSAIET, Computer Dept, Navsari Page 3


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

H..Hit Finish to complete installation.

Step 3— Create the Data Folders to Store our Databases

A. Navigate to the C Drive on your


computer using Explorer and create
a new folder called data here.

Step 4 — Setup Alias Shortcuts for Mongo and Mongodb.

Once installation is complete, we’ll need to set up MongoDB on the

local system

A. Open up your Hyper terminal running Git Bash.


B. Change directory to your home
directory with the following
command:
cd ~

C. Here, we’re going to create a file called


.bash_profile using the following
command:

SSAIET, Computer Dept, Navsari Page 4


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

touch .bash_profile

[Link] the newly created .bash_profile with vim using the following

command:

vim .bash_profile

[Link] vim, hit the I key on the keyboard to enter insert mode.

D. In your explorer go to C → Program Files → MongoDB →


Server
Now you should see the version of your MongoDB.

[Link] in the following code into vim, make sure your replace the 4.0

with your version that you see in explorer

H. Hit the Escape key on your keyboard to exit the insert mode. Then type
:wq!

to save and exit Vim

Step 5 — Verify That Setup was Successful

A. Close down the current Hyper terminal and quit the


application.

SSAIET, Computer Dept, Navsari Page 5


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

B. Re-launch Hyper.
C. Type the following commands into the Hyper terminal:
mongo –version

SSAIET, Computer Dept, Navsari Page 6


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

PRACTICAL: - 2
AIM:- Write the Mongo dB queries for creating database, collection, inserting
documents, updating document, deleting documents.

Create Database & Collection:

Insert Documents

SSAIET, Computer Dept, Navsari Page 7


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

Update & Delete Documents:

SSAIET, Computer Dept, Navsari Page 8


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

SSAIET, Computer Dept, Navsari Page 9


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

PRACTICAL: - 3
AIM:- Write the MongoDB queries for the given collection.

A. Find the document wherein the name of


student has value ‘Fletcher Mcconnell’.

[Link] name of students from student collection

SSAIET, Computer Dept, Navsari Page 10


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

C. Display name of student with id of the student having id value 22.

D. Display documents with students id with 1 to 3.

E. display documents with students name is ‘Tressa Schwing’ and


‘exam’ score greater than 85.35.

F. dispaly all documents with ‘homework’ type score is less than 95.

SSAIET, Computer Dept, Navsari Page 11


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

G. retrieve all documents with ‘quize’ score between 80 and 90


inclusively.

SSAIET, Computer Dept, Navsari Page 12


Enrollment No. 221230107020 [BIG DATA ANALYSTICS (3170722)]

PRACTICAL: -4
AIM:- Solve following queries using various MongoDB methods such as Count,
Limit, Sort and Skip.

[Link] documents in the

B Display documents in the descending order of name.

SSAIET, Computer Dept, Navsari Page 13


Enrollment No. 221230107020 [ BIG DATA ANALYSITCS (3170722)]

C. Display documents first in the ascending order of _id and then descending order of
name.

D. Display all documents except first two from students collection

SSAIET, Computer Dept, Navsari Page 14


Enrollment No. 221230107020 [BIG DATA ANALYSTICS (3170722)]

E. Display 5th and 6th documents from the students collection

F. Display total number of documents in the students collection

G. Display last three documents from the students collection

SSAIET, Computer Dept, Navsari Page 15


Enrollment No. 221230107020 [ BIG DATA ANALYSITCS (3170722)]

PRACTICAL: - 5
AIM:- Write MongoDB queries similar to LIKE predicate in SQL.

Find the ids of students whose name begins with the letter “A”.

Display all documents in which student name ends with the letter ’r’.

Find all documents in student name contains ‘t’ in any position.

SSAIET, Computer Dept, Navsari Page 16


Enrollment No. 221230107020 [BIG DATA ANALYSTICS (3170722)]

PRACTICAL: - 6
AIM:- To Demonstrate the Installation and Configuration of Single node Hadoop.

Download Hadoop binaries

The first step is to download Hadoop binaries from the official website. The
binarypackage size is about 342 MB.

After finishing the file download, we should unpack the package using two steps. First, we
should extract the [Link] library, and then, we should unpack the extracted tar file.
The tar file extraction may take some minutes to finish. In the end, you may see
some warnings about symbolic link creation. Just ignore these warnings since
they are not related to windows.

Since we are installing Hadoop 3.2.1, we should download the

files locatedin [Link] and copy them into


the “hadoop-3.2.1\bin” directory.
After unpacking the package, we should add the Hadoop native IO libraries, which can be
found in the following GitHub repository: [Link]

SSAIET, Computer Dept, Navsari Page 17


Enrollment No. 221230107020 [ BIG DATA ANALYSITCS (3170722)]

Setting up environment variable.

After installing Hadoop and its prerequisites, we should configure the


environment variables to define Hadoop and Java default paths.
To edit environment variables, go to Control Panel > System and Security > System (or right-
click > properties on My Computer icon) and click on the “Advanced systemsettings” link

There are two variables to define:


• JAVA_HOME: JDK installation folder path
• HADOOP_HOME: Hadoop installation folder path

SSAIET, Computer Dept, Navsari Page 18


Enrollment No. 221230107020 [BIG DATA ANALYSTICS (3170722)]

Configuring Hadoop Cluster.


There are four files we should alter to configure Hadoop cluster:
%HADOOP_HOME%\etc\hadoop\[Link]
%HADOOP_HOME%\etc\hadoop\[Link]
%HADOOP_HOME%\etc\hadoop\[Link]
%HADOOP_HOME%\etc\hadoop\[Link]

As we know, Hadoop is built using a master-slave paradigm. Before altering


the HDFS configuration file, we should create a directory to store all master node
(name node) data and another one to store data (data node). In this example, we
created the following directories:

• C:\hadoop-env\hadoop-3.2.1\data\dfs\namenode
• C:\hadoop-env\hadoop-3.2.1\data\dfs\datanode

Now we can edit our [Link] file for further config. Open file add edit as
below:
<property>
<name>[Link]</name>
<value>1</value>
</property>
<property>
<name>[Link]</name>
<value>[Link]
</property>
<property>
<name>[Link]</name>
<value>[Link]
</property>

Core site Configuration:

<property>
<name>[Link]</name>
<value>hdfs://localhost:9820</value>

SSAIET, Computer Dept, Navsari Page 19


Enrollment No. 221230107020 [ BIG DATA ANALYSITCS (3170722)]

</property>
Map Reduce site Configuration

<property>
<name>[Link]</name>
<value>yarn</value>
<description>MapReduce framework name</description>
</property>

Yarn site Configuration:

<property>
<name>[Link]</name>
<value>mapreduce_shuffle</value>
<description>Yarn Node Manager Aux Service</description>
</property>

Formatting Name Node:

hdfs namenode -format

this commad may give you some error then we must fix those. If you had done all well
then you will get message like below
Let start Hadoop services and see it’s working or not:

Just navigate to “%HADOOP_HOME%\sbin” directory. Then we will run the


following command to start the Hadoop nodes:
.\[Link] command prompt windows will open (one for the name node and
one for the data node) as follows:

SSAIET, Computer Dept, Navsari Page 20


Enrollment No. 221230107020 [BIG DATA ANALYSTICS (3170722)]

./[Link]

To make sure that all services started successfully, we can


run the following command: Jps
14560 DataNode
4960 ResourceManager
5936 NameNode
768 NodeManager
14636 Jps
It will show above services running and it’s all for single node Hadoop setup.

SSAIET, Computer Dept, Navsari Page 21


Enrollment No. 221230107020 [ BIG DATA ANALYSITCS (3170722)]

PRACTICAL: - 7
AIM:- To demonstrate the configuration of multimode Hadoop cluster.

Data, data and Data. Across every sectors people are dealing with huge and colossal
amount of data which is also termed as Big data. Hadoop is a very well known and
widespread distributed framework for big data processing . But when it comes to
Hadoop installation, most of us feel that it is quite cumbersome job. This article will
provide you some easy and quick steps for a multi node Hadoop cluster setup.

Multi-Node Cluster in Hadoop 3.x (3.1.3)

A Multi Node Cluster in Hadoop contains two or more data nodes in a distributed Hadoop
environment. This is used in organisations to store and analyse their massive amount of data.
So knowing how to setup a multi-node Hadoop cluster is an important task

Prerequisites

We will need the following software and hardware as prerequisite to perform the
activities. Ubuntu 18.04.3 LTS (Long Term Support)

• Hadoop-3.1.3

• JAVA 8

• SSH

• At least 2 laptop/desktop connected by LAN/Wi-Fi

Installations Steps

Installation of Ubuntu/OS in the machines.

This step is very self-explanatory, as a first step we need to install Ubuntu or


any other flavor of Linux you have chosen in both the nodes
(Laptop/Desktop — will be referred as nodes from hereon). You can also
install a lighter version of Ubuntu — Lubuntu (Light weight Ubuntu)( if you
are using old hardware where you are having difficulty installing Ubuntu.

In my case I was using an old laptop of mine as the slave node and I had to
install Lubuntu and it worked without any issues.

Please create an admin user in both the nodes preferably with the same username

SSAIET, Computer Dept, Navsari Page 22


Enrollment No. 221230107020 [BIG DATA ANALYSTICS (3170722)]

Configuring host names.

Once OS is installed as a next step, we should set the


hostname for both the nodes. In my case I named the nodes
as —

● Masternode
● Slave

Reboot of the node is required after the hostname is updated.

* This step is optional if you have already put the hostnames during OS installation

Configuring IP address in the hosts file of the nodes.

Next, we need to add the IPs of masternode and slave node in the

/etc/hosts file in both the nodes.

Command:

sudo vi /etc/host
Comment out all other entries you have in the hosts file in
both the nodes. Command to see the IP of the node:
ip addr show

Restart the sshd service in both the nodes.

Command:
service sshd restart

Create the SSH key in the master node and publish it in the slave node.

For this activity follow the below steps:

● Command to generate SSH key in masternode: ssh-keygen

● It will ask for folder location where it will copy the keys, I entered
/home/username/.ssh/id_rsa

● It will ask for pass phrase, keep it empty for simplicity.

● Next copy the newly generated public key to


auth file in your users home/.ssh directory.
Command: cat
$HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

SSAIET, Computer Dept, Navsari Page 23


Enrollment No. 221230107020 [ BIG DATA ANALYSITCS (3170722)]

● Next execute — ssh localhost to check if the key is working.

● Next, we need to publish the key to the slave node.


Command: ssh- copy-id -i

$HOME/.ssh/id_rsa.pub <username>@slave

● First time it will prompt you to enter the password and publish the key.

● Execute ssh <username>@slave again to


check if you are able to loging without
password. This is very important. Without
public key working,the slave node cannot
be added to the cluster later.

Download and install Java.

Download and install Open JDK 8 and set the JAVA_HOME


path in your .bashrc file of the user under which you are
installing hadoop.

Download the Hadoop 3.1.3 package in all nodes.

Login to each node and download and untar the Hadoop package

wget
[Link]
op- [Link] tar -xzf hadoop- [Link]

Add the Hadoop And Java paths in bash file (.bashrc) on all nodes.

Command: sudo
vi .bashrc
Environment
Variables to Set in .bashrc

SSAIET, Computer Dept, Navsari Page 24


Enrollment No. 221230107020 [BIG DATA ANALYSTICS (3170722)]

Set NameNode Location

Update your ~/hadoop/etc/hadoop/[Link] file to set the NameNode location to node-


master on port9000

Set Path for HDFS.

Edit ~/hadoop/etc/hadoop/[Link] to add the following for the masternode

For the data node please put the following

SSAIET, Computer Dept, Navsari Page 25


Enrollment No. 221230107020 [ BIG DATA ANALYSITCS (3170722)]

Please note the difference between the configuration properties of masternode


and slave.

Set YARN as Job Schedular.

Configure YARN.

Edit ~/hadoop/etc/hadoop/[Link], which contains the configuration options for


YARN. In the value field for the [Link], replace
[Link] with the IP address of node-master that you have:

Configure Workers

The file worker is used by startup scripts to start required daemons on all nodes. Edit

~/hadoop/etc/hadoop/workers of the masternode to include hostnames of both of the

SSAIET, Computer Dept, Navsari Page 26


Enrollment No. 221230107020 [BIG DATA ANALYSTICS (3170722)]

Update the JAVA_HOME in [Link].

Edit ~/hadoop/etc/hadoop/[Link] and update the value for the


JAVA_HOME of your installation for both the nodes.

Format HDFS NameNode

Start and Stop HDFS.

Ok. So now you are almost there. Only thing left is starting
the daemons. To start all the daemons and bring up your
hadoop cluster use the below command:
Command: [Link]
Once the command prompt is back, to check the daemons
running use the following command: Command: jps
This is what you see in the masternode:

This is what you will see in the slave node:

SSAIET, Computer Dept, Navsari Page 27


Enrollment No. 221230107020 [ BIG DATA ANALYSITCS (3170722)]

If you are not seeing the above daemons running,then something has
gone wrong in your [Link],you need to check the previous
steps again.

URL after modifying the IP with that of your masternode:


[Link] overview

SSAIET, Computer Dept, Navsari Page 28


Enrollment No. 221230107020 [BIG DATA ANALYSTICS (3170722)]

Put and Get Data to HDFS.

To start with you have to create the user directory in your HDFS cluster. This user directory
should be in the same username under which you have installed and running the cluster. Use
the following command

Command: hdfs dfs -mkdir /user/username

Once user directory is created you can use any of your hdfs dfs commands and start using your
HDFS cluster

SSAIET, Computer Dept, Navsari Page 29


Enrollment No. 221230107020 [ BIG DATA ANALYSITCS (3170722)]

PRACTICAL: - 8
AIM:- To Develop a Map Reduce program for Word count for Hadoop cluster.

Steps to execute MapReduce word count for Hadoop Cluster:

• Create a text file in your local machine and write some text into it.
$ nano [Link]

• Check the text written in the [Link] file. $ cat [Link].

In this example, we find out the frequency of each word exists in this text file

• Create a directory in HDFS, where to kept text file.


$ hdfs dfs -mkdir /test
• Upload the [Link] file on HDFS in the specific directory.
$ hdfs dfs -put /home/codegyani/[Link] /test

• Write the MapReduce program using any java IDE or Editor.


WC_Mapper.java

package [Link];
import [Link];

import [Link];

SSAIET,
importComputer Dept, Navsari
[Link]; Page 30
Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

WC_Reducer.ja

SSAIET, Computer Dept, Navsari Page 31


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

WC_Runner.java

Download the source code:

• Create the jar file of this program and name it [Link].


• Run the jar file hadoop
jar home/codegyani/[Link]
[Link].WC_Runner /test/[Link]/r_output
• The output is stored in /r_output/part-00000
• Now execute the command to see the
output. hdfs dfs -cat /r_output/part-

00000

SSAIET, Computer Dept, Navsari Page 32


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

OUTPUT:

SSAIET, Computer Dept, Navsari Page 33


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

PRACTICAL: - 9
AIM:- To show the Installation steps for the SPARK on single node system.

Apache Spark is an open-source, distributed processing system used for big


data workloads. It utilizes in-memory caching and optimized query execution
for fast queries against data of any size. Simply put, Spark is a fast and
general engine for large-scale data processing.

Steps to install apache Spark as single node:


Download Apache Spark.

Open a browser and navigate to [Link]


Under the Download Apache Spark heading, there are two
drop-down menus. Use the current non-preview version.

Click the [Link] link

Now Verify Spark Software File.


Verify the integrity of your download by checking the checksum
of the file. This ensures you are working with unaltered,
uncorrupted software.
certutil -hashfile c:\users\username\Downloads\[Link]
SHA512

Compare the code to the one you opened in a new browser tab. If
they match, your download file is uncorrupted.

SSAIET, Computer Dept, Navsari Page 34


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

Install Apache Spark

Installing Apache Spark involves extracting the downloaded file to the


desired location.

Create a new folder named Spark in the root of your C: drive. From a command line,enter the
following

In Explorer, locate the Spark file you downloaded.

Right-click the file and extract it to C:\Spark using the tool you
have on your system.

Now, your C:\Spark folder has a new folder spark-2.4.5- bin-


hadoop2.7 with the necessary files inside.

Now we need to configure path.

Go to Control Panel -> System and Security -> System -> Advanced
Settings -> Environment Variables.

Add %SPARK_HOME%\bin to the path variable

Spark needs a piece of Hadoop to run. For Hadoop 2.7, you need to install
[Link].
So, if you are using Hadoop old version then please download
this software, install it and set environment path for it.

SSAIET, Computer Dept, Navsari Page 35


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

Testing Time:

Open command line and type spark-shell, you get the result as below.

We have completed spark installation on Windows system.

SSAIET, Computer Dept, Navsari Page 36


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

PRACTICAL: - 10
AIM:- Develop a word count program using SPARK.

Steps to develop Word Count using SPARK:


• Create a text file in your local machine and write some text into it
$ nano [Link]

• Check the text written in the [Link] file. $ cat [Link]

• Create a directory in HDFS, where to kept text file.


$ hdfs dfs -mkdir /spark
• Upload the [Link] file on HDFS in the specific directory.
$ hdfs dfs -put/home/codegyani/[Link]/spark

• Now, follow the below command to open the spark in Scala mode.

• Let's create an RDD by using the following command.

SSAIET, Computer Dept, Navsari Page 37


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

• scala > val data = [Link]("[Link]") Here, pass any file name that contains
the data.
• Now, we can read the generated result by using the following command. scala >
[Link];

• Here, we split the existing data in the form of individual words by using the following comman

scala > val splitdata = [Link](line => [Link](" "));


• Now, we can read the generated result by using the following command. scala>

[Link]

• Now, perform the map [Link] > val mapdata = [Link](word => (word,1));
Here, we are assigning a value 1 to each word.
Now, we can read the generated result by using the following command. scala >

[Link]

• Now, perform the reduce operation


scala > val reducedata = [Link](_+_);

SSAIET, Computer Dept, Navsari Page 38


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

Here, we are summarizing the generated data.

• Now, we can read the generated result by using the


following command. scala > [Link];

OUTPUT:-

SSAIET, Computer Dept, Navsari Page 39


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

CASE STUDY

Case Study 1: Predictive Analytics in Patient Care – Mount Sinai Hospital

Background:
The healthcare sector generates massive volumes of data daily through electronic health
records (EHRs), diagnostic imaging, wearable devices, lab reports, and patient feedback.
Traditionally, this information remained fragmented, making it difficult for doctors to use it
effectively for early diagnosis and treatment planning.

Implementation:
Hospitals can integrate big data analytics into its clinical workflows to improve patient care.
By applying machine learning algorithms to large sets of patient records, the hospital
developed a predictive model that could identify patients at high risk of developing chronic
conditions, such as heart failure. The system processed data including lab test results,
medication history, lifestyle indicators, and hospital visits to generate a risk score for each
patient.

Results:

• Enabled doctors to intervene earlier and provide preventive treatment.


• Improved patient outcomes by reducing avoidable hospital readmissions.
• Enhanced efficiency in resource allocation, allowing doctors and nurses to prioritize
high-risk patients.
• Contributed to cost savings by minimizing unnecessary tests and emergency
admissions.

Conclusion:
This case highlights how big data analytics transforms healthcare by turning raw medical
information into actionable insights. Predictive analytics not only improves clinical decision-
making but also supports personalized treatment plans, paving the way for a more efficient
and patient-centered healthcare system.

Case Study 2: Walmart – Big Data for Personalized Customer Experience &
Inventory Management

Background:
Retail businesses face challenges in understanding customer preferences, predicting demand,
and managing large-scale inventory across multiple locations. With millions of daily
transactions, analyzing consumer behavior manually is impossible. Walmart, one of the
world’s largest retailers, adopted big data analytics to enhance both customer experience and
supply chain efficiency.

SSAIET, Computer Dept, Navsari Page 40


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

Implementation:
Walmart collects and processes petabytes of data from online purchases, in-store transactions,
social media interactions, and mobile app usage. By leveraging big data tools such as Hadoop
and machine learning algorithms, Walmart analyzes customer buying patterns in real time.
This allows the company to provide personalized product recommendations, optimize store
layouts, and forecast demand for seasonal or trending products. Additionally, Walmart applies
predictive analytics to manage inventory by anticipating which products will sell in specific
regions and time frames.

Results:

• Enhanced customer satisfaction through personalized recommendations and targeted


promotions.
• Reduced stockouts and overstocking by improving demand forecasting.
• Achieved significant cost savings in supply chain management through optimized
logistics and inventory planning.
• Improved decision-making by integrating real-time insights into business operations.

Conclusion:
Walmart’s case demonstrates how big data analytics enables retailers to move beyond
traditional sales analysis toward predictive, customer-centric strategies. By leveraging data-
driven insights, Walmart successfully creates a balance between consumer satisfaction and
operational efficiency, making it a global leader in retail innovation.

Case Study 3: Smart City Initiative in Surat, Gujarat – Big Data for Urban
Planning & Traffic Management

Background:
As one of the fastest-growing cities in Gujarat, Surat faces challenges such as rapid
urbanization, increasing traffic congestion, waste management issues, and the need for better
public services. To address these concerns, the Surat Municipal Corporation (SMC) adopted
smart city solutions, with big data analytics playing a central role in improving urban planning
and governance.

Implementation:
Surat deployed an Integrated Command and Control Centre (ICCC) that collects and
analyzes data from IoT sensors, CCTV cameras, GPS devices, and public service systems
across the city. Real-time traffic data helps in managing congestion by controlling traffic
signals dynamically, while waste management data ensures efficient garbage collection using
GPS-enabled trucks. Big data analytics also supports air quality monitoring, emergency
response, and predictive maintenance of city infrastructure. Citizen feedback and complaints
from mobile apps are also analyzed to improve service delivery.

Results:

• Improved traffic flow and reduced congestion through smart traffic signal management.
• Enhanced efficiency in waste management with real-time tracking of collection
vehicles.

SSAIET, Computer Dept, Navsari Page 41


Enrollment No. 221230107020 [ BIG DATA ANALYSTICS (3170722)]

• Increased safety through surveillance data analysis, aiding law enforcement and
disaster management.
• Better decision-making in urban planning with insights from data-driven reports.

Conclusion:
Surat’s Smart City initiative highlights how big data analytics transforms traditional city
management into a more efficient, transparent, and citizen-friendly system. By leveraging real-
time data, Surat has set a benchmark for other Indian cities in terms of sustainable growth,
resource optimization, and improved quality of life for its citizens.

SSAIET, Computer Dept, Navsari Page 42

You might also like