2203031247014

Abhishek Ghosh

HPC LAB MANUALD

FACULTY OF ENGINEERING AND

TECHNOLOGY
BACHELOR OF TECHNOLOGY

High Performance Computing (203105430)

6th SEMSETER

COMPUTER SCIENCE ENGINEERING & TECHNOLOGY

0
 2203031247014
Abhishek Ghosh

CERTIFICATE

This is to certify that

Mr./Ms Ghosh Abhishek S. with enrolment no. (2203031247014) has
successfully completed his/her laboratory experiments in the High-
Performance Computing (203105430) from the department of CSE – AI
during the academic year 2022-25.

Date of Submission:…...................... Staff In charge:…........................

Head Of Department:…........................................

1
 2203031247014
Abhishek Ghosh

Table of Content
Sr Practical Title Page Date of Date of Sign Marks
No. No. Performance Completion
1. Study the facilities provided by
Google Colab.
2. Demonstrate basic Linux
Commands.
3. Using Divide and Conquer
Strategies design a class for
Concurrent Quick Sort using
C++.
4. Write a program on an unloaded
cluster for several different
numbers of nodes and record the
time taken in each case.
Draw a graph of execution time
against the number of nodes.
5. Write a program to check task
distribution using Gprof.
6. Use Intel V-Tune Performance
Analyzer for Profiling.
7. Analyze the code using Nvidia-
Profilers.
8. Write a program to perform load
distribution on GPU using
CUDA.
9. Write a simple CUDA program to
print “Hello World!”
10. Write a CUDA program to add
two arrays.

2
 2203031247014
Abhishek Ghosh

PRACTICAL-1

Aim:- Study the facilities provided by Google Colab.

Theory:-
• Google Colab, short for Colaboratory, is a free, cloud-based platform provided by Google
that allows users to write and execute Python code in a collaborative environment. It offers a
variety of services and features, making it a popular choice for data scientists, researchers,
and students. Here are some key services and features of Google Colab:
• Free Access to GPU and TPU: Colab provides free access to Graphics Processing Units
(GPUs) and Tensor Processing Units (TPUs), which can significantly accelerate the training
of machine learning models.
• Cloud-Based: Colab is entirely cloud-based, meaning that you can access it from any device
with an internet connection. There's no need to install any software locally.
• Jupyter Notebooks: Colab supports Jupyter notebooks, allowing users to create and share
documents that contain live code, equations, visualizations, and narrative text.
• Collaborative Editing: Multiple users can collaborate on the same notebook in real-time.
Changes made by one user are instantly visible to others who have access to the notebook.

• Pre-installed Libraries: Colab comes with many popular Python libraries pre-installed,
including NumPy, Pandas, Matplotlib, TensorFlow, and PyTorch, among others.

• Integration with Google Drive: Colab is integrated with Google Drive, making it easy to
save and share your work. Notebooks are automatically saved to your Google Drive account.

• Sharing and Publishing: You can share your Colab notebooks just like you would with
Google Docs or Sheets. Colab notebooks can be published to the web, and anyone with the
link can view them.

• Code Snippets and Examples: Colab provides access to a wide range of code snippets and
examples. You can access these through the "Code snippets" tab, which can be helpful for
learning and exploring new techniques.
3
 2203031247014
Abhishek Ghosh

• Markdown Support: Colab supports Markdown, allowing users to add formatted text,
images, and links to their notebooks for better documentation and communication.

• BigQuery Integration: Colab can be used to query large datasets using BigQuery, which is
Google's fully-managed, serverless data warehouse.

These features make Google Colab a powerful tool for various tasks, including data analysis,
machine learning, and collaborative research. Keep in mind that Google Colab has some
limitations, such as session timeouts and restrictions on resource usage for free accounts. However,
for many users, especially those getting started with data science and machine learning, Colab
provides a convenient and accessible environment.

Google Colab is a free cloud-based platform for data analysis, research, and machine learning.
With Colab, researchers can access powerful hardware, develop and run Python code in a Jupyter
notebook environment, and share and collaborate on projects with ease. Researchers should be
aware of Colab's limits, which include a monthly utilization cap on CPU, GPU, and TPU usage as
well as a session runtime cap and memory restrictions. Colab is nonetheless a potent tool for
academics despite these drawbacks, offering a userfriendly design, compatibility for widely used
libraries, and simple access to resources.
What is a Notebook in Google Colab?
In Google Colab, a notebook is a web-based environment for creating and running code.
Notebooks are similar to scripts or code files in other programming environments but offer some
unique advantages. Notebooks allow you to write and execute code in a web browser, displaying
the output in real time. This makes it easy to iterate on your code and visualize the results as you
go. Colab notebooks also support markdown, allowing you to include formatted text, equations,
and images alongside your code. You can also add comments and notes to your code, which makes
it easier to understand and collaborate with others. Overall, notebooks are a powerful tool for data
scientists and machine learning practitioners, providing a flexible and interactive environment for
writing and testing code.
Advantages :-
Users can build and run Python programs on the free cloud-based Google Colaboratory platform,
also known as Colab. Machine learning, data analysis, and research are its main uses. Using Colab
has a number of major advantages, including:
4
 2203031247014
Abhishek Ghosh
• Powerful hardware is available to customers through Colab, which can considerably speed up
the training of machine learning models.
• Teams can easily work together on a project using Colab notebooks, which allow for easy
collaboration and sharing.
• Google Drive integration allows Colab notebooks to be loaded from and saved to Google Drive,
making it simple to access and share files.
• Free to use: There are no setup fees or usage charges when using Colab
. • Popular libraries are supported by Colab, making it simple to start working on machine learning
• projects. These libraries include TensorFlow, PyTorch, and Keras.
• Convenient interface: The data science and machine learning community frequently uses the
Jupyter notebook interface, which Colab offers. Alternatives:- Google Colaboratory (Colab) can be
used for machine learning, data analysis, and research, although there are a number of alternatives.
Here are a few well-liked choices:
• Kaggle Kernels: Users of the cloud-based Kaggle Kernels platform can create and run code in a
Jupyter notebook setting. It has a sizable user base and is largely used for data science and machine
learning competitions.
• Microsoft Azure Notebooks: Users can create and use Jupyter notebooks on the cloudbased
platform known as Microsoft Azure Notebooks. Both Azure Machine Learning services and access
to powerful hardware are incorporated.
• Jupyter Notebook: Jupyter Notebook is an open-source web tool that enables users to create and
share documents with live code, equations, visuals, and text. You have the option of running it
locally on your computer or online.
• Databricks: A cloud-based platform that enables the creation and execution of Apache Spark
workloads. It offers a collaborative setting for analytics, machine learning, and data engineering.
• IBM Watson Studio: A cloud-based tool called IBM Watson Studio enables users to create,
develop, and employ machine learning models. It integrates with IBM's existing AI services and
offers access to potent hardware like GPUs and TPUs.
• DataCamp: DataCamp is a cloud-based platform that offers practice problems for data science
and machine learning as well as interactive coding training.
GPUs and TPUs on Google Colab Ask anyone who uses Colab why they love it. The answer is
unanimous – the availability of free GPUs and TPUs.
5
 2203031247014
Abhishek Ghosh
Training models, especially deep learning ones, takes numerous hours on a CPU. We’ve all faced
this issue on our local machines. GPUs and TPUs, on the other hand, can train these models in a
matter of minutes or seconds.
If you still need a reason to work with GPUs, check out this excellent explanation by Faizan
Shaikh.
It gives you a decent GPU for free, which you can continuously run for 12 hours. For most data
science folks, this is sufficient to meet their computation needs. Especially if you are a beginner,
then I would highly recommend you start using Google Colab.
Google Colab gives us three types of runtime for our notebooks:
• CPUs,
• GPUs, and
• TPUs
As I mentioned, Colab gives us 12 hours of continuous execution time. After that, the whole virtual
machine is cleared and we have to start again. We can run multiple CPU, GPU, and TPU instances
simultaneously, but our resources are shared between these instances.

How to Use Google Colab?

You can go to Google Colab using this link. This is the screen you’ll get when you open Colab:

6
 2203031247014
Abhishek Ghosh
Click on the NEW NOTEBOOK button to create a new Colab notebook. Upload your local
notebook to Colab by clicking the upload button:

You can also import your notebook from Google Drive or GitHub, but they require an
authentication process.

You can rename your notebook by clicking on the notebook name and change it to anything you
want.
I usually name them according to the project I’m working on.
Google Colab Runtimes – Choosing the GPU or TPU Option
The ability to choose different types of runtimes is what makes Colab so popular and powerful.
Here are the steps to change the runtime of your notebook:
Step 1: Click ‘Runtime’ on the top menu and select ‘Change Runtime Type’:

7
 2203031247014
Abhishek Ghosh

Step 2: Here you can change the runtime according to your need:

8
 2203031247014
Abhishek Ghosh
A wise man once said, “With great power comes great responsibility.” I implore you to shut down
your notebook after you have completed your work so that others can use these resources because
various users share them. You can terminate your notebook like this:
Step 3:Uploading Files and Datasets:-
Here’s a must-know aspect for any data scientist. The ability to import your dataset into Colab is
the first step in your data analysis journey.
The most basic approach is to upload your dataset to Colab directly:

You can use this approach if your dataset or file is very small because the upload speed in this
method is quite low. Another approach that I recommend is to upload your dataset to Google Drive
and mount your drive on Colab:
You can do this in just one click of your mouse:

9
 2203031247014
Abhishek Ghosh

You can also upload your dataset to any other platform and access it using its link. I tend to go with
the second approach more often than not (when feasible).
Step 4: Saving Your Notebook:-
All the notebooks on Colab are stored on your Google Drive. The best thing about Colab is that
your notebook is automatically saved after a certain time period and you don’t lose your progress.
If you want, you can export and save your notebook in both *.py and *.ipynb formats:

10
 2203031247014
Abhishek Ghosh
Not just that, you can also save a copy of your notebook directly on GitHub, or you can create a
GitHub Gist:

I love the variety of options we get.

Step 5: Exporting Data/Files from Google Colab:-
You can export your files directly to Google Drive, or you can export it to the VM instance and
download it by yourself

11
 2203031247014
Abhishek Ghosh

Exporting directly to the Drive is a better option when you have bigger files or more than one file.
You’ll pick up these nuances as you work on bigger projects in Colab.
Step 6:Sharing Your Notebook:
Google Colab also gives us an easy way of sharing our work with others. This is one of the best
things about Colab:

Just click the Share button, and it gives us the option of creating a shareable link that we can share
through any platform. You can also invite others using their email IDs. It’s exactly the same as
sharing a Google Doc or Google Sheet. The intricacies and simplicity of Google’s ecosystem are
astounding!

12
 2203031247014
Abhishek Ghosh

13
 2203031247014
Abhishek Ghosh

Example of Hello World C Program:

Example of Hello World Python Program:

14
 2203031247014
Abhishek Ghosh

INPUT:-
1. !pip install numba
• OUTPUT:-

2. import numba
from numba import cuda
print([Link])
#Device 0 means GPU is free
• OUTPUT:-

3. [Link].select_device(0)
• OUTPUT:-

4. !pip install git+[Link]

• OUTPUT:-

15
 2203031247014
Abhishek Ghosh

5. %load_ext nvcc_plugin
• OUTPUT:-

6. %%cu
#include<stdio.h>
#include<cuda.h>
#include "curand.h"
#include<cuda_runtime_api.h>
__global__ void add(int *a, int *b, int *c)
{
*c = *a + *b;
}
int main() { int a, b, c; // host copies of a, b, c
int *d_a, *d_b, *d_c; // device copies of a, b, c
int size = sizeof(int); // Allocate space for device copies of a, b, c
cudaMalloc((void **)&d_a, size);
cudaMalloc((void **)&d_b, size);
cudaMalloc((void **)&d_c, size);
// Setup input values
a = 10;
b = 40;
// © NVIDIA Corporation 2011
// Addition on the Device: main()
// Copy inputs to device
cudaMemcpy(d_a, &a, size, cudaMemcpyHostToDevice);
16
 2203031247014
Abhishek Ghosh
cudaMemcpy(d_b, &b, size, cudaMemcpyHostToDevice);
// Launch add() kernel on GPU
add<<<1,1>>>(d_a, d_b, d_c);
// Copy result back to host cudaMemcpy(&c, d_c, size, cudaMemcpyDeviceToHost);
// Cleanup cudaFree(d_a);
cudaFree(d_b); cudaFree(d_c);
printf("%d",c);
return 0;
}
• OUTPUT:-

7. !nvidia-smi
• OUTPUT:-

17
 2203031247014
Abhishek Ghosh
Practical-2

Aim:- Demonstrate basic Linux Commands.

Theory:-
Command shell: A program that interprets commands is Command shell.

Shell Script: Allows a user to execute commands by typing them manually at a terminal, or
automatically in programs called shell scripts. A shell is not an operating system. It is a way to
interface with the operating system and run Commands.

BASH (Bourne Again Shell)

• Bash is a shell written as a free replacement to the standard Bourne Shell (/bin/sh)
originally written by Steve Bourne for UNIX systems.

• It has all of the features of the original Bourne Shell, plus additions that make it easier to
program with and use from the command line. Since it is Free Software, it has been
adopted as the default shell on most Linux systems.

1. PWD – Print / Present Working Directory

Description : - PWD is used to shows the full pathname of Current Working Directory.

starting from the root (/). Use the PWD command to find your way in the

Linux filesystem structure maze or to pass the working directory in a Bash

script.

Syntax:-

pwd

Output : -
18
 2203031247014
Abhishek Ghosh

2. cd : -Change Directory

Description :- It Allows you to Change your Working Directory. You use it to move around

Within the Hierarchy of your file system.

Syntax:- cd

Desktop

Output:-

3. cd .. :- Change Directory to home directory

Description :- It allows you to Change your Working Directory and ‘..’ is use to redirect to

home directory. Syntax : cd ..

Output :-

19
 2203031247014
Abhishek Ghosh

4. ls :- list all the Files & directories

Description :- List all the files and folders in current directory in

column format. Syntax :- ls

Output :-

5. cat :- it’s use to read any file

Description :- cat stands “catenate”.It Reads data from files,& outputs their contents. It

is the simplest way To Display the contents of a file at the command line.

Syntax :- cat filename

Note :- 1. Print the contents of files [Link] and

[Link] Cat [Link] [Link]

2. Print The cpu information using cat

command Cat/ proc/cpuinfo

3. Print the memory information using cat

command Cat/ proc/meminfo

Output :-

20
 2203031247014
Abhishek Ghosh

6. head

Description :- head by default, it prints the 10 lines of each file to standard output. With

more then one File, it precedes each set of output with a header identifying

the file name.

If no file is specified, or when file is specified as a dash (‘-‘), heads reads from

standard input. Syntax :- head [option]….[file/directory]

Output : -

7. tail

Description :- tail is a command which prints the last few number of lines (10 lines

by default) of a certain file, then terminates.

Ex ~ Output the last 100 lines of the file

[Link]. tail [Link] -n 100

21
 2203031247014
Abhishek Ghosh
Syntax : - tail

[option]….[file/directory] Output :

22
 2203031247014
Abhishek Ghosh

8. mv : Moving (and Renaming) Files

Description :- The mv command lets you move a file from one directory location to another.
It also lets you rename a file (there is no separate rename
command).

Syntax :- mv [option] source directory

Note : - 1. Moves the file [Link] to the directory

destination-directory.
Mv [Link] destination _ directory

2. Move the file [Link] into the parent

directory. mv [Link] ../

Output :-

9. mkdir :- to create a folder or directory

Description :- If the specified directory does not already exist, mkdir creates it. More
than one directory may be specified when calling mkdir.

Syntax:- mkdir

ram Output :-

23
 2203031247014
Abhishek Ghosh

10. cp
:- copy files
Description :- The cp command is used to make copy of files and

directories. Syntax :- cp [option] source directory

Output :-

11. rmdir :- remove the folder / directory

Description :- The rmdir command is used to remove a directory that contains other

files or directories. Syntax :- rmdir 123

Output :-

24
 2203031247014
Abhishek Ghosh

12. gedit :- it use to open notepad in liunx.

Description :- The gedit command is used to create and

open a file. Syntax :- gedit _filename.txt

Output :-

13. man

Description :- it is use as manual in liunx. & shows that usage of any command.

Displays on online manual page or manpage.

Syntax :- man command

Output :-

1
 2203031247014
Abhishek Ghosh

14. clear

Description :- it is use to clear the terminal. & clear

the screen . Syntax :- clear

Output :-

15. whoami

Description :- Print the user name associated with the current

effective user ID. Syntax :- whoami

Output :-

2
 2203031247014
Abhishek Ghosh

16. wc

Description :- print newline,word & byte count for each FILE,& a total line if more than one
FILE is

Specified. A word is a non-zero length sequence of printable characters delimited

by white space.

Syntax:- wc --char

[Link] Output :-

17. grep

Description :-grep searches for PATTERNS of each file. Patterns is one or more patterns

separated by newline Characters.& grep prints the each line that matches a

pattern. Typically PATTERNS should be Quoted when grep is used in a shell

command.

Syntax:- grep a

[Link] Output :-

3
 2203031247014
Abhishek Ghosh

18. free

Description :- free displays the total amount of of free & used physical swap memory &

swap memory in System, as well as the buffers and caches used by the kernel.

Syntax:- free

Output :-

19. echo

Description :- Display the text into terminal

screen. Syntax: echo “anything you write”

Output :-

4
 Ghosh Abhishek S.
2203031247014

20. pipe ( | )

Description :- Pipe command is used to send output of one program as a input toanother.

Pipes “|” help combine 2 or more commands

Syntax :- ls -l | grep

[Link] Output :-

21. touch

Description :- touch command is use to create a file . a file arguments that

does not created Empty . unless -c or -h is supplied. A file

arguments string of -s is handled Specially and causes touch to

change the times of the file associated with Standard output.

Syntax:- touch

[Link] Output :-
 Ghosh Abhishek S.
2203031247014
Practical-3

AIM: Using Divide and Conquer Strategies design a class for Concurrent Quick Sort using C++.
Divide and Conquer:
The Divide and Conquer strategy is a fundamental problem-solving technique in computer science
and mathematics. It involves breaking down a complex problem into smaller, more manageable
subproblems, solving them independently, and then combining the solutions to obtain the final
result. The strategy follows a recursive approach, where the problem is divided into subproblems,
each of which is solved independently. The solutions to the subproblems are then combined to
solve the original problem. This approach is particularly useful for solving problems that exhibit
overlapping subproblems and can be efficiently solved using smaller instances of the same
problem.
The Divide and Conquer strategy offers several advantages. It allows for efficient problem-solving
by reducing the complexity of the original problem into smaller, more easily solvable subproblems.
The parallelizability of the approach enables the use of concurrent or distributed computing
resources, further improving the efficiency of the solution. Additionally, the modular nature of
Divide and Conquer allows for code reusability and extensibility, as the same technique can be
applied to various problems by appropriately defining the divide, conquer, and combine steps.
Overall, the Divide and Conquer strategy is a powerful and widely applicable technique that
provides elegant solutions to complex problems across different domains.
Multi-threading in Parallel Computing:
In many applications today, software needs to make decisions quickly. And the best way to do that
is through parallel programming in C/C++ and multithreading.

Multithreaded Programming on a Single Processor:

Multithreading on a single processor gives the illusion of running in parallel. In reality, the
processor is switching by using a scheduling algorithm. Or, it’s switching based on a combination
of external inputs (interrupts) and how the threads have been prioritized.

Multithreading on Multiple Processors:

Multithreading on multiple processor cores is truly parallel. Individual microprocessors work
together to achieve the result more efficiently. There are multiple parallel, concurrent tasks
happening at once.
 Ghosh Abhishek S.
2203031247014
Algorithm:
DAC
(a, i, j)
{
if(small(a, i, j))
return(Solution(a, i, j))
else
mid = divide(a, i, j) // f1(n)
b = DAC(a, i, mid) // T(n/2)
c = DAC(a, mid+1, j) // T(n/2)
d = combine(b, c) // f2(n)
return(d)
}

Quick-Sort :
QuickSort is a sorting algorithm based on the Divide and Conquer algorithm that picks an element
as a pivot and partitions the given array around the picked pivot by placing the pivot in its correct
position in the sorted array.
The key process in quickSort is a partition(). The target of partitions is to place the pivot (any
element can be chosen to be a pivot) at its correct position in the sorted array and put all smaller
elements to the left of the pivot, and all greater elements to the right of the pivot.
Partition is done recursively on each side of the pivot after the pivot is placed in its correct position
and this finally sorts the array.

Quick Sort Algorithm:-

procedure quickSort(left, right)
if right-left <= 0
return
else
pivot = A[right]
partition = partitionFunc(left, right, pivot)
quicksort(left,partition-1)
quicksort(partition+1,right)
end if
end procedure
Steps to Perform QuickSort:- 
Select the pivot element, here for the sake of simplicity, we are selecting the rightmost element as
the pivot.
Rearrange the elements of the array in a way that all the elements smaller than the pivot are on its
left and that all the greater ones are on the right. They need not be sorted. After the rearrangement,
the array would look like this
In this step, the sublist is divided into two parts – sublist before the pivot element, and sublist after
the pivot element.
 Ghosh Abhishek S.
2203031247014

Program:-
INPUT:-
Ghosh Abhishek S.
2203031247014
 Ghosh Abhishek S.
2203031247014

OUTPUT:-
 Ghosh Abhishek S.
2203031247014
Practical-9
Aim: Write a simple CUDA program to print “Hello World!”

Theory:
CUDA Programming:
CUDA programming is a specialized skill that involves harnessing the computational power of
NVIDIA GPUs (Graphics Processing Units) to accelerate parallel processing tasks. CUDA stands
for "Compute Unified Device Architecture," and it allows developers to write programs that can be
executed on the GPU, taking advantage of its massively parallel architecture. The basic idea behind
CUDA programming is to offload certain compute-intensive tasks from the CPU to the GPU, which
can handle a large number of parallel threads, thus significantly speeding up the overall processing.

The process of writing a simple CUDA program involves several key steps. First, the developer
needs to set up the development environment, which includes installing the CUDA toolkit and
ensuring that the appropriate NVIDIA drivers are installed. Once the environment is ready, the
developer must identify the portions of the code that can be parallelized and executed on the GPU.
These sections are typically enclosed in functions called "kernels." The kernel functions are written
in C/C++ with some additional CUDA- specific syntax. The data is transferred from the CPU to the
GPU memory, the kernel function is launched with a specified number of threads, and the results
are then transferred back to the CPU memory. This data transfer between the CPU and GPU can be
a significant overhead, so efficient memory management is crucial in CUDA programming.
Overall, CUDA programming empowers developers to leverage the GPU's parallelism and
accelerate a wide range of scientific, engineering, and machine learning applications.

CUDA programming is a specialized field that enables developers to utilize NVIDIA GPUs'
parallel processing capabilities for high-performance computing. By writing CUDA kernel
functions in C/C++ with additional CUDA-specific syntax, specific compute-intensive tasks can be
offloaded from the CPU to the GPU. The process involves setting up the CUDA development
environment, identifying parallelizable sections of code, and transferring data between CPU and
GPU memory. Once launched with a specified number of threads, the GPU executes the kernel
function in parallel, resulting in significant speedup compared to CPU execution. CUDA's efficient
memory management ensures optimal performance, making it an essential tool for accelerating a
diverse range of applications, including scientific simulations, image processing, and deep learning
tasks.
 Ghosh Abhishek S.
2203031247014
Step to perform code:

Step 1: go to Google Colab

[Link]

Step 2: Select Runtime > Change runtime type> Hardware accelerator> Select :
 Ghosh Abhishek S.
2203031247014

T4 GPU
Step 3: Write CUDA Program

%%writefile hello_world.cu
#include <cuda.h>
#include <iostream>
#include <stdio.h>
int main(void){
printf("Hello world!");
return 0;
}

%%writefile :
In Google Colab, %%writefile is a "magic command" used to write the contents of a cell
to a file on the Colab virtual machine's file system. When you use the %%writefile magic
command followed by a file path, the contents of the cell are saved to that file.
 Ghosh Abhishek S.
2203031247014

#include <cuda.h>:
This line includes the CUDA runtime library header, which provides the necessary
functions and types for CUDA programming.

Step 4: Write CUDA Program commond to compile .cu file and name output file with -o

Step 5: Write ./filename to run the program.