Periods/Week
Course Code Course Title Semester Category C
L T P R
PARALLEL PROCESSING
2311CSC501J (Common to CSE, AIADS, CSE(AIML), V PC 3 0 2 0 4
CSE(CS), CSD, CSBS & IT)
Course
Pre-requisite Nil THEORY CUM PRACTICAL
Type
Course Objectives: This course will help the students
1. Understand the fundamentals of parallel computing and the need for parallelism in modern computing
systems.
2. To explore various parallel architectures, programming models, and interconnection networks.
3. To develop skills for designing parallel algorithms and evaluating their performance.
4. To gain hands-on experience with parallel programming tools such as MPI, OpenMP, and CUDA.
5. To study real-world applications and recent advances in parallel and distributed computing.
Course Outcomes:
Upon completion of the course, the students will be able to:
CO
Course Outcomes
Number
CO1 Classify different parallel architectures and models.
CO2 Implement parallel algorithms for computational problems.
CO3 Apply parallel programming tools (MPI/OpenMP/CUDA) to solve real-world problems.
CO4 Evaluate the performance of parallel applications using speedup and efficiency metrics.
CO5 Choose suitable parallelization strategies for the given domains.
Course Content:
Unit I Introduction to Parallel Processing 9
Need for parallel processing -Concepts of concurrency, parallelism, and multitasking-Flynn's classification (SISD,
SIMD, MISD, MIMD)-Parallel computing models-Shared Memory-Distributed Memory-Hybrid Models-Parallel
programming models (Thread, Task, Data parallelism)
Unit II Parallel Architecture 9
Taxonomy of parallel architectures -Interconnection networks (Bus-based, Crossbar, Multistage, Hypercube, Mesh,
Torus)-Memory hierarchy and cache coherence-Symmetric and asymmetric multiprocessing-Multicore processors
and GPUs-SIMD and MIMD architectures
Unit III Parallel Algorithms and Design 9
Principles of parallel algorithm design -Performance measures: Speedup, Efficiency, Scalability, Amdahl’s Law,
Gustafson’s Law -Granularity and decomposition-Load balancing and task scheduling
Case studies: Parallel search, Matrix multiplication, Sorting
Unit IV Programming Models and Tools 9
Message Passing Interface (MPI) -OpenMP (Shared memory programming)-CUDA programming basics (GPUs)-
Kubernets based parallel workloads, Tensor RT, CUDA-X-AI
Unit V Applications of Parallel Database 9
Scientific computing and simulations -Parallel databases and data mining-Real-time systems and embedded parallel
systems-Big Data and parallel processing with Hadoop/Spark-Recent trends: Federated Learning
Total Periods 45
Platform Needed:
List of Experiments
[Link]. Title of the Experiments
1 Matrix Multiplication Using OpenMP
2 Parallel Sorting Algorithm Using MPI
3 Image Processing with CUDA
4 Parallel Prime Number Generation
5 Implementation of Parallel Reduction (Sum, Min, Max) using CUDA
Total Periods 30
Learning Resources:
Text Books
1 “Parallel Programming in C with MPI and OpenMP" by Michael J. Quinn – McGraw-Hill, 2024
“Introduction to Parallel Computing" by Ananth Grama, Anshul Gupta, George Karypis, and Vipin
2
Kumar – Pearson Education.,2023
Reference Books
"Programming Massively Parallel Processors" by David B. Kirk and Wen-mei W. Hwu – Morgan
1
Kaufmann., 2020
“CUDA by Example: An Introduction to General-Purpose GPU Programming" by Jason Sanders and
2
Edward Kandrot – Addison-Wesley,2020
Online Resources (Web Links)
[Link] – NVIDIA CUDA Developer Zone for GPU programming
1
resources.
Name of the Internal Expert 1 Dr S Ahamed Ali Signature
Name of the Internal Expert 2 Ms Fathima Vincy Signature