Parallel Programming module 4

GPU PROGRAMMING WITH CUDA

GPU (Graphics Processing Unit) is a parallel processor designed to execute thousands of
lightweight threads at the same time. Traditional CPU → few powerful cores, GPU →
thousands of small cores optimized for parallel workloads. CUDA (Compute Unified
Device Architecture) is NVIDIA’s parallel computing platform and API for GPU
programming. It allows developers to write code in C, C++, Python to run functions
(called kernels) on the [Link] follows heterogeneous computing:
CPU = Host GPU = Device CUDA programming focuses on data-parallel solutions
where large datasets can be processed simultaneously.
Key Concepts In CUDA
Kernel
A function executed by a large number of threads on the GPU.
__global__ void add(int *a, int *b, int *c) { ... }
Threads, Blocks, and Grid
 Thread → smallest execution unit.
 Block → collection of threads (usually 128–1024 threads).
 Grid → collection of blocks.
Memory Types
 Global Memory: Large but slow.
 Shared Memory: Fast on-chip memory shared by threads.
 Registers: Fastest memory but limited.
Parallel Execution Model
CUDA uses SIMT (Single Instruction Multiple Threads) architecture.
Real-Time Example
Real-Time Scenario 1: Image Processing
 GPU used to:
o Remove noise from an image
o Apply filters like blur, sharpen
o Perform edge detection
 Millions of pixels are processed in parallel, giving 10–50x faster results than CPU.
Real-Time Scenario 2: Weather Forecasting
 Climate models require:
o Complex differential equations
o Massive matrix calculations

Dept Of. CSE (DS), AIET 17

 Parallel Programming module 4
 GPUs accelerate simulations used by IMD, NASA, etc.
Real-Time Scenario 3: Medical Imaging – CT/MRI Reconstruction
 GPU helps reconstruct images instantly using parallel math operations.
Real-Time Scenario 4: Deep Learning
 Training neural networks with millions of parameters.
 TensorFlow and PyTorch internally use CUDA kernels.

Figure 1 cuda architecture

CUDA Programming Example – with Output

Program: Vector Addition on GPU
Adds two arrays A and B and stores result in C.
CUDA C Code
#include <stdio.h>
__global__ void vectorAdd(int *A, int *B, int *C, int n) {
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i < n) {
C[i] = A[i] + B[i];
}
}
int main() {
int n = 5;
int h_A[5] = {1, 2, 3, 4, 5};
int h_B[5] = {10, 20, 30, 40, 50};
int h_C[5];

Dept Of. CSE (DS), AIET 18

 Parallel Programming module 4
int *d_A, *d_B, *d_C;
size_t bytes = n * sizeof(int);
cudaMalloc(&d_A, bytes);
cudaMalloc(&d_B, bytes);
cudaMalloc(&d_C, bytes);
cudaMemcpy(d_A, h_A, bytes, cudaMemcpyHostToDevice);
cudaMemcpy(d_B, h_B, bytes, cudaMemcpyHostToDevice);
vectorAdd<<<1, 256>>>(d_A, d_B, d_C, n);
cudaMemcpy(h_C, d_C, bytes, cudaMemcpyDeviceToHost);
printf("Result: ");
for (int i = 0; i < n; i++)
printf("%d ", h_C[i]);
cudaFree(d_A);
cudaFree(d_B);
cudaFree(d_C);
return 0;
}
Output
Result: 11 22 33 44 55
Explanation
 256 GPU threads will run the same vectorAdd kernel.
 Each thread adds one element:
o Thread 0 → 1 + 10 = 11
o Thread 1 → 2 + 20 = 22
o Thread 2 → 3 + 30 = 33
 GPU does all operations parallelly, reducing execution time.
Limitations of CUDA / GPU Programming
 Hardware limitation: Only works on NVIDIA GPUs.
 Cost: High-performance GPUs (RTX, A100, H100) are expensive.
 Memory limit: GPU memory is less compared to CPU RAM.
 Learning Curve: Requires understanding of threads, blocks, memory hierarchy.
 Power Consumption: GPUs consume more energy than CPUs.
 Debugging Complexity: Harder to debug parallel code.
Advantages of GPU Programming
 Massive parallelism: Thousands of threads at once.

Dept Of. CSE (DS), AIET 19

 Parallel Programming module 4
 10x–100x speedup: for large mathematical workloads.
 Real-time processing :of big data, images, and videos.
 Scalability: Can increase performance by adding more GPUs.
 Support for AI/ML: through CUDA libraries (cuDNN, cuBLAS).
Applications of CUDA GPU Programming
1. Artificial Intelligence & Deep Learning
 Training CNNs, RNNs, Transformers.
 Frameworks like TensorFlow, PyTorch use CUDA internally.
2. Scientific Computing
 Physics simulations
 Chemical reactions
 Computational fluid dynamics (CFD)
3. Medical & Healthcare
 MRI reconstruction
 Tumor detection
 3D medical imaging processing
4. Finance
 Risk modeling
 Stock market prediction
 Option pricing (Monte Carlo simulations)
5. Gaming & Virtual Reality
 Realistic graphics
 Ray tracing
 Physics engines
6. Big Data Processing
 GPU-accelerated Spark
 Parallel database queries
 Faster ETL processes
7. Automotive – Self Driving Cars
 Real-time object detection
 Sensor fusion
 Path planning
Real-Time Example
Self-driving Cars Using CUDA
 Cars use:

Dept Of. CSE (DS), AIET 20

 Parallel Programming module 4
o LIDAR
o Camera feeds
o Radar
 GPUs process thousands of images per second.
 Tasks:
o Object detection
o Lane detection
o Vehicle tracking
o Obstacle avoidance
 CUDA accelerates deep learning models like YOLO, ResNet, MobileNet.
Example Speed:
 CPU image processing → 200 ms
 GPU CUDA processing → 5 ms
This makes real-time driving possible.

CONCLUSION
CUDA GPU programming is one of the most powerful modern computing methods. It
enables high-performance parallel computing for AI, simulations, image processing,
scientific research, and real-time systems. With thousands of cores, GPUs outperform
CPUs for large-scale parallel workloads. Although it has limitations like hardware
dependency and complexity, its advantages in performance, scalability, and speed make
CUDA essential in today’s computing world. GPU programming continues to grow with
applications in AI, healthcare, robotics, finance, and scientific research, shaping the future
of high-performance computing.

Dept Of. CSE (DS), AIET 21