Graphics Processing Unit (GPU)
Course Teacher:
Md. Obaidur Rahman, Ph.D.
Professor
Department of Computer Science and Engineering (CSE)
Dhaka University of Engineering & Technology (DUET), Gazipur.
Course ID: CSE - 4503
Course Title: Microprocessors and Assembly Language
Department of Computer Science and Engineering (CSE),
Islamic University of Technology (IUT), Gazipur.
Graphics Processing Unit (GPU)
A specialized circuit designed to rapidly manipulate and
alter memory
Accelerate the building of images in a frame buffer
intended for output to a display
GPU is microprocessor that has been designed specifically
for the processing of 3D graphics.
The processor is built with integrated transform, lighting,
triangle setup/clipping, and rendering engines, capable of
handing millions of math-intensive processes per second.
GPUs form that heard of modern graphics card, relieving
the CPU of much of the graphics processing load
2 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)
Why GPU?
To provide a separate dedicated graphics resources
including a graphics processor and memory.
To relieve some of the burden of the main system
resources, namely CPU, Main Memory, and the
System Bus, which would otherwise get saturated
with graphical operations and I/O requests.
However, the abstract goal of a GPU is to enable
representation of a 3D world as realistically as
possible.
3 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)
Components of GPU
Graphics Processor How many processing units?
Graphics Co-processor Lots of
Graphics Accelerator How many ALUs?
Frame Buffer Hundreds
Memory Do you need a cache?
Graphics BIOS Sort of
Digital-to-Analog Converter (DAC) What kind of memory?
Display Connector Very fast
Computer Bus Connector
4 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)
Conventional CPU Architecture
Space devoted to control logic
instead of ALU
CPUs are optimized to
minimize the latency of a single
thread
Can efficiently handle control
flow intensive workloads
Multi level caches used to hide
latency
Limited number of registers
are used due to smaller
number of active threads
5 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)
Modern GPU Architecture
Generic many core GPU
Less space devoted to control logic
and caches
Large register files to support
multiple thread contexts
Low latency hardware managed for
thread switching
Large number of ALU per “core”
with small user managed cache per
core
Memory bus optimized for
bandwidth
~150 GBps bandwidth allows us to
service a large number of ALUs
simultaneously
6 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)
Modern GPU Architecture
SP: Scalar Processors in ‘CUDA core’
architecture, each executes one thread
SM: Streaming multiprocessor
32xSP (or 16, 48 or more)
Fast local ‘shared memory’ (shared between
SPs) 16 KB (or 64 KB)
Parallelization
Decomposition to threads
Memory
Shared memory, global memory
Enormous processing power
Thread communication
Synchronization, no interdependencies
7 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)
Factors Behind GPU Revolutions
The GPU is specialized for computing-intensive with highly
parallel computation (exactly what graphics with multi-frames
rendering is about)
So, more transistors can be devoted to data processing rather
than data caching and flow control
The fast-growing video game industry exerts strong
economic pressure that forces constant innovation
8 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)
GPU Vendors
9 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)
NVIDIA GPU Architecture
Compute Unified Device Architecture (CUDA)
Framework
A general purpose parallel computing architecture
A new parallel programming model and instruction set
architecture
Leverages the parallel compute engine in NVIDIA GPUs
Software environment that allows developers to use C as a
high-level programming language
CUDA Capable Devices: Tesla, Quadro, NVS, GeForce
10 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)
NVIDIA GPU Architecture
Device = GPU = Set of multiprocessor
Warp = A scheduling unit of up to 32 threads
Multiprocessor = Set of processors & shared
memory
Kernel = GPU program (Typically OS)
Grid = array of thread blocks that execute a kernel
Thread block = group of SIMD threads that execute a
kernel and can communicate via shared memory
11 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)
NVIDIA Architecture
GPUDirect NVIDIA GPUDirect™ Accelerated
Communication with Network And Storage Devices
12 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)
CPU
13 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)
GPU (Two Cores)
14 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)
GPU (Four Cores)
15 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)
CPU vs. GPU
16 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)
GPU Pipeline
Host Interface stage is the communication bridge of
GPU with the CPU and receives the command from CPU.
Vertex Processing stage receives the vertices from the
host interface in object space and outputs them in screen
space.
In Triangle Setup stage information becomes raster
information
17 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)
GPU Pipeline
Pixel Processing stage provides all kinds of pixel related
and color related processing including texture mapping
and math operations.
Memory Interface stage is the communicating bridge
for final output towards the display.
18 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)
GPU vs. CPU
Main advantage of CPU is caches
Fast single thread performance, but also helps with multithreaded apps
Disadvantage is complexity, limiting number of cores per chip
Also have fast synchronization
Main advantage of GPU is high throughput
Each instruction for an SM executes on 8 scalar units (32 data elements)
Disadvantage is need to move data explicitly into (small) SM memory
from large shared memory
Also have support for gather/scatter from memory and special functional
units (e.g., texture sampling, math ops)
Performance measurements for GPU assume data already in
GPU memory
Overall performance of GPU (geometric mean) is 2.5X of
CPU
19 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)
Thank You !!
20 CSE-4503: Microprocessors and Assembly Language
Islamic University of Technology (IUT)