Shader Performance Analysis on a Modern GPU Architecture

Victor Moya1, Carlos Gonzalez, Jordi Roca, Agustin Fernandez, Roger Espasa2
Department of Computer Architecture, Universitat Politècnica de Catalunya
{vmoya, cgonzale, jroca, agustin, roger}@[Link]

Abstract from the research literature [24], producing a microar-

chitecture that closely tracks today’s GPUs without
being an exact replica of any particular product avail-
This paper presents an analysis of the performance of
the shader processing units in a modern Graphics Proc- able or announced. We have then implemented this
essor Unit (GPU) architecture using real graphic appli- microarchitecture in full detail in a cycle-level, execu-
cations. The architecture of a modern GPU is described tion-driven simulator. In order to feed this simulator,
and a simulator and associated framework used to eval- we have also produced an OpenGL framework (library,
uate the architecture is introduced. The paper analyses driver and trace capture tool) able to run full applica-
the effects in performance of different configurations of tions (i.e., commercial games) on our GPU microarchi-
the shader processing units and compares a classic tecture. Finally we have used this simulator to evaluate
GPU with a unified shader GPU. The evaluated unified the performance of a unified shader architecture and
shader architecture proves to be 15% to 30% more effi- basic performance parameters of the shader microarchi-
cient, in terms of area, with a 2% to 7% improvement in tecture.
performance when compared with a similar non-unified
The reminder of this paper is organized as follows:
architecture.
Section 2 introduces the 3D rendering algorithm. Sec-
tion 3 describes the GPU pipeline and briefly discusses
1. Introduction our ATTILA GPU architecture. Section 4 introduces
the simulator and the associated OpenGL framework.
The microarchitecture of the shader units in modern Section 5 describes in detail the architecture of the
GPUs is becoming a key research topic as they evolve shader units and the unified shader programming
to support more complex and generic programming model. In Section 6 the performance and efficiency of
models. Recently presented by ATI and MS the GPU different shader architecture configurations is evaluated
for the XBOX360 game platform implements a unified using a UT2004 trace. Finally Sections 7 and 8 present
shader architecture. However, there are no proper eval- related work and conclusions.
uations of the performance differences between the
classic GPU architecture (implementing separated ver-
tex and fragment shader units) and a unified shader
2. 3D Rendering
GPU architecture. The present work compares the per-
formance and efficiency of both architectural models GPUs are designed as specific purpose processors
along with other parameters of the architecture. implementing a specific 3D rendering algorithm. The
We have developed a generic GPU microarchitec- 3D rendering algorithm implemented takes as input a
ture that contains most of the advanced hardware fea- stream of vertices that defines the geometry of the
tures seen in today’s major GPUs. We have liberally scene. The input vertex stream passes through a com-
blended techniques from all major vendors and also putation stage that transforms and computes some of
the vertex attributes generating a stream of transformed
vertices. The stream of transformed vertices is assem-
This work has been supported by the Ministry of Science and Technol-
bled into a stream of triangles, each triangle keeping the
ogy of Spain and the European Union (FEDER funds) under contracts
TIC2001-0995-C02-01 and TIN2004-07739-C02-01. attributes of its three vertices. The stream of triangles
1
Research work supported by the Department of Universities, Re- may pass through a stage that performs a clipping test.
search and Society of the Generalitat de Catalunya and the European Then each triangle passes through a rasterizer that gen-
Social Fund. erates a stream of fragments, discrete portions of the tri-
2
Intel Labs Barcelona (BSSAD), Intel Corp.
angle surface that correspond with the pixels of the Streamer

rendered image. Fragment attributes are derived from Index Buffer

the triangle vertex attributes. Vertex Cache Vertex Request Buffer

This stream of fragments may pass through a num-

ber of stages performing a number of visibility tests
(stencil, depth, alpha and scissor) that will remove non
visible fragments and then the stream of fragments will Primitive Assembly

pass through a second computation stage. This second

fragment computation stage may modify the fragment Clipping
Register
Texture
Filter

attributes using additional information from n-dimen-

File
Texture
Cache

sional arrays stored in memory (textures). The stream Texture

Address
Triangle Setup
of shaded fragments will, finally, update the frame-
buffer. Shader

Modern GPUs implement the two described compu- Fragment Generation

Hierarchical
Z buffer
Shader

tation stages as programmable stages named vertex Shader

shading and fragment shading. The programmability of Shader

Hierarchical Z HZ Cache
these stages and the streaming nature of the rendering
algorithm allows the implementation of other stream
based algorithms over modern GPUs [30][31]. How-
ever those implementations may not be optimal. The Z Cache Z Cache

non programmable stages are configurable using a lim- Z test Z test

Shader
ited and predefined set of parameters.
The shading stages are programmed using a shader,
or shader program, a relatively small program written Interpolator

in either assembly-like (legacy) or high level C-like

languages for graphics that describes how the input
attributes of a processing element (a vertex or a frag-
ment) are used to compute its output attributes. Color Color
Cache Cache
Graphics applications use software APIs (OpenGL Blend Blend

or Direct3D) that present an interface for the described

rendering algorithm and map the algorithm to the mod-
MC0 MC1 MC2 MC3
ern GPU hardware capabilities.
The 3D rendering algorithm is embarrassingly par-
Figure 1. ATTILA Architecture.
allel and shows parallelism at multiple levels. The larg-
est source of parallelism comes from the data and
control independency of the processing elements: verti- 3. ATTILA Architecture
ces are independent of each other, triangles are mostly
independent (except for transparent surfaces) and frag- We will now briefly describe our ATTILA imple-
ments from the same triangle are independent. mentation of the 3D rendering pipeline. We have
GPUs exploit four forms of parallelism: the pipeline blended techniques and ideas from different vendors
is divided into hundreds of single cycle stages to and publications [24] and we have made educated
increase the throughput and the GPU clock frequency guesses in those areas where information was specially
(pipeline parallelism); the pipeline stages are replicated scarce. Our implementation correlates in most aspects
to process in parallel multiple vertices, triangles and with current real GPUs.
fragments (data parallelism); multiple processing ele- The ATTILA architecture supports both hard parti-
ments are stored and processed concurrently to hide tioning of vertex and fragment shaders (the norm in
memory latencies in a specific stage or processing unit current GPUs) or a unified shader model (that will be
(multithreading); and independent instructions in a implemented in future GPUs). Figure 1 shows the
shader program may be executed in parallel (instruction ATTILA GPU graphic pipeline for the unified shader
level parallelism). model. The input and output processing elements, the
bandwidth and the latency in cycles of the different
ATTILA stages can be found at Table 1. Table 2 shows
the sizes of some of the input queues in those stages and fast rate without accessing GPU memory. The HZ
the number of threads supported in the vertex and frag- buffer is stored as on chip memory and supports resolu-
ment/unified shader units. The diagram and the table tions up to 4096x4096 (256 KB).
data corresponds to a reference architecture implement- The processing element for the next stages is the
ing 4 vertex shaders (non unified), 2 shader units (frag- fragment quad, a tile of 2x2 fragments. Most modern
ment or unified), 2 ROPs and four 32-bit DDR GPUs use this working unit for memory locality and the
channels. computation of the texture LOD in the Texture Unit.
Two GPU units are not shown in Figure 1 the Com- The Z and stencil test stage removes as early as pos-
mand Processor that controls the whole pipeline, pro- sible non visible fragments thereby reducing the com-
cessing the commands received from the system main putational load in the fragment shaders. Figure 1 shows
processor and the DAC unit that consumes bandwidth the datapath for early fragment rejection. However
for screen refreshes and outputs the rendered frames another path exists to perform the tests after fragment
into a file. shading. ATTILA only supports a depth and stencil
buffer mode: 8 bits for stencil and 24 bits buffer for
Table 1: Queue sizes and number of threads in the depth. The Z and Stencil test unit implements a 16 KB,
ATTILA reference architecture 64 lines, 4-way set associative cache. The cache sup-
ports fast depth/stencil buffer clear and depth compres-
Unit Size Element width
sion. The architecture is derived from the methods
Streamer 48 16×4×32 bits described for ATI GPUs [18][19].
Primitive Assembly 8 3×16×4×32 bits The Interpolator unit uses perspective corrected lin-
Clipping 4 3×4×32 bits ear interpolation [5] to generate the fragment attributes
from the triangle attributes. However other implemen-
Triangle Setup 12 3×4×32 bits
tations may interpolate the fragment attributes in the
Fragment Generation 16 3×4×32 bits
Fragment Shader [4]. The interpolated fragment quads
Hierarchical Z 64 (2×16+4×32)×4 bits are fed into the fragment or unified shader pool. The
Z Test 64 (2×16+4×32)×4 bits Texture Unit attached to each fragment or unified
Interpolator - - shader supports n-dimensional and cubemap textures,
Color Write 64 (2×16+4×32)×4 bits mipmapping, bilinear, trilinear and anisotropic filtering.
The Texture Cache architecture is based on
Shader (vertex) 12+4 16×4×32 bits
[20][21][22] and is configured as a 64 lines, 4-way set
Shader (fragment/unified) 112+16 10×4×32 bits
associative, 16 KB cache. Relatively small texture
caches are known to work well [20]. Compressed tex-
The Streamer unit reads streams of vertex input tures are also supported [23].
attributes from GPU or system memory and feeds them
to a pool of vertex or unified shader units (Figure 1). Table 2: Inputs, outputs and latencies in cycles in
The streamer also supports an indexed mode that allows the reference ATTILA architecture
reusing vertices shaded and stored in a small post shad-
ing cache. After shading the Primitive Assembly stage Unit Input BW Output BW Latency
converts the shaded vertices into triangles and the Clip- Streamer 1 index 1 vertex Mem
per stage performs a trivial triangle rejection test. Primitive Assembly 1 vertex 1 triang. 1
The rasterizer stages generate fragments from the Clipping 1 triang. 1 triang 6
input triangles. The rasterization algorithm is based on
Triangle Setup 1 triang. 1 triang 10
the 2D Homogeneous rasterization algorithm [14]
Fragment Generation 1 triang 2×64 frag. 1
which allows for unclipped triangles to be rasterized.
The Triangle Setup stage calculates the triangle edge Hierarchical Z 2×64 frag. 2×64 frag. 1
equations and a depth interpolation equation while the Z Test 4 frag. 4 frag. 2+Mem
Fragment Generator stage traverses the whole triangle Interpolator 2×4 frag. 2×4 frag. 2 to 8
generating tiles of fragments. ATTILA supports two Color Write 4 frag. 2+Mem
fragment generation algorithms: a tile based fragment
Shader (vertex) 1 vertex 1 vertex variable
scanner [16] and a recursive algorithm [15] (used for
Shader (fragment/unified) 4 frag. 4 frag. variable
the paper experiments).
After fragment generation a Hierarchical Z buffer
[17] is used to remove non visible fragment tiles at a The Color Write stage basic architecture is similar
to the Z and Stencil test stage architecture but color trace file. Frames, disregarding data preload in mem-
compression is not supported. ory, are mostly independent from each other and groups
The Memory Controller interfaces with the ATTILA of frames can be simulated independently. A PC cluster
memory and the main computer memory system. The with 80 nodes is used to simulate dozens of frames in
ATTILA memory interface simulates a simplified parallel. The current implementation of the simulator
GDDR memory where banks are not being modeled. can simulate up to 50 frames at 1024x768 of a UT2004
The memory access unit is a 64 byte transaction: a sin- trace, equivalent to 200-300 million cycles, in 24 hours
gle 4 cycle 8 32-bit word burst from a single GDDR in a single node (P4 Xeon @ 2 GHz).
channel. The number of channels and the channel We have developed an OpenGL framework (trace
interleaving is configurable. Read to write and write to capturer, library and driver) for our ATTILA architec-
read penalties are implemented. A number of queues ture (D3D is in the works).
and dedicated buses conform a complex crossbar that Figure 2 shows the whole framework and the pro-
services the memory requests for the different GPU cess of collecting traces from real graphic applications,
stages. verifying the trace, simulating the trace and verifying
Collect Verify Simulate Analyze
the simulation result.
OpenGL Application
Our OpenGL stack bridges the gap between the
OpenGL API and the ATTILA architecture translating
GLInterceptor
each OpenGL call into one or more low-level control
commands and maintaining and updating OpenGL
Trace state. The driver software organization is layered: the
GLPlayer top layer manages all OpenGL state while the lower
layer offers basic services to configure the graphics
Stats
Vendor OpenGL Driver Vendor OpenGL Driver Attila OpenGL Driver hardware and a basic memory allocation model. The
Signals
features supported by our OpenGL library are: basic
ATI R400/NVidia NV40 ATI R400/NVidia NV40 Attila Simulator
OpenGL functionality, about 200 API calls supported;
ARB Vertex and Fragment program extensions; vertex
Framebuffer Framebuffer Framebuffer Signal
Visualizer arrays and vertex buffer objects; legacy vertex and frag-
ment fixed function API emulated with library gener-
CHECK! CHECK!
ated shader programs [25]; texturing; stencil test, Z test
Figure 2. ATTILA Simulation Framework. and blending functions; and alpha test and fragment fog
emulated using library generated shaders.
The GLInterceptor tool uses an OpenGL stub
4. ATTILA Simulator and OpenGL Frame- library to capture a trace of all the OpenGL API calls
work and data that the graphic application is generating as
shown in Figure 2. All this information is stored in an
We have developed a highly accurate, cycle-level output file (a trace). To allow the graphic application
and execution driven simulator for the ATTILA archi- continue its normal execution GLInterceptor also
tecture described in the previous section. passes on all the OpenGL commands and data to the
The model is highly configurable (over 100 parame- original library. To verify the integrity and faithfulness
ters) and modular, to enable fast yet accurate explora- of the recorded trace a second tool, GLPlayer, can be
tion of microarchitectural alternatives. used to reproduce the trace.
The simulator is “execution driven” in the sense that After the trace is validated, it is feed by our
real data travels through signals from box to box. A OpenGL stack into the simulator. Using traces from
box uses the data received from signals and the data it graphic applications isolates our simulator from any
stores on its local structures to call the associated func- non GPU system related effects (for example CPU lim-
tional module that creates new or modified data that ited executions, disk accesses, memory swapping). Our
continues flowing through the pipeline. The same (or simulator uses the simulated DAC unit to dump the ren-
equivalent) accesses to memory, hits and misses and dered frames into files. The dumped frame is used for
bandwidth usage that a real GPU are generated. This verifying the correctness of our simulation and archi-
key feature of our simulator allows to verify that the tecture.
architecture is performing the expected tasks.
Our simulator implements a “hot start” technique
that allows the simulation to be started at any frame of a
5. Unified Shader Architecture nent swizzling and negation and absolute value modifi-
ers. The destination operand supports full per
Our shader architecture follows the OpenGL ARB component swizzling and masking of the operation
specifications for vertex [27] and fragment [28] shader result. The ISA supports two types of operations: 4
programs. component SIMD operations, (ADD, CMP, DP3, DP4,
MAD, etc.) and scalar operations (COS, EX2, RCP,
Constant Input etc.).
Register File Register File There are a few differences between the vertex and
96×4×32 bits 10/16×4×32 bits
fragment program specifications. Fragments can access
texture data with the TEX, TXB, and TXP instructions
while vertices can’t. Texture instructions, in our archi-
tecture, use the SIMD ALU for the texture address
Texture ALU Temporal
Unit ALU
ALU
ALU Register File
computation and then the texture request is issued to the
32×4×32 bits Texture Unit. The Texture Unit processes the request,
accesses the Texture Cache and, optionally, memory
and performs the filtering of the sampled texels. For
Output
fragment programs a KILL instruction is defined, used
Register File Physical Register to ‘stop’ the processing of a fragment Texture and
2/10×4×32 bits
File
KILL instructions use vectorial operands. An addi-
Figure 3. Shader architecture. tional instruction modifier _SAT is defined only for
fragment programs to inexpensively implement the
required clamping (to the [0, 1] range) of color result
The ARB vertex and fragment program specifica-
values.
tions define assembly alike instructions that can be used
Our unified shader architecture implements the
to program how the vertex and fragment output regis-
super set of both vertex and fragment program models,
ters can be calculated from per vertex and fragment
however our current OpenGL framework is limited to
input registers and a set of per batch constant parame-
the ARB vertex and fragment program features.
ters. There are four defined register banks (as shown in
The support for a non unified shader model is
Figure 3): the input register bank, a read only bank,
implemented capping a unified shader unit to work as a
stores the vertex and fragment input attributes; the out-
vertex shader unit from a current GPU would. The uni-
put register bank, write only, stores the vertex and frag-
fied model not only creates a coherent programming
ment output attributes; the temporal register bank,
model for both fragment and vertex processing but also
which supports reading and writing, is used to store
simplifies the architecture design, and allows a more
intermediate values; and a constant parameter bank
flexible and efficient use of the shading units. As the
stores parameters that are constant for a whole batch. A
workload balance of vertices and fragments changes
shader register is a 4 component 32-bit float point vec-
from batch to batch more shader units can be allocated
tor, limiting the ARB shader program models to support
to the more demanding task. The unification of the ver-
only float point data. The program size is limited to a
tex and fragment programming models is the target for
few hundred instructions and changes in the execution
future APIs (Shader Model 4.0 [26] in Direct3D and
flow control (loops, branches, functions) are not sup-
OpenGL glSlang) and GPU architectures.
ported.
The abundance of parallelism inherent to shader
The OpenGL specification supports a high level
processing (all processing elements are always inde-
shader language, glSlang [29], that offers additional
pendent) is exploited via multithreading. The reference
programming features (vertex textures, looping,
architecture used for the paper experiments implements
branching, functions) but our OpenGL framework
128 threads per (unified or fragment) shader unit to
doesn’t support it yet. The glSlang programming lan-
hide texture (memory) access latency. The vertex
guage virtualizes all the hardware resources available
shaders for the non unified architecture implement 12
for the shader architecture and tasks the compiler and
threads to hide the instruction execution latency. The
optimizer with accommodating the requested resources
ARB programming model specifies a relatively large
with the resources available in the target architecture.
number of temporal registers, but in most cases most of
The ARB instructions are defined as an operation
those registers aren’t used by the shader program. For a
opcode, a destination operand and up to three source
more efficient use of the available transistor budget we
operands. The source operands support full per compo-
have implemented per program static allocation of tem-
poral live registers from a per shader unit physical reg- The shader execution pipeline consists of the fol-
ister file to each thread. The number of available lowing single cycle stages: a fetch stage; a decode
threads for multithreaded execution changes as the stage; a register read stage; a variable number of execu-
shader program requirements for live temporal registers tion stages (1 to 9 depending on the instruction latency)
changes. The reference architecture implements 4 tem- and a register write back stage. Split hardware pipe-
poral registers per thread and the allocation granularity lines are implemented to receive the shader inputs (ver-
is set at two registers per thread. tex and fragment input attributes) from the feeding
stage (streamer or interpolator) and send the shader
3 results (vertex and fragment output attributes) to the
2,8 2sh
2,6
8sh next rendering stages (post shading vertex cache or the
relative performance

uni2sh
2,4
6sh
4sh ROP units).
2,2
2
uni4sh
GPU hardware vendors don’t disclose the number
6sh
1,8
1,6
4sh
uni6sh of supported threads or temporal registers in their archi-
1,4 8sh tectures. However they do disclose information about
1,2 uni8sh
1
2sh the organization of the shader hardware ALUs. While
1-way 1-way + 2-way 4-way there are many differences between the GPU architec-
scalar
tures, most allow grouping multiple ARB instructions
(up to 5 or 6) in single cycle issue to a number of paral-
Figure 4. Performance scaling number of
lel SIMD ALUs (even implementing partitioned 2+2
shader units and issue width.
operations), scalar ALUs and special ‘mini-ALUs’
The shader units process in parallel groups of four implementing per operand modifier operations (e.g.
threads (each thread corresponding with a vertex or a vector normalization). Our OpenGL framework
fragment) because of a requirement of fragment pro- doesn’t support such level of optimization and the
cessing (texture LOD derivative computation). A sin- implemented arrangement of ALUs in the shader units
gle PC is kept per group to fetch and issue the same is relatively simple (swizzle, modifier, SIMD | scalar,
instruction for the four threads implementing an addi- write mask).
tional SIMD level to the architecture. For the non uni-
fied vertex shader architecture no grouping of vertices 6. Analysis of shader performance
is performed and each thread has an associated state
and PC. A group may be in one of four states: free (no 6.1. Unified shader architecture
fragments or vertices allocated), ready (instructions can
be fetched), blocked or finished (waiting for the thread The experiments in the current section are per-
results to be sent out of the shader unit). formed using a 450 frame trace from an Unreal Tourna-
We support two configurable fetch and issue modes ment 2004 Primeval map time demo. The Unreal game
in our simulated architecture: fetch and issue of a SIMD engine supports both the OpenGL and Direct3D APIs.
instruction and a scalar instruction per cycle and group The engine (for UT2004) is limited to the OpenGL
or fetch and issue of 1 to n instructions (disregarding fixed function API and doesn’t use shader programs.
the type) per cycle and group. Texture instructions can However our OpenGL framework generates shader
only be issued one per cycle and group and after decode programs that emulate all the fixed function API calls.
the group becomes blocked, waiting for the Texture The vertex and texture load of UT2004 is comparable
Unit to return the result. (or even higher) to other current games.
The shader instruction decoder detects dependen- The vertex shader programs generated for UT2004
cies and conflicts accessing the register bank ports and are relatively large (in some cases up to 100 instruction
may request the shader fetch unit to refetch instructions long) and implement multiple lights and complex trans-
that can’t be issued in a given cycle. Instructions are formations. Meanwhile the fragment programs ar at
fetched and issued for any group that is ready in a per top a dozen instructions long and implement mostly
shader unit thread group window, supporting the disor- texture instructions with a few arithmetic instructions
dered execution of groups. Instructions in the execu- combining the texture and input colors and performing
tion flow of a group are always fetched and issued in the alpha test.
order. The instructions are fetched from a small sized The first 50 and the last 20 frames of the trace aren’t
(not below 512 instructions) instruction memory where used in the experiments because they correspond to
shader programs are explicitly loaded before starting load or exit screens. The trace is simulated at a
the rendering batch.
 tions when comparing 1-way to 2-way execution.
1,7 Going to a 4-way configuration does not yield any addi-
1,6
1,5
1-way/1-way+scalar 1-w ay tional benefit over the 2-way configuration for our trace
2-way uni 1-w ay
set. It must be noted though that the fragment programs
fps/MTransistors

1,4
1-w ay+sca
1,3
1,2
uni 1-w ay+sca in the trace are relatively small and our current OpenGL
2-w ay
1,1
4-way uni 2-w ay
framework only performs a limited instruction reorder-
1
0,9
4-w ay ing optimization over them. With a better optimizer
uni 4-w ay
0,8 and larger fragment programs the improvement for the
0,7
2-way and 4-way configurations might be higher.
2sh 4sh 6sh 8sh
The difference between equivalent configurations
a) averaged frames/s per 1 Mtransistors. for non-unified and unified shader architectures shows
that the unified architecture can use the larger shader
170 pool to shade vertices at a faster rate. The improvement
1-w ay
150
uni 1-w ay is small though, ranging from a 1% to an 8% depending
130
1-w ay+sca on the configuration. The reason is that the frames in
8sh uni 1-w ay+sca
our trace are mostly limited by fragment shading.
fps

110 2-w ay
6sh
90
uni 2-w ay
Another reason is that the same configuration is used
4sh 4-w ay

70 uni 4-w ay for the geometry stages in both architectures and is cur-
50
2sh linear
rently limited to a throughput of 1 vertex and 1 triangle
30 80 130 180 per cycle. The vertex data fetch from memory may also
MTransistors
become a bottleneck for not properly aligned or inter-
b) frames per second vs Mtransistors. leaved streams.

Figure 5. Performance vs. transistors. 6.2. Area comparisons

1024x768 resolution with 8X Anisotropic Filtering
enabled. Four regions of 40 frames were rendered for We have built a rough estimation of the transistor
the experiments: frames 100 to 139, 200 to 239, 300 to cost of a shader unit based on the difference in transis-
339 and 400 to 439. The simulation CPU time (P4 tors between the ATI R400 (160 million transistors for
Xeon @ 2 GHz) was 24 hours per region and configu- 6 vertex shaders and 4 fragment shader units) and ATI
ration. RV400 (120 million transistors for 4 vertex shaders and
Figure 4 compares the performance of 32 configura- 2 fragment shader units) as detailed in [5]. Our estima-
tions in three axis: unified and non unified shader archi- tion puts 2.5 million transistors per vertex shader, 15
tectures, number of shader units and the shader million transistors per shader unit (fragment or unified)
instruction fetch/issue width. All the configurations are and a 15% increase in transistors per extra SIMD ALU
based on the reference architecture we have described and a 5% increase for the extra scalar ALU. Using this
in sections 3 and 5: two ROP units (working on frag-
ment quads) and four 32-bit memory channels. All the 100
90
non unified shader configurations implement 4 vertex 80

shaders. Figure 4 shows the performance improvement 70

instructions

60
of each configuration relative to the baseline configura- 50
fragment
vertex
tion: a non-unified 2 fragment shader 2-way architec- 40
30
ture. 20

As expected, the main performance gain comes 10

0
from increasing the number of shading units as the 1 51 101 151 201 251

inherent parallelism of the independent fragment (or batches frame 330

vertices) is exploited. We can clearly see a two fold

Figure 6. Vertex and fragment program
increase in performance going from 2 to 4 shader units.
instructions for frame 330.
From 2 to 6 the gain is below linear and it drops further
for 2 to 8. The reason, as we will discuss in the next estimation, Figure 5 compares the same 32 configura-
subsections, is that the rendered frame becomes limited tions in terms of transistor area and efficiency.
by the memory system rather than by fragment shading. Figure 5 a) compares the configurations in terms or
Increasing the fetch/issue width of the shader units frames per second per million of transistors. Figure 5
allows for up to 8% improvement in some configura- b) tallies the simulated frames per second with the esti-
mated area of the configuration. The linear perfor- maximum data rate is 1 datum being read or written per
mance line shows a linear improvement of performance cycle; maximum shader execution is 1 instruction per
per additional transistor based on the non unified 2 cycle in 8 b) and 2 instructions per cycle in 8 d); maxi-
shader 1-way configuration. Figure 5 demonstrates that mum ROP bandwidth is 8 fragments/cycle in all config-
a unified shader architecture is more efficient, ranging urations. Figures 8 a) and 8 c) are normalized to the
from 30% more efficient for the 1-way configurations maximum per cycle bandwidth also for all the configu-
to 15% more efficient for the 4-way configurations, rations tested so far: 64 bytes per cycle.
than a non unified architecture in terms of performance We compare two configurations implementing a
per area. A unified architecture becomes more efficient unified shader architecture, the first with two shaders
as the work load (vertices or fragments) of the rendered units and 1-way fetch/issue width and the second with
frame becomes more unbalanced from batch to batch. eight shader units and 2-way fetch/issue width.
The UT2004 trace we use is limited in this aspect and Figure 6 shows that most fragment shaders are 6 to
other applications may get additional improvements 10 instruction long, so given that the ROPs process
from a unified architecture. fragments at a rate of 8 per cycle the GPU would
require a pool of 12 to 20 1-way shader units to peak
0,8 the ROPs. As we can see in figures 8 b) and d) the
0,7 ROPs only reach a 70% utilization at the start of the
0,6 frame with the 8 shader configuration. However, even
0,5
execVSh
if the two shader configuration is clearly limited by
0,4
execFSh fragment shading the eight shader configuration isn’t.
0,3

0,2
Figure 8 d) shows that the average utilization of the
0,1 shaders is a 30% with peaks of over 50% utilization. If
0 we consider that even for a 2-way shader configuration
1 51 101 151 201 251 301 351
the shader unit still executes on average 1 instruction
Figure 7. Workload balance in frame 330. per cycle and thread, this 30% is equivalent to a 60%
utilization on a 1-way architecture. What keeps the 8
shader configuration from becoming fragment shader
6.3. Detailed performance analysis limited is the memory system, that as we can se in Fig-
ure 8 c) is at 60-70% utilization with a peak for a small
In this section we will analyze in detail what are the fillrate limited zone of 90%. The memory system is
bottlenecks in the configurations from the previous sec- unable to go beyond that 60-70% because of inefficien-
tion. We will use trace frame 330 for the detailed anal- cies and conflicts, and because the increased latency of
ysis. memory requests can not be hidden by the shader unit
Figure 6 shows the number of vertex and fragment threads. Using a pure fill rate benchmark, drawing a
program instructions per batch in frame 330 (260 full screen quad with forced reading and writing of Z
batches are rendered in total). Figure 7 shows the and color buffers, the memory subsystem seems to peak
workload balance between the vertex and fragment at 80%-90%.
shader units in the non-unified 8 shader 2-way configu- If we analyze the bandwidth usage in Figures 8 a)
ration for frame 330, sampled at 10Kcycle intervals. and c) texture data dominates in both cases, followed
Batches 120 to 200 (large vertex programs) in Figure 6 by color data and z data. Color data dominates over z
correspond with the vertex dominated zone at Figure 7 data because our architecture supports z compression
(cycles 2900K to 3100K). This limited zone accounts and fast early rejection using an on chip hierarchical Z
for most of the performance difference between the uni- buffer, and both techniques save a large amount of z
fied and non-unified shader architectures. bandwidth. The bandwidth usage for the vertex limited
Figure 8 shows the average utilization, sampled at zone of frame 330, see Figures 6 and 7, shows a clear
10K cycle intervals, of the three key GPU subsystems: increase in the amount of vertex data read from mem-
the memory system, the shader execution pipeline and ory.
the ROP (color and z) pipeline, Figure 8 also shows the
bandwidth consumed for vertex, texture, z and color 6.4. Increasing memory bandwidth
data for frame 330. The pipeline utilization graphics
are normalized to the maximum data, instruction and To verify that the memory system is the bottleneck
fragment operation rates. For the memory subsystem, for the six and eight shader configurations we per-
formed an experiment increasing the available band-
 0,5 1
0,9
0,4 0,8
0,7
other
0,6
0,3 z
0,5
color 0,4
0,2 texture 0,3
vertex 0,2
0,1 0,1
0
1 101 201 301 401 501 601 701 801 901
0
1 101 201 301 401 501 601 701 801 901 mem execSh rop

a) effective bandwidth usage for two shaders 1-way b) per GPU unit utilization for two shaders 1-way

1 1
0,9 0,9
0,8 0,8
0,7
0,7 other
0,6
0,6 z
0,5
0,5 color 0,4
0,4 texture 0,3
0,3 vertex 0,2
0,2 0,1
0
0,1
1 51 101 151 201 251 301
0
1 51 101 151 201 251 301 mem execSh rop

c) effective bandwidth usage for eight shaders 2-way d) per GPU unit utilization for eight shaders 2-way

Figure 8. Frame 330 characterization for two unified shader architectures.

width. If the six and eight shader configuration are scenario the 8 shader configurations deliver more than
unbalanced in terms of bandwidth per shader unit the a three fold improvement over the base architecture.
experiment should show an improvement in perfor-
mance when the additional bandwidth is provided. 7. Related work
In modern GPUs the number of 32-bit memory
channels ranges from 1 in the lower-end segments to 4 NVidia presented their first implementation of a
in the high-end implementations of the architecture. vertex shader for the NV2x GPU architecture [3].
More than 4 channels add a large number of extra pins Information obtained from available patents [4] and the
in the package and is too expensive to implement with analysis of the shaders performance and architecture
current technology. Therefore for a high end architec- using shader benchmarks [31] is limited. Some of the
ture the offered bandwidth can only be modified by information about NVidia and ATI implementations
changing the memory frequency. Modern GPUs allow surfaces on unofficial Internet forums [5][6]. Beyond
running the memory subsystem with a different, faster, shader microarchitecture, some recent work can be
clock than the GPU pipeline to provide more than 64 found: T. Aila et al. proposed delay streams [7] and
bytes per cycle to the GPU pipeline. However our cur- Akenine-Möller described a graphic rasterizer for
rent implementation of the simulator only supports run- mobile phones [8]. We can find research on other
ning the memory subsystem at the same frequency as graphic algorithms: a Reyes renderer was implemented
the GPU pipeline. For this reason we are limited to on the Imagine [1] stream processor by Owens [11], the
increasing the number of channels in order to simulate SaarCor [2] group presented a FPGA [13] implementa-
an increase in memory bandwidth. tion of their raytracing architecture and ray tracing and
Figure 9 shows the improvement of adding 1 and 2 photon mapping [12] has been implemented on a mod-
additional memory channels, normalized to the non ern GPU.
unified two shader 1-way architecture. The bandwidth On the side of simulators Stanford has a public soft-
is increased by 25% and 50% and goes from 64 to 80 ware implementation of the OpenGL library that can be
and 96 bytes per cycle. The graphic demonstrates that used for profiling. GLSim [9] and the GLTrace tool are
the 6 and 8 shader configurations are limited by mem- used in university courses for limited experiments on
ory as the increase in bandwidth produces a a 7% to the graphic pipeline but support for the last OpenGL
14% increase in performance. In this high bandwidth API specification or extensions is not available. QSil-
ver [10] is a GPU simulator framework that combines a [6] DIRECTXDEV mail list. [Link]
flexible and programmable OpenGL trace capturing chives/[Link]
and profiling tool, based on the Chromium tool, that [7] T. Aila, V. Miettinen and P. Nordlund. Delay streams for
feeds a preprocessed trace into a cycle-timer simulation graphics hardware. ACM Transactions on Graphics, 2003.
[8] T. Akenine-Möller and J. Ström Graphics for the masses: a
model. QSilver doesn’t emulate the GPU functionality
hardware rasterization architecture for mobile phones. ACM
and uses statistics and probability distributions as
Transaction on Graphics, 2003.
inputs to the model. [9] Stanford University GLSim & GLTrace. [Link]
3,4
[Link]/courses/cs448a-01-fall/[Link]
[10] J. W. Sheaffer, et al. A Flexible Simulation Framework
3,2
for Graphics Architectures. Graphics Hardware 2004.
3
2-way 6sh
[11] J. Owens, B. Khailany, et ak. Comparing Reyes and
2,8
uni 2-way 6sh OpenGL on a Stream Architecture. Graphics Hardware 2002.
2-way 8sh
uni 2-way 8sh [12] T. J. Purcell, I. Buck, W. R. Mark, P. Hanrahan. Ray
2,6
Tracing on Programmable Graphics Hardware. ACM Trans-
2,4
actions on Graphics, 2002.
2,2 [13] Jörg Schmittler, et al. Realtime Ray Tracing of Dynamic
4 5 6

32-bit memory channels Scenes on a FPGA Chip. Graphics Hardware, 2004.

[14] Marc Olano, Trey Greer. Triangle Scan Conversion using
Figure 9. Performance increase with 2D Homogeneous Coordinates. Graphics Hardware, 2000.
bandwidth increase. [15] Michael D. McCool, et al. Incremental and Hierarchical
Hilbert Order Edge Equation Polygon Rasterization. Proceed-
8. Conclusions ings Graphics Hardware 2001.
[16] J. McCorkmack, et al. Neon: A (Big) (Fast) Single-Chip
3D Workstation Graphics Accelerator. WRL Research 1998.
Research on the microarchitecture of the shader
[17] Green, N. et al. Hierarchical Z-Buffer Visibility. Pro-
units in modern GPU has become a hot topic in the last ceedings of SIGGRAPH 1993.
couple of years. Future GPUs will change from the [18] S. Morein. ATI Radeon Hyper-z Technology. In Hot3D
current non-unified shader model towards a unified Proceedings - Graphics Hardware Workshop, 2000.
shader model. This work evaluates the differences in [19] US20030038803: System, Method, and apparatus for
performance and efficiency between both models. compression of video data using offset values. ATI Tech.
The experiments in this paper show that the main [20] Ziyad S. Hakura, Anoop Gupta. The Design and Analysis
source of performance improvement in modern GPUs of a Cache Architecture for Texture Mapping. ISCA 1997.
comes, as expected, from increasing the number of [21] Homan Igehy, et al. Prefetching in a Texture Cache Ar-
shader units working in parallel. The increase is near chitecture. Proceedings of the 1998 Eurographics/SIG-
linear as long as the other GPU subsystems, for exam- GRAPH Workshop on Graphics Hardware
[22] Se-Jeong Park et al. A reconfigurable multilevel parallel
ple memory, don’t become the bottleneck. The experi-
texture cache memory with 75-GB/s parallel cache replace-
ments show that exploiting ILP with superscalar shader ment bandwidth. Solid-State Circuits, IEEE Journal 2002.
units adds an additional 8% performance increase. The [23] S3TC compression: [Link]
unified shader architecture shows a little performance ple/registry/EXT/texture_compression_s3tc.txt
benefit, at least for the tested trace, over the non-unified [24] Stanford University CS488a Fall 2001 Real-Time Graph-
architecture but the largest gain comes from improved ics Architecture. Kurt Akeley, Path Hanrahan.
efficiency per area, up to 30% better. [25] Lars Ivar Igesund, Mads Henrik Stavang. Fixed function
pipeline using vertex programs. November 22. 2002
9. References [26] Microsoft Meltdown 2003, DirectX Next Slides
[27] ARB Vertex Program extension: [Link]
projects/ogl-sample/registry/ARB/vertex_program.txt
[1] Ujval Kapasi, et al. The Imagine Stream Processor. Pro-
[28] ARB Fragment Program extension: [Link]
ceedings 2002 IEEE Intl. Conference on Computer Design.
projects/ogl-sample/registry/ARB/fragment_program.txt
[2] Jörg Schmittler, et al. SaarCOR A Hardware Architecture
[29] OpenGL Shading Language v 1.10.
for Ray Tracing. Graphics Hardware 2002.
[30] GPGPU. [Link]
[3] Erik Lindholm, et al. An User Programmable Vertex En-
[31] K. Fatahalian, J. Sugerman, and P. Hanrahan. Under-
gine. ACM SIGGRAPH 2001.
standing the Efficiency of GPU Algorithms for Matrix-Matrix
[4] WO02103638: Programmable Pixel Shading Architecture,
Multiplication. Graphics Hardware 2004.
December 27, 2002, NVIDIA CORP.
[5] Beyond3D Graphic Hardware and Technical Forums. ht-
tp://[Link]