See discussions, stats, and author profiles for this publication at: [Link]

net/publication/332378350

Evaluating the Performance and Energy Efficiency of OpenGL and Vulkan on a

Graphics Rendering Server

Conference Paper · February 2019

DOI: 10.1109/ICCNC.2019.8685588

CITATIONS READS

5 248

4 authors, including:

Dayuan Chen
Texas State University
6 PUBLICATIONS 19 CITATIONS

SEE PROFILE

All content following this page was uploaded by Dayuan Chen on 10 December 2024.

The user has requested enhancement of the downloaded file.

 Evaluating the Performance and Energy Efficiency of
OpenGL and Vulkan on a Graphics Rendering Server
Michael Lujan1, Michael Baum1, Dayuan Chen2, and Ziliang Zong1
1
Department of Computer Science, Texas State University, San Marcos, TX
2
Department of Computer Science, Harbin Institute of Technology at Weihai, China
1
{mal290, mcb114, zz11}@[Link] [Link]@[Link]

Abstract— Open Graphics Library (OpenGL) has been the [2]. Since then, Vulkan has gained enormous support from
de facto industry standard of 2D and 3D graphics rendering industry and is becoming another industry standard for high
for decades. As OpenGL continued to evolve, it becomes performance graphics rendering.
easier to use and more powerful by accommodating The design philosophies of Vulkan include: 1) Vulkan
heterogeneous hardware and hiding optimization details from empowers rather than isolates the capability of developers. It
developers. As a side effect, its overhead gradually increases assumes the application developers have the most accurate
and its performance becomes unpredictable. To address the high-level knowledge about the work to be done, not the
weaknesses of OpenGL, Vulkan was proposed as a thin driver. 2) Vulkan is thread friendly. 3) Vulkan reduces
graphics API to reduce the overhead and allow developers to repeated work in the driver by allowing explicitly re-use of the
gain grainer control over rendering process. This paper rendering work from the application. These new features
conducts a comprehensive evaluation on the performance and enable Vulkan with great potential to boost performance and
energy efficiency of OpenGL and Vulkan on a graphics save energy compared to OpenGL. However, Vulkan’s great
rendering server. Our experimental results show that 1) power comes with the great responsibility on software
OpenGL’s performance is unpredictable; 2) Vulkan can save developers. Vulkan developers must completely abandon their
significant amount of energy while maintaining the same traditional programming modalities and take care of many
performance; and 3) OpenGL cannot keep up with Vulkan’s details when building a Vulkan application. In fact, migrating
performance when an extremely high frame rate is required. an OpenGL application to a Vulkan application is quite time
consuming because Vulkan API is verbose and the learning
Keywords-performance; energy efficiency; OpenGL; Vulkan; curve is steep.
This paper aims to quantitatively evaluate the performance
I. INTRODUCTION and energy efficiency of OpenGL and Vulkan on a graphics
Since being developed in early 1990’s, Open Graphics rendering server, which can help developers determine if
Library (OpenGL) has become the industry standard of high writing a Vulkan application is worthwhile. Our experimental
performance graphics rendering for more than two decades. results show that 1) OpenGL’s performance is unpredictable;
With the continuous efforts of numerous companies and 2) Vulkan can save significant amount of energy while
organizations (e.g. Nvidia and Microsoft etc.), OpenGL has maintaining the same performance; and 3) OpenGL cannot
been significantly improved by releasing 18 different versions keep up with Vulkan’s performance when an extremely high
(Version 1.1 to 4.6) [1]. Undoubtedly, OpenGL has made frame rate is required. Specifically, our results demonstrate
great contributions to many important fields such as game that Vulkan can save more than 50% of power on the CPU
design, VR and AR. side without degrading performance or achieve 3 times of
However, one of the key design philosophies of OpenGL frame rate than OpenGL when power is not a limiting factor.
is to make the implementation details transparent to The remainder of this paper is organized as follows.
developers, which helps usability of the API but limits the Section II discusses related work. Section III describes the
capability of developers. OpenGL acts like a black box to server hardware specifications, the software configurations,
developers. It has limited knowledge about the developers’ and the details about the profiling tool. Section IV describes
rendering work. Meanwhile, the developers have no idea how the design of the orbiting rocks application and our
the OpenGL will optimize their code for better performance. experimental methodology. Section V analyzes the results.
As a result, OpenGL is forced to incorporate more and more Section VI concludes our study.
optimizations and try different ones to ensure high
performance can be achieved for various workloads. This II. RELATED WORK
causes the gradual increase in overhead and makes Compared to decades of development of OpenGL and
performance unpredictable sometimes. Additionally, OpenGL OpenGL ES, Vulkan is relatively new. It was first announced
has the inherent weakness of not supporting parallelism well, in 2015 [2] and Khronos released the first specification of
largely because multi-threading was not available when Vulkan in 2016 [3]. To the best of our knowledge, there are
OpenGL was first designed in early 1990’s. only a few studies that compared the performance and energy
To address the weaknesses (thick driver, unpredictable efficiency of Vulkan and OpenGL. Larabel compared the
performance, poor parallelism etc.) of OpenGL and OpenGL performance of Vulkan and OpenGL using a suite of Linux
ES, Vulkan was first formally introduced by the Khronos games (e.g. Dota 2, Mad Max, and Dawn of War 3) supporting
Group in February 2015 at the Game Developers Conference both graphics APIs [4]. Shiraef conducted a preliminary study
to compare the performance of Vulkan, OpenGL, and DirectX A. Orbiting Rocks Application Design
11 [5]. Blacker analyzed the impact of using multiple threads 1) Rock Data
in Vulkan on performance improvement [6]. ARM reported a The rock data contains an array of n rock objects where
15% of overall energy saving when using Vulkan to render a each rock object has unique attributes about its position,
graphic application instead of using OpenGL ES [7]. Most of rotation direction, scale, and texture index.
existing literature either only focused on performance or
2) Context
reported their results at the coarse-grained level. Our work
The context initializes the graphics API and connects it to
provides a complementary detailed study to evaluate both
the device in which to be rendered. This includes choosing the
performance and energy efficiency of Vulkan and OpenGL.
API version, Vertical-Sync options, GPU device, and surface
III. SYSTEM CONFIGURATION AND PROFILING TOOL to render to.
A. Hardware Specification 3) Meshes
We run the experiments on a Dell Precision 5820 The mesh of an object is the shape of the object
workstation, which is designed for 3D modeling and graphics represented by an interconnected web of points. We use one
mesh for the planet and one mesh for the rocks. Each rock
rendering tasks. It contains a quad-core Intel Xeon E5-1620
processor (3.5GHz) that supports hyper-threading (presents up uses the same mesh. Meshes are loaded into GPU memory.
to 8 threads), a 32GB DDR4 DRAM, a Nvidia GeForce GTX 4) Textures
1080 GPU, and a 2TB hard disk drive. A texture, in our case, is the image that is pasted on a
mesh. We use one texture for the planet and one texture array
B. Software Configuration for the rocks. There are 7 different textures for the rocks. Each
We run the experiments on Windows 10 (version rock is given a texture based on its texture index attribute.
10.0.17134). We use the LunarG SDK (version 1.1.73) for Textures are loaded into GPU memory.
Vulkan and the PowerVR SDK (version 3.2) for OpenGL ES. 5) Camera
The Graphics driver is NVIDIA driver (version 397.44). The camera sets the position and direction of the camera
C. Profiling Tool and loads the data into GPU memory.
We profile the orbiting rocks application (see Fig. 1) 6) Rendering Pipeline
using Intel’s PowerLog command-line tool and the NVIDIA This involves setting up GPU programs called shaders to
Management Library (NVML). Intel’s Powerlog tool use the rock data, meshes, textures, and camera information.
measures the CPU core frequency, utilization, and power The pipeline can be viewed as a function that takes the time
consumption. GPU core frequency, utilization, and power difference between the current time and start time of the
consumption are measured using commands from NVML. program as input and outputs the image of the orbiting rocks
for that specific time.
B. Rendering Methods
1) Instanced Draw Calls
An instanced draw call can draw multiple objects of the
same mesh in one draw call. In this case, one instanced draw
call renders all n rocks. Each frame performs two draw calls.
One for the planet and one for the rocks. This type of
rendering moves the rock orientation calculations to the GPU
and is expected to produce the least CPU workload for
displaying the output.
2) Indexed Draw Calls
One indexed draw call corresponds to one object being
Fig. 1. The screenshot of orbiting rocks application.
rendered. Each frame performs n + 1 draw calls. One for the
planet and n for the rocks. If multiple threads are used, the
IV. EXPERIMENTAL DESIGN draw calls are distributed across k threads. So, each thread
We designed a series of experiments that allow us to executes at least n / k indexed draw calls for the rocks. The
control CPU workload, the number of CPU threads, and main thread takes care of the planet draw call.
rendering methods. Fig. 1 shows the orbiting rocks application C. Experiments
that we developed to evaluate Vulkan and OpenGL. It displays 1) Static Instanced Draw Calls
a primary planet in space with rocks orbiting around it, which Renders a fixed number of rocks per frame using
refers to Willem’s collection of Vulkan examples and uses the instanced draw calls. Frame rate is locked at 60 frames per
same meshes and textures [8]. Each rock’s orientation second (fps).
(position and rotation) must be calculated for every frame. The
2) Dynamic Instanced Draw Calls
following subsections discuss the design of our application,
Gradually increases the amount of rocks rendered per
the two rendering methods, and the six different experiments
frame starting from 0 rocks to the desired maximum amount
that we conduct to evaluate Vulkan and OpenGL.
of rocks using instanced draw calls. Frame rate is 60 fps. and 1024 rocks. The highest and lowest power saving using
3) Static Indexed Draw Calls Vulkan in these tests is 26.45% and 54.66%, respectively.
Renders a fixed number of rocks per frame using indexed B. Dynamic Instanced Draw Calls
draw calls. Frame rate is 60 fps. Fig. 2 shows the CPU power draw in a dynamic 0 to 2048
4) Dynamic Indexed Draw Calls instanced rocks test. Vulkan conserves more power overall
Gradually increases the amount of rocks rendered per than OpenGL ES in this test. Unlike in subsection A, the
frame starting from 0 rocks to the desired maximum amount power of OpenGL ES does not decrease. Fig. 3 plots the
of rocks using indexed draw calls. Frame rate is 60 fps. power consumption of GPU, which is almost identical for
5) Frame Rate Instanced Draw Calls Vulkan and OpenGL ES.
Gradually increases the amount of rocks rendered per
frame starting from 0 rocks to the desired maximum amount Table I. The power consumption of Vulkan and OpenGL ES
when using instanced static draw calls.
of rocks using instanced draw calls. Frame rate is unlocked.
The application will render as many frames per second as
possible. # of Rocks VK AVG (W) GL AVG (W) Power Savings %

6) Frame Rate Indexed Draw Calls 32 16.604386 36.275135 54.226534

Gradually increases the amount of rocks rendered per
frame starting from 0 rocks to the desired maximum amount 64 18.220012 37.666771 51.628421
of rocks using indexed draw calls. Frame rate is unlocked. The
application will render as many frames per second as possible. 128 16.431131 36.236222 54.655508

Experiments using instanced draw calls are devised to 256 16.681142 36.424264 54.203215
show the performance comparison when they are rendering
the same object. One draw call is needed to render instances of 512 17.609447 24.751033 28.853690
the same object. Experiments using indexed draw calls are
devised to show the difference between Vulkan and OpenGL 1024 19.386995 26.359397 26.451296
when they are rendering unique objects, since one draw call is
2048 19.490728 38.619033 49.530771
needed per unique object. While the visual output in instanced
experiments is the same as the output of indexed experiments,
we can interpret each rock to be a unique object with a unique
mesh.
For experiments 1, 2, 3, and 4 we chose 2048 rocks to be
the maximum amount of rocks rendered since OpenGL ES
cannot render 4096 rocks without dropping the frame rate
under 60 fps. Experiments 1, 2, 3, and 4 run for 30 minutes
each and experiments 5 and 6 execute for 15 minutes each.
V. EXPERIMENTAL RESULTS
This section presents our experiments and analyzes the
results. Subsections A and B study the power draw of baseline
CPU and GPU usage of instanced draw calls. Subsections C
and D evaluate the performance and energy efficiency of
indexed draw calls. Subsections E and F compare the number Fig. 2. CPU power of dynamic instanced draw calls (0 to 2048 rocks).
of frames per second of OpenGL ES and Vulkan when power
is not a limiting factor.
A. Static Instanced Draw Calls
Table I shows the average power consumption of the
orbiting rocks application using Vulkan and OpenGL ES for
32, 64, 128, 256, 512, 1024, and 2048 rocks over 30 minutes
using instanced draw calls. Initially, we expect the power
consumption of Vulkan and OpenGL ES would be similar
since instanced draw calls render any amount of rocks with as
little CPU workload as possible. Surprisingly, the results show
that Vulkan conserves significant amount of power in all tests.
Specifically, the power consumption of Vulkan (VK) stays
around 17 W in all tests and OpenGL stays around 36 W in
most cases and decreases to around 25 W when rending 512
Fig. 3. GPU power of dynamic instanced draw calls (0 to 2048 rocks).
C. Static Indexed Draw Calls E. Frame Rate Instanced Draw Calls
Table II shows the power consumption of Vulkan and Fig. 5 shows for both APIs, the frames per second in a
OpenGL ES for rendering 32, 64, 128, 256, 512, 1024, and dynamic 0 to 16384 instanced rocks test with the frame rate
2048 rocks respectively using indexed draw calls on a single unlocked. Fig. 6 shows the GPU core utilization. As more time
thread. The results show that Vulkan power consumption stays elapses, the CPU and GPU have more to process. This can
around 17 W in all tests. The power consumption of OpenGL lead to both CPU and GPU bottlenecks. We can see in the
ES decreases as the number of rocks increases from 32 to 256 beginning the frame rate of OpenGL ES reaches a CPU
then increases when rendering 512, 1024, and 2048 rocks. The bottleneck, which is evidenced that the CPU is running at its
highest and lowest power savings using Vulkan in these tests maximum frequency (~ 3.5 GHz) throughout the entire test on
is 15.82% and 55.49%, respectively. both APIs and the GPU core utilization is not at 100%. The
bottleneck caps OpenGL ES top frame rate at around 3,000
Table II. The power consumption of Vulkan and OpenGL ES fps. Vulkan, however, can reach the maximum frame rate of
when using indexed draw calls 9000 fps, which is three times higher than the frame rate of
OpenGL ES.
# of Rocks VK AVG (W) GL AVG (W) Energy Savings %

32 18.17837 37.41813 51.4183

64 16.36975 36.77535 55.48716

128 18.53716 25.33994 26.84609

256 16.54105 19.64873 15.81619

512 18.18385 23.39965 22.29005

1024 17.52684 26.41409 33.64587

2048 18.15172 34.52804 47.42905

D. Dynamic Instanced Draw Calls

Fig. 4 shows for both APIs, the CPU power draw in a Fig 5. Frame rate of dynamic instanced draw calls (0 to 16384 rocks).
dynamic 0 to 2048 indexed rocks test. Because draw calls are
expensive operations for the CPU, we expect to see power
consumption increases proportionally with the number of draw
calls. However, as has been seen in subsections A and C,
OpenGL’s performance with respect to power consumption is
difficult to predict. It follows the proportional increase for
some phases but would suddenly drop at arbitrary point. With
2048 rocks, Vulkan is not being pushed enough to have a
significant increase in power consumption.

Fig 6. GPU utilization of dynamic instanced draw calls (0 to 16384 rocks).

F. Frame Rate Indexed Draw Calls

Fig. 7 shows for both APIs, the frames per second in a
dynamic 0 to 16,384 indexed rocks test with the frame rate
unlocked. Fig. 8 shows the GPU utilization. Figs. 9 and 10 are
the same as Figs. 7 and 8 except that two CPU threads are
used to execute the draw calls, which feed the GPU two times
faster with rendering tasks. It is obvious that multi-threading is
Fig 4. CPU power of dynamic indexed draw calls (0 to 2048 rocks).
useful on the Vulkan side, which can render more frames with
two threads than a single thread. This is evidenced by the
higher GPU core utilization rate. Clearly, OpenGL ES cannot
keep up with Vulkan when the frame rate is unlocked.

Fig 10. GPU utilization of dynamic indexed draw with two CPU threads.

VI. CONCLUSION
This paper studies the performance and power
Fig. 7. Frame rate of dynamic indexed draw calls (0 to 16384 rocks). consumption of OpenGL ES and Vulkan on rendering a
graphic application. Experiments 1, 2, and 4 show that the
performance of OpenGL ES is unpredictable. Experiments 1,
2, 3, and 4 demonstrate that Vulkan is able to save significant
amount of power (more than 50% with certain workload)
without degrading performance. Experiments 5 and 6 prove
that OpenGL ES cannot keep up with Vulkan’s performance
when power is not a concern. In summary, Vulkan is more
predictable, energy efficient, and performant than OpenGL in
general. If these qualities are important to the product, the
benefits of using Vulkan will far outweigh its longer
development time and steeper learning curve.
REFERENCES
[1] [Link]
[2] [Link]
[3] [Link]
specification
[4] [Link]
Fig 8. GPU utilization of dynamic indexed draw calls (0 to 16384 rocks). july2018&num=1
[5] Joseph A. Shiraef, “An exploratory study of high performance graphics
application programming interfaces”, Master thesis of University of
Tennessee, Chattanooga.
[6] Axel Blackert, "Evaluation of multi-threading in Vulkan",
[Link]
[7] [Link]
[8] Sascha Willems, Vulkan, (2017), GitHub repository,
[Link]

Fig 9. Frame rate of indexed draw calls with two CPU threads.

View publication stats