Web GPUAttacks
Web GPUAttacks
Daniel Gruss
Graz University of Technology
ABSTRACT KEYWORDS
In recent years, the use of GPUs for general-purpose computations Side Channels, Cache Attacks, GPU computing
has steadily increased. As security-critical computations like AES
ACM Reference Format:
are becoming more common on GPUs, the scrutiny must also in-
Lukas Giner, Roland Czerny, Christoph Gruber, Fabian Rauscher, Andreas
crease. At the same time, new technologies like WebGPU put easy Kogler, Daniel De Almeida Braga, and Daniel Gruss. 2024. Generic and
access to compute shaders in every web browser. Prior work has Automated Drive-by GPU Cache Attacks from the Browser. In ACM ASIA
shown that GPU caches are vulnerable to the same eviction-based Conference on Computer and Communications Security (ASIA CCS 2024), July
attacks as CPUs, e.g., Prime+Probe, from native code. 1–5, 2024, Singapore. ACM, New York, NY, USA, 13 pages. [Link]
In this paper, we present the first GPU cache side-channel attack [Link]
from within the browser, more specifically from the restricted We-
bGPU environment. The foundation for our generic and automated
1 INTRODUCTION
attacks are self-configuring primitives applicable to a wide variety
of devices, which we demonstrate on a set of 11 desktop GPUs In the last decades, Graphics Processing Units (GPUs) have seen an
from 5 different generations and 2 vendors. We leverage features important evolution. While they were initially designed for the spe-
of the new WebGPU standard to create shaders that implement all cific purpose of graphic rendering, most modern discrete GPUs offer
building blocks needed for cache side-channel attacks, such as tech- the possibility of general-purpose computing. With the introduc-
niques to distinguish L2 cache hits from misses. Beyond the state tion of NVIDIA’s CUDA [1] in 2007 and OpenCL [2] in 2009, GPUs
of the art, we leverage the massive parallelism of modern GPUs have become commonplace for workloads that benefit from the
to design the first parallelized eviction set construction algorithm. massive parallelism they can offer. While the individual execution
Based on our attack primitives, we present three case studies: First, speed is still slow compared to recent CPUs, current-generation
we present an inter-keystroke timing attack with high F1 -scores, cards offer thousands of cores, enabling a huge performance boost
i.e., 82 % to 98 % on NVIDIA. Second, we demonstrate a generic, for parallelizable operations.
set-agnostic, end-to-end attack on a GPU-based AES encryption The increasing number of use cases of general-purpose GPU
service, leaking a full AES key in 6 minutes. Third, we evaluate a computing includes computations on potentially secret informa-
native-to-browser data-exfiltration scenario with a Prime+Probe tion, e.g., neural networks [3] or cryptographic applications [4, 5].
covert channel that achieves transmission rates of up to 10.9 kB/s. Thus, general-purpose GPU computing also becomes a more in-
Our attacks require no user interaction and work in a time frame teresting attack target. Recent research confirms these security
that easily enables drive-by attacks while browsing the Internet. concerns, as GPUs have become a recurrent target of side-channel
Our work emphasizes that browser vendors need to treat access to attacks, exploiting various shared components [3, 6–12]. Further-
the GPU similar to other security- and privacy-related resources. more, attackers may also leverage the GPU to attack other system
components [13, 14]. As on CPUs, the GPU cache is a particularly
CCS CONCEPTS interesting resource for side channels. Consequently, prior work
also replicated well-known CPU cache side-channel attacks on
• Security and privacy → Side-channel analysis and counter-
GPUs [6, 10, 12, 15, 16], albeit only in native code so far.
measures; Browser security; Information flow control.
While native code has direct access to a large variety of GPU
APIs, e.g., CUDA, Vulkan, Metal, and Direct3D, acquiring native
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed code execution is a significant hurdle for any attacker. Instead,
for profit or commercial advantage and that copies bear this notice and the full citation the browser has become a more interesting attack vector, as users
on the first page. Copyrights for components of this work owned by others than ACM routinely run untrusted third-party code on their devices within
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or a the browser. Since GPU computing can also offer advantages for
fee. Request permissions from permissions@[Link]. computations within websites, browser vendors decided to expose
ASIA CCS 2024, July 1–5, 2024, Singapore the GPU to JavaScript through APIs like WebGL and the upcom-
© 2024 Association for Computing Machinery.
ACM ISBN 978-1-4503-XXXX-X/18/06. . . $15.00 ing WebGPU standard. WebGPU is not only available on desk-
[Link] top browsers but is also already partially supported on mobile
ASIA CCS 2024, July 1–5, 2024, Singapore Giner et al.
devices in Chrome Canary version 117. As the future standard templating approach enables us to profile the T-tables in 13 s on
for web-based general-purpose interaction with GPUs, WebGPU average, with the remaining time (5.7 min) dedicated to the last
aims to lay solid foundations for performance and security. The round attack. We perform the attack on 2 recent GPUs, a NVIDIA
standard already has explicit mitigations against timing side chan- RTX 3060 Mobile and a NVIDIA RTX 3060 Ti, with similar results.
nels [17], e.g., disabling timer access (making it a trusted feature), Lastly, we demonstrate a covert channel with true channel capaci-
and mimicking the JavaScript mitigation against malicious use ties between 7.3 kB/s and 10.9 kB/s on the NVIDIA RTX 2070 Super,
of the SharedArrayBuffer [18–21]. Previous work demonstrated NVIDIA RTX 3080 and NVIDIA RTX 3060 Ti.
native code side-channel attacks on GPUs, where the browser trig- Our attacks require no user interaction and work within a realis-
gered L1 and L2 cache activity, e.g., through WebGL [12]. How- tic time frame a user might spend on a website, e.g., in the range
ever, the feasibility of a browser-based GPU cache side-channel of multiple minutes. Therefore, they can easily be implemented as
attack, targeting a victim running in native code or another browser drive-by attacks, targeting arbitrary users while browsing the Inter-
window, nor the possibility of an attack with the upcoming We- net. Furthermore, since our attacks are based on WebGPU, they are
bGPU standard [22] have been demonstrated yet. Considering the applicable to all operating systems and browsers implementing the
ubiquitous attack surface browsers offer to attackers, we need to WebGPU standard and, as we demonstrate, to a broad range of GPU
investigate the following questions: devices. Consequently, it becomes clear that browser vendors need
Can GPU cache side-channel attacks also be mounted from within to reassess their approach to offer GPU access to untrusted websites
a restrictive browser environment using APIs like WebGPU? Can without user consent. Instead, we recommend a security-centric in-
these attacks be made generic enough to work on the wide spectrum teractive approach that is already applied to all other security- and
of GPU hardware? To what extent can an attacker leverage GPU privacy-related resources, such as the microphone and the camera.
parallelization to enhance attacks? In summary, our paper makes the following main contributions:
In this work, we answer these questions by presenting the first (1) We present the first end-to-end cache attacks on GPU caches
end-to-end cache side-channel attacks from within browsers, lever- from the browser, using the restrictive WebGPU API.
aging the new WebGPU standard. Despite the inherent restrictions (2) We evaluate our attack primitives and attacks on a wide
of the JavaScript and WebGPU environment, we construct new range of GPU architectures and explore where the massive
attack primitives enabling cache side-channel attacks with an effec- parallelism of GPUs can improve attacks.
tiveness comparable to traditional CPU-based attacks. Our attacks (3) Based on our insights, we develop the first parallel eviction
are generic and automated, in the sense that our 2 attack primitives set construction algorithm and the first Prime+Probe attack
automatically determine GPU-specific configuration parameters re- on the L2 cache of a single, dedicated GPU.
quired for an attack, i.e., the cache hit-miss threshold, the cache size, (4) We describe a novel templating approach that we use in an
and the number of cache sets. Consequently, our attacks work on a attack on an AES T-table GPU implementation. Using pre-
wide variety of devices, which we demonstrate in our evaluation: dictable LRU cache set eviction cascades on GPUs, our attack
We show that our 2 basic attack primitives work on 11 desktop can skip the lengthy set-construction phase by exploiting
GPUs from 5 different generations and 2 vendors, NVIDIA and only contention in sets that are actually used by the victim.
AMD. We demonstrate that based on these, we can also identify Outline. Section 2 provides the background and Section 3 our
cache sets and monitor cache set collisions directly from a browser threat model. Section 4 presents the primitives for Prime+Probe on
on a variety of NVIDIA GPUs. the GPU from the browser. Section 5 explores an inter-keystroke
We introduce 3 techniques to exploit cache contention on the timing attack. Section 6 evaluates our attack for an AES key recov-
L2 cache of discrete GPUs from JavaScript via WebGPU compute ery and Section 7 in a covert channel scenario. Section 8 discusses
shaders. First, we highlight that significant cache eviction, often limitations and mitigations. Section 9 concludes.
induced by graphical rendering, can enable attackers to discern in-
stances of re-rendering. Second, we implement a templating attack 2 BACKGROUND
within the browser, designed to monitor memory access patterns.
Lastly, we present the first Prime+Probe attack on discrete GPUs 2.1 GPU architecture
executed from a browser. For all 3 attacks, we evaluate whether The architecture of discrete GPUs may vary by brand. Hereafter, we
using the GPU’s parallelism improves the basic attacks. For the focus on giving an insight into discrete GPUs architecture, tackling
eviction set construction in particular, we extend the state of the both computation and memory management. We default to the
art by leveraging the massive parallelism of modern GPUs with the concepts and notations adopted by NVIDIA, but similar concepts
first parallelized eviction set construction algorithm. are used by other manufacturers, such as AMD.
We evaluate our attacks in 3 distinct scenarios covering both A GPU consists of multiple Streaming Multiprocessors (SMs),
low-frequency non-repeatable events, as well as repeatable and called Compute Units (CUs) on AMD cards. Each SM has its ded-
high-frequency events: an inter-keystroke timing attack, AES key icated memory subsystem, including shared memory (SM-local
extraction, and the establishment of a covert channel, all initiated memory), caches, and functional units, to execute multiple threads
from a browser, i.e., through an attacker-controlled website. Our in parallel, operating under the SIMD paradigm. On GPUs, threads
keystroke monitoring attack detects inter-keystroke timings with are organized into thread blocks (also called workgroups on We-
F1 -scores in the range of 82 % to 98 %, and a sampling time below bGPU) that are assigned their own SM when executed. SMs consist
15 ms, fast enough to distinguish even very fast typing. We suc- of multiple processing blocks (4 on recent NVIDIA and AMD GPUs).
cessfully extract AES keys in 6 min with a precision of 100 %. Our Each processing block is a separate SIMD execution unit with its
Generic and Automated Drive-by GPU Cache Attacks from the Browser ASIA CCS 2024, July 1–5, 2024, Singapore
GPU
both consumer- and business-oriented NVIDIA GPUs. The high
SM SM
market share of NVIDIA GPUs and the ease of use of CUDA makes
processing
block
processing
block
processing
block
processing
block
processing
block
processing
block
processing
block
processing
block it the currently most widely used framework for general-purpose
computation on GPUs. Apple recently dropped support for OpenGL
L0 L0 L0 L0 L0 L0 L0 L0
in favor of Metal. Similarly, Vulkan was released in 2016 as a mod-
ern alternative to OpenGL. While these APIs have their differences,
L1 L1
typical calls include operations on texture mappings, rasterization,
L2
and memory management on the GPU.
Web APIs. WebGL is the current baseline JavaScript API giving
DRAM System Memory access to the GPUs rendering. As its name suggests, it was origi-
nally designed with a specific goal: graphic rendering in browsers.
Figure 1: Modern GPUs (here NVIDIA) have an L0 cache per Hence, its API is limited and does not provide support for generic
processing block, an L1 cache per SM, and a shared L2 cache. computations on the GPU [26]. This was the motivation behind
the WebGL 2.0 Compute initiative [27]: “to bring compute shader
support to the web via the WebGL rendering context”. Due to the
own load and store units capable of running 32 threads in parallel. emergence of new native rendering APIs, the diminishing promi-
Thread blocks are divided into warps, groups of 32 threads that nence of OpenGL, and the perceived constraints of WebGL, the
are scheduled on processing blocks. Processing blocks have warp project contributors decided to deprecate it [27] in favor of a more
schedulers, hardware schedulers that schedule warps in and out contemporary alternative, namely WebGPU.
of the processing block. When a warp has to wait for a memory Like WebGL, WebGPU provides access to the GPU graphics ca-
access or register dependencies, the warp scheduler schedules a pabilities in the browser. It is, however, not a mere wrapper around
different warp that is ready to execute to keep the SIMD units busy. OpenGL. More than that, it aims at being cross-platform and sup-
The constant rescheduling of warps allows for latency hiding and, porting modern graphic APIs, such as Vulkan, Metal, and DirectX,
therefore, more efficient use of processing blocks [1]. through JavaScript. Compared to WebGL, it offers a cleaner API,
Prior to the Volta architecture, all threads in a warp share the significantly better performance, and a more generic application
same instruction unit with a single program counter, i.e., instruc- range. At the time of writing, the standard is still under active
tions execute in lockstep [23]. If threads in the same warp diverge, deployment. However, the involvement of major browsers in this
they are masked until they converge again. If some threads execute process, and the promising performance, foreshadow a widespread
the if-branch and some execute the else-branch, the entire warp deployment in the next years. Chrome, Chromium, and Microsoft
executes the if-branch and the else-branch with threads masked ac- Edge already support WebGPU in their official release, and Firefox
cordingly. Volta introduced independent thread scheduling, with a has it in its Nightly version [28]. Support of mobile GPUs is also in
per-thread program counter and call stack, allowing the processing progress, with recent deployment on Android [29].
block to interleave execution of diverging branches [23]. Developers can create rendering pipelines and manage GPU
Similar to CPUs, GPUs use caches to reduce the latency for resources with WebGPU. WebGPU has its own shader language
memory accesses. Namely, each SM has a dedicated L1 cache that is called WebGPU Shading Language (WGSL) to write custom shaders
shared between processing blocks, and each processing block has that are compiled at runtime. While WebGPU provides access to
access to a smaller private L0 cache. Finally, GPUs share one global GPUs through native APIs, implementations of the standard may
L2 cache (LLC) between the SMs. Figure 1 illustrates the cache restrict the available GPU resources, e.g., memory and runtime,
hierarchy of Nvidia Turing GPUs. However, we note a few relevant for security reasons. Without restrictions, big WebGPU workloads
peculiarities of GPU caches. First, there is no coherency protocol could significantly impact the useability of the host system [30], as
between caches, and maintaining coherency is the responsibility of most GPUs only allow for one active shader at a time.
developers. Second, unlike the classical 64-byte cache line in CPUs,
GPUs’ LLC commonly has 128-byte cache lines [24, 25].
2.3 Prime+Probe
2.2 GPU APIs In the last decades, microarchitectural attacks have been studied ex-
GPUs can be called through different APIs depending on the context. tensively. Prime+Probe [31, 32] is a cache-based attack that exposes
We distinguish two main API families: native APIs (e.g., OpenGL, the memory access patterns of a process by exploiting cache con-
Vulkan and CUDA), and web APIs (e.g., WebGL and WebGPU). tention to leak the cache set accesses. This technique is particularly
Native APIs. The most straightforward and efficient way to in- useful for attackers with limited control over the victim’s machine,
teract with a GPU is through dedicated native APIs. They enable since it has low requirements and does not need shared memory or
the use of GPUs for either graphic rendering or generic computing. direct control over the cache with a flush instruction. Because of
OpenGL was introduced in 1992 to support GPU-assisted rendering these weak assumptions, it is well-suited for browser-based attacks,
on Linux and Apple platforms, while Windows uses the Direct3D where an attacker controls JavaScript on a web page [33, 34].
framework. For general-purpose computing, OpenCL, released in Assuming the attacker can execute code on the same processor
2008, provides support for all major GPU vendors and is widely as the victim, the attack works as follows. First, the attacker primes
used. In 2007, NVIDIA released CUDA [1], a compute language the cache by filling well-chosen cache sets with its own data. Then,
specifically designed for NVIDIA GPUs. CUDA is supported by they wait for the victim to make memory accesses. Finally, the
ASIA CCS 2024, July 1–5, 2024, Singapore Giner et al.
attacker probes their data to access the same cache sets as before. 1 if global_id.x != 0 {
If the victim accessed one of the sets monitored by the attacker, 2
3
var time: u32 = 0;
atomicStore(&timer, 0);
they will have evicted some of the attacker’s data, causing a longer 4 while (atomicLoad(&stop) != stop_value) {
latency in the probing phase. In the context of a covert channel, the 5 for (var a: u32 = 0; a < 100000; a++) {
6
attacker would run both sender and receiver, and use the contention
time++;
7 atomicStore(&timer, time);
on the cache sets to build the channel. 8 } } }
9 else {
10 start = atomicLoad(&timer);
11
2.4 Related Work
var c : u32 = atomicLoad(&buffer); //access
12 if c != 0 { //prevent optimization
Covert and side channels on GPUs. Naghibijouybari et al. [11] 13
14 }
return;
describe multiple covert and side-channel attacks on GPUs. Many 15 end = atomicLoad(&timer);
works consider a spy outside of the targeted GPU. Jiang et al. [7–9] 16 atomicStore(&stop, stop_value);
17
present a cache-based attack, a shared memory attack, and a bank-
}
conflict attack, all leading to a key recovery attack on AES. Similarly, Listing 1: Counting thread implementation in WGSL based
Ahn et al. [6] exploit cache conflicts to recover an AES key from on the global thread id and atomic operations.
a GPU implementation. While they rely on cycle-accurate timers,
our attack works from the browser without a timer. In addition,
their spy uses the native API, while we perform our attack from the
in multiple aspects. First, they focus on a SoC system and use con-
browser. More similar to our approach, Dutta et al. [35] perform
tention on the system-level cache, which is shared between the
Prime+Probe on Intel’s integrated GPU through contention on the
CPU cores and its peripherals (namely the GPU) in ARM systems.
LLC shared between the CPU and GPU with native OpenCL. They
This enables them to create contention from the CPU, whereas we
also demonstrate ring-bus interconnect covert channel reaching
consider spy and attacker to be co-located on the GPU. Second, the
the LLC. Dutta et al. [16] present a cross-multi-GPU Prime+Probe
cache occupancy of the system-level cache is significantly different,
covert channel based on L2 contention. Our threat model is different,
resulting in different challenges to overcome. Finally, they exploit
assuming a spy co-located on the same GPU in a drive-by attack
it using WebGL code, while we focus on its successor, WebGPU,
from the web.
which claims to consider and address the side-channels threat. Re-
Naghibijouybari et al. [10, 12] present the first attacks in the
cently, Taneja et al. [36] demonstrated hybrid side channels on the
co-located setting. Their first work [10] presents an in-depth study
CPU and GPU, based on how they adjust their frequency, power,
of General Purpose GPUs and highlights various ways to build
and temperature depending on the workload. They demonstrate
covert channels on GPUs using caches and functional units. In their
that GPUs exhibit instruction and data-dependent throttling. Their
following work [12], they demonstrate the ability of an attacker to
JavaScript attack assumes a victim in the browser but still relies
implement website fingerprinting based on GPU memory usage and
on the ability of the attacker to access native APIs to monitor the
performance counters. They also demonstrate the practical impact
power consumption and frequency of the GPU.
of their attack by tracking keystrokes from users and recovering
some internal parameters of a neural network running on the GPU.
3 THREAT MODEL
Wei et al. [3] present a similar approach, using the GPU context-
switching impact on performance counters to enhance the leakage As we target WebGPU, our primary requirement is a browser with
and recover the complete structure of a neural network. WebGPU support. As of writing, this includes Chrome releases since
Despite the groundbreaking nature of these works, our contribu- version 112, Chromium, Edge, and Firefox Nightly. By targeting
tions differ in key aspects. All aforementioned contributions rely on web browsers, our threat model includes any scenario where a
the attacker having access to the native APIs of the GPU through browser might run while sensitive information is being processed.
CUDA or OpenGL. This enables them to monitor high-precision Because the entire system usually shares the GPU, this can include
performance counters. Our attack works entirely from the browser anything rendered (such as websites or applications) and general-
using JavaScript, with the corresponding API limitations (e.g., we purpose computing operations. We show that our attack can be
do not have an accurate timer). This results in better portability but done in a drive-by manner, simply by visiting a website for a while.
also a weaker attacker in our threat model. We assume that the victim will visit an attacker’s page for several
Browser-based cache attacks on GPUs. The growth of web- minutes, e.g., reading a blog with malicious WebGPU code. We
based API usage to offer GPU-enhanced rendering inadvertently do not assume that WebGPU provides any interface for hardware
enables attackers to run GPU-based attacks through JavaScript, by- timers. To further constrain our attacker, we assume that WebGPU
passing its existing limitations. To our knowledge, all existing works provides no workgroup memory in reaction to prior work [35]. In
exploit the GPU through the WebGL API. Frigo et al. [13] leverage this paper, we attack dedicated NVIDIA and AMD GPUs, whereas
the integrated GPU to mount Rowhammer attacks from browsers some other works [13] have focused on integrated mobile GPUs.
on mobile devices, using the WebGL timing APIs. In response, major
browsers disabled this timer. Cronin et al. [15] presented a browser- 4 WEBGPU PRIMITIVES
based attack with assumptions similar to ours. They target SoC To build advanced cache attacks in WebGPU, we need several key
platforms and leverage system-level cache occupancy to build a primitives. The first is a timer accurate enough to reliably distin-
covert channel and fingerprint websites. They differ from our work guish a cache hit from a miss. Using this timer, we can then detect
Generic and Automated Drive-by GPU Cache Attacks from the Browser ASIA CCS 2024, July 1–5, 2024, Singapore
Table 1: Timing thread counter value for the 98th and 5th per- copies of the same data in their L1 caches. Maintaining a coherent
centile for L2 cache hits and misses, respectively, for a variety state by synchronizing the data is left to the developers. Therefore,
of GPUs and the methods add and store. A good threshold without coherency, our counting thread would increment the timer
can be found when the distributions are clearly separable. in its private L1 cache, unobservable from the outside.
𝑛 = 1 000 000 hits and misses were recorded each. C3. Optimization. The WGSL compiler aggressively optimizes
the code, such that a counting while loop may be replaced with
Add Store
the final result, and memory accesses may be replaced by registers.
GPU hit>98% miss<5% hit>98% miss<5% Solutions. In their OpenCL implementation, Dutta et al. [35] solve
RX 6800 XT 6 7 9 11 the first challenge by executing enough counting threads to fill one
AMD
RX 6900 XT 5 7 9 11 or more warps. Then, they conduct the attack in a separate warp
GTX 1070 5 8 62 95 within the same SM, so each warp only executes the same branches,
GTX 1650 7 13 75 94 avoiding warp divergence. To address the other challenges, they
GTX 1660 Ti 7 11 74 94 simply store the counter in a shared memory region available to all
GTX 1660 Ti Lin 4 7 10 18 threads in the same workgroup.
RTX 2070 SUPER 6 8 80 106 In line with our goal to get a portable and low-assumption attack,
NVIDIA
6 5
⋅10 ⋅10
1 8
hit (add) hit (store) hit (add) hit (store)
0.8 6
miss (add) miss (store) miss (add) miss (store)
samples
samples
0.6
4
0.4
0.2 2
0 0
4 6 8 10 12 14 16 18 20 0 22 60 95 119 160
timer count timer count
(a) AMD RX 6800 XT (b) NVIDIA RTX 3080
Figure 2: WebGPU cache hit and cache miss histograms for different GPUs with counting thread for 1 million samples. Adding
to a memory location provides less resolution than storing a register value. Higher counts show higher timer resolution.
parameters as possible. An important parameter for all further Table 2: Our WebGPU cache-size finding algorithm on a vari-
sections is the cache size. It determines how many sets we can ety of GPUs, 𝑛 = 10. With one exception, the correct size is
expect (Section 4.3) and lets us derive suitable buffer sizes for cache almost always found on all cards.
eviction detections (Section 4.4).
Size Runtime
We assume standard LRU, as suggested in previous work [40],
and fill the cache with a large array of 10 MB. The buffer size choice Actual Detected Correct 𝜇 𝑥¯ 𝜎
is motivated by our observation that most GPUs have below 8 MB GPU MB MB % ms ms
of L2 cache. In the same shader execution, we now iterate over the AMD RX 6800 XT 4.0 4.0 100 179.3 19.21
array forward and then backward, counting hits. Going backward RX 6900 XT 4.0 4.0 100 185.6 26.20
avoids self-eviction of an entire set after a single miss and allows GTX 1070 2.0 2.0 100 192.6 26.15
us to accurately measure the number of cache lines that remain in GTX 1650 1.0 1.0 100 422.2 31.56
the cache. If the hit rate is very high (> 95 %), we increase the test GTX 1660 Ti 1.5 1.5 100 283.6 11.02
size in steps to 40 MB, 80 MB, and 100 MB. This keeps measurement 4.0 4.0 100 189.9 6.15
NVIDIA
times low for most cards, while allowing accurate detection even RTX 3060 Mobile 3.0 2.975 90 285.3 15.03
for larger caches. Finally, we match this approximate size to the RTX 3060 Ti 4.0 2.975 0 276.8 9.50
closest larger size within a list of known sizes. RTX 3080 5.0 5.0 100 257.4 9.81
Table 2 shows that for most cards, we can reliably determine RTX 4090 72.0 72.0 100 1729.6 60.23
Quadro P620 1.0 1.0 100 251.7 23.25
the cache size in less than 400 ms. Of interest among the outliers
is the NVIDIA RTX 3060 Ti. It reliably returns a size of 3 MB, and
indeed, we never see any hits more than exactly 3 MB, though the
official L2 cache size is 4 MB. A simple explanation is that both our AMD or even mobile GPUs may follow an entirely different scheme
NVIDIA RTX 3060 Ti models, only have 3 MB of L2 cache. Another altogether. Like the work by Dutta et al. [16], we also do not have
possibility is that these cards have a different mapping function, the advantage of relying on physically contiguous memory, or any
and some part of their cache is only reachable for much larger total specific page size.
VRAM allocations. We will encounter this again in Section 4.3. In keeping with our generic approach, we do not attempt to rely
on any known mapping functions or page sizes. Instead, we employ
a generic set-finding algorithm based on prior work for CPU caches.
4.3 L2 Cache Eviction Set Construction Given a timer accurate enough to distinguish cache hits and misses,
The next step in building a Prime+Probe attack is to find a set of an attacker should be able to create eviction sets efficiently, similar
addresses that map to the same cache set. To make sure this set to the methods presented by Qureshi and Purnal et al. [47, 48].
of addresses replaces all the current entries in the cache set, the While this approach works well for CPUs, we encountered vari-
cardinality of the set should at least match the cache associativity ous challenges to efficiently port it to GPUs. Hereafter, we describe
𝑊 . We call this an eviction set. On CPUs, much work has been done a novel approach to compute fast and reliable eviction sets on GPUs.
to reverse engineer the mapping from virtual-to-physical addresses In particular, we describe how to leverage the powerful parallelism
to cache sets [41–43]. Comparable work on GPUs [16, 40, 44–46] that GPUs offer to speed up this process.
however has shown that their cache set mapping can be much The basis of our implementation is the Group-Elimination Method
more complex. Jain et al. [40] used a modified driver to reverse- (GEM) [47]. The goal of GEM is to find an eviction set for a target
engineer the hash functions for mapping addresses to both cache address. To this end, a large set of addresses S >> 𝑊 that evicts the
and VRAM on an NVIDIA GTX 1070 and 1080. However, our tests target address is partitioned into 𝑊 + 1 groups. As a full eviction
suggest these functions differ in newer generations of NVIDIA set of 𝑊 addresses must be contained in some combination of ≤ 𝑊
GPUs. In particular, many GPUs need to employ different non- out of the 𝑊 groups, (at least) one group can be eliminated without
linear (or linear, but different by address range) mapping functions affecting the eviction. GEM tries to remove each of the 𝑊 +1 groups
due to their non-power-of-two cache and VRAM sizes. Additionally, from the set S until one is found that does not influence the eviction
Generic and Automated Drive-by GPU Cache Attacks from the Browser ASIA CCS 2024, July 1–5, 2024, Singapore
1 𝒮 ← {1.5x cacheSize, 128B steps} The goal of preprocessing is to separate an initially large set
2
3
Buckets ← {{}}
while 𝒮 != {}
S of addresses 𝑠𝑖 = ∣𝑆∣ (1.5x the cache size in 128 B steps) into
4 ℬ ← 𝒮, 𝒫 ← ℬ[0] 1 //initialize B, select a pivot P buckets with no overlapping sets. This partitioning facilitates the
5 while |ℬ| > targetSize independent examination of each bucket for sets, circumventing
6
inter-thread interference. The process follows a similar approach
𝒢 ← {}
7 do
8 shuffle(ℬ) as GEM and is delineated in two main steps.
9 ℬ ← ℬ ∪ 𝒢, 𝒢 ← ℬ[0:1/2𝑊 ∣ℬ∣], ℬ ← ℬ\{𝒫 ∪ 𝒢} Starting with B = S, the first ( 1 ) step involves selecting a random
10 access(𝒫), parallelAccess(ℬ) //access pivot, then B
11 while isCached(𝒫) 2 element from the set as our pivot. This pivot address guarantees the
12 if optCondition() 2b presence of at least one complete set within the bucket, although
hits ← accessAndMeasure({ℬ ∪ 𝒫})
13
14 ℬ ← ℬ\hits //remove addresses not part of eviction sets
it probably contains several more. The second ( 2 ) step consists
15 ℬ ← ℬ ∪ 𝒫, 𝒮 ← 𝒮\ℬ in removing a portion of the set, i.e., a group, and verifying if the
16 Buckets ← Buckets ∪ {ℬ} pivot element is still evicted by the residual B. If eviction is not
Listing 2: Parallel Set Construction. This simplified pseudo- observed, we reiterate with another group. Contrary to GEM, we
code algorithm partitions an initial set of addresses S into find that eliminating 1/2𝑊 ∣𝐵∣ rather than 1/𝑊 + 1 better mitigates
several buckets B whose addresses do not share cache sets. the excessive removal of elements in later steps ( 2b ).
We also incorporate several optimizations not found in GEM.
Until B diminishes to 3/4𝑠𝑖 , we exploit parallelism by accessing all set
1 Buckets = ParallelSetConstruction() elements concurrently using 30 threads, measuring only the pivot
2 originalBuckets ← Buckets, EvictionSets ← {} at the end. When ∣B ∣ < 1/6𝑠𝑖 or on every fourth iteration when
3
∣B ∣ < 3/4𝑠𝑖 (optCondition is met, Line 12), we measure not just
Pivots ← {ℬ[0] | ℬ ∈ Buckets)} A
4 Buckets ← {{ℬ\𝒫} | (ℬ,𝒫) ∈ (Buckets,Pivots)}
5 while ∃ℬ : ℬ ≠ {} the pivot but all other elements . We only do this sparingly because
6 Gs ← {{}} measuring elements in addition to accessing them has a significant
7 foreach ℬ ∈ Buckets : ℬ ≠ {}
8 if |ℬ| > 1000 overhead, and, as mentioned, LRU-related observations can’t be
9 𝒢 ← ℬ[0:1/2𝑊 ∣ℬ∣] B parallelized while sets are still unknown. However, this enables a
10
11
else
𝒢 ← ℬ[0] B2
crucial optimization: the removal of all set elements that register
12 Gs ← Gs ∪ {𝒢}, ℬ ← ℬ\𝒢 a cache hit ( 2b ). Given a consistent access sequence and a cache
13 AllHits = parallelMeasure(Pivots, Buckets) C replacement policy approximating LRU, all persisting elements in
14
B are now part of full eviction sets.
foreach (𝒫,Hits,ℬ,𝒢) ∈ (Pivots,AllHits,Buckets,Gs)
15 if isCached(𝒫)
16 ℬ ← ℬ ∪ 𝒢 This procedure can be iterated until B is below a predetermined
17 shuffle(B) threshold. Empirical evaluations suggest a bucket size of 3500
18 else
19 ℬ ← ℬ\Hits D (equivalent to 145 − 206 sets) works for the majority of GPUs.
20 if |ℬ| == 𝑊 //bucket has reduced down to one set Upon completion, B is subtracted from S. We are left with a bucket
EvictionSets ← EvictionSets ∪ {ℬ ∪ 𝒫}
21
22 ℬ = originalBucket\{ℬ ∪ 𝒫} //refill Bucket
of the desired size, exclusively containing eviction sets. The residual
23 shuffle(ℬ) segment of S does not include overlapping sets with the bucket.
24 𝒫 ← ℬ[0], ℬ ← ℬ\𝒫 A Repeating the previous steps ensures that the final buckets consist
25
only of non-overlapping eviction sets, collectively representing
else if |𝒢| == 1 && |Hits| == 𝑊 E
26 EvictionSets ← EvictionSets ∪ {𝒢 ∪ ℬ} //free set!
nearly all cache sets.
Listing 3: Parallel Bucket Sifting. This algorithm sifts sets in Parallel Bucket Sifting. With full eviction sets sorted into roughly
parallel from the previously separated buckets. equal buckets, we can now begin to extract single sets from them.
Since there is no more overlap between the cache sets in the buckets,
we can now run measurements on them in parallel. For the NVIDIA
of the target. This is repeated until only 𝑊 addresses remain in S, RTX 3080 and its 5 MB cache for example, the previously mentioned
forming an eviction set for the target address. target bucket size produces around 16 buckets of 160 sets each, with
Our implementation differs from GEM in two significant ways. up to 24 addresses per set. This means we can start a loop on our 16
First, we aim to find all sets, and we, therefore, try to find more input buckets with the following broad steps running in parallel for
than one eviction set at a time. Similar to Prime+Prune+Probe [48], each bucket. First, we once again shuffle the elements in each bucket
we make use of the predictable behavior of LRU for this. Second, B and pick a pivot element to find an eviction set for ( A ). Second,
we parallelize parts of the algorithm to multiple threads. Many like before, we remove groups of 1/2𝑊 ∣𝐵∣ elements until we find
constants in the following are empirically determined values that one that doesn’t affect the pivot’s eviction ( B ). Third, to determine
work on a variety of GPUs, not optimal values. eviction, we measure the access latency for all addresses remaining
Parallel Set Construction. See Listing 2. When we access many in all buckets in parallel. ( C ) Fourth, addresses in buckets that
addresses in parallel on the GPU (or the CPU), ordering between show cache hits are also removed, such that all remaining addresses
them is not guaranteed. This means that when a set is split between still form eviction sets within B ( D ). Buckets are shrunk in parallel
different threads, we can no longer expect to observe effects stem- this way until a bucket’s size goes below 1000 elements. At this
ming from LRU. In effect, eviction measurements that rely on access point, instead of 1/2𝑊 ∣𝐵∣, we remove only a single element per loop
order become meaningless. We, therefore, add a preprocessing step ( B2 ). This allows us to make use of the cascading eviction effect of
to the eviction-set construction algorithm.
ASIA CCS 2024, July 1–5, 2024, Singapore Giner et al.
the LRU replacement policy: when we remove only one address, Table 3: Our WebGPU set-construction algorithm on a variety
that together with 𝑊 other addresses in B forms an eviction set, of GPUs, 𝑛 = 10. All but one card reliably find > 80% of sets.
those 𝑊 addresses will now show up as cache hits ( E ). In effect, we
Sets Runtime
have found an entire eviction set in a large bucket B by removing a
single element. This allows us to sift out many sets for “free” while Overall Found 𝑥¯ Found 𝜎 𝑥¯ 𝜎
trimming the bucket to find the pivot element’s eviction set. We GPU % % min min
continue decreasing the bucket size until either a complete eviction GTX 1070 1024 96.0 2.1 11.8 4.8
set for the pivot remains, or some false measurement has left us GTX 1650 512 82.9 2.2 4.2 3.9
with an incomplete set. At this point, we refill the bucket with all GTX 1660 Ti 768 96.4 2.1 12.1 3.8
NVIDIA
discarded addresses that could not be attributed to a complete set RTX 2070 SUPER 2048 98.7 1.0 7.0 2.1
and start again at step one. The algorithm terminates when either RTX 3060 Mobile 1536 99.9 0.1 2.3 0.4
all buckets are empty, or no new sets have been found for too long. RTX 3060 Ti 1536 94.5 5.3 2.6 1.5
This sifting method is so effective, in fact, that it finds signifi- RTX 3080 2560 99.3 1.9 2.8 1.2
Quadro P620 512 50.8 24.5 13.7 9.0
cantly more sets than the number of pivots chosen. On our NVIDIA
RTX 3080, for example, we might search 17 buckets for eviction sets
with 90 chosen pivots (an average of 5.2 bucket “refills”), but sift be used as a side channel to user activity. We describe an inter-
out 2465 sets on the way. The change at 1000 elements represents keystroke timing attack based on this primitive in Section 5.
an empirically found trade-off between fast bucket-shrinking and a
high amount of sets found through sifting. When the number is too 5 KEYSTROKE MONITORING WITH FULL
high, the time to find sets will needlessly increase, as most sets start
with an average of 24 addresses, but can only be detected when just
EVICTION
16 are left in the bucket. When it is too low, many sets are lost to Starting from the observation that drawing elements on screen
the sifting method through the removal of many elements. evicts a significant part of the cache, we build an attack that can
Combining these optimizations, we can map most sets in the L2 record inter-keystroke timings by observing cache contention. As
cache of all NVIDIA GPUs in WebGPU in a reasonable time frame, prior work has shown [49–52], inter-keystroke timings carry a sig-
as shown in Table 3. The notable exception is the NVIDIA RTX nificant amount of information and can lead to password recovery.
4090, as the enormous cache size presented problems not found in While subsequent sections of this paper present conventional
other cards. Likewise, both AMD cards fail this important step to benchmarks for high-frequency side channels, this section focuses
further attacks and are therefore not included in the more advanced on low-frequency benchmarking. Despite the infrequent occurrence
attacks. One possible explanation is that the timing difference to of events, achieving a high detection rate is crucial for accurately
other cards, which can already be seen in Table 1, causes more measuring inter-keystroke timings. In addition, keystroke profiling
noise, as the hit and miss distributions are closer together. While represents a practical application of our attack, as our setup mirrors
we believe that from the basic timing difference, it is clear that all the most prevalent end-user scenario: a computer equipped with
our attacks could run on these cards, we only had temporary and a single discrete GPU engaged in internet browsing. As the full
time-restricted remote access to these GPUs, which did not allow WebGPU standard becomes increasingly integrated into mobile
for analyzing the underlying problem. The NVIDIA RTX 3060 Ti devices, this scenario will gain further relevance in the future. Our
also sticks out, as it consistently finds close to 1536 sets even when approach for this attack is similar to Naghibijouybari et al. [12].
looking for 2048. This is consistent with 3 MB of L2 cache found in
our experiments (see Section 4.2). 5.1 Construction
We see that the percentage of sets we find varies along with the The attack is based on the following observation: for each character
time, though a majority of sets can almost always be found within typed, the text box is re-rendered. We can measure this as the
5 minutes. With this additional primitive, attackers can implement eviction of a certain amount of the cache, up to the entire cache,
Prime+Probe to build a covert channel, as we show in Section 7, or correlated with the size of the rendered area. To see this effect,
execute some other cache attack, e.g., Rowhammer [13]. we use a buffer that covers a part of the cache size and repeatedly
measure its hit rate. Whenever we see a hit rate below a well-chosen
threshold, e.g., 50 %, we record the timestamp as an event. The time
resolution of this attack is determined by how fast our attacking
4.4 Full Cache Evictions shader can complete its measurement, which is determined by
One of the first observations while measuring cache hit rates on the total buffer size. Though on some GPUs we see that a small
GPUs is that some events evict a sizeable portion of the cache. percentage of the cache is already enough to observe keystrokes,
Whenever an element on the screen is redrawn or the frame buffer we find that for most, 35 % is a good tradeoff between detection and
is refreshed for some other reason, this occupies a large part of speed. The screen resolution, size of the text box and zoom level all
the cache. Depending on the total size of the L2 cache and what is contribute to the amount of evicted cache lines.
being drawn, this may even evict the entire cache. On the one hand, After recording raw traces, we filter based on two observations.
this presents as noise during some attacks; each measurement that First, very close measurements (<25 ms difference) are unlikely to
happens after a draw event is tainted. On the other hand, these be separate keystroke events. Second, after a short break in typing,
evictions are indicators of activity on screen and can therefore the cursor starts blinking at a 530 ms interval on Windows. Filtering
Generic and Automated Drive-by GPU Cache Attacks from the Browser ASIA CCS 2024, July 1–5, 2024, Singapore
these sources of noise removes most false positives. Figure 3 shows last-round attack, we assume the attacker has access to the victim’s
the trace of an attacker typing at varying speeds compared to the ciphertexts, but not the plaintext or the key.
ground truth on our NVIDIA RTX 3080. We can see that while we
measure some spurious events, most timings are accurate. 6.2 AES Implementation Details
The native encryption service is an AES CUDA implementation,
5.2 Evaluation which uses combined T-tables for all rounds. This increases the dif-
We tested this attack with a small text box and generated input ficulty of the attack compared to implementations that use separate
directly injected from javascript, randomly drawing inter-keystroke tables for the last round, as all other rounds influence cache hits on
timings from distributions similar to the patterns observed by table entries. As GPU cache line width is usually 128B, and each
Song et al. [49]. Table 4 shows the tested GPUs and their 𝐹 1 scores table is composed of 256 4-byte entries, each table fits in exactly 8
and inter-keystroke timing errors. During this test, no other visual cache lines, for a total of 32 cache lines filled with table entries.
noise was present, similar to the static login pages of many websites.
The consistently high recall shows that virtually no keystrokes are 6.3 Attack Methodology
missed on most cards. However, even after filtering, the recall shows Our strategy, akin to the keystroke attack (Section 5), involves allo-
that there is a low average of false positives for most cards. AMD cating a large buffer to occupy a significant cache portion, executing
once again behaves differently. Despite the high recall, with the an AES encryption, and then identifying evicted buffer offsets using
low precision, we can consider this attack mostly failed or severely Prime+Probe. In an ideal scenario with a minimalistic AES kernel,
degraded. The results suggest either a high level of noise or, more evicted offsets would correlate with the table or the encryption’s
likely, frequent misclassification of hits as misses due to the close inputs and outputs. This is because, from our observation, GPUs
timing differences visible in Table 1. implement a deterministic LRU-like eviction policy. This means that
An interesting example for the timing resolution is the NVIDIA when one address from a full set is evicted, measuring all others in
RTX 4090. Because of its large L2 cache of 72 MB, simply measuring the same order used to place them in the set will cause a cascade of
cache contention requires a disproportionately large measurement cache misses, as each new access will result in a miss, overwriting
set. This is because the cache footprint of a text box does not in- the next address. In practice, we find that kernel loading introduces
crease with the cache size. While all other cards easily reach a substantial cache occupancy, leading to measurement noise. Our
sampling rate below 15 ms, the huge buffer means that each mea- primary challenge is discerning the tables amidst this noise, and
surement takes more than 200 ms, making an inter-keystroke timing profiling each table’s cache lines to track their access. We employ
attack with this method impractical. chosen keys and specially crafted plaintexts to deduce the relation-
We also observe that on Windows, the blinking of the cursor ship between our offsets and table entries. With this mapping, we
causes slightly less eviction than a typed character. One possible can execute the traditional last-round attack [53, 54]. Hereafter, we
explanation is that instead of re-rendering the entire text box, the delve into each step and the optimizations we employed to achieve
cursor is drawn on top. a reliable key recovery in a drive-by manner.
Profiling T-Tables. The initial profiling phase involves allocating a
6 AES KEY RECOVERY WITH sizable array in the browser, ensuring kernel loading and encryption
BROWSER-BASED TEMPLATING evict specific offsets. This step templates the AES encryption’s
Recovering AES keys from vulnerable T-table implementations has memory accesses. Optimal array size varies across models, even
become a benchmark for assessing how fine-grained side-channel with similar cache sizes.
attacks and microarchitectural attacks are. This case study has also Using random plaintexts, we would expect a random distribu-
been adopted in GPUs [6–9]. Additionally, AES has been proposed tion of the access to each entry of the tables, thereby causing a
as a use case for general-purpose GPU computing since 2007 [4, 5]. predictable eviction frequency of the set-congruent offsets (at a fre-
Unlike previous research, our method adopts a set-agnostic ap- quency of 0.995). On the contrary, the offsets that are set-congruent
proach, eliminating the need to understand cache set sizes or map- to the memory required for kernel loading would be evicted every
pings. Traditional set-based strategies would require extensive pro- time, and other noise artifacts should be sparse. Our differential ac-
filing of cache sets and mapping of T-table accesses. Our method- cess templating, using a fixed key and chosen plaintexts, enhances
ology bypasses this initial step, focusing on locating addresses profiling reliability and efficiency.
congruent with T-table lines. The strategy involves pre-generating 32 plaintexts 𝑝𝑖 with a fixed
key, ensuring the encryption of each plaintext access all but one
cache line within the tables, and a reference plaintext 𝑝𝑟 , which
6.1 Threat Model encryption accesses all cache lines. Comparing memory accesses
Like earlier, we assume our attacker embeds some malicious JavaScript during the encryption of 𝑝𝑖 and 𝑝𝑟 reveals offsets congruent to
in a webpage the victim is browsing for several minutes. The vic- cache line 𝑖. This process identifies offsets that are set congruent to
tim runs a GPU-based AES implementation that can be queried each cache line, though some cache lines may remain undetected
for encryption with a chosen plaintext and key. The attacker aims due to kernel loading noise. This refined offset list streamlines the
to recover the AES key used by a victim. This scenario could be attack, focusing on a reduced subset of offsets.
found in the case of an SFTP server, where the chosen plaintext and Last Round Attack. As we are performing a last-round attack,
key represent downloading our own file. In order to implement a the only requirement is that we can make measurements during
ASIA CCS 2024, July 1–5, 2024, Singapore Giner et al.
Table 4: Efficacy of WebGPU inter-keystroke timing detection on a variety of GPUs for 100 keystrokes.
Table 5: Evaluation of the AES Last Round Attack (LRA) on needed are 9300 and 9800, respectively. The uniformity of results
two NVIDIA cards. All values are average across 𝑛 = 50 runs. across GPUs, coupled with the low standard deviation, underscores
the stability and reproducibility of our attack.
GPU Measurements Time (min)
The profiling of the T-table with the templating attack takes on
(x1000) Profiling LRA Total
average 13 s. The variability in this phase predominantly stems from
RTX 3060 Mobile 9.3±1.6 0.23±0.1 5.7±0.9 6.0±1.0 the inconsistent repetition of profiling until an optimal buffer size
RTX 3060 Ti 9.8±3.4 0.23±0.1 5.9±1.9 6.1±2.0 is identified, enabling sufficient eviction observation. Typically, a
single profiling session lasts 6 s. Once profiling is complete, the same
session can be repurposed to divulge multiple AES keys, thereby
encryptions by the victim, and observe the ciphertext. The attack reducing the attack duration to the sample collection time needed
aligns with the non-elimination method presented by Neve and in the concluding step.
Seifert [55]. The idea is, given a collection of ciphertexts and the Profiling often does not provide a complete mapping for every
access to T-tables entries that happened during the encryptions, cache line. The disparities in measurements and time allocated for
to remove possible values on the key bytes by looking for cache the last-round attacks correlate directly with the number of cache
lines that were not accessed in the process. For each cache line not lines we can monitor. On average, we can spy on 20/32 cache lines.
accessed during the encryption, we can remove all last-round key The NVIDIA RTX 3060 Ti exhibits marginally less consistent results,
bytes that would have resulted in a memory access during the last occasionally mapping fewer cache lines, leading to an elongated
round, based on the ciphertext value. Given the cache line size of attack duration and increased standard deviation. Our evaluation
4
GPUs, we get up to 2 bit of information on the last-round key on both cards consistently had a 100 % success rate.
every time a cache line is not accessed.
The more cache lines we can monitor, the more likely we are to 7 A PRIME+PROBE COVERT CHANNEL
reduce the search space for the last-round key. Once we get below
40 A covert channel is a channel that is constructed on top of some
2 candidates, we switch to an exhaustive search of the key. shared resource that is not meant for data transmission. This allows
an attacker to transmit data between two domains that should be
6.4 Evaluation isolated or strictly monitored. Because both sender and receiver
For our evaluations, we focus on a CUDA-based target implemen- work together to transmit data, covert channels are a valuable
tation, rendering evaluations on AMD cards infeasible. Somewhat benchmark for any side channel’s bandwidth. In a traditional Prime+
breaking with the theme of this work, the nature of this attack Probe cache covert channel, the sender transmits bits by priming
necessitates some parameter adjustments, which adds complexity (evicting) cache sets to transmit a binary 1, which the receiver can
and extends the evaluation duration compared to other attacks. later detect by probing (measuring) its own lines in the same set.
Therefore, we settle on evaluating our attack on two recent cards: With our reliable timer and a method to find the required eviction
NVIDIA RTX 3060 Ti and NVIDIA RTX 3060 Mobile. All our exper- sets (see Section 4), we can now construct a Prime+Probe cache
iments are conducted on Ubuntu 22.04, using Chromium 117. We covert channel for the L2 cache.
also observe consistent and similar results across multiple Chrome The sender is a C++ application that uses CUDA. In this scenario,
versions, ranging from 112 to 115. it is a malicious application without network privileges but with
Table 5 showcases our findings. It highlights the average duration access to the GPU. The sender’s goal is to exfiltrate sensitive data via
of the attack’s primary steps and the mean number of encryptions a GPU covert channel. The receiver runs in a website the user visits
required for successful key recovery. Notably, both cards yield at the same time. This may be a legitimate website with injected
similar results, recovering the key in 6 min. The average encryptions malicious JavaScript, or a website the user is led to in some way.
Generic and Automated Drive-by GPU Cache Attacks from the Browser ASIA CCS 2024, July 1–5, 2024, Singapore
2,000
raw detections inferred keystrokes true keystrokes
Δ𝑡 [ms]
1,000
0
5,630 6,310 7,080 7,950 8,920 10,000 11,200 12,600 14,100
time [ms]
Figure 3: Inter-keystroke timing recovery on a NVIDIA RTX 3080. The raw activity detections (green) show prominent cursor
blinking that can be filtered out very well, which leaves an accurate trace of inter-keystroke timings (red).
7.1 Construction window length of 5 ms for our packets. This length is limited not
We write the Sender 𝒮 in C++ and CUDA, making full use of only by the accuracy of the timer but also by the time it takes for a
the native high-resolution timer. The browser-based Receiver ℛ shader to run. To compensate for the long packet duration, we use
uses a combination of JavaScript and WGSL. Using our eviction set the GPU’s parallelism to transmit on 1024 sets concurrently.
construction (see Section 4.3), ℛ starts by mapping all cache sets. While eviction for 𝒮 is as simple as accessing many addresses in
Setup - CJAG. As neither ℛ nor 𝒮 have absolute labels for their parallel in a loop, ℛ still needs to measure time. Challenge C1 (see
respective eviction sets, the first step is to communicate the shared Section 4.1) means that we need to separate each parallel thread into
sets from 𝒮 to ℛ. For this, we implement a GPU-friendly version different workgroups to prevent lockstep execution. Additionally,
of the cache jamming agreement (CJAG) proposed by Maurice et al. individual sets always need to be measured by the same thread, as
[56]. In CJAG, 𝒮 alternates between jamming a set, i.e., evicting ordering between these accesses is crucial for the eviction policy.
it continuously for some time, and probing the set for a slightly As observed in Section 4.4, GUI-related events can introduce
longer period. Meanwhile, ℛ probes all sets continuously until the undesirable noise. Similarly, the operating system can deschedule
jammed set is detected. Then, ℛ switches to a longer period of 𝒮 for periods of time. To reduce such noise, we adopt the following
jamming, so that 𝒮 knows the set has been received and moves on. strategies. First, we employ a majority vote measurement approach
Unlike the CJAG approach on the CPU, distinct shaders do not where each set is measured as often as possible within a transmis-
execute concurrently on the GPU, making simultaneous detection sion window. By counting evictions and non-evictions, we obtain
and jamming infeasible. Rather than employing shaders that con- the result through a majority vote. Second, we access the addresses
tinuously loop through jamming or detection, we need to segment within each set in an alternating order. This ensures a consistent
them into single invocations. Depending on the frequency of driver read from the most to the least-recently-used cache line, precluding
interruptions, we might otherwise see long shader executions that the cascade of self-evictions that would arise if the oldest cache
rarely interface with each other. line were evicted. This lets us determine how much of a set was
Additionally, we want to use a large number of sets (e.g., 1024). evicted and easily identify low-level noise. Lastly, we use a dif-
Serial transmission, as implemented in CJAG, is, therefore, imprac- ferential measurement scheme. Here, a pair of sets transmit 1 bit,
tical. Instead, we enhance the CJAG framework by leveraging the and measurements wherein neither or both sets are evicted are
inherent parallelism of our GPU, enabling both 𝒮 and ℛ to jam and discarded. In a valid transmission, precisely one set is evicted for
detect all sets concurrently. Here, copying to and from shaders is the every pair, effectively halving our transmission rate and resulting
main bottleneck, with 3 ms on average. Thus, the time difference in a total packet length of 64 B. Consequently, the raw transmission
between measuring a single set versus 64 sets per shader invocation speed is fixed by the parameters to a default of 12.8 kB/s.
is marginal. So, we combine both as a trade-off and measure sets in
parallel on 16 threads. At this stage, 𝒮 also swaps out any sets that
are not detected from ℛ’s jamming, thus ensuring that all sets are
7.2 Evaluation
fully functioning for both parties. After a selection of 1024 sets has We evaluate the covert channel on 3 NVIDIA GPUs; the RTX 2070
been communicated, 𝒮 switches to jamming on only half of all sets. SUPER, 3060 Ti and 3080. The GTX 1070 and Quadro P620’s Pascal
The specific half is dictated by the current bit in the index number architecture does not support all the instructions used by the CUDA
of its cache sets. In this way, 𝒮 can transmit the order of all 1024 sender. AMD cards do not support CUDA, though as set-finding fails
sets in log2 (1024) = 10 steps by jamming different 512 sets for each on AMD (see Section 4.3) the attack would not work either way. The
bit. After all sets have been communicated, data transmission can GTX 1650 and GTX 1660 Ti both support the instructions as well
begin. Table 6 shows that it takes 14 s to 28 s to transmit 1024 sets. as set-finding, but we could not reliably establish communication
Transmission. After the set jamming agreement has been com- because of malfunctioning jamming detection in CUDA.
pleted, the transmission is entirely one-way. We opt for a channel Table 6 shows the configuration and transmission details for
design where sender and receiver are synchronized with the wall all tested devices. We can see that as we shrink the transmission
clock. In native C++, this provides at least µs accuracy, while in window, average reads in the window go down, and the error rate
browsers, this is limited to 100 µs. We choose a default transmission increases. At 4 ms, the NVIDIA RTX 3080 shows a decrease in
true channel capacity compared to 5 ms for this reason. Because
ASIA CCS 2024, July 1–5, 2024, Singapore Giner et al.
Table 6: Transmission results of our Prime+Probe covert channel from a native CUDA sender to a WebGPU receiver in the
browser, 𝑛 = 10. True channel capacity can vary widely either due to general noise, incorrect set transmission during CJAG or
too few correct reads per window, i.e., the number of correctly received pairs within the transmission window.
RTX 3060 Ti 1024 6.0 2.9 16.4 10 666.7 9004.9 271.4 2.3 0.5
1024 5.0 1.9 15.7 12 800.0 7272.0 1252.5 9.1 3.1
RTX 3080 1024 5.0 2.7 27.8 12 800.0 10 897.5 698.3 2.2 1.0
1024 4.0 1.9 28.2 16 000.0 5964.5 1048.6 15.9 2.7
of it’s higher clock speed, the 3080 supports faster transmission requires permission before use. For WebGL and WebGPU, this is
than the two other cards. Its average true bandwidth in its fastest not currently the case (Firefox 114, Chrome 115, Chromium 117).
configuration is, therefore, 10.9 kB/s, at a BER of 2.2 %. Though This would also prevent malicious parties from stealthily using
our channel is non-optimal and slower than prior work, it clearly local computing resources for, e.g., cryptomining.
demonstrates the viability of using WGSL code embedded in a Disclosure. We have disclosed our results to Mozilla, AMD, NVIDIA
website as a covert channel receiver. and the Chromium team.
8 DISCUSSION 9 CONCLUSION
Supported Devices. Our research primarily targets recent NVIDIA
GPUs have become a ubiquitous computation resource and as such
GPUs, leading to worse results on AMD cards, as we only had very
require increased security scrutiny. We showed that it is possible
limited access. Despite these architectural differences, WebGPU
to mount powerful GPU cache side-channel attacks directly from
clearly enables generic cache attacks from browsers. At the time
within the browser. We demonstrated that our basic attack prim-
of writing, WebGPU is already integrated into Android’s Chrome
itives are generic and automated to the extent that we can run
Canary, though some features are not yet available. Once parity
them without manual intervention on a set of 11 desktop GPUs
is achieved, the potential for browser-based GPU attacks could
from 5 different generations and 2 vendors, running in the browser
significantly increase.
through WebGPU. We showed that the massive parallelism of mod-
Limitations. We evaluated our proof-of-concept on various op-
ern GPUs can be leveraged in parallelized eviction set construction
erating systems using Chrome and Chromium versions 112-117.
algorithms. Our three case studies emphasized the relevance of our
Despite identifying functional combinations for all devices, the We-
work: Our inter-keystroke timing attack, with F1 -scores between
bGPU implementation remains inconsistent, as evidenced by our
82 % and 98 %, exposes sensitive user input to an attacker. Our set-
experiments. The same code might succeed in one version and unex-
agnostic end-to-end attack on GPU-based AES encryption leaks
pectedly fail in another, potentially due to variations in WebGPU’s
full AES keys in 6 min, showing that also cryptographic secrets are
code compilation beyond user control. We observed notable differ-
exposed to browser-based attackers. Our native-to-browser Prime+
ences between Linux and Windows (see Table 1). While the exact
Probe covert channel shows that the bandwidth of this channel
cause—whether driver, browser, or WebGPU’s underlying frame-
can reach average transmission rates of up to 10.9 kB/s. Since our
work (e.g., Vulkan vs. DirectX)—remains unclear, the fundamental
attacks require no user interaction, they can be implemented as
time discrepancy supports the viability of these attacks.
dangerous drive-by attacks. Thus, we conclude that GPU access
Countermeasures. The attacks shown in this paper are generic
should be treated as a similar security and privacy risk as other
and rely on only a few assumptions. Nevertheless, steps can be
devices and resources that require explicit user consent.
taken to limit the attack surface. As already suggested in the current
WebGPU draft, timers can be made optional, very coarse, or ideally
removed altogether [17]. However, as we have shown, as long
as coherent memory is available between concurrent threads, it is ACKNOWLEDGMENTS
possible to construct a timer. If, however, the coherency mechanism This research is supported in part by the European Research Council
(in our case, atomic operations) were to be changed, such a timer (ERC project FSSec 101076409), and the Austrian Science Fund (FWF
would quickly fail. Of course, this could cause normal workloads project NeRAM I-6054-N). Additional funding was provided by a
to malfunction unless specifically redesigned. generous gift from Red Hat. Any opinions or recommendations
The simplest and most effective solution against a drive-by attack expressed are those of the authors and do not necessarily reflect
scenario is, in our opinion, to treat GPU access in the browser the views of the funding parties. We also thank Gregor Heindl for
as a sensitive resource, like microphone or camera access, that his generous donation of time and hardware.
Generic and Automated Drive-by GPU Cache Attacks from the Browser ASIA CCS 2024, July 1–5, 2024, Singapore
REFERENCES [36] H. Taneja, J. Kim, J. J. Xu, S. van Schaik, D. Genkin, and Y. Yarom, “Hot Pixels:
[1] NVIDIA, “CUDA C++ Programming Guide,” 2023. Frequency, Power, and Temperature Attacks on GPUs and ARM SoCs,” in USENIX
[2] Khronos, “OpenCL,” 2023. [Online]. Available: [Link] Security, 2023.
[3] J. Wei, Y. Zhang, Z. Zhou, Z. Li, and M. A. A. Faruque, “Leaky DNN: Stealing [37] P. Vila, B. Köpf, and J. Morales, “Theory and Practice of Finding Eviction Sets,” in
Deep-Learning Model Secret with GPU Context-Switching Side-Channel,” in S&P, 2019.
DSN, 2020. [38] M. Lipp, D. Gruss, R. Spreitzer, C. Maurice, and S. Mangard, “ARMageddon: Cache
[4] C. Tezcan, “Optimization of advanced encryption standard on graphics processing Attacks on Mobile Devices,” in USENIX Security, 2016.
units,” IEEE Access, vol. 9, pp. 67 315–67 326, 2021. [39] W3C, “WebGPU Shading Language - Terminology and Concepts,” 2023. [Online].
[5] T. Yamanouchi, “GPU Gems 3 - AES Encryption and Decryption on the GPU,” Available: [Link]
2007. [Online]. Available: [Link] [40] S. Jain, I. Baek, S. Wang, and R. Rajkumar, “Fractional GPUs: Software-based
vi-gpu-computing/chapter-36-aes-encryption-and-decryption-gpu compute and memory bandwidth reservation for GPUs,” in IEEE Real-Time and
[6] J. Ahn, C. Jin, J. Kim, M. Rhu, Y. Fei, D. Kaeli, and J. Kim, “Trident: A hybrid Embedded Technology and Applications Symposium (RTAS), 2019.
correlation-collision GPU cache timing attack for AES key recovery,” in HPCA, [41] C. Maurice, N. Le Scouarnec, C. Neumann, O. Heen, and A. Francillon, “Reverse
2021. Engineering Intel Complex Addressing Using Performance Counters,” in RAID,
[7] Z. H. Jiang, Y. Fei, and D. Kaeli, “A complete key recovery timing attack on a 2015.
GPU,” in HPCA, 2016. [42] G. Irazoqui, T. Eisenbarth, and B. Sunar, “Systematic reverse engineering of cache
[8] ——, “A novel side-channel timing attack on GPUs,” in Proceedings of the on Great slice selection in Intel processors,” in Euromicro Conference on Digital System
Lakes Symposium on VLSI, 2017, pp. 167–172. Design, 2015.
[9] ——, “Exploiting bank conflict-based side-channel timing leakage of gpus,” ACM [43] Y. Yarom, Q. Ge, F. Liu, R. B. Lee, and G. Heiser, “Mapping the Intel Last-Level
TACO, 2019. Cache,” Cryptology ePrint Archive, Report 2015/905, 2015.
[10] H. Naghibijouybari, K. N. Khasawneh, and N. B. Abu-Ghazaleh, “Constructing [44] X. Mei and X. Chu, “Dissecting GPU memory hierarchy through microbench-
and characterizing covert channels on GPGPUs,” in MICRO, 2017. marking,” IEEE Transactions on Parallel and Distributed Systems, vol. 28, no. 1,
[11] H. Naghibijouybari, E. M. Koruyeh, and N. B. Abu-Ghazaleh, “Microarchitectural 2016.
Attacks in Heterogeneous Systems: A Survey,” ACM Comput. Surv., vol. 55, no. 7, [45] Z. Jia, M. Maggioni, J. Smith, and D. P. Scarpazza, “Dissecting the NVidia Turing
pp. 142:1–142:40, 2023. T4 GPU via microbenchmarking,” arXiv:1903.07486, 2019.
[12] H. Naghibijouybari, A. Neupane, Z. Qian, and N. Abu-Ghazaleh, “Rendered [46] Z. Jia, M. Maggioni, B. Staiger, and D. P. Scarpazza, “Dissecting the NVIDIA volta
Insecure: GPU Side Channel Attacks are Practical,” in CCS, 2018. GPU architecture via microbenchmarking,” arXiv:1804.06826, 2018.
[13] P. Frigo, C. Giuffrida, H. Bos, and K. Razavi, “Grand Pwning Unit: Accelerating [47] M. K. Qureshi, “New attacks and defense for encrypted-address cache,” in ISCA,
Microarchitectural Attacks with the GPU,” in S&P, 2018. 2019.
[14] S. van Schaik, A. Kwong, D. Genkin, and Y. Yarom, “SGAxe: How SGX fails in [48] A. Purnal, L. Giner, D. Gruss, and I. Verbauwhede, “Systematic Analysis of
practice,” 2020. Randomization-based Protected Cache Architectures,” in S&P, 2021.
[15] P. Cronin, X. Gao, H. Wang, and C. Cotton, “An Exploration of ARM System-Level [49] D. X. Song, D. Wagner, and X. Tian, “Timing Analysis of Keystrokes and Timing
Cache and GPU Side Channels,” in ACSAC, 2021. Attacks on SSH,” in USENIX Security, 2001.
[16] S. B. Dutta, H. Naghibijouybari, A. Gupta, N. B. Abu-Ghazaleh, A. Marquez, and [50] D. Gruss, R. Spreitzer, and S. Mangard, “Cache Template Attacks: Automating
K. J. Barker, “Spy in the GPU-box: Covert and Side Channel Attacks on Multi-GPU Attacks on Inclusive Last-Level Caches,” in USENIX Security, 2015.
Systems,” in ISCA, 2022. [51] M. Lipp, D. Gruss, M. Schwarz, D. Bidner, C.-m.-t.-n. Maurice, and S. Mangard,
[17] W3C, “WebGPU - W3C Working Draft - Timing attacks,” 2023. [Online]. “Practical Keystroke Timing Attacks in Sandboxed JavaScript,” in ESORICS, 2017.
Available: [Link] [52] J. Monaco, “SoK: Keylogging Side Channels,” in S&P, 2018.
[18] B. Gras, K. Razavi, E. Bosman, H. Bos, and C. Giuffrida, “ASLR on the Line: [53] J. Bonneau and I. Mironov, “Cache-collision timing attacks against AES,” in CHES,
Practical Cache Attacks on the MMU.” in NDSS, 2017. 2006.
[19] M. Schwarz, C. Maurice, D. Gruss, and S. Mangard, “Fantastic Timers and Where [54] G. Irazoqui, M. S. Inci, T. Eisenbarth, and B. Sunar, “Wait a minute! A fast, Cross-
to Find Them: High-Resolution Microarchitectural Attacks in JavaScript,” in FC, VM attack on AES,” in RAID, 2014.
2017. [55] M. Neve and J.-P. Seifert, “Advances on access-driven cache attacks on AES,” in
[20] A. van Kesteren, “Safely reviving shared memory,” 2020. [Online]. Available: SAC. Springer, 2007.
[Link] [56] C. Maurice, M. Weber, M. Schwarz, L. Giner, D. Gruss, C. Alberto Boano, S. Man-
[21] Mozilla, “SharedArrayBuffer,” 2012. [Online]. Available: [Link] gard, and K. Römer, “Hello from the Other Side: SSH over Robust Cache Covert
org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer Channels in the Cloud,” in NDSS, 2017.
[22] W3C, “WebGPU Security Considerations,” 2023. [Online]. Available: https:
//[Link]/TR/webgpu/#security-considerations
[23] NVIDIA, “NVIDIA Tesla v100 GPU architecture,” 2017. [Online]. Available:
[Link]
[Link]
[24] ——, “Kernel Profiling Guide,” 2023.
[25] AMD, “AMD RDNA Whitepaper,” 2023. [Online]. Available: [Link]
m/system/files/documents/[Link]
[26] K. Group, “WebGL Specification,” [Link]
2023.
[27] K. W. W. Group, “WebGL 2.0 Compute,” [Link]
cs/latest/2.0-compute/, 2021.
[28] Mozilla, “WebGPU API,” 2023. [Online]. Available: [Link]
en-US/docs/Web/API/WebGPU_API
[29] Google, “Chrome ships WebGPU,” 2023. [Online]. Available: [Link]
[Link]/blog/webgpu-release/
[30] W3C, “WebGPU,” 2023. [Online]. Available: [Link]
[31] F. Liu, Y. Yarom, Q. Ge, G. Heiser, and R. B. Lee, “Last-Level Cache Side-Channel
Attacks are Practical,” in S&P, 2015.
[32] D. A. Osvik, A. Shamir, and E. Tromer, “Cache Attacks and Countermeasures:
the Case of AES,” in CT-RSA, 2006.
[33] Y. Oren, V. P. Kemerlis, S. Sethumadhavan, and A. D. Keromytis, “The Spy in the
Sandbox: Practical Cache Attacks in JavaScript and their Implications,” in CCS,
2015.
[34] A. Shusterman, A. Agarwal, S. O’Connell, D. Genkin, Y. Oren, and Y. Yarom,
“Prime+Probe 1, JavaScript 0: Overcoming browser-based side-channel defenses,”
in USENIX Security, 2021.
[35] S. B. Dutta, H. Naghibijouybari, N. Abu-Ghazaleh, A. Marquez, and K. Barker,
“Leaky buddies: Cross-component covert channels on integrated cpu-gpu systems,”
in ISCA, 2021.