Secure, Precise, and Fast Floating-Point

Operations on x86 Processors

Ashay Rane, Calvin Lin, and Mohit Tiwari, The University of Texas at Austin
[Link]

This paper is included in the Proceedings of the

25th USENIX Security Symposium
August 10–12, 2016 • Austin, TX
ISBN 978-1-931971-32-4

Open access to the Proceedings of the

25th USENIX Security Symposium
is sponsored by USENIX
 Secure, Precise, and Fast
Floating-Point Operations on x86 Processors

Ashay Rane, Calvin Lin Mohit Tiwari

Department of Computer Science Dept. of Electrical and Computer Engineering
The University of Texas at Austin The University of Texas at Austin
{ashay, lin} @[Link] tiwari@[Link]

Abstract Zero Normal Subnormal Infinity NaN

7 11 153 7 7
Floating-point computations introduce several side chan-
nels. This paper describes the first solution that closes Table 1: Latency (in cycles) of the SQRTSS instruction for
these side channels while preserving the precision of various operands.
non-secure executions. Our solution exploits micro-
architectural features of the x86 architecture along with
novel compilation techniques to provide low overhead. Both x861 and ARM2 provide variable-latency
Because of the details of x86 execution, the evaluation floating-point instructions. This variable latency stems
of floating-point side channel defenses is quite involved, from the desire to have graceful floating-point arithmetic
but we show that our solution is secure, precise, and fast. behavior, which, as we explain in Section 3, requires the
Our solution closes more side channels than any prior so- use of so-called subnormal values [8], which are pro-
lution. Despite the added security, our solution does not cessed using special algorithms. Since subnormal values
compromise on the precision of the floating-point oper- are rare, hardware vendors typically support such values
ations. Finally, for a set of microkernels, our solution is in microcode, so as not to slow down the common case.
an order of magnitude more efficient than the previous The resulting difference in instruction latency creates a
solution. timing side channel, which has been used to infer cross-
origin data in browsers and to break differential privacy
guarantees of a remote database [3].
1 Introduction However, variable latency floating-point instructions
represent only a part of the problem, since higher level
To secure our computer systems, considerable effort has
floating-point operations, such as sine and cosine, are
been devoted to techniques such as encryption, access
typically implemented in software. Thus, the implemen-
control, and information flow analysis. Unfortunately,
tation of these floating-point operations can leak secret
these mechanisms can often be subverted through the use
information through other side channels as well. De-
of side channels, in which an adversary, with the knowl-
pending on the secret values, programs can throw excep-
edge of the program, monitors the program’s execution
tions, thereby leaking the presence of abnormal inputs
to infer secret values. These side channels are signifi-
through termination. Programs can also contain condi-
cant because they have been used to discover encryption
tional branches, which can leak secrets through the in-
keys in AES [26], RSA [27], and the Diffie-Hellman key
struction pointer, branch predictor, or memory access
exchange protocol [14], thereby rendering these sophis-
count. Finally, programs that index into lookup tables
ticated schemes useless.
can leak secrets through the memory address trace.
Numerous side channels exist, including instruction
To prevent information leaks in both floating-point in-
and data caches [27, 26], branch predictors [2], mem-
structions and floating-point software, a strong solution
ory usage [12, 35], execution time [31, 4], heat [22],
should ensure at least four key properties, which cor-
power [15], and electromagnetic radiation [9], but one
respond to the side channels that we discussed above:
particularly insidious side channel arises from the exe-
cution of variable-latency floating-point instructions [3, 1 [Link] [Link]
10], in which an instruction’s latency varies widely de- 2 [Link]

pending on its operands, as shown in Table 1. [Link].ddi0344k/[Link]

USENIX Association 25th USENIX Security Symposium 71

(1) fixed-time operations that are independent of secret floating-point operations, Escort is 16× faster than
values, (2) disabled exceptions, (3) sequential control the previous solution [3].
flow, and (4) uniform data accesses that are independent
of the value of secret variables. Previous solutions [3, 5] 5. As an ancillary contribution, we introduce a
are inadequate because they do not ensure all four prop- methodology for evaluating the precision and se-
erties, are slow, are orders of magnitude less precise, or curity of floating-point operations, which is fraught
are difficult to implement. with subtleties.
This paper presents a novel solution that closes side
The rest of this paper is organized as follows. Sec-
channels arising from both hardware and software im-
tion 2 describes our threat model, related work, and sys-
plementations of floating point operations, providing all
tem guarantees. We provide background in Section 3 be-
four properties mentioned above. Our compiler-based
fore presenting our solution in Section 4. We evaluate
solution has two components.
our solution in Sections 5–7 . Finally, we conclude in
The first component creates building blocks of ele-
Section 8.
mentary floating-point operations for instructions that are
natively supported by the hardware (addition, subtrac-
tion, multiplication, division, square root, and type con- 2 Threat Model and Related Work
version). Our solution leverages unused SIMD lanes so
that fast operations on normal operands are accompanied This section begins by describing our threat model,
by slower dummy computations on subnormal operands, which shapes our subsequent discussion of related work
yielding a consistent yet low instruction latency for all and of Escort’s security guarantees.
types of operands.
The second component is a software library of higher- Threat Model. Our goal is to prevent secret floating-
level floating-point operations like sine and cosine. point operands from leaking to untrusted principals that
The key to creating this second component is a new either read digital signals from the processor’s pins or
code transformation that produces fixed-latency func- that are co-resident processes.
tions through normalized control flows and data access We assume that the adversary is either an external en-
patterns. Code generated by our compiler closes digital tity that monitors observation-based side channels (e.g.
side-channels, which have been defined to be those side time [14], memory address trace [11], or the /proc
channels that carry information over discrete bits [28]. pseudo-filesystem [12]) or a co-resident process/VM that
Whereas previous work in closing digital side channels monitors contention-based side channels (e.g. cache [27]
employs a runtime system [28], our solution shifts much or branch predictor state [2]).
of the work to compile time, yielding a significantly For off-chip observation-based channels, we assume
smaller runtime overhead. that the processor resides in a sealed and tamper-proof
This paper makes the following contributions: chip that prevents the adversary from measuring physi-
cal side channels like heat, power, electromagnetic radi-
1. We present a novel compiler-based system, called ation, etc. We assume that the CPU encrypts data trans-
Escort, for closing digital side channels that arise ferred to and from DRAM. All components other than
from the processing of floating-point instructions. the processor are untrusted, and we assume that the ad-
versary can observe and tamper with any digital signal.
2. Secure: We demonstrate that our solution is secure For on-chip contention-based channels, we assume that
not just against timing but also against digital side the OS is trusted and does not leak the victim process’s
channels. We demonstrate Escort’s capabilities by secret information. We also assume that the adversary
defeating a machine-learning side-channel attack, cannot observe or change the victim process’s register
by defending against a timing attack on the Firefox contents. Our trusted computing base includes the com-
web browser, by conducting extensive performance pilation toolchain.
measurements on an x86 processor, and by verify-
ing our solution’s code using typing rules.
Side-Channel Defenses. Decades of prior research
3. Precise: We show that Escort provides precision have produced numerous defenses against side channels,
that is identical to that of the standard C math li- the vast majority of which close only a limited number
brary. By contrast, the previous solution’s precision of side channels with a single solution. For instance,
is off by several million floating-point values. numerous solutions exist that close only the cache side
channel [6, 36, 39, 37, 16] or only the address-trace
4. Fast: We show that our solution is fast. On a side channel [33, 20, 32, 29]. Raccoon [28] is the first
set of micro-benchmarks that exercise elementary solution that closes a broad class of side channels—in

72 25th USENIX Security Symposium USENIX Association

particular, the set of digital side channels—with a sin- does transform loops that leak information through trip
gle solution. Similar to Raccoon, Escort also closes counts. Escort is unable to handle programs contain-
digital side channels with a single solution, but unlike ing irreducible control flow graphs (CFGs), but standard
Raccoon, Escort focuses on closing floating-point digi- compiler transformations [24] can transform irreducible
tal side channels, which can arise from variable latency CFGs into reducible CFGs. Escort assumes that the in-
floating-point instructions and from software implemen- put program does not use vector instructions, does not
tations of floating-point libraries, in which points-to set exhibit undefined behavior, does not terminate abnor-
sizes are typically small. Given Escort’s narrower focus mally through exceptions, and is free of race conditions.
on floating-point computations, Escort is faster than Rac- Given a program that abides by these limitations, Es-
coon by an order of magnitude. cort guarantees that the transformed code produces iden-
tical results as the original program, does not leak se-
crets through timing or digital side channels, and that the
Timing Side-Channel Defenses. Prior defenses
transformed code does not terminate abnormally.
against timing side-channel attacks utilize new algo-
rithms [30], compilers [23], runtime systems [21], or
secure processors [18]. However, these solutions only 3 Background
address one source of timing variations—either those
stem from the choice of the algorithm [31] or those The variable latency of floating-point instructions creates
that stem from the microarchitectural design [10]. By security vulnerabilities. In this section, we explain sub-
contrast, Escort closes timing variations from both normal numbers, which are the cause of the variable la-
sources. tency, and we explain the difficulty of fixing the resulting
vulnerability. We also explain how the Unit of Least Pre-
cision (ULP) can be used to quantify the precision of our
Floating-Point Side-Channel Defenses. Andrysco et and competing solutions.
al. [3] present libfixedtimefixedpoint (FTFP), the
first software solution for closing the floating-point tim- Next smallest Next smallest
ing channel. FTFP has some weaknesses, as we now positive number positive number

discuss, but the main contribution of their paper is the Large gap Equal gaps
10-38 10-45
demonstration of the significance of this side channel, ... ...
as they use variable-latency floating-point operations to 0 Small gap 0
10-45
break a browser’s same-origin policy and to break dif- Smallest Smallest
positive positive
ferential privacy guarantees of remote databases. FTFP number number
is a fixed-point library that consists of 19 hand-written (a) Without subnormal values. (b) With subnormal values.
functions that each operates in fixed time, independent
of its inputs. FTFP is slow, it is imprecise, and it ex- Figure 1: Impact of allowing subnormal numbers. With-
poses secrets through other side channels, such as the out subnormal values, there exists a much larger gap be-
cache side channel or the address trace side channel. tween zero and the smallest positive number than be-
Cleemput et al. [5] introduce compiler transformations tween the first two smallest positive numbers. With sub-
that convert variable-timing code into fixed-timing code. normal numbers, the values are more equally spaced.
Their technique requires extensive manual intervention, (The figure is not drawn to scale.)
applies only to the division operation, and provides weak
security guarantees. Both solutions require manual con-
struction of fixed-time code—a cumbersome process that
3.1 Subnormal Numbers
makes it difficult to support a large number of operations.
By contrast, Escort implements a fixed-time floating- Subnormal numbers have tiny exponents, which result
point library, while preventing information leaks through in floating-point values that are extremely close to zero:
timing as well as digital side channels. Escort includes a 10−45 < |x| < 10−38 for single-precision numbers and
compiler that we have used to automate the transforma- 10−324 < |x| < 10−308 for double-precision numbers.
tion of 112 floating-point functions in the Musl standard Subnormal values extend the range of floating-point
C library, a POSIX-compliant C library. Escort also pro- numbers that can be represented, but more importantly,
vides precision identical to the standard C library. they enable gradual underflow—the property that as
floating-point numbers approach zero along the number
Escort’s Guarantees. Escort rejects programs that scale, the difference between successive floating-point
contain unsupported features—I/O operations and recur- numbers does not increase3 . Figures 1a and 1b show the
sive function calls. Unlike prior work [18, 28], Escort 3 [Link] [Link]

USENIX Association 25th USENIX Security Symposium 73

differences between zero and the two smallest positive code for higher-level operations and converts it into se-
floating-point numbers. With subnormal numbers, the cure code. This compiler combines a code transforma-
gap between any two consecutive floating-point values is tion technique with Escort’s secure elementary opera-
never larger than the values themselves, thus exhibiting tions. Third, we present an example that shows the syn-
Gradual Underflow. Subnormal numbers are indispens- ergy among Escort’s components.
able because gradual underflow is required for reliable
equation solving and convergence acceleration [8, 13].
To avoid the added hardware complexity of supporting
4.1 Elementary Operations
subnormal numbers, which occur infrequently, vendors The key insight behind Escort’s secure elementary opera-
typically process subnormal values in microcode, which tions is that the latencies of SIMD instructions are deter-
is orders of magnitude slower than hardwired logic. mined by the slowest operation among the SIMD lanes
The resulting difference in latencies creates a security (see Figure 2), so the Escort compiler ensures that each
vulnerability. An adversary that can measure the latency elementary instruction runs along side a dummy instruc-
of a floating-point instruction can make reasonable esti- tion whose operand will produce the longest possible la-
mates about the operand type, potentially inferring secret tency. Our analysis of 94 x86 SSE and SSE2 instruc-
values using the timing channel. While subnormal values tions (which includes single- and double-precision arith-
occur infrequently in typical program execution, an ad- metic, comparison, logical, and conversion instructions)
versary can deliberately induce subnormal values in the reveals: (1) that only the multiplication, division, square
application’s inputs to enable subnormal operand timing root, and single-precision to double-precision conver-
attacks. sion (upcast) instructions exhibit latencies that depend
on their operands and (2) that subnormal operands in-
3.2 Floating-Point Error Measurement duce the longest latency.
In particular, Escort’s fixed-time floating-point opera-
Unlike real (infinite precision) numbers, floating-point tions utilize SIMD lanes in x86 SSE and SSE2 instruc-
numbers use a limited number of bits to store values, tions. Our solution (1) loads genuine and dummy (sub-
thus making them prone to rounding errors. Rounding normal) inputs in spare SIMD lanes of the same input
errors in floating-point numbers are typically measured register, (2) invokes the desired SIMD instruction, and
in terms of the Unit of Least Precision (ULP) [25]. The (3) retains only the result of the operation on the genuine
ULP distance between two floating-point numbers is the inputs. Our tests confirm that the resulting SIMD instruc-
number of distinct representable floating-point numbers tion exhibits the worst-case latency, with negligible vari-
between them, which is simply the result of subtracting ation in running time (standard deviation is at most 1.5%
their integer representations. If the result of the subtrac- of the mean). Figure 3 shows Escort’s implementation of
tion is zero, the floating-point numbers must be exactly one such operation.
the same. Escort includes Raccoon’s conditional data copy op-
eration (see Figure 4) which does not leak information
4 Our Solution: Escort through digital side channels. This operation copies the
contents of one register to another register if the given
Escort offers secure counterparts of ordinary non-secure condition is true. However, regardless of the condition,
floating-point operations, including both elementary op- this operation consumes a fixed amount of time, executes
erations and higher-level math operations. The elemen- the same set of instructions, and does not access applica-
tary operations include the six basic floating-point op- tion memory.
erations that are natively supported by the ISA—type
conversion, addition, subtraction, multiplication, divi- 4.2 Compiling Higher-Level Operations
sion, and square root—and a conditional data copy op-
eration. The 112 higher-level math operations are those Escort’s compiler converts existing non-secure code into
that are implemented using a combination of native in- secure code that prevents information leakage through
structions. Examples of higher-level functions include digital side channels. First, our compiler replaces all ele-
sine, cosine, tangent, power, logarithm, exponentiation, mentary floating-point operations with their secure coun-
absolute value, floor, and ceiling. terparts. Next, our compiler produces straight-line code
The next subsections describe Escort’s design in three that preserves control dependences among basic blocks
parts. First, we describe the design of Escort’s secure el- while preventing instruction side effects from leaking se-
ementary operations. These operations collectively form crets. Our compiler then transforms array access state-
the foundation of Escort’s security guarantees. Second, ments so that they do not leak information through mem-
we describe Escort’s compiler, which accepts non-secure ory address traces. Finally, our compiler transforms

74 25th USENIX Security Symposium USENIX Association

 A*B A*B
C*D C*D
(intended After (intended P*Q P*Q
operation) (intended transformation operation) (intended
operation) (dummy operation) (dummy

time

time
time

time
[next instr.] operation) operation)

[next instr.]

[next instr.] [next instr.]

(a) Original (b) Transformed

(non-secure) code (secure) code

Figure 2: The key idea behind Escort’s secure elementary operations. The operation is forced to exhibit a fixed latency
by executing a fixed-latency long-running operation in a spare SIMD lane.

double escort_mul_dp(double x, double y) { 01: copy(uint8_t pred, uint32_t t_val, uint32_t f_val) {
const double k_normal_dp = 1.4; 02: uint32_t result;
const double k_subnormal_dp = 2.225e-322; 03: __asm__ volatile (
04: "mov %2, %0;"
double result; 05: "test %1, %1;"
__asm__ volatile( 06: "cmovz %3, %0;"
"movdqa %1, %%xmm14;" 07: "test %2, %2;"
"movdqa %2, %%xmm15;" 08: : "=r" (result)
"pslldq $8, %1;" 09: : "r" (pred), "r" (t_val), "r" (f_val)
"pslldq $8, %2;" 10: : "cc"
"por %3, %1;" 11: );
"por %4, %2;" 12: return result;
"movdqa %2, %0;" 13: }
"mulpd %1, %0;"
"psrldq $8, %0;"
"movdqa %%xmm14, %1;" Figure 4: Code for conditional data copy operation that
"movdqa %%xmm15, %2;" does not leak information over digital side channels. This
: "=x" (result), "+x" (x), "+x" (y)
: "x" (k_subnormal_dp), "x" (k_normal_dp) function returns t val if pred is true; otherwise it re-
: "xmm15", "xmm14"); turns f val. The assembly code uses AT&T syntax.
return result;
}

Figure 3: Escort’s implementation of double-precision

LLVM IR. For every Φ-node that produces an operand
multiplication, using the AT&T syntax.
for an arithmetic expression, Escort creates one copy of
the expression for each input to the Φ-node. If the solver
reports that no operand can have a subnormal value, then
loops whose trip count reveals secrets over digital side
Escort skips instrumentation of that floating-point opera-
channels. We now describe each step in turn.
tion.

4.2.1 Step 1: Using Secure Elementary Operations We set a timeout of 40 seconds for each invocation of
the SMT solver. If the solver can prove that the instruc-
The Escort compiler replaces x86 floating-point type- tion never uses subnormal operands, then Escort skips
conversion, multiplication, division, and square root as- replacing that floating-point instruction with its secure
sembly instructions with their Escort counterparts. How- counterpart. Figure 5 shows the percentage of floating-
ever, Escort’s secure elementary operations can be up point instructions in commonly used math functions that
to two orders of magnitude slower than their non-secure are left untransformed by Escort.
counterparts. Hence, our compiler minimizes their usage
by using taint tracking and by employing the quantifier- This optimization is conservative because it assumes
free bit-vector logic in the Z3 SMT solver [7], which is that all floating-point instructions in the program have
equipped with floating-point number theory. If the solver subnormal operands unless proven otherwise. The cor-
can prove that the operands can never be subnormal val- rectness of the optimization is independent of the code’s
ues, then Escort refrains from replacing that instruction. use of pointers, library calls, system calls, or dynamic
In effect, the Escort compiler constructs path-sensitive values. The static analysis used in this optimization is
Z3 expressions for each arithmetic statement in the flow-sensitive, path-sensitive, and intra-procedural.

USENIX Association 25th USENIX Security Symposium 75

 100 % 1: for each basic block bb in function do
2: if entry block(bb) then
Optimized floating-point instructions

80 % 3: pred[bb] ← true
4: else
60 % 5: pred[bb] ← false
6: end if
40 % 7: end for
8:
20 %
9: for each basic block bb in function do
10: br ← branch(bb)
0%
11: if unconditional branch(br) then
exp10f

log10f
exp10

exp2f
log10

floorf
log2f
exp2
fabsf

powf

floor
expf

log2
fabs

ceilf
pow

cosf

tanf
logf

sinf
ceil

cos

tan
log

sin
12: {s} ← successors(bb)
13: pred[s] ← pred[s] ∨ pred[bb]
Figure 5: Percentage of instructions that are left uninstru- 14: pred[s] ← simpli f y(pred[s])
mented (without sacrificing security) after consulting the 15: else  Conditional Branch.
SMT solver. 16: {s1 , s2 } ← successors(bb)
17: if loop condition branch(br) then
18:  Skip branches that represent loops.
4.2.2 Step 2: Predicating Basic Blocks 19: pred[s1 ] ← pred[s1 ] ∨ pred[bb]
20: pred[s2 ] ← pred[s2 ] ∨ pred[bb]
Basic block predicates represent the conditions that dic-
21: else
tate whether an instruction should execute. These pred-
22: p ← condition(br)
icates are derived by analyzing conditional branch in-
23: pred[s1 ] ← pred[s1 ] ∨ (pred[bb] ∧ p)
structions. For each conditional branch instruction that
24: pred[s2 ] ← pred[s2 ] ∨ (pred[bb] ∧ ¬p)
evaluates a predicate p, the Escort compiler associates
25: end if
the predicate p with all basic blocks that execute if the
26: pred[s1 ] ← simpli f y(pred[s1 ])
predicate is true, and it associates the predicate ¬p with
27: pred[s2 ] ← simpli f y(pred[s2 ])
all basic blocks that execute if the predicate is false. For
28: end if
unconditional branches, the compiler copies the predi-
29: end for
cate of the previous block into the next block. Finally,
if the Escort compiler comes across a block that already Figure 6: Algorithm for predicating basic blocks.
has a predicate, then the compiler sets the block’s new
predicate to the logical OR of the input predicates. At
each step, the Escort compiler uses Z3 as a SAT solver to Memory Access Side Effects. To ensure proper mem-
simplify predicates by eliminating unnecessary variables ory access side effects, the Escort compiler replaces store
in predicate formulas. Figure 6 shows the algorithm for instructions with conditional data-copy operations that
basic block predication. are guarded by the basic block’s predicate, so memory
is only updated by instructions whose predicate is true.
Unfortunately, this naı̈ve approach can leak secret in-
4.2.3 Step 3: Linearizing Basic Blocks formation when the program uses pointers. Figure 7
The Escort compiler converts the given code into illustrates the problem: If store instructions are not al-
straight-line code so that every invocation of the code lowed to update a pointer variable when the basic block
executes the same instructions. To preserve control de- predicate is false, then the address trace from subsequent
pendences, the basic blocks are topologically sorted, and load instructions on the pointer variable will expose the
then the code is assembled into a single basic block with fact that the pointer variable was not updated.
branch instructions removed. The Escort compiler prevents such information leaks
by statically replacing pointer dereferences with loads or
stores to each element of the points-to set4 . Thus Escort
4.2.4 Step 4: Controlling Side Effects replaces the statement in line 8 (Figure 7) with a store
operation on b. When the points-to set is larger than a
We now explain how Escort prevents side effects from
4 Escort uses a flow-sensitive, context-insensitive pointer analysis:
leaking secrets. Here, side effects are modifications to
[Link] Replacing a pointer derefer-
the program state or any observable interaction, includ- ence with a store operation on all elements of the points-to set is feasi-
ing memory accesses, exceptions, function calls, or I/O. ble for Escort because points-to set sizes in the Musl C library are very
Escort controls all side effects except for I/O statements. small.

76 25th USENIX Security Symposium USENIX Association

 1: p ← &a 4.2.5 Step 5: Transforming Array Accesses
2: secret ← input()  Assume input() returns true.
3: if secret = true then Array index values reveal secrets as well. For instance,
4: ... if the adversary observes that accesses to array[0] and
5: else array[secret index] result in accesses to locations 10
6: ... and 50, then the adversary knows that secret index =
7: p ← &b  Instruction does not update pointer p, 40. To eliminate such information leaks, the Escort com-
since basic block’s execution-time predicate is false. piler transforms each array access into a linear sweep
8: ∗p ← 10  Accesses a instead of b! over the entire array, which hides from the adversary the
9: end if address of the program’s actual array index.
Of course, the transformed code is expensive, but this
Figure 7: The use of pointers can leak information. If approach is feasible because (1) math library functions
store instructions are not allowed to access memory typically use only a few small lookup tables, thus requir-
when the basic block’s predicate is false, then pointer ing relatively few memory accesses and (2) the proces-
p will dereference the address for a instead of b, thus sor’s caches and prefetchers dramatically reduce the cost
revealing that secret is true. of sweeping over the arrays.

4.2.6 Step 6: Transforming Loops

singleton set, Escort uses the conditional data copy op-
eration on all potential pointees i.e. the elements of the Some loops introduce timing channels because their trip
points-to set. The predicate of the conditional copy oper- counts depend on secret values. The Escort compiler
ation checks whether the pointer points to the candidate transforms such loops using predictive mitigation [38].
pointee. If the predicate is false, the pointee’s existing The loop body executes as many times as the smallest
value is overwritten, whereas if the predicate is true, the power of 2 that is greater than or equal to the loop trip
new value is written to the pointee. count. For instance, if the actual loop trip count is 10,
then the loop body is executed 16 times. The basic block
Function Call Side Effects. Adversaries can observe predicate ensures that dummy iterations do not cause side
the invocation of functions (or lack thereof) using side effects. With this transformed code, an adversary that ob-
channels like the Instruction Pointer. Thus, a solution in- serves a loop trip count of l can infer that the actual loop
capable of handling function calls will leak information trip count l  is between l and 0.5 × l. However, the exact
to the adversary. While inlining functions is a potential value of l  is not revealed to the adversary.
solution, inlining is impractical for large applications. Unfortunately, this naive approach can still leak infor-
Escort handles side effects from function calls by mation. For instance, if two distinct inputs cause the loop
propagating the predicate from the calling function to the to iterate 10 and 1000 times respectively, the transformed
callee. Thus, each user-defined function is given an ad- codes will iterate 16 and 1024 times respectively—a
ditional argument that represents the predicate of the call large difference that may create timing variations. To
site’s basic block. The callee ensures correct handling of mitigate this problem, Escort allows the programmer to
side effects by ANDing its own predicates with the caller’s manually specify the minimum and maximum loop trip
predicate. counts using programmer annotations. These annota-
tions override the default settings used by the Escort
compiler.
Side Effects from Exceptions. Program termination
caused by exceptions will leak the presence or absence
of abnormal operands. To prevent such information leak- 4.3 Example Transformation: exp10f
age, Escort requires that exceptions not occur during pro- We now explain how Escort transforms an exam-
gram execution5 . ple non-secure function (Figure 8a) into a secure func-
Escort manages floating-point and integer exceptions tion (Figure 8c). To simplify subsequent analyses and
differently. Escort requires that the programmer disable transformations, the Escort compiler applies LLVM’s
floating-point exceptions (e.g. using feclearexcept()). mergereturn transformation pass, which unifies all exit
For integer exceptions, Escort borrows ideas from nodes in the input function (see Figure 8b).
Raccoon by replacing abnormal operands with benign First, the Escort compiler replaces elementary
operands (e.g. Escort prevents integer division-by-zero floating-point operations in lines 8 and 10 with their se-
by replacing a zero divisor with a non-zero divisor). cure counterpart function shown in lines 21 and 22 of the
5 Escort
assumes that the input program does not throw exceptions, transformed code. Second, using the algorithm shown in
so masking exceptions does not change the semantics of the program. Figure 6, the Escort compiler associates predicates with

USENIX Association 25th USENIX Security Symposium 77

float e10(float x) {
float n, y = mf(x, &n);
if (int(n) >> 23 & 0xff < 0x82) { 12: float e10(float x) {
float p = p10[(int) n + 7]; 13: return e10_cloned(x, true);
if (y == 0.0f) { 14: }
return p; 15:
} 16: float e10_cloned(float x, uint pred) {
return exp2f(3.322f * y) * p; 17: float n, y = mf_cloned(x, &n, pred);
} 18: float p = write(int(n) >> 23 & 0xff
return exp2(3.322 * x); < 0x82, stream_load(p10, (int) n + 7]));
} 19: bool p2 = y == 0.0f;
20: write(pred & p1 & p2, p, &result);
21: write(pred & p1 & !p2,
(a) Original code for exp10f(). escort_mul(
escort_mul(
exp2f_cloned(3.322f,
pred & p1 & !p2),
01: float e10(float x) {
y),
02: float n, y = mf(x, &n);
p),
03: if (int(n) >> 23 & 0xff < 0x82) {
&result);
04: float p = p10[(int) n + 7];
05: if (y == 0.0f)
22: write(!p1,
06: result = p;
escort_mul(
07: else
exp2_cloned(3.322, pred & !p1),
08: result =
escort_upcast(x))),
exp2f(3.322f * y) * p;
result);
09: } else
10: result = exp2(3.322 * x);
23: return result;
11: return result;
24: }
12: }

(c) Result of the Escort compiler’s transformation.

(b) Result after applying LLVM’s mergereturn pass. This
code becomes the input for the Escort compiler.

Figure 8: Escort’s transformation of exp10f().

Line # Predicate
2, 3, 11 TRUE
A: y = mf(x, &n) 4, 5 (n >> 23 & 0xff) < 0x82
6 (n >> 23 & 0xff) < 0x82 ∧ y = 0
8 (n >> 23 & 0xff) < 0x82 ∧ y = 0
(n >> 23 Yes 10 ¬((n >> 23 & 0xff) < 0x82)
& 0xff) B: p = p10[n + 7]
< 0x82?
Table 2: Predicates per line for function in Figure 8b.
No
Yes
C: result = exp2 y = 0? D: result = p
(3.332 * x)

No

E: result = exp2f
(3.332f * y) * p
each basic block, which we list in Table 2. Third, the Es-
cort compiler linearizes basic blocks by applying a topo-
F: return result logical sort on the control flow graph (see Figure 9) and
fuses the basic blocks together. Finally, the Escort com-
Figure 9: Control flow graph with labeled statements for piler replaces the array access statement in line 4 with a
the code in Figure 8b. A, B, D, E, C, F is one possible function that sweeps over the entire array. The resulting
sequence of basic blocks when linearized by the Escort code, shown in Figure 8c, eliminates control flows and
compiler. data flows that depend on secret values. In addition to
closing digital side channels, the code also uses secure
floating-point operations.

78 25th USENIX Security Symposium USENIX Association

5 Security Evaluation between Mx and Md . The difference between the first and
the third quartiles (Q3 − Q1 ) is called the Inter-Quartile
This section demonstrates that Escort’s floating-point op- Range, RIQ . Tukey’s method states that any value v, such
erations run in fixed time and do not leak information that v > Q3 + 3×RIQ or v < Q1 − 3×RIQ is a probable
through digital side channels. Since precise timing mea- outlier. In our evaluation, we weaken our outlier elim-
surement on x86 processors is tricky due to complex ination process (i.e. we count fewer values as outliers),
processor and OS design, we take special measures to by (1) setting the RIQ to be at least equal to 1.0, and
ensure that our measurements are accurate. In addi- (2) classifying v as an outlier when v > Q3 + 20×RIQ
tion to Escort’s timing and digital side channel defense, or v < Q1 − 20×RIQ . Results presented in the following
we also demonstrate Escort’s defense against a floating- sections use the relaxed Tukey method described above.
point timing channel attack on the Firefox web browser.
Mean Median Std. Dev.
Different 847,323 1,066,270
5.1 Experimental Setup Operands (0.81%) (1.02%)
381,467

We run all experiments on a 4-core Intel Core i7-2600 Same 929,703 1,139,961
364,192
(Sandy Bridge) processor. The processor is clocked at Operands (0.89%) (1.09%)
3.4 GHz. Each core on this processor has a 32 KB pri-
vate L1 instruction cache, a 32 KB private L1 data cache, Table 3: Number of discarded outliers from 100 million
and a 256 KB private L2 cache. A single 8 MB L3 cache double-precision square-root operations. The results in-
is shared among all four cores. The host operating sys- dicate that our outlier elimination process is statistically
tem is Ubuntu 14.04 running kernel version 3.13. We im- independent of the input operand values.
plement compiler transformations using the LLVM com-
piler framework [17] version 3.8. To demonstrate that our outlier elimination process
We measure instruction latencies using the RDTSC in- does not bias conclusions, we compare the distribution
struction that returns the number of elapsed cycles since of outliers between (a) 100 million operations using
resetting the processor. Since the latency of executing randomly-generated operands, and (b) 100 million op-
the RDTSC instruction is usually higher than the latency erations using one fixed operand. The two experiments
of executing operations, our setup measures the latency do not differ in any way other than the difference in their
of executing 1024 consecutive operations and divides the input operands. Table 3 shows the mean, median, and
measured latency by 1024. Our setup uses the CPUID in- standard deviation of outliers for the double-precision
struction and volatile variables for preventing the pro- square-root operation. Results for other floating-point
cessor and the compiler from reordering critical instruc- operations are similar and are elided for space reasons.
tions. Finally, our setup measures overhead by execut- Since the difference in mean values as well as the dif-
ing an empty loop body—a loop body that contains no ference in median values is within a quarter of the stan-
instructions other than those in the test harness. By plac- dard deviation from the mean, we conclude that the dis-
ing an empty volatile asm block in the empty loop carded outlier count is statistically independent of the in-
body, our setup prevents the compiler from deleting the put operand values.
empty loop body.
5.2 Timing Assurance of Elementary Op-
5.1.1 Outlier Elimination erations
Many factors outside of the experiment’s control, like in- Since exhaustively testing all possible inputs for each op-
terrupts, scheduling policies, etc., may result in outliers eration is infeasible, we instead take the following three-
in performance measurements. We now explain our pro- step approach for demonstrating the timing channel de-
cedure for eliminating outliers, before demonstrating that fense for Escort’s elementary operations: (1) We char-
the elimination of these outliers does not bias the conclu- acterize the performance of Escort’s elementary opera-
sions. tions using a specific, fixed floating-point value (e.g. 1.0),
We use Tukey’s method [34] for identifying outliers, (2) using one value from each of the six different types
but we adapt it to conservatively classify fewer values as of values (zero, normal, subnormal, +∞, -∞, and not-
outliers (thus including more values as valid data points). a-number), we show that our solution exhibits negligi-
The original Tukey’s method first finds the minimum ble variance in running time, and (3) to demonstrate that
(Mn ), median (Md ), and maximum (Mx ) of a set of values. each of the six values in the previous experiment is rep-
The first quartile, Q1 , is the median of values between Mn resentative of the class to which it belongs, we generate
and Md . The third quartile, Q3 , is the median of values 10 million normal, subnormal, and not-a-number (NaN)

USENIX Association 25th USENIX Security Symposium 79

values, and show that the variance in running time among Native
Function Escort
each set of 10 million values is negligible. Our key find- (SSE)
ings are that Escort’s operations run in fixed time, are add-sp 0 0
fast, and that their performance is closely tied to the per- add-dp 0 0
formance of the hardware’s subnormal operations. sub-sp 0 0
sub-dp 0 0
mul-sp 0 49.2 (175%)
mul-dp 0 49.2 (175%)
div-sp 0.66 (0.4%) 65.67 (163%)
div-dp 1.66 (0.8%) 69.08 (164%)
sqrt-sp 1.49 (0.8%) 62.7 (170%)
sqrt-dp 2.98 (1.5%) 66.87 (169%)
upcast 0 40.99 (178%)

Table 4: Comparison of standard deviation of running

times of elementary operations across six types of values
(zero, normal, subnormal, +∞, −∞, and not-a-number).
Numbers in parenthesis show the standard deviation as
a percentage of the mean. The -sp suffix identifies
single-precision operations while the -dp suffix identi-
fies double-precision operations. Compared to SSE op-
Figure 10: Comparison of running times of elementary erations, Escort exhibits negligible variation in running
operations. sp identifies Escort’s single-precision opera- times.
tions, dp identifies Escort’s double-precision operations,
and fix identifies FTFP’s fixed-point operations. Num-
bers at the top of the bars show the total cycle count. We 3.1% of the mean). We thus conclude that our chosen
see that Escort’s execution times are dominated by the values for each of the six classes faithfully represent their
cost of subnormal operations, and we see that FTFP’s class.
overheads are significantly greater than Escort’s.
14,000
FTFP
Escort-SP
Figure 10 compares the running times of elementary 12,000 Escort-DP

operations of Escort and of previous solutions (FTFP). 10,000

First, we observe that the running times of Escort’s
Processor Cycles

8,000
single- and double-precision operations are an order-of-
magnitude lower than those of FTFP’s fixed-precision 6,000

operations. Second, Escort’s running time is almost en-

4,000
tirely dominated by the processor’s operation on subnor-
mal numbers. Third, conversion between fixed-point and 2,000

floating-point takes a non-trivial amount of time, further 0

increasing the overhead of FTFP’s operations. Overall,
10
or

2
s

w
p
il

n
log

sin
fab

log

co
ce

ex

po
ta
log
flo

Escort elementary operations are about 16× faster than

FTFP’s. Figure 11: Comparison of running times of commonly
Table 4 shows the variation in running time of ele- used higher-level functions. Error bars (visible for only
mentary operations across six different types of inputs a few functions) show the maximum variation in running
(zero, normal value, subnormal value, +∞, −∞, and not- time for different kinds of input values.
a-number value) and compares it with the variation of
SSE (native) operations. While SSE operations exhibit
high variation (the maximum observed standard devia-
tion is 176% of the mean), Escort’s operations show neg- 5.3 Timing Assurance of Higher-Level Op-
ligible variation across different input types. erations
Finally, we measure Escort’s running time for 10 mil-
lion random normal, subnormal, and not-a-number val- Using different types of floating-point values (zero, nor-
ues. We observe that the standard deviation of these mea- mal, subnormal, +∞, −∞, and not-a-number), Figure 11
surements, shown in Table 5, is extremely low (at most compares the performance of most of the commonly used

80 25th USENIX Security Symposium USENIX Association

 Fn. NaN Normal Subnormal exp10f(), exp2(), and exp2f()) use lookup tables
add-sp 0.21 (3.1%) 0.21 (2.9%) 0.19 (2.7%) that are susceptible to address-trace-based side-channel
add-dp 0.21 (3.0%) 0.20 (2.9%) 0.21 (3.0%) information leaks, so the code transformed by Escort
sub-sp 0.18 (2.6%) 0.19 (2.7%) 0.20 (2.9%) sweeps over these lookup tables for each access to the
sub-dp 0.19 (2.7%) 0.19 (2.7%) 0.19 (2.7%) table. Finally, we see that the cost of control flow ob-
mul-sp 0.98 (0.7%) 0.94 (0.7%) 1.05 (0.7%) fuscation (i.e. the cost of executing all instructions in the
mul-dp 0.90 (0.6%) 1.04 (0.7%) 1.02 (0.7%) program) contributes the least to the total overhead.
div-sp 1.22 (0.6%) 1.27 (0.7%) 1.23 (0.6%)
div-dp 1.39 (0.7%) 1.37 (0.6%) 1.17 (0.6%) 5.4 Side-Channel Defense in Firefox
sqrt-sp 1.15 (0.6%) 1.13 (0.6%) 1.14 (0.6%)
sqrt-dp 1.29 (0.7%) 1.41 (0.7%) 1.33 (0.7%) We now evaluate Escort’s defense against the timing
upcast 1.03 (0.9%) 0.89 (0.8%) 0.95 (0.8%) channel attack by Andrysco et al. [3] on the Firefox web
browser. The attack reconstructs a two-color image in-
Table 5: Standard deviation of 10 million measurements side a victim web page using only the timing side chan-
for each type of value (normal, subnormal, and not-a- nel in floating-point operations. The attack convolves the
number). All standard deviation values are within 3.1% given secret image with a matrix of subnormal values.
of the mean. Furthermore, the mean of these 10,000,000 The convolution step for each pixel is timed using high
measurements is always within 2.7% of the representa- resolution Javascript timers. By comparing the measured
tive measurement. time to a threshold, each pixel is classified as either black
or white, effectively reconstructing the secret image.
We integrate Escort into Firefox’s convolution code7
single- and double-precision higher-level operations6 . and re-run the timing attack. The results (see Figure 13c)
Overall Escort’s higher-level operations are about 2× show that Escort successfully disables the timing attack.
slower than their corresponding FTFP operation, which
is the price for closing side channels that FTFP does not
close. 5.5 Control- and Data-Flow Assurance
We now show that Escort’s operations do not leak infor-
baseline (non-secure) execution
15,000
control flow obfuscation mation through control flow or data flow. We first use
data access obfuscation
subnormal operands inference rules over the LLVM IR to demonstrate non-
interference between secret inputs and digital side chan-
Processor Cycles

10,000
nels. We run a machine-learning attack on Escort and
demonstrate that Escort successfully disables the attack.

5,000
5.5.1 Non-Interference Using Inference Rules
Since Escort’s elementary operations are small and
0
simple—they are implemented using fewer than 15 lines
exp10f
log10f

exp10
MEAN
exp2f

log10
floorf

log2f

exp2
fabsf

powf
floor

expf
log2
fabs

ceilf

pow

cosf
tanf
logf

exp

sinf
ceil

cos
tan
log

sin

of assembly code, they do not access memory, and they

do not contain branch instructions—they are easily veri-
Figure 12: Performance breakdown of Escort’s com- fied for non-interference between secret inputs and digi-
monly used higher-level functions. The baseline (non- tal side channels. Using an LLVM pass that applies the
secure) execution and exception handling together cost inference rules from Table 6, tracking labels that can be
less than 250 cycles for each function, making them too either L (for low-context i.e. public information) or H (for
small to be clearly visible in the above plot. high-context i.e. private information), we verify that Es-
cort’s higher-level operations close digital side channels.
Figure 12 shows the breakdown of the performance This compiler pass initializes all function arguments with
of commonly used higher-level functions. We observe the label H, since arguments represent secret inputs.
that the performance of most higher-level functions is Inference rules for various instructions dictate updates
dominated by the latency of operations on subnormal to the labels. The environment Γ tracks the label of each
operands, which is closely tied to the performance of the pointer and each address. The Escort compiler tags load
underlying hardware. A handful of routines (exp10(),
7 Specifically, we replace three single-precision multiplication oper-
6 We
exclude the exp2() (6,617 cycles), exp10() (14,910 cycles), ations with invocations to the equivalent Escort function. All source
exp2f() (1,693 cycles), and exp10f() (9,134 cycles) from Figure 11 code changes are limited to the code in the ConvolvePixel() function
because FTFP does not implement these operations. in [Link].

USENIX Association 25th USENIX Security Symposium 81

 (b) Reconstructed image (c) Reconstructed images in 3 independent, consec-
(a) Original image.
using timing attack. utive experiments after patching Firefox with Escort.

Figure 13: Results of attack and defense on a vulnerable Firefox browser using timing-channel information leaks
arising from the use of subnormal floating-point numbers.

and store instructions as secret if the pointer is tainted, implementation, the classifier’s accuracy drops to 33%,
or public otherwise. Unlike a public load or store in- which is no better than randomly guessing one of the
struction, a secret load or store instruction is allowed to three secret input values.
use a tainted pointer since Escort generates correspond-
ing loads and stores to all statically-determined candi-
date values in the points-to set. The sanitization rule re-
sets the value’s label to L and is required to suppress false
6 Precision Evaluation
alarms from Escort’s loop condition transformation. Es-
cort’s transformed code includes instructions with spe- We examine the precision of Escort and FTFP by com-
cial LLVM metadata that trigger the sanitization rule. paring Escort’s and FTFP’s results with those produced
During verification, the compiler pass iterates over by a standard C library.
each instruction and checks whether a rule is applica-
ble using the rule’s antecedents (the statement above the
horizontal line); if so, it updates its local state as per 6.1 Comparison Using Unit of Least Preci-
the rule’s consequent (the statement below the horizon- sion
tal line). If no applicable rule is found, then the com-
piler pass throws an error. The compiler pass processes Methodology. We adopt an empirical approach to esti-
the code for Escort’s 112 higher-level operations without mate precision in terms of Unit of Least Precision (ULP),
throwing errors. since formal derivation of maximum ULP difference re-
quires an intricate understanding of theorem provers and
5.5.2 Defense Against Machine-Learning Attack floating-point algorithms. We run various floating-point
operations on 10,000 randomly generated pairs (using
We use the TensorFlow [1] library to design a machine- drand48()) of floating-point numbers between zero and
learning classifier, which we use to launch a side-channel one. For elementary operations, we compare the outputs
attack on the execution of the expf() function, where of Escort and FTFP with the outputs of native x86 in-
the input to the expf() function is assumed to be secret. structions. For all other operations, we compare the out-
Using three distinct inputs, we run this attack on the im- puts of Escort and FTFP with the outputs produced by
plementations in the (non-secure) Musl C library and in corresponding function from the Musl C library.
the (secure) Escort library. We first use the Pin dynamic
binary instrumentation tool [19] to gather the full instruc-
tion address traces of both expf() implementations8 . We
Results. We observe that Escort’s results are identi-
train the TensorFlow machine-learning classifier by feed-
cal to the results produced by the reference implemen-
ing the instruction address traces to the classifier, asso-
tations, i.e. the native (x86) instructions and the Musl
ciating each trace with the secret input to expf(). We
C library. More precisely, the ULP difference between
use cross entropy as the cost function for TensorFlow’s
Escort’s results and reference implementation’s results
training phase. In the subsequent testing phase, we ran-
is zero. On the other hand, FTFP, which computes
domly select one of the collected address traces and ask
arithmetic in fixed-point precision, produces output that
the classifier to predict the secret input value.
differs substantially from the output of Musl’s double-
We find that for the Musl implementation, the classi- precision functions (see Table 7). The IEEE 754 standard
fier is accurately able to predict the correct secret value requires that addition, subtraction, multiplication, divi-
from the address trace. On the other hand, for the Escort sion, and square root operations are computed with ULP
8 Using the md5sum program, we observe that Escort’s address traces difference of at most 0.5. Well-known libraries compute
for all three inputs are identical. results for most higher-level operations within 1 ULP.

82 25th USENIX Security Symposium USENIX Association

 P = ptset(ptr) Function Min. Median Max.
Γ(ptr) = L m = max Γ(addr)
T-PUBLIC-LOAD addr∈P add 16 1,743,272 210,125,824
Γ = Γ[val → m] sub 1,312 6,026,976 84,089,503,744
Γ  val := public-load ptr : Γ mul 317 8,587,410 112,134,679,849
∀ addr ∈ ptset(p) div 829 5,834,095 30,899,033,427
Γ(ptr) = L m = max(Γ(val), Γ(addr)) sqrt 562 2,815,331 21,257,836,468
T-PUBLIC-STORE Γ = Γ[addr → m]
Γ  public-store ptr, val : Γ
floor 0 0 0
ceil 0 0 0
log 1,698 5,908,547 2,705,277,8104
Γ = Γ[val → H] log2 262 5,812,840 13,890,632,367
T-SECRET-LOAD
Γ  val := secret-load ptr : Γ log10 981 10,105,199 40,631,590,323
exp 132 1,409,624 6,066,894
∀ addr ∈ ptset(p)
sin 1,316 4,173,786 40,138,955,131
T-SECRET-STORE Γ = Γ[addr → H]
cos 2,166 2,241,360 10,127,702
Γ  secret-store ptr, val : Γ
tan 717 5,576,540 40,126,401,802
pow 522 3,425,870 26,876,068,127
Γ(cond) = L fabs 352 3,129,984 40,134,770,688
T-BRANCH
Γ  br cond,block1,block2 : Γ
Table 7: Floating-point difference for 10,000 operations
Γ = Γ[x → Γ(y)] on random inputs in terms of Unit of Least Precision
T-OTHER (ULP) in FTFP versus Musl C library. Since we ob-
Γ  x:=y : Γ
serve zero ULP distance between Escort’s results and
Musl’s results, this table omits Escort’s results.
Γ  S1 : Γ , Γ  S2 : Γ
T-COMPOSITION
Γ  S1 ; S2 : Γ
< 10−5 to 10−3 to 100 to >
Γ = Γ[x → L]
10−5 10−3 100 103 103
T-SANITIZER
Γ  S(x) : Γ
49% 9% 21% 10% 11%

Table 8: Distribution of differences in answers produced

Table 6: Inference rules for verifying the security of Es- by M INPACK -FTFP and M INPACK -C. In all, 321 values
cort’s higher-level operations. differ between the outputs of the two programs.

6.2 Comparison of Program Output

lyze all 321 differences between M INPACK -FTFP and
Methodology. Since differences in program outputs M INPACK -C by classifying them into the following five
provide an intuitive understanding of the error intro- categories: (1) smaller than 10−5 , (2) between 10−5 and
duced by approximate arithmetic operations, we com- 10−3 , (3) between 10−3 and 100 , (4) between 100 and
pare the output of the test suite of Minpack9 , a li- 103 , and (5) larger than 103 . As seen in Table 8, almost
brary for solving non-linear equations and non-linear half of the differences (49%) are extremely small (less
least squares problems. We generate three variants of than 10−5 ), possibly arising from relatively small dif-
Minpack: M INPACK -C uses the standard GNU C li- ferences between fixed-point and floating-point calcula-
brary, M INPACK -E SCORT uses the Escort library, and tions. However, we hypothesize that differences amplify
M INPACK -FTFP uses the FTFP library. We run the 29 from propagation, since nearly 42% of the differences
programs in Minpack’s test suite and compare the out- are larger than 10−3 .
puts produced by the three program variants.

Results. We observe that M INPACK -E SCORT produces

output that is identical to M INPACK -C’s output. We also 7 Performance Evaluation
observe that all outputs of M INPACK -FTFP differ from
M INPACK -C. Specifically, 321 values differ between the
outputs of M INPACK -FTFP and M INPACK -C. We ana- We now evaluate the end-to-end application performance
impact of Escort’s floating-point library and Escort’s
9 [Link] control flow obfuscation.

USENIX Association 25th USENIX Security Symposium 83

 Static (LLVM) Overhead for Overhead for
Test Case
Escort Floating-Point Training Classification
Application
Overhead Instruction #1 8.66× 1.34×
Count #2 30.24× 0.96×
[Link] 29.33× 2,791 #3 1.41× 1.11×
[Link] 57.32× 9,647 #4 12.75× 0.92×
[Link] 20.31× 21,963 GEO
8.28× 1.07×
[Link] 4.74× 4,177 MEAN
[Link] 82.53× 25,671
[Link] 56.19× 711 Table 10: Overhead of Escort on SVMlight program.
480.sphinx3 52.46× 629
32.63× 9,370
MEAN tation of Support Vector Machines in C, using the four
(geo. mean) (arith. mean)
example test cases documented on the SVMlight web-
Table 9: Overhead of SPEC-E SCORT (SPECfp2006 site11 . We mark the training data and the classification
using Escort operations) relative to SPEC-L IBC data as secret. Before replacing floating-point computa-
(SPECfp2006 using libc). tions, Escort’s taint analysis discovers all floating-point
computations that depend on the secret data, thus re-
ducing the list of replacements. We also instruct Es-
7.1 Impact of Floating-Point Library cort to query the Z3 SMT solver to determine whether
candidate floating-point computations could use subnor-
This section evaluates the performance impact of Escort mal operands. Escort then replaces these computations
on the SPEC floating point benchmarks, as well as on a with secure operations from its library. We compile the
security-sensitive program SVMlight , a machine-learning baseline (non-secure) program using the Clang/LLVM
classifier. 3.8 compiler with the -O3 flag, and we disable auto-
vectorization while compiling SVMlight with Escort. We
Evaluation Using SPEC Benchmarks. We use the C measure the total execution time using the RDTSC instruc-
and C++ floating-point applications in the SPEC CPU tion. Table 10 shows that Escort’s overhead on SVMlight .
2006 benchmark suite with reference inputs. We gen- We observe that Escort’s overhead on SVMlight is sub-
erate two versions of each program—the first version stantially lower than that on SPEC benchmarks. Using
(SPEC-L IBC) uses the standard C library functions, and the md5sum program, we verify that the output files be-
the second version (SPEC-E SCORT) uses functions from fore and after transformation of SVMlight are identical.
the Escort library10 . We compile the SPEC-L IBC pro-
gram using the Clang/LLVM 3.8 compiler with the -O3 7.2 Impact of Control Flow Obfuscation
flag, and we disable auto-vectorization while compil-
ing the SPEC-E SCORT program. The following results To compare the performance impact of Escort’s con-
demonstrate the worst case performance overhead of Es- trol flow obfuscation technique with that of Raccoon,
cort for these programs, since we transform all floating- we use the same benchmarks that were used to eval-
point operations in SPEC-E SCORT to use the Escort li- uate Raccoon [28], while compiling the baseline (non-
brary. More precisely, we do not reduce the number of transformed) application with the -O3 optimization flag.
transformations either using taint tracking or using SMT Although both Escort and Raccoon obfuscate control
solvers. flow and data accesses, we compare the cost of control
Table 9 shows that Escort’s overhead is substantial, flow obfuscation only, since both Escort and Raccoon ob-
with a geometric mean of 32.6×. We expect a lower av- fuscate data accesses using the identical technique. Ta-
erage overhead for applications that use secret data, since ble 11 shows the results.
taint tracking would reduce the number of floating-point We find that programs compiled with Escort have a
operations that would need to be transformed. significantly lower overhead than those compiled with
Raccoon. Escort’s geometric mean overhead is 32%,
Evaluation Using SVMlight . To evaluate Escort’s while that of Raccoon is 5.32×. The worst-case over-
overhead on a security-sensitive benchmark, we mea- head for Escort is 2.4× (for ip-tree).
sure Escort’s performance on SVMlight , an implemen- The main reason for the vast difference in overhead
is that Raccoon obfuscates branch instructions at execu-
10 We also ran the same programs using the FTFP library, but the
tion time, which requires the copying and restoring of
programs either crashed due to errors or ran for longer than two hours,
after which they were manually terminated. 11 [Link]

84 25th USENIX Security Symposium USENIX Association

 Raccoon Escort Acknowledgments. We thank our shepherd Stephen
Benchmark
Overhead Overhead McCamant and the anonymous reviewers for their help-
ip-tree 1.01× 2.40× ful feedback. We also thank David Kohlbrenner for giv-
matrix-mul 1.01× 1.01× ing us the Firefox timing attack code. We are grateful
radix-sort 1.01× 1.06× to Jia Chen for providing us the pointer analysis library,
findmax 1.01× 1.27× and to Joshua Eversmann for help with code and discus-
crc32 1.02× 1.00× sions. This research was funded in part by NSF Grants
genetic-algo 1.03× 1.03× DRL-1441009, CNS-1314709, and CCF-1453806, C-
heap-add 1.03× 1.27× FAR (one of the six SRC STARnet Centers sponsored
med-risks 1.76× 1.99× by MARCO and DARPA), and a gift from Qualcomm.
histogram 1.76× 2.26×
map 2.04× 1.01× References
bin-search 11.85× 1.01× [1] A BADI , M., ET AL . Tensorflow: Large-scale machine learn-
heap-pop 45.40× 1.44× ing on heterogeneous distributed systems. Computing Research
classifier 53.29× 1.24× Repository abs/1603.04467 (2016).
tax 444.36× 1.67× [2] ACIIÇMEZ , O., KOÇ , Ç. K., AND S EIFERT, J.-P. On the Power
of Simple Branch Prediction Analysis. In Symposium on Informa-
dijkstra 859.65× 1.10× tion, Computer and Communications Security (2007), pp. 312–
GEO MEAN 5.32× 1.32× 320.
[3] A NDRYSCO , M., KOHLBRENNER , D., M OWERY, K., J HALA ,
Table 11: Performance comparison of benchmarks com- R., L ERNER , S., AND S HACHAM , H. On Subnormal Floating
piled using Raccoon and Escort. We only compare the Point and Abnormal Timing. In Symposium on Security and Pri-
vacy (S&P) (2015), pp. 623–639.
control flow obfuscation overhead, since both Raccoon
and Escort use the same technique for data access obfus- [4] B RUMLEY, D., AND B ONEH , D. Remote Timing Attacks are
Practical. Computer Networks 48, 5 (2005), 701–716.
cation.
[5] C LEEMPUT, J. V., C OPPENS , B., AND D E S UTTER , B. Com-
piler Mitigations for Time Attacks on Modern x86 Processors.
Transactions on Architecture and Code Optimization 8, 4 (Jan.
the stack for each branch instruction. Since the stack 2012), 23:1–23:20.
can be arbitrarily large, such copying and restoring adds [6] C RANE , S., H OMESCU , A., B RUNTHALER , S., L ARSEN , P.,
substantial overhead to the running time of the program. AND F RANZ , M. Thwarting Cache Side-Channel Attacks
Through Dynamic Software Diversity. In Network and Dis-
On the other hand, Escort’s code rewriting technique ob-
tributed System Security Symposium (2015).
fuscates code at compile time using basic block predi-
[7] D E M OURA , L., AND B JØRNER , N. Z3: An Efficient SMT
cates, which enables significant performance boosts on Solver. In International Conference on Tools and Algorithms for
the above benchmarks. the Construction and Analysis of Systems (2008), pp. 337–340.
[8] D EMMEL , J. W. Effects of Underflow on Solving Linear Sys-
tems. Tech. Rep. UCB/CSD-83-128, EECS Department, Univer-
8 Conclusions sity of California, Berkeley, Aug 1983.
[9] G ANDOLFI , K., M OURTEL , C., AND O LIVIER , F. Electromag-
In this paper, we have presented Escort, a compiler-based netic Analysis: Concrete Results. In Third International Work-
shop on Cryptographic Hardware and Embedded Systems (2001),
tool that closes side channels that stem from floating- pp. 251–261.
point operations. Escort prevents an attacker from in- [10] G ROSSSCH ÄDL , J., O SWALD , E., PAGE , D., AND T UNSTALL ,
ferring secret floating-point operands through the tim- M. Side-Channel Analysis of Cryptographic Software via Early-
ing channel, though micro-architectural state, and also terminating Multiplications. In International Conference on In-
through off-chip digital side channels, such as memory formation Security and Cryptology (2010), pp. 176–192.
address trace. [11] I SLAM , M. S., K UZU , M., AND K ANTARCIOGLU , M. Access
Pattern Disclosure on Searchable Encryption: Ramification, At-
Escort uses native SSE instructions to provide speed tack and Mitigation. In Network and Distributed System Security
and precision. Escort’s compiler-based approach enables Symposium, NDSS (2012).
it to support a significantly larger number of floating- [12] JANA , S., AND S HMATIKOV, V. Memento: Learning Secrets
point operations (112) than FTFP (19). from Process Footprints. In Symposium on Security and Privacy
Escort’s design motivates further research into hard- (S&P) (2012), pp. 143–157.
ware support for side-channel resistant systems. For ex- [13] K AHAN , W. Interval Arithmetic Options in the Proposed
IEEE Floating-Point Arithmetic Standard. Interval Mathematics
ample, by allowing software to control the timing of in- (1980), 99–128.
teger instruction latencies and their pipelined execution,
[14] KOCHER , P. C. Timing Attacks on Implementations of Diffie-
Escort’s guarantees could be extended to instructions be- Hellman, RSA, DSS, and Other Systems. In Advances in Cryp-
yond floating-point instructions. tology (1996), pp. 104–113.

USENIX Association 25th USENIX Security Symposium 85

[15] KOCHER , P. C., JAFFE , J., AND J UN , B. Differential Power [33] S TEFANOV, E., VAN D IJK , M., S HI , E., F LETCHER , C. W.,
Analysis. In 19th Annual International Cryptology Conference R EN , L., Y U , X., AND D EVADAS , S. Path ORAM: An Ex-
on Advances in Cryptology (1999), pp. 388–397. tremely Simple Oblivious RAM Protocol. In Conference on Com-
[16] KONG , J., ACIIÇMEZ , O., S EIFERT, J., AND Z HOU , H. puter and Communications Security (2013), pp. 299–310.
Hardware-Software Integrated Approaches to Defend Against [34] T UKEY, J. Exploratory Data Analysis. Pearson, 1977.
Software Cache-Based Side Channel Attacks. In International
[35] WANG , Y., F ERRAIUOLO , A., AND S UH , G. E. Timing Chan-
Conference on High-Performance Computer Architecture (2009),
nel Protection for a Shared Memory Controller. In International
pp. 393–404.
Symposium on High Performance Computer Architecture (2014),
[17] L ATTNER , C., AND A DVE , V. S. LLVM: A Compilation Frame- pp. 225–236.
work for Lifelong Program Analysis & Transformation. In In-
[36] WANG , Z., AND L EE , R. B. New Cache Designs for Thwarting
ternational Symposium on Code Generation and Optimization
Software Cache-based Side Channel Attacks. In International
(2004), pp. 75–88.
Symposium on Computer Architecture (2007), pp. 494–505.
[18] L IU , C., H ARRIS , A., M AAS , M., H ICKS , M., T IWARI , M.,
[37] WANG , Z., AND L EE , R. B. A Novel Cache Architecture with
AND S HI , E. GhostRider: A Hardware-Software System for
Enhanced Performance and Security. In International Symposium
Memory Trace Oblivious Computation. In International Confer-
on Microarchitecture (2008), pp. 83–93.
ence on Architectural Support for Programming Languages and
Operating Systems (2015), pp. 87–101. [38] Z HANG , D., A SKAROV, A., AND M YERS , A. C. Predictive
[19] L UK , C., ET AL . Pin: Building Customized Program Analysis Mitigation of Timing Channels in Interactive Systems. In Confer-
Tools with Dynamic Instrumentation. In Conference on Program- ence on Conference on Computer and Communications Security
ming Language Design and Implementation (2005), pp. 190–200. (2011), pp. 563–574.

[20] M AAS , M., L OVE , E., S TEFANOV, E., T IWARI , M., S HI , E., [39] Z HANG , Y., AND R EITER , M. K. Duppel: Retrofitting Com-
A SANOVIC , K., K UBIATOWICZ , J., AND S ONG , D. PHAN- modity Operating Systems to Mitigate Cache Side Channels in
TOM: Practical Oblivious Computation in a Secure Processor. In the Cloud. In Conference on Computer and Communications Se-
Conference on Computer and Communications Security (2013), curity (2013), pp. 827–838.
pp. 311–324.
[21] M ARTIN , R., D EMME , J., AND S ETHUMADHAVAN , S. Time-
Warp: Rethinking Timekeeping and Performance Monitoring
Mechanisms to Mitigate Side-Channel Attacks. In International
Symposium on Computer Architecture (2012), pp. 118–129.
[22] M ASTI , R. J., ET AL . Thermal Covert Channels on Multi-core
Platforms. In USENIX Security Symposium (2015), pp. 865–880.
[23] M OLNAR , D., P IOTROWSKI , M., S CHULTZ , D., AND WAG -
NER , D. The Program Counter Security Model: Automatic De-
tection and Removal of Control-Flow Side Channel Attacks. In
International Conference on Information Security and Cryptol-
ogy (2005), pp. 156–168.
[24] M UCHNICK , S. Advanced Compiler Design and Implementation.
Morgan Kaufmann Publishers Inc., 1997.
[25] M ULLER , J.-M. On the definition of ulp(x). Tech. Rep. 2005-
009, ENS Lyon, February 2005.
[26] O SVIK , D. A., S HAMIR , A., AND T ROMER , E. Cache Attacks
and Countermeasures: the Case of AES. In RSA Conference on
Topics in Cryptology (2006), pp. 1–20.
[27] P ERCIVAL , C. Cache Missing for Fun and Profit. In Proceedings
of the Technical BSD Conference (2005).
[28] R ANE , A., L IN , C., AND T IWARI , M. Raccoon: Closing Dig-
ital Side-channels Through Obfuscated Execution. In USENIX
Conference on Security Symposium (2015), pp. 431–446.
[29] R EN , L., Y U , X., F LETCHER , C., VAN D IJK , M., AND D E -
VADAS , S. Design Space Exploration and Optimization of Path
Oblivious RAM in Secure Processors. In International Sympo-
sium on Computer Architecture (2013), pp. 571–582.
[30] S AKURAI , K., AND TAKAGI , T. A Reject Timing Attack on an
IND-CCA2 Public-key Cryptosystem. In International Confer-
ence on Information Security and Cryptology (2003), pp. 359–
374.
[31] S CHINDLER , W. A Timing Attack Against RSA with the Chi-
nese Remainder Theorem. In International Workshop on Crypto-
graphic Hardware and Embedded Systems (2000), pp. 109–124.
[32] S HI , E., C HAN , T. H., S TEFANOV, E., AND L I , M. Oblivious
RAM with O((logN)3 ) Worst-Case Cost. In Advances in Cryp-
tology (2011), pp. 197–214.

86 25th USENIX Security Symposium USENIX Association